Pushing to many connections

Keeping one socket open is easy. Keeping a million open means knowing which machine holds each socket, getting updates to that machine, and surviving its restart. Separate the servers doing application work from the servers holding connections so each can scale for its own workload.

Connection servers

A connection server owns live sockets and a local map from user or session id to connections. It authenticates new connections, checks subscriptions, and writes outgoing events. A user may have a phone, a laptop and several tabs, so one user can map to several sockets on different servers.

App servers validate commands and commit durable state. They don't need to keep a socket open for every recipient. A routing or pub/sub layer between the two delivers committed events to whichever connection servers currently need them.

Where does the event go?

Two common designs:

• Subscriptions. A connection server subscribes to a room or user channel while it has local listeners. Redis Pub/Sub broadcasts to all subscribers. It's fast, but a disconnected subscriber misses events; keep durable history elsewhere.
• A routing directory. Store user-to-server mappings with expiry. A router looks up recipients and forwards events to the appropriate servers, which deliver to their local sockets. Update mappings on connect, disconnect and lease expiry.

Kafka can carry the durable event stream into routing workers. But one consumer group distributes partitions among workers; it doesn't broadcast every event to every connection server. The workers must route onward, or the design must arrange independent subscriptions. State which behavior you need.

Sticky routing

Once established, a WebSocket stays on the server holding its connection. You don't rebalance individual frames across servers. Sticky sessions can help long-poll requests return to the same server when it holds local waiting state; a shared state layer can remove that requirement.

Stickiness doesn't route events from other app servers, and it doesn't survive a failed server. Reconnects can land anywhere. Drain servers during deploys, stop accepting new connections, and spread reconnect attempts with backoff and jitter.

Sizing connections

An asynchronous server might hold tens of thousands of mostly idle connections, perhaps 50,000 as a planning assumption. That's not a benchmark: TLS state, application buffers, heartbeat frequency and message rate all matter. Load-test the actual workload.

A million mostly idle sockets and a million sockets each receiving ten updates a second are very different systems. Also size outbound bandwidth, CPU, file descriptors and per-client buffers. Bound those buffers; disconnect slow readers and let them resume from history rather than exhausting memory.

Presence and heartbeats

Send periodic heartbeats, say every 30 seconds, and renew a presence lease for 90 seconds. Missing a few beats makes the session expire. The product should treat presence as an approximate hint; a device can disappear just after renewing.