Pattern · Communication
Real-time updates
Getting changes to users within moments. Choosing between polling, SSE and WebSockets, routing events to the right connection server, and handling reconnects and offline users.
11 min read
Chat messages, live scores, collaborative cursors, ride locations, notifications and dashboards all need the server to tell clients about changes as they happen. The pattern has two halves: the last hop from server to client, and the fan-out inside your system that gets each event to the server holding the right connection.
Part 1: the last hop
Polling
- Plain HTTP
- Stateless servers
- Easy to cache
- Delay up to the interval
- Wasted requests when nothing changed
Pick it when: Updates every 10+ seconds are fine, or as a fallback.
Long polling
- Near-instant
- Works through any proxy
- Held requests
- Reconnect after every message
Pick it when: Low message rates where WebSockets are unavailable.
Server-sent events
- Simple server-to-client stream over HTTP
- Automatic reconnect with Last-Event-ID
- One direction
- Text only
Pick it when: Feeds, notifications, dashboards, AI token streams.
WebSockets
- Two-way, low overhead per message
- Stateful connections
- Load balancers and deploys need care
Pick it when: Chat, games, collaboration, high message rates both ways.
Start from the simplest that meets the latency need. Many "real-time" features are fine with SSE or a 5-second poll.
Part 2: fan-out
Persistent connections live on specific servers. When an event happens anywhere, it must reach the servers holding the connections of everyone who should see it.
- Keep connection servers thin: authenticate, hold sockets, forward events. Business logic stays in stateless API services.
- Each connection server subscribes to channels for its connected users or rooms (Redis pub/sub, NATS, or Kafka with routing).
- Alternatively, keep a connection registry (user → server) in Redis and send each event directly to the right server.
- For huge fan-out (a live event with millions of viewers), go through a tree of relays or a managed service, and batch updates.
- Idle WebSocket connections per servermemory and file descriptors are the limit
- ~100,000+
- Heartbeat intervalkeeps NATs and load balancers from closing idle sockets
- 20–30 s
- Polling costeven when nothing changes
- 100,000 clients ÷ 5 s = 20,000 req/s
Part 3: reliability
- Reconnects are normal. Phones switch networks constantly. Clients reconnect with backoff and send the id of the last event they saw; the server replays from a short retention buffer or tells the client to resync.
- Persist first, then push. Write the message to durable storage, then publish. A missed push is recoverable; a lost write is not.
- Deduplicate on the client by event id, since replays and retries can deliver twice.
- Order per conversation or room with a sequence number from one owner.
- Offline users get push notifications (APNs, FCM) instead, and catch up on next open.
- Presence uses heartbeats with expiring keys rather than trusting disconnect events.
Try it
Chat Service HLD
Design a chat service with delivery, ordering and presence.