LESSON 31.3
Chat app end to end
Applying the delivery framework to design a real-time messaging platform with persistent WebSocket connections, message routing, and offline sync.
7 min read
Real-time messaging (such as WhatsApp, Slack, or Discord) is a premier system design interview challenge. It moves beyond standard stateless HTTP request-response patterns into persistent stateful connections, distributed message routing, presence tracking, and strict ordering guarantees.
Here is the complete delivery framework walkthrough for a distributed chat service.
1. Requirements
- Functional Requirements:
- 1-on-1 direct messaging and group messaging (up to 500 members per group).
- Real-time message delivery with delivery receipts (sent, delivered, read).
- Offline message storage and synchronization when disconnected users come back online.
- User online/offline presence tracking.
- Non-Functional Requirements:
- Low latency: Sub-100 ms message delivery when both users are online.
- Guaranteed delivery: Messages must never be lost (at-least-once delivery with deduplication).
- Strict ordering: Messages within a conversation must be displayed in causal chronological order.
- Scale: 50 million concurrent active connections.
2. Estimates and scale
- Concurrent connections: 50 million connected users simultaneously.
- Connection memory: Each idle WebSocket connection consumes roughly 10 KB of kernel memory. 50M * 10 KB ≈ 500 GB of RAM. Spread across servers with 64 GB RAM (each hosting 500k connections), we require roughly 100 WebSocket gateway nodes.
- Message throughput: Average 10 messages per user per day, resulting in 500M messages per day, roughly 6,000 messages per second average and 25,000 messages per second peak.
- Storage math: 500M messages * 200 bytes ≈ 100 GB of new text per day, totaling roughly 36.5 TB per year.
3. API design and data model
WebSocket events for real-time interaction:
SEND_MESSAGEpayload:client_msg_id, conversation_id, contentMESSAGE_DELIVEREDpayload:msg_id, conversation_id, statusSYNC_MESSAGES:GET /v1/conversations/:id/messages?after_id=m98(for offline catch-up).
Data model:
conversations:(conversation_id, type [DIRECT|GROUP], created_at)conversation_members:(conversation_id, user_id, last_read_message_id)messages:(message_id, conversation_id, sender_id, content, created_at)with primary key(conversation_id, message_id).- Use 64-bit Snowflake IDs for
message_idso that IDs are naturally time-ordered within each conversation.
4. High-level design
Deep dive 1: Connection management and message routing
When Alice sends a message to Bob, Alice's WebSocket is connected to Gateway-3, while Bob's WebSocket is connected to Gateway-87. How does Gateway-3 route the message to Gateway-87?
- Session Registry in Redis: When Bob connects to Gateway-87, Gateway-87 writes a mapping in Redis:
SET user:bob:session "gateway-87" EX 300. Gateway-87 sends periodic heartbeats to refresh the key. - Lookup and routing: When Alice's message arrives at the Message Service, it checks Redis for Bob's session.
- Internal dispatch: The service publishes the message to an internal queue or gRPC channel subscribed by Gateway-87 (
dispatch/gateway-87). Gateway-87 receives the payload and pushes it down the active WebSocket connection to Bob. - Offline handling: If Bob has no active entry in Redis, the Message Service immediately publishes a push notification event to Firebase Cloud Messaging (FCM) or Apple Push Notification service (APNs).
Deep dive 2: Message ordering and offline sync
In mobile environments, network connections drop constantly and clocks drift. Guaranteeing correct message ordering requires careful design:
Sender Local Timestamp
- Captures client perception of send time
- Vulnerable to malicious or drifted client phone clocks; causes messages to appear out of order or far in the future
Pick it when: Never use for server message ordering.
Server-Assigned Snowflake ID
- Monotonically increasing 64-bit integer assigned by the server upon receipt
- Guarantees deterministic, tamper-proof ordering within each conversation
- Minor clock drift across generator nodes handled with sequence counters
Pick it when: All production messaging systems.
When a user reconnects after being offline for hours, the client sends GET /messages?conversation_id=C1&after_id=M100. The server queries the primary key index on (conversation_id, message_id) where message_id > M100, returning only the delta of missed messages in a single sequential read.
To build this design step by step, with a load test at the end, open the WhatsApp lab.
Try it
Chat Service HLD
Or design it freely in the practice workspace.