Huge audiences and hot channels

Earlier lessons covered pushing updates to users. This one is about the extreme: one channel with millions of listeners, or an event stream far faster than anyone can read.

Cost is messages times viewers

The number that matters is deliveries per second: events per second multiplied by recipients.

At the top end, the design question changes from "how do we deliver everything?" to "what should each viewer actually see?".

Coalesce, sample, summarize

Three techniques cut deliveries without hurting the experience:

Prioritize what the individual viewer cares about: their own messages, replies to them, and messages from moderators should always be delivered, even when everything else is sampled.

Tiered fan-out

One publisher can't open a connection to two million clients. Fan-out is done in tiers, like a tree:

The event is published once, delivered to a few hundred edge servers, and each edge server writes it to its own tens of thousands of connections. Each edge server subscribes to a channel only if at least one of its clients is watching it, so quiet channels cost nothing. For the largest events, an extra tier of relay servers sits between pub/sub and the edges.

Reconnect storms

When a server or region restarts, every client it held reconnects at once. If they all retry immediately, they overload the connection servers, and every reconnect also triggers work behind them: authentication, subscriptions, fetching missed messages.

• Jittered backoff on the client spreads reconnects over tens of seconds.
• Admission control on servers accepts new connections at a bounded rate.
• Cheap resumption: clients reconnect with their last-seen id and get a compact catch-up, or just the current state, instead of replaying everything.