Pattern · Feeds
Fan-out and Feed Delivery
Delivering events to millions of recipients. Fan-out on write vs fan-out on read, the celebrity problem, hybrid feeds, push vs pull in chat, and write amplification math.
11 min read
Fan-out describes the pattern where a single action produced by one user must be distributed to thousands or millions of interested recipients: social timelines (Twitter, Instagram), group chat messages (Slack, Discord), live notifications, and event broadcasts. Choosing how and when to fan out determines whether your system collapses under write amplification or stumbles under heavy read queries.
The core dilemma: push vs pull
When a user posts a photo or publishes a message, there are two fundamental ways to build their followers' feeds:
Fan-out on Write (Push)
- Feed reads are instant O(1) lookups from an in-memory list or Redis sorted set
- Low read latency, zero runtime joins
- Huge write amplification: 1 post × N followers = N writes
- Inactive users waste storage and write IOPS
Pick it when: Read-heavy networks where users have modest follower counts (average user on Instagram or Facebook).
Fan-out on Read (Pull)
- Post creation is a single O(1) database write
- Zero wasted writes for inactive followers
- Feed generation requires querying all followed users, merging, and sorting at runtime
- High P99 read latency
Pick it when: Write-heavy systems, or accounts with massive follower counts.
The celebrity problem (hot user skew)
The weakness of pure fan-out on write is the celebrity problem:
- An average user has 200 followers: posting creates 200 lightweight cache entries.
- A celebrity (an athlete, musician, or news outlet) has 50 million followers: a single post triggers 50 million writes across your database or Redis cluster.
If a celebrity posts three times in ten minutes, background worker queues back up for hours. Regular users wait minutes to see their friends' posts because workers are drowning in millions of fan-out jobs for a single account.
The hybrid architecture
Modern social networks (such as Twitter/X and Instagram) solve this dilemma with a hybrid fan-out model:
- Threshold check: When user A posts, the system checks user A's follower count against a threshold (such as 25,000 followers).
- Push for regular users: If user A has under 25,000 followers, an asynchronous worker queue fans out the post ID into each follower's precomputed timeline in Redis (e.g.
ZADD timeline:user_id post_id timestamp). - Write once for celebrities: If user A is a celebrity, the post is written only once to user A's personal posts table. No background fan-out occurs.
- Merge on read: When follower B opens their feed:
- Fetch follower B's precomputed timeline from Redis (instant O(1) sorted set slice of 50 post IDs).
- Fetch the list of celebrities follower B follows from a fast cache.
- Fetch the latest posts from those specific celebrities.
- Merge the two sorted arrays in memory and return the top 20 items.
This hybrid approach protects the background queue from write explosions while keeping 99% of feed reads sub-15ms.
Fan-out in chat and notifications
The same push-versus-pull trade-off governs messaging and notification systems:
- 1-on-1 and small group chat (under 100 members): Fan out on write. The sender sends a message, and the chat service pushes it to each member's private channel or connection gateway.
- Large community channels (Discord servers, Twitch streams with 100k+ users): Fan-out on write is impossible (one chat message would require 100,000 writes every few milliseconds). Instead, active viewers maintain open WebSockets connected to edge connection gateways. The gateway subscribes to an internal pub/sub topic and broadcasts the packet over open sockets, completely bypassing database persistence for inactive members.
Write amplification math
- Average user write amplification
- 200–500 timeline writes per post
- Celebrity write amplification
- 10,000,000+ writes if not isolated
- Redis timeline memory size
- 800 bytes per timeline (800 post IDs in a ZSET)
- Active working set cap
- keep timelines only for users active in last 30 days
- Read feed latency (hybrid)
- 5–15 ms from Redis timeline + in-memory celebrity merge
Where it bites
- Inactive users consuming storage: If you fan out posts to all 500 million registered accounts, 80% of your writes and RAM are wasted on users who haven't opened the app in a year. Fan out only to active users (logged in within the last 30 days). For dormant users, generate their feed on read when they log back in.
- Unfollow and delete cascades: If a post is deleted or a user unfollows someone, scrubbing that post from millions of precomputed timelines is prohibitively expensive. Instead, leave the ID in the timeline and filter it out dynamically when fetching post metadata.
- Lagging fan-out queues: If a major global event occurs and millions of users post simultaneously, worker queues can lag by tens of minutes. Always monitor fan-out delay (the time difference between post creation and timeline delivery).
Try it
News Feed HLD
Design a scalable news feed that balances write amplification with sub-50ms read latency.