Design a distributed cache

Advanced60 minBuilds Distributed CacheRead first: 7.1 Why caches work, 7.2 Read and write patterns, 7.3 Keeping caches correct, 7.4 When caches fail, 24.1 The hash ring in depth

STEP 1 OF 13

The brief

You are designing a distributed in-memory cache tier (such as a Redis or Memcached cluster) that sits between application API servers and durable databases. Downstream databases cannot handle the full volume of read traffic without becoming hopelessly saturated; the cache absorbs the vast majority of reads to deliver sub-millisecond query responses.

How big it is

Peak read traffic: 40,000 / second
Peak write traffic: 500 / second
Average item payload: 1 KB
Target p99 response time: under 100 ms

What makes distributed caching tricky

Hot keys and partition skew. When millions of users simultaneously request the same key (such as a viral post or breaking news banner), hashing can focus all traffic onto a single node, exhausting its CPU and network NIC.
Cache stampedes (thundering herds). When a popular key expires or is invalidated, thousands of concurrent requests miss at the exact same millisecond and surge downstream into the database simultaneously.
Node churn and rebalancing. Adding or losing a cache server must not flush the entire cache or cause massive key reshuffling.
Staleness and race conditions. Asynchronous invalidation events can lag or race with database reads, risking permanent storage of stale values in cache memory.

All changes saved

0 components · 0 connections

A blank canvas. A better system.

Add components from the palette, then drag between them to connect the request and event flow.

React Flow

Design a distributed cache

Advanced60 minBuilds Distributed CacheRead first: 7.1 Why caches work, 7.2 Read and write patterns, 7.3 Keeping caches correct, 7.4 When caches fail, 24.1 The hash ring in depth

STEP 1 OF 13

The brief

How big it is

Peak read traffic: 40,000 / second
Peak write traffic: 500 / second
Average item payload: 1 KB
Target p99 response time: under 100 ms

What makes distributed caching tricky

Hot keys and partition skew. When millions of users simultaneously request the same key (such as a viral post or breaking news banner), hashing can focus all traffic onto a single node, exhausting its CPU and network NIC.
Cache stampedes (thundering herds). When a popular key expires or is invalidated, thousands of concurrent requests miss at the exact same millisecond and surge downstream into the database simultaneously.
Node churn and rebalancing. Adding or losing a cache server must not flush the entire cache or cause massive key reshuffling.
Staleness and race conditions. Asynchronous invalidation events can lag or race with database reads, risking permanent storage of stale values in cache memory.

All changes saved

0 components · 0 connections

A blank canvas. A better system.

Add components from the palette, then drag between them to connect the request and event flow.

React Flow