LESSON 7.3
Keeping caches correct
TTLs, explicit invalidation, versioned keys and eviction policies, and how much staleness each one allows.
6 min read
Phil Karlton's joke that cache invalidation is one of the two hard problems in computer science is popular because it's true. A cache holds a copy, and copies go stale. The goal is not to eliminate staleness but to bound it to something the product can live with, and to know exactly what that bound is.
Three ways to keep a cache fresh
TTL only
- Trivial
- Self-healing
- Data is stale for up to the TTL after every change
Pick it when: Data where minutes of staleness are fine: trending lists, public profiles.
Invalidate on write (plus a TTL)
- Fresh almost immediately after changes
- Every write path must remember to invalidate
- Races in edge cases
Pick it when: The usual choice for entity data such as users, products and links.
Versioned keys
- No invalidation needed: a new version is a new key
- Readers must know the current version
- Old keys linger until evicted
Pick it when: Static assets, configuration and computed results tied to a version.
The race nobody notices
Even "update the database, then delete the key" has a narrow window for trouble:
- Reader A misses the cache and reads the old value from the database.
- Writer B updates the database and deletes the cache key.
- Reader A, slightly delayed, writes the old value into the cache.
- The cache now holds stale data until the TTL expires.
It's rare, which is why the TTL matters: it caps how long the mistake lives. Systems that need tighter bounds use a short TTL, a delayed second delete a moment after the write, or leases where the cache refuses a set from a reader that started before the most recent invalidation (Facebook's Memcache papers describe this).
Invalidating across many copies
With one distributed cache, a delete is one call. With per-server in-memory caches or caches in several regions, every copy must hear about the change. Common approaches: publish invalidation events through pub/sub or a change stream that every cache subscribes to, or keep in-process TTLs short enough that staleness doesn't matter.
Eviction: what leaves when memory is full
A cache is smaller than the data. When it fills, an eviction policy chooses what to drop:
- LRU (least recently used) drops whatever was untouched longest. A great default for skewed workloads.
- LFU (least frequently used) keeps long-term favorites even if they weren't read in the last minute. Resists one-off scans that would flush LRU.
- TTL-based eviction removes expired entries first.
Redis lets you choose (allkeys-lru, allkeys-lfu, volatile-ttl and others). Size memory so the hot set fits comfortably; a cache that evicts its hot keys has a collapsing hit ratio.
Negative caching
When lookups for things that don't exist are frequent (unknown short codes, bots probing random IDs), cache the absence too, briefly. Otherwise every miss goes to the database. Keep the TTL short so newly created items appear quickly.
Try it
Distributed Cache HLD
Design the cache itself: partitioning, replication and eviction.