Caching and CDNs
Caches trade freshness for speed. Know who fills the cache, how stale data gets in, and what happens when a hot key expires or one key gets more traffic than a shard can serve.
Key points
- 1
Cache-aside: the app reads the cache, loads from the database on a miss and populates the cache; writes update the database and then delete the key. Read-through, write-through and write-back move those steps into the cache layer.
- 2
Invalidation races are real. A reader that loads an old value before a write and sets it after the delete leaves stale data until TTL expiry. Leases or versioned compare-and-set prevent it, and short TTLs bound it.
- 3
Stampede size ≈ request rate × rebuild time. Bound it to one rebuild with request coalescing (a per-key lock or single-flight), stale-while-revalidate, or probabilistic early refresh (XFetch).
- 4
Cache penetration (lookups for keys that don't exist) needs negative caching with a short TTL, a Bloom filter of real keys, and rate limiting.
- 5
A hot key lives on one shard, so adding shards does not help. Use local in-process L1 caches, key replicas with random suffixes, or replica reads.
- 6
For CDNs, the cache key is the URL plus every header named in Vary. Keep Vary minimal and normalise the headers it lists;
privatekeeps responses out of shared caches entirely.
Common traps
Moving the delete before the database write does not fix the stale-set race; it just moves the window.
TTL jitter fixes many keys expiring together, not a single hot key expiring.
Pure LRU is flushed by one large scan; use a frequency-aware policy such as W-TinyLFU or keep batch jobs away from the cache.
Read the source
Test yourself on Caching and CDNs
Ten questions, with the answer and explanation after each one.