Caching
Some answers are slow to get, so programs keep a copy close by. Follow one price through the cache-aside pattern: the app asks a Redis cache first, misses, loads from the database and keeps a copy for 60 seconds, then hits. See why copies go stale, how a time to live and invalidation keep them fresh, how a full cache evicts the least recently used copy, and where caches sit, from your browser to the database.
5 quick questions
Pick an answer to see why. Your score saves with your progress.
01In get_price, cache.get("tea") returns None. What happens next?
None means the copy is not in the cache: a miss. In cache-aside the app itself runs db.load, then cache.set keeps a copy for next time.
02What does ex=60 in cache.set(item, p, ex=60) do?
EX sets an expiry in seconds. When it runs out the key is deleted, so the next read misses and loads a fresh value.
03Another program changes tea from $3 to $4 straight in the database. The cache still holds $3 for another 30 seconds. What does a read return?
The cache was not told about the change, so the read is a hit on the old copy. The copy stays stale until its TTL runs out or someone deletes it.
04Why does set_price delete the cached copy after saving the new price?
Deleting the copy on a write is invalidation. The next get_price misses, loads the fresh price and caches it again.
05A three-slot LRU cache holds tea (last used at t = 90 s), cake (t = 80 s) and pie (t = 85 s). Jam misses. Which copy is evicted?
LRU evicts the least recently used copy. Cake was used longest ago, at t = 80 s, so cake goes and jam takes its slot.
Glossary
- Cache
- A small, fast store that keeps copies of answers that are slow or costly to get again.
- Cache hit / miss
- A hit finds the copy in the cache; a miss does not, so the answer must come from the slower source.
- Cache-aside
- A pattern where the app checks the cache first and, on a miss, loads from the database and puts a copy in the cache itself.
- TTL (time to live)
- How long a cached copy may be kept. When it runs out the copy is deleted.
- Stale data
- A cached copy that no longer matches the source of truth, because the source changed.
- Invalidation
- Removing or updating a cached copy when the data behind it changes.
- LRU (least recently used)
- An eviction rule: when the cache is full, drop the copy that was used longest ago.
- CDN
- Content delivery network: servers spread around the world that keep copies of a site's files near its users.
Notes and sources
- The figure's cache has exactly three slots so eviction is easy to see. A real Redis is limited by memory (maxmemory), and by default it refuses new writes when full (noeviction) unless you choose a policy such as allkeys-lru.
- Redis does not track exact LRU order: it samples a few keys and evicts the oldest of the sample, which comes close to true LRU. The figure shows exact LRU.
- With redis-py, cache.get returns bytes unless the client is made with decode_responses=True. db.load and db.save stand for the app's own database functions.
- Peter Norvig's table gives about 100 ns to fetch from main memory and about 8 ms (8,000,000 ns) to fetch from a new spot on a disk. A real database query can be faster (its own cache) or slower (network, heavy work).
- "There are only two hard things in Computer Science: cache invalidation and naming things." Phil Karlton, as quoted by Martin Fowler.
- Microsoft Azure Architecture Center: Cache-Aside pattern
- AWS whitepaper, Database Caching Strategies Using Redis: Caching patterns
- redis-py documentation: Redis commands (get, set, delete)
- Redis documentation: SET (EX seconds)
- Redis documentation: EXPIRE
- Redis documentation: Key eviction (maxmemory policies, approximated LRU)
- Arpaci-Dusseau, Operating Systems: Three Easy Pieces, ch. 22: Beyond Physical Memory: Policies (LRU)
- MDN: HTTP caching
- RFC 9111: HTTP Caching
- MDN Glossary: CDN
- PostgreSQL documentation: Resource Consumption (shared_buffers)
- Python documentation: functools.lru_cache
- Peter Norvig, Teach Yourself Programming in Ten Years (approximate timings table)
- Martin Fowler, TwoHardThings (Phil Karlton's quote)
Quiz Unlocks after the last step.