Caching Strategies

Why Caching Works

Most workloads are wildly skewed: a tiny fraction of data gets the vast majority of reads (the 80/20 rule, often more like 95/5). A cache keeps those hot answers in fast storage (usually RAM) so repeated requests skip the slow path — the database query, the API call, the expensive computation.

Recall the latency table from Topic 01: RAM ≈ 100 ns, SSD ≈ 100 µs. A cache hit can be 1,000× faster than the query it replaces — and it spares your database the work entirely.

The Caching Layers (You're Already Using Five)

Cache Update Strategies

The hard part of caching isn't reading — it's keeping the cache and the database telling the same story. Four patterns:

1. Cache-aside (lazy loading) — the default

def get_user(user_id):
 user = cache.get(f"user:{user_id}")
 if user is None: # miss
 user = db.query("SELECT … WHERE id = %s", user_id)
 cache.set(f"user:{user_id}", user, ttl=3600)
 return user

The app owns the logic: check cache → on miss, load from DB → populate cache. Only requested data gets cached, and a dead cache just means slow (not broken). Risks: stale data until TTL expires, and three round trips on every miss. On writes, the usual move is to invalidate (delete) the key, not update it.

2. Write-through

Every write goes to the cache, and the cache synchronously writes to the DB before confirming. The cache is always fresh and reads after writes never miss. Cost: every write pays double latency, and you cache data that may never be read.

3. Write-behind (write-back)

Writes land in the cache and are acknowledged immediately; a background process flushes them to the DB later, often batched. Spectacular write throughput. Risk: the cache crashes before flushing and those writes are gone forever. Use for tolerable-loss data: view counters, analytics, "last seen" timestamps.

4. Refresh-ahead

For hot keys, the cache proactively re-fetches the value shortly before its TTL expires, so popular items never go cold. Works great when you can predict what stays popular; wasteful when you can't.

Eviction: Who Gets Kicked Out?

RAM is finite. When the cache fills, an eviction policy chooses the victim:

Redis vs Memcached

The Three Classic Cache Disasters

Cache stampede (thundering herd)

A hot key expires. Ten thousand concurrent requests all miss at once and all hammer the database with the same query, possibly knocking it over. Fixes: a mutex/lock so only one request recomputes while others wait or serve stale; or probabilistic early refresh — each request has a small chance of refreshing before expiry, smearing the recompute over time.

Cache penetration

Requests for keys that don't exist anywhere (random IDs, often malicious) bypass the cache every time and hit the DB. Fixes: cache the nulls (set("user:999", NOT_FOUND, ttl=60)), or keep a Bloom filter of all valid keys and reject impossible IDs before they touch storage.

Cache avalanche

Thousands of keys share the same TTL (e.g., all cached at deploy time) and expire in the same second — or the cache node itself reboots empty. The DB takes the entire load at once. Fixes: add jitter to TTLs (ttl = 3600 + random(0, 300)), warm caches before traffic, and run replicated cache clusters.

Cache-Control, decoded

# Static asset with hashed filename — cache forever
Cache-Control: public, max-age=31536000, immutable

# API response — cache 60s, then revalidate with ETag
Cache-Control: private, max-age=60
ETag: "a1b9c3"

# Sensitive page — never store
Cache-Control: no-store

Redis Data Structures — Supercharged Caching

Redis isn't just a key-value store — it's a toolbox of purpose-built data structures, each unlocking a different system design pattern:

Redis Cluster — How Data Shards Across Nodes

One Redis instance maxes out a single core and a single machine's RAM. Redis Cluster shards the keyspace across nodes using 16,384 hash slots: CRC16(key) % 16384 = slot number. Each master node owns a range of slots.

• Replicas: each master has 1+ replica for HA. If a master dies, its replica is promoted automatically.
• Resharding: slots can be moved live between nodes (MIGRATE command). Zero downtime.
• Multi-key operations: MGET/MSET only work when all keys live on the same slot. Fix with hash tags: {user:123}:name and {user:123}:age hash on just user:123, so they land on the same slot.
• Typical setup: 6 nodes (3 masters + 3 replicas) for HA. A cluster like this can handle millions of ops/sec.

Cache Stampede — The Thundering Herd Problem, In Depth

The scenario: a popular item's TTL expires. 10,000 concurrent requests all miss simultaneously → all 10,000 hit the database with the same query → DB overwhelmed → cascade failure. Three production-grade fixes:

Redis-based mutex in Python

import redis, json, time

r = redis.Redis()

def get_with_lock(key, loader_fn, ttl=3600):
 val = r.get(key)
 if val: return json.loads(val)

 lock_key = f"{key}:lock"
 lock = r.set(lock_key, "1", nx=True, ex=10) # 10s lock

 if lock:
 data = loader_fn()
 r.setex(key, ttl, json.dumps(data))
 r.delete(lock_key)
 return data
 else:
 time.sleep(0.1)
 return get_with_lock(key, loader_fn, ttl) # retry

Cache Penetration — Attacking the DB with Nonexistent Keys

The scenario: a malicious or buggy client queries keys that never exist (user_id: 9999999999). The cache always misses — there's nothing to cache — so every request hammers the DB with useless queries.

Cache Avalanche — Mass Expiry, In Depth

The scenario: you bulk-loaded your cache with TTL=3600. Every entry expires at exactly t=0 + 3600s. Traffic shifts entirely to the DB, the DB overloads, the cache can't refill fast enough, and the system collapses. Four defenses:

• TTL jitter: instead of EX 3600, use EX random(3000, 3600). Keys expire scattered over 10 minutes, not all at once.
• Staggered cache warmup: don't load every key at startup — warm them gradually or on-demand.
• Circuit breaker at the DB layer: if DB response time exceeds a threshold, stop accepting cache-miss queries and return an error or stale data. Caps the damage even if an avalanche starts.
• Redis persistence (RDB/AOF): on restart, Redis reloads from disk — the cache is not cold after a reboot.

HTTP Cache Headers — The Browser Cache Layer

The earlier Cache-Control primer covered the common cases; here's the full vocabulary:

How Twitter Uses Redis

• Home timeline: a sorted set per user, score = tweet timestamp. ZADD timeline:user:1 timestamp tweet_id.
• Who-to-follow: intersection of mutual follow sets to find suggestions (SINTERSTORE).
• Rate limiting: sliding window rate limiter on a sorted set (score = timestamp, member = request_id).
• Trending topics: ZINCRBY on each hashtag mention, ZREVRANGE for the top 10.

Memcached vs Redis — The Full Scorecard

Interview Q&A

Three caching questions interviewers actually ask — click to reveal strong answers.

Anti-Patterns to Avoid

Caching Interview Questions

Key interview questions on caching strategies, implementation, and optimization.

Core Conceptual Questions

Scenario-Based Questions

Practical Implementation

Further Reading

Interactive: Cache Hit / Miss Visualizer

Click any URL button to simulate a request. The first request always misses (hits the DB at 85ms). Repeat the same URL for a cache hit at 2ms. Fill up the 6-slot cache to see LRU eviction kick in — the least recently used entry is evicted when you add a 7th key.