51
Cache Stampede / Thundering Herd
Coalesce, probabilistic early expiration, SWR.SavedSaved on this device — Saved on this device
01Clarifications
What would you ask before drawing a single box?
Ambiguity you would resolve with the interviewer: scope, scale, who uses it, what counts as done.
AI staff engineer
Enter to send · Shift+Enter for a new line
About Cache Stampede / Thundering Herd
Coalesce, probabilistic early expiration, SWR.
- Difficulty
- intermediate
- Time
- about 50 minutes
- Stages
- 10
- Topic
- Caching, Proxies & the Edge
How this problem is worked
Ten stages, from the questions you would ask an interviewer to the trade-offs you would defend. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.
- 01ClarificationsWhat would you ask before drawing a single box?
- 02Functional reqsWhat must this system actually do?
- 03Non-functionalWhat must it promise about speed, uptime and correctness?
- 04Capacity estimationHow much load and data does this have to hold?
- 05API designWhat does the outside world call, and what comes back?
- 06Data modelWhat gets stored, and what is it looked up by?
- 07Use-case breakdownHow does each requirement actually get served?
- 08High-level designWhich components handle a request, and in what order?
- 09Deep divesWhich part breaks first, and what do you do about it?
- 10Trade-offsWhat did this design cost, and what breaks at 10×?
Primary sources for this problem
- Vattani et al. — Optimal Probabilistic CSP
More in Caching, Proxies & the Edge
Everything between the client and the origin: in-memory caches, CDNs, load balancers and service proxies — and the three ways a cache betrays you.
- Cache Invalidation Across a FleetWrite-through vs write-behind. Two generals.
- Build Build RedisAn in-memory data-structure server: one thread, rich types, optional persistence, async replication. Internalize the cost of single-threaded simplicity and a dozen caching/HA decisions get easier.
- Build Build a CDNA globally-distributed reverse proxy whose only job is to (a) terminate the user's TCP/TLS milliseconds away and (b) serve a cached origin response so origin never sees the request. Internalize edge caching, anycast, TTL, revalidation, SWR, purge, the Vary footgun, origin shield, bypass, and hit ratio — and the dozen ways to misconfigure each.
- Build Build a Service Mesh (Envoy / Istio style)Every microservice request crosses two proxies. This curriculum is what they do: routing, load balancing, timeout-and-retry-budget, circuit breakers, outlier detection, token-bucket rate limits, mTLS with workload identity, and a control plane that streams config to all of them. Build it in the order the production problems show up — and feel why Envoy plus a control plane has eaten the east-west world.
- Build Build a gRPC-style RPC frameworkEvery microservice talks over RPC, and the framework you ship determines half the system's failure modes. Build an RPC framework with codec, streams, deadlines, cancellation, retries, interceptors, and load-aware client-side balancing — and feel why gRPC ate the polyglot RPC market and why Thrift and JSON-over-HTTP linger.
Browse the full problem catalog, or see what the simulator does and does not model.