Build your own Redis
An in-memory data-structure server: one thread, rich types, optional persistence, async replication. Internalize the cost of single-threaded simplicity and a dozen caching/HA decisions get easier.
- Scenes
- 10 interactive scenes
- Time
- about 70 minutes
- Topic
- Caching, Proxies & the Edge
What you are building, and why
You are designing an in-memory data-structure server: clients send commands, the server mutates rich types (strings, lists, hashes, sets, sorted sets, streams) held in RAM, and optionally persists to disk. This is the canonical "your data structures, on a server" system — once you have built it, you understand why a single thread is fast (and where it is fatal), why "Redis transactions" are not ACID, why default Redis can lose ~1 second of writes, why Sentinel is not CP, why Cluster has no proxy, and how every distributed-systems decision in Redis flows from one foundational choice: one thread.
Resist the urge to "describe Redis." Make decisions yourself, defend them, and let the AI push back. The point is to make every choice the Redis authors made and feel why they made it.
What you will be able to explain afterwards
- single-threaded event loop
- data-structure encodings
- RDB + AOF persistence
- sampled LRU / LFU eviction
- async PSYNC replication
- Sentinel quorum + majority
- Cluster hash slots & client routing
The loop
One thread, one command — and the encoding under it.
- 01Foundations — what Redis is, words you'll hearAn in-memory data-structure server, the eight core nouns, and the six canonical types. Orientation before you touch the internals.~7 min
- 02One thread, one command at a time — the Redis event loop and slow-command stallsWhy a single event loop is fast — and why one slow command (KEYS *, big LRANGE, slow Lua) stalls every client.~7 min
- 03Encodings flip under youListpack ↔ hashtable, intset ↔ hashtable, embstr ↔ raw — crossing a threshold silently rewrites memory and op-cost.~7 min
Memory
Persistence, eviction, TTL — what makes RAM disappear.
- 04Persistence — fork, CoW, and the 1-second windowRDB snapshots via fork+CoW, AOF fsync policies, and why default Redis can lose ~1s of writes on crash.~7 min
- 05Eviction is sampled, not exactmaxmemory + sampled LRU/LFU — Redis only inspects N keys per pass; tunable via `maxmemory-samples`.~7 min
- 05aTTL and cleanup — lazy, active, and the freer threadPassive + 10Hz active sampler expire keys; DEL of a big value freezes the loop unless `lazyfree-*`/UNLINK offloads to the background freer thread. Cache stampede + jitter / SET NX rebuilder lock.~7 min
HA
Replication and Sentinel — async by design.
- 06Replication is async — acked writes can vanishPSYNC, replication backlog, and the AP-not-CP gotcha: WAIT doesn't fix it; min-replicas-to-write does (at the cost of unavailability).~7 min
- 07Sentinel — quorum detects, majority electsSDOWN/ODOWN ladder, epoch-based election, and the operator footgun: quorum ≠ majority.~7 min
Shard & ship
Cluster slots, MOVED/ASK, and the design canvas.
- 08Cluster — 16384 slots and the client routesCRC16 mod 16384, MOVED vs ASK (permanent vs transient), hash tags, configEpoch — and why sharding alone is not HA.~7 min
- 09Design your Redis deploymentCapstone: pick persistence, HA, and sharding for a stated SLO; the verifier traces each choice back to the scene that taught it.~7 min
Where you'll use this
Product designs whose trade-offs turn on what this curriculum teaches.
- URL ShortenerShorten a long URL. Read-heavy. Don't collide.
- Distributed Rate LimiterEnforce a per-key request limit across a fleet of enforcers — accurately, in under a millisecond, without becoming the outage.
- Twitter / X TimelinePush or pull? Both. The canonical fanout problem.
- Uber / Lyft — Match Drivers and RidersMatch a rider to the closest acceptable driver in under 3 s. Geohash, S2, surge.
- WhatsApp / MessengerHundreds of millions of long-lived sockets, sub-second 1:1 + group delivery, E2E-encrypted, multi-device, multi-region active-active.
- Slack / DiscordChannels and history. Push or pull — and how a hot-channel fanout doesn't melt the gateway.
More in Caching, Proxies & the Edge
Everything between the client and the origin: in-memory caches, CDNs, load balancers and service proxies — and the three ways a cache betrays you.
- Cache Invalidation Across a FleetWrite-through vs write-behind. Two generals.
- Build Build a CDNA globally-distributed reverse proxy whose only job is to (a) terminate the user's TCP/TLS milliseconds away and (b) serve a cached origin response so origin never sees the request. Internalize edge caching, anycast, TTL, revalidation, SWR, purge, the Vary footgun, origin shield, bypass, and hit ratio — and the dozen ways to misconfigure each.
- Build Build a Service Mesh (Envoy / Istio style)Every microservice request crosses two proxies. This curriculum is what they do: routing, load balancing, timeout-and-retry-budget, circuit breakers, outlier detection, token-bucket rate limits, mTLS with workload identity, and a control plane that streams config to all of them. Build it in the order the production problems show up — and feel why Envoy plus a control plane has eaten the east-west world.
- Build Build a gRPC-style RPC frameworkEvery microservice talks over RPC, and the framework you ship determines half the system's failure modes. Build an RPC framework with codec, streams, deadlines, cancellation, retries, interceptors, and load-aware client-side balancing — and feel why gRPC ate the polyglot RPC market and why Thrift and JSON-over-HTTP linger.
Prefer to design it yourself?
The same subject as a staged workspace: draw the architecture, and a simulator traces requests through the boxes you drew.
Open the Build Redis workspace