The Full Journey · 81 stops
One route through all of system design.
Sequenced so every concept is taught before it's needed. Ride the line stop by stop, or tap any station to jump — progress counts either way.
- done
- 0 done
- in progress
- 0 in progress
- of the route
- 0% of the route
Choose your line
Start here· Foundations
URL Shortener
Zero prerequisites and the single richest on-ramp — one easy CRUD system seeds caching, sharding, replication, load balancing, consistent hashing, IDs and queues for the whole course.
Foundations: Your First Systems
Give a complete beginner the shared web-scale vocabulary — caching, sharding, replication, load balancing, IDs, rate limits, idempotency and capacity math — on warm-up problems where nothing can go badly wrong.
URL Shortener
Up nextZero prerequisites and the single richest on-ramp — one easy CRUD system seeds caching, sharding, replication, load balancing, consistent hashing, IDs and queues for the whole course.
Beginner ~25mDesigncaching / cache-asidesharding & consistent hashingleader-follower replication / read replicas+4Pastebin
Reuses the LB/scaling you just learned and layers on durable object storage, CDN read path and the outbox/CDC/DLQ trio on a still-easy system.
Beginner ~45mDesignobject storagecdn & write-through cachecache invalidation+4Distributed Rate Limiter
A fast, self-contained win — token-bucket and sharded counters, the first time state must be shared correctly across servers.
Intermediate ~35mDesignrate limitingtoken bucketsliding window+1Submit Order (Prevent Double-Charge)
The smallest possible correctness problem — one endpoint plus retries and an idempotency key turns 'charged twice' into 'charged once', the most-reused reliability trick.
Intermediate ~40mDesignidempotency keysexactly-once vs at-least-onceoptimistic concurrency control+1Distributed Unique ID Generator
A compact, very common interview warm-up whose needs are all met — Snowflake IDs force first contact with clock sync and graceful degradation.
Intermediate ~45mDesigndistributed id generation (deep)clock synchronizationleader election (preview)+1Build a Bitcask-style KV store
The build track's zero-prereq great-first-problem — storage from first principles (append-only log, hash index, compaction, fsync) that every later database reuses, bridging into Phase 2.
Beginner ~60mBuildlog-structured storagehash indexcompaction / tombstones+3
Storage Engines From Scratch
Open the black box every design draws as a single 'database': build the storage substrate bottom-up so replication, quorum, CAP and anti-entropy are learned cheaply while building the engines, plus the index and object store used later.
Build a B-tree storage engine (SQLite-style)
Bitcask taught append-only logs; the B-tree is its read-optimized dual — paged storage, buffer pool, WAL and MVCC underpin every relational engine ahead.
Beginner ~90mBuildb-treewrite-ahead logbuffer pool / page cache+3Build an LSM-tree storage engine (LevelDB / RocksDB style)
Fuses bitcask's log with btree's pages into the LSM tree, adding bloom filters and leveled compaction that Cassandra/RocksDB/Kafka all demand.
Beginner ~75mBuildlsm treebloom filterprobabilistic data structures+3Build a wide-column store (Cassandra / DynamoDB family)
Distribute the LSM engine: the canonical home for quorums, leaderless replication, CAP, gossip and Merkle anti-entropy — one problem that unlocks tunable consistency for the entire design track.
Beginner ~90mBuildleaderless replicationquorum reads/writestunable consistency+4Build a document store (MongoDB-style)
PreviewA document store on the B-tree with oplog replication and quorum (all now taught) — adds scatter-gather and the embedding-vs-referencing tradeoff.
Intermediate ~85mBuilddocument data modelreplication log / oplogscatter-gather+1Build a Prometheus-style time-series database
A specialized LSM + inverted index for metrics — introduces time-series storage, downsampling and the cardinality trap the observability track reuses.
Beginner ~80mBuildtime-series storagedownsamplingcompression+2Build an inverted index (Lucene-style)
PreviewNeeds only hashing + sorted structures; the read-side index that powers every search product later (Elasticsearch, autocomplete, logging, yelp).
Beginner ~75mBuildinverted indexpostings listterm dictionary / fst+3Build an S3-style distributed object store
The object store other systems treat as a black box, built while replication is fresh — erasure coding, content-addressable chunking and read-after-write consistency the file/media capstones lean on.
Advanced ~95mBuildobject storage (deep)erasure codingcontent-addressable storage+2
Caching & the Edge
Open the boxes that sit in front of every service — CDN, load balancer, cache-failure modes, a distributed cache fleet, Redis itself — and close with fleet-wide cache correctness. A coherent, enter-sideways caching module.
Build a CDN
Pastebin used a CDN as a black box; open it — anycast, TTL, ETag/conditional GET, stale-while-revalidate and cache-key design (authored anchor for the phase).
Intermediate ~75mBuildcdn internalsanycastetag / conditional get+3Build a load balancer (HAProxy / NGINX style)
Previewurl-shortener named load balancing; now build it — L4/L7, health checks, power-of-two-choices and connection draining.
Intermediate ~80mBuildl4 vs l7 load balancingpower-of-two-choices / least-connectionshealth checks+3Cache Stampede / Thundering Herd
PreviewWith caching + TTL + hot-key internalized, confront the classic production failure: coordinated expiry and thundering herds.
Intermediate ~45mDesigncache stampederequest coalescing / singleflightprobabilistic early expirationNegative Caching
PreviewA short companion to stampede — cache the absence of data behind a bloom filter to kill pointless lookups.
Intermediate ~45mDesignnegative cachingbloom-filter gatingBuild a distributed cache (Memcached / Pelikan style)
PreviewAssemble consistent hashing + TTL + stampede defenses into a Memcached-style fleet; adds rendezvous hashing.
Intermediate ~75mBuilddistributed cache designrendezvous hashingreplicated cache nodesBuild Redis
The caching capstone — single-threaded event loop, data-structure encodings, COW snapshots and replication/failover (needs CAP from build-wcs and leader election from pastebin, both now taught).
Advanced ~75mBuildsingle-threaded event loopdata-structure encodingscache eviction+2Cache Invalidation Across a Fleet
The hard version of everything above — keeping a global cache fleet coherent with two-generals, leases, write-behind and consistency monitoring; all its needs (pub/sub, cdc, eventual consistency) are now taught.
Advanced ~45mDesigntwo generals problemleases (cache)write-behind cache+1
Async Backbone: Queues, Pub/Sub & Delivery
Decouple services with queues and get delivery semantics right — the exactly-once/idempotency/DLQ machinery every real-time, counter and transaction system downstream assumes — then apply it to reliable fan-out delivery.
Build a Message Queue (RabbitMQ / SQS)
Build the queue everything else assumes: at-least-once vs exactly-once, leases, dead-letter queues and backpressure (authored anchor).
Intermediate ~80mBuildmessage queue internalsexactly-once vs at-least-oncedead letter queue+2Build a pub/sub system (Google Pub/Sub / Redis Pub/Sub style)
PreviewExtend the queue to one-to-many — fan-out on write, push vs pull and delivery-semantics tradeoffs, seeding fan-out for feeds and chat.
Intermediate ~70mBuildpub/subfan-out on writepush vs pull+1Webhook Delivery
PreviewFirst real use of the queue+idempotency stack — retries with backoff, DLQ and the outbox pattern for reliable external delivery.
Intermediate ~45mDesignretries with backoffdead letter queue (delivery)outbox patternDistributed Job Scheduler
PreviewRun work exactly once across a fleet using the queue plus a lightweight leader — a gentle first taste of leader election and DAG scheduling.
Intermediate ~45mDesigndistributed schedulingdag schedulingexactly-once execution (jobs)+1Notification System
The canonical fan-out consumer — queues, pub/sub, DLQ, rate limiting, consumer-group rebalancing and provider bulkheads at scale.
Intermediate ~60mDesignmulti-channel fan-outconsumer-group rebalancingbulkheads+1Push Notification Fanout (1 → 100M)
PreviewPush the fan-out to 100M (build-pubsub already taught fan-out on write) — amplification, dedup, backpressure and DLQ under extreme skew.
Advanced ~45mDesignmassive fan-out on writedelivery dedupbackpressure under fan-out
Real-Time: Presence & Live Delivery
Push data to live clients over WebSockets. Everything here needs only websockets + pub/sub + caching, so it slots in before consensus and stays approachable; the authored online-indicator carries the load-bearing presence concept.
Online Indicator
Authored presence anchor that introduces WebSockets and sharded presence state at social-graph scale, establishing the primitives the rest of the phase reuses.
Intermediate ~45mDesignconnection state / presencewebsocketsheartbeats & expiry+2Typing Indicators
PreviewThe minimal ephemeral case now that presence is understood — TTL-based expiry and debounce with almost no failure modes.
Beginner ~45mDesignttl-based ephemeralitydebouncingpresence heartbeatsActive-on-Document Avatars
PreviewSame presence primitives fanned out to a document's viewers with live cursor broadcast — the on-ramp to collaboration.
Intermediate ~45mDesignpresence fan-outcursor broadcastStock Ticker / Price Feed
PreviewFirst real backpressure problem — a firehose feed where slow clients must not stall fast ones; introduces causal ordering.
Intermediate ~45mDesignclient backpressurepush vs pull tradeoffscausal orderingLive Auction Bidding
PreviewReal-time plus correctness — optimistic concurrency, bid ordering, anti-sniping and fairness over WebSockets.
Intermediate ~45mDesignbid orderingcontended optimistic concurrencyanti-sniping / fairnessConcurrent Hotel Viewers
A live counter under extreme hot-key load — edge fan-in, request coalescing and HyperLogLog for approximate concurrency.
Intermediate ~45mDesignedge aggregation / fan-insingleflight coalescingbroadcast coalescing+1Live Comments / Score Updates
High-volume live fan-out introduces load shedding and fail-open-vs-closed under overload.
Intermediate ~55mDesignload sheddingfail-open vs fail-closedbulkheadingLive Sports Scores to Millions
Broadcast to millions — thundering-herd control on reconnect, last-event-id replay, anycast edge and tiered storage.
Intermediate ~55mDesignthundering-herd controllast-event-id replayanycast edge+1
Counters, Feeds, Timelines & Chat
Master the fan-out spectrum (write/read/hybrid) and high-write aggregation. The iconic interview problems live here — twitter, instagram, slack, whatsapp — so a learner is interview-competent by mid-course, not after a capstone slog.
Like Button at Scale
The first hot-key write problem — sharded counters, CRDT counters and idempotent dedup on top of the queue + replication you now have.
Intermediate ~45mDesignsharded countershot-key write mitigationcrdt counters+2Read Receipts
PreviewFan-out on write meets per-recipient state and exposes the write-amplification tradeoff in miniature.
Intermediate ~45mDesignper-recipient statewrite amplificationView Count on a Video/Post
Introduces the lambda architecture — a speed layer (sharded counters + HLL) reconciled by a batch/stream layer; seeds stream processing for the whole data-intensive tier.
Intermediate ~45mDesignhyperloglog / approximate countinglambda architecturestream processing+2Reddit / Hacker News
Vote-driven ranking sits right on top of sharded counters, adding a bloom-filter seen-check and feed materialization.
Intermediate ~45mDesignvote-driven rankingmaterialized-view rankingbloom-filter dedupTwitter / X Timeline
THE feed problem — fan-out on write vs read vs hybrid, push-vs-pull and the celebrity hot key, plus cursor pagination.
Intermediate ~45mDesignfan-out on write / on readhybrid fan-outcursor pagination+1Instagram News Feed
Extends the timeline with two-stage retrieval, an online feature store and graceful degradation under deadline pressure.
Intermediate ~45mDesigntwo-stage retrievalonline feature storedeadline propagation+1Slack / Discord
Real-time chat combining presence (from Phase 5), per-channel fan-out and cell-based architecture — a top-5 interview problem placed the moment its prereqs are ready.
Intermediate ~50mDesigncell-based architectureper-channel fan-out at scaleconnection routingWhatsApp / Messenger
The messaging capstone — per-conversation FIFO, hybrid fan-out and multi-region active-active; needs fan-out on read, which twitter/instagram just taught.
Advanced ~50mDesignper-conversation FIFO orderingdelivery / read statemulti-region active-active messaging
Consensus, Coordination & Service Infrastructure
Derive consensus once (Raft) with replication/WAL/failover already understood, then reuse it for every lock, leader, coordination service and CRDT — and build the RPC + mesh substrate that ties fleets together. The conceptual keystone of the course.
Build Raft — consensus you can defend
The heart of the course — with replication/WAL/failover in hand, derive consensus: leader election, replicated state machine and linearizability.
Advanced ~100mBuildraft consensusreplicated state machineleader election+4Distributed Lock
Build a correct lock on quorum + leases + fencing tokens and see exactly why clock skew breaks the naive version.
Advanced ~45mDesigndistributed lockfencing tokensleases+2Leader Election for a Singleton Job
PreviewApply Raft to the smallest useful goal — one active worker — confronting split-brain, lease renewal and fencing grounded in the lock and Raft you just built.
Advanced ~45mDesignsplit-brainlease renewalfencing tokens (applied)Build a coordination service (ZooKeeper / etcd style)
PreviewPackage Raft + locks + leases into a ZooKeeper/etcd — watches, ephemeral nodes and sessions that later systems depend on.
Advanced ~90mBuildcoordination servicewatches / edge-triggered notificationsephemeral nodes+2Build a CRDT library
PreviewThe conflict-free path to convergence — join-semilattice merge and vector clocks — placed right before the collaborative products that consume CRDTs. (Justified exception: this is the canonical introduction of the vector clocks it nominally needs.)
Advanced ~80mBuildcrdtsjoin-semilattice mergevector clocks+1Build a gRPC-style RPC framework
Build the call layer every service rides — partial failure, deadline/cancellation propagation, streaming and retry budgets that prevent retry storms.
Advanced ~85mBuildrpc & stubspartial failuredeadline / cancellation propagation+3Build a Service Mesh (Envoy / Istio style)
Wrap every RPC call — sidecars, control/data plane, circuit breakers, mTLS and traffic splitting for fleet-wide resilience and identity.
Advanced ~90mBuildsidecar patterncontrol vs data planecircuit breaker / outlier detection+3Distributed Cron — Mass Scheduled Email
Distributed cron at scale — fencing + leader election + idempotency + batch reconciliation to fire millions of jobs exactly once, applying the whole coordination toolkit.
Advanced ~75mDesigndistributed scheduling at scalebulkhead isolationbatch reconciliation (jobs)
Transactions, Consistency & Money
Money-grade correctness. CDC/outbox solve the dual-write problem, then sagas/2PC/durable-execution build up to distributed SQL and TrueTime-backed external consistency.
Build a CDC pipeline (Debezium + outbox)
Solve the dual-write problem at the source — CDC + outbox + schema registry stream DB changes reliably; the backbone of every saga ahead.
Intermediate ~75mBuildcdcoutbox patterndual-write problem+2Ticketmaster / Hotel Booking
First multi-step transaction — reservation/escrow, distributed locks, sagas and 2PC on contended inventory.
Intermediate ~55mDesignescrow / reservationsagastwo-phase commit+2Payment / Wallet System
Money-grade correctness — double-entry ledger, idempotent sagas, 2PC and batch reconciliation where mistakes cost real dollars.
Advanced ~45mDesignledger / double-entryidempotency (financial)sagas (financial)+2Build a workflow engine (Temporal / Airflow / Cadence style)
Generalize sagas into durable execution — deterministic replay, event sourcing, durable timers and workflow versioning (also teaches the event sourcing stock-exchange needs next).
Advanced ~90mBuilddurable executiondeterministic replayevent sourcing+3Stock Exchange / Order Matching
Determinism under contention — single-writer matching, deterministic replay and event sourcing with Raft-backed failover at microsecond scale.
Advanced ~45mDesignsingle-writer principledeterministic replay (matching)clock synchronization (trading)+1Build a distributed SQL engine (CockroachDB-style)
PreviewAssemble Raft + MVCC + 2PC into distributed SQL — ranges, leaseholders, serializable (SSI) isolation and hybrid logical clocks.
Advanced ~100mBuildrange-based shardinghybrid logical clocksserializable (ssi) isolation+2Build a Spanner-style strongly consistent distributed database
PreviewStrong consistency with physics — TrueTime / bounded clock uncertainty for external consistency over 2PC + Raft.
Advanced ~100mBuildtruetime / bounded clock uncertaintyexternal consistencysnapshot reads+1
Data-Intensive: Streaming, Search & Analytics
Build the log, stream, columnar and observability engines, then the analytical and search products on top — everything here needs the storage engines from P2, the counter/stream primitives from P6, and consensus from P7.
Build Kafka
The messaging/log capstone — a replicated partitioned log with consumer offsets, leader election and exactly-once, unifying the whole async track (needs leader election, now taught).
Advanced ~90mBuildevent log (deep)consumer offsetsisr / leader-follower replication+1Build a stream processor (Flink / Kafka Streams style)
PreviewFormalize the speed layer met in view-count — watermarks, windowing, keyed state and barrier-snapshot checkpointing.
Advanced ~95mBuildwatermarkswindowingevent-time vs processing-time+2Ad Click Aggregator
Exactly-once analytics — event-time watermarks and lambda reconciliation for billing-grade click counts over the Kafka log.
Advanced ~45mDesignevent-time watermarkslambda + batch reconciliation (billing)checkpointing+1Trending Topics
Approximate top-k over a stream — count-min sketch, windowing, hysteresis and an abuse gate.
Advanced ~75mDesigncount-min sketch / top-kwindowing (trends)hysteresis+1Live Viewer Count (YouTube/Twitch)
Capstone counter combining HLL, stream processing, sampling and load-shedding to count millions of concurrent viewers.
Advanced ~75mDesignsamplingkappa architectureload shedding+2Build a columnar OLAP store (ClickHouse / Druid style)
Rotate storage 90 degrees — columnar layout, compression and vectorized execution for analytical scans (needs HLL from view-count).
Advanced ~95mBuildcolumnar storageolapcolumn compression+3Metrics / Monitoring System
PreviewA Prometheus-style product ties your TSDB to push-vs-pull scraping and downsampling end-to-end.
Intermediate ~45mBuildmetrics pipelinepush vs pull scrapingdownsampling (retention)Build a distributed logging stack (ELK / Loki)
An ELK/Loki pipeline — backpressure, cardinality, tiered storage and sampling over the inverted index + object store.
Intermediate ~80mBuildlog ingest backpressurecardinality controltiered storage+2Build a distributed tracing system (Jaeger / Zipkin style)
PreviewTrace requests across services — context propagation and head-vs-tail sampling on a columnar store, closing the observability loop.
Intermediate ~75mBuilddistributed tracingcontext propagationhead/tail samplingBuild a distributed search engine (Elasticsearch / OpenSearch style)
Distribute the inverted index — sharded segments, scatter-gather and distributed top-k for full-text search.
Beginner ~80mBuildsearch rankingscatter-gatherdistributed top-k approximation+1Autocomplete / Typeahead
PreviewPrefix search product on the inverted index + a trie/FST with prefix sharding.
Intermediate ~35mDesigntrie / fstprefix shardingsearch-as-you-type rankingRecent Searches / People Also Searched
PreviewPer-user history with TTL, privacy/retention and cold-start handling.
Intermediate ~45mDesignper-user storageprivacy / data retentioncold startDid You Mean / Spell Correction
PreviewSpell correction over the index — edit distance and n-gram indexing on top of the ranking Elasticsearch/autocomplete just taught.
Intermediate ~45mDesignedit distancen-gram indexingquery-log miningBuild a vector database (Pinecone / Weaviate / pgvector style)
Approximate nearest-neighbor search — HNSW, IVF and product quantization on a sharded index.
Intermediate ~85mBuildvector search / annembeddingshnsw+3Web Crawler
A massive distributed graph traversal with bloom-filter dedup, politeness rate-limiting and frontier sharding — it produces the corpus the index consumes and seeds graph traversal for the capstones.
Advanced ~45mDesigngraph traversalfrontier / dedup at scalepoliteness rate limitingRecommendation System
PreviewTwo-stage candidate-gen + ranking using embeddings + ANN (from build-vectordb) and a lambda feature pipeline.
Advanced ~45mDesigncandidate generationembedding retrievalranking pipeline
Grand Capstones: Whole Products & Frontier Systems
Compose everything into whole products and frontier systems — file sync, streaming video, graph/geo matching, collaboration and agentic AI. Nothing here is tractable until the earlier phases land; each opens on an authored capstone.
Dropbox / Google Drive
File-sync capstone on the S3 you built — content-addressable chunking, erasure coding and cursor-based delta sync (authored anchor for the phase).
Advanced ~55mDesignblob chunking (sync)content-addressable storagereference counting+2YouTube / Netflix Streaming
Media capstone — hot/cold tiering, adaptive bitrate, signed URLs and CDN at massive read scale.
Advanced ~60mDesignhot/cold tieringadaptive bitrate streamingsigned urls+1Build a graph database (Neo4j / Dgraph-style)
Index-free adjacency and graph partitioning — placed here so its distributed-transactions and pessimistic-locking needs (from Phase 8) and graph traversal (from web-crawler) are actually satisfied.
Intermediate ~80mBuildindex-free adjacencyproperty graph modelgraph partitioning (vertex/edge cut)+1Yelp / Nearby Places
PreviewGeo warm-up for the spatial cluster — geohashing/quadtree over the inverted index.
Intermediate ~45mDesigngeohashing / spatial indexquadtreeUber / Lyft — Match Drivers and Riders
Geospatial matching capstone — spatial index, gossip dispatch, escrow matching and real-time dispatch in one design.
Advanced ~60mDesigngeohashing / spatial index (matching)gossip-based dispatchgeo-partitioningGoogle Maps / Routing
PreviewRouting capstone — contraction hierarchies over a road graph (graph traversal from web-crawler) plus map tiles and ETA.
Advanced ~45mDesigncontraction hierarchiesmap tileseta predictionCollaborative Editor (Google Docs)
Real-time collaboration capstone — OT vs CRDT (from build-crdt), causal ordering and single-writer serialization with lease ownership.
Advanced ~60mDesignoperational transformcrdt editingsingle-writer serialization+2AI Agent Platform
The final capstone — durable execution + sandboxing + capability scoping + cost governance stitched over the entire stack.
Advanced ~45mDesigndurable execution (agents)sandbox isolationcapability scoping / least privilege+2
Stops marked Preview are brief today — the order still holds, and their concepts are covered by neighboring problems.