The Full Journey · 81 stops

One route through all of system design.

Sequenced so every concept is taught before it's needed. Ride the line stop by stop, or tap any station to jump — progress counts either way.

done
0
done
in progress
0
in progress
of the route
0%
of the route

Choose your line

Start here

URL Shortener

Zero prerequisites and the single richest on-ramp — one easy CRUD system seeds caching, sharding, replication, load balancing, consistent hashing, IDs and queues for the whole course.

Start

Foundations: Your First Systems

Give a complete beginner the shared web-scale vocabulary — caching, sharding, replication, load balancing, IDs, rate limits, idempotency and capacity math — on warm-up problems where nothing can go badly wrong.

0/6 done
  1. URL Shortener

    Up next

    Zero prerequisites and the single richest on-ramp — one easy CRUD system seeds caching, sharding, replication, load balancing, consistent hashing, IDs and queues for the whole course.

    Beginner ~25mDesign
  2. Pastebin

    Reuses the LB/scaling you just learned and layers on durable object storage, CDN read path and the outbox/CDC/DLQ trio on a still-easy system.

    Beginner ~45mDesign
  3. Distributed Rate Limiter

    A fast, self-contained win — token-bucket and sharded counters, the first time state must be shared correctly across servers.

    Intermediate ~35mDesign
  4. Submit Order (Prevent Double-Charge)

    The smallest possible correctness problem — one endpoint plus retries and an idempotency key turns 'charged twice' into 'charged once', the most-reused reliability trick.

    Intermediate ~40mDesign
  5. Distributed Unique ID Generator

    A compact, very common interview warm-up whose needs are all met — Snowflake IDs force first contact with clock sync and graceful degradation.

    Intermediate ~45mDesign
  6. Build a Bitcask-style KV store

    The build track's zero-prereq great-first-problem — storage from first principles (append-only log, hash index, compaction, fsync) that every later database reuses, bridging into Phase 2.

    Beginner ~60mBuild

Storage Engines From Scratch

Open the black box every design draws as a single 'database': build the storage substrate bottom-up so replication, quorum, CAP and anti-entropy are learned cheaply while building the engines, plus the index and object store used later.

0/7 done
  1. Build a B-tree storage engine (SQLite-style)

    Bitcask taught append-only logs; the B-tree is its read-optimized dual — paged storage, buffer pool, WAL and MVCC underpin every relational engine ahead.

    Beginner ~90mBuild
  2. Build an LSM-tree storage engine (LevelDB / RocksDB style)

    Fuses bitcask's log with btree's pages into the LSM tree, adding bloom filters and leveled compaction that Cassandra/RocksDB/Kafka all demand.

    Beginner ~75mBuild
  3. Build a wide-column store (Cassandra / DynamoDB family)

    Distribute the LSM engine: the canonical home for quorums, leaderless replication, CAP, gossip and Merkle anti-entropy — one problem that unlocks tunable consistency for the entire design track.

    Beginner ~90mBuild
  4. Build a document store (MongoDB-style)

    Preview

    A document store on the B-tree with oplog replication and quorum (all now taught) — adds scatter-gather and the embedding-vs-referencing tradeoff.

    Intermediate ~85mBuild
  5. Build a Prometheus-style time-series database

    A specialized LSM + inverted index for metrics — introduces time-series storage, downsampling and the cardinality trap the observability track reuses.

    Beginner ~80mBuild
  6. Build an inverted index (Lucene-style)

    Preview

    Needs only hashing + sorted structures; the read-side index that powers every search product later (Elasticsearch, autocomplete, logging, yelp).

    Beginner ~75mBuild
  7. Build an S3-style distributed object store

    The object store other systems treat as a black box, built while replication is fresh — erasure coding, content-addressable chunking and read-after-write consistency the file/media capstones lean on.

    Advanced ~95mBuild

Caching & the Edge

Open the boxes that sit in front of every service — CDN, load balancer, cache-failure modes, a distributed cache fleet, Redis itself — and close with fleet-wide cache correctness. A coherent, enter-sideways caching module.

0/7 done
  1. Build a CDN

    Pastebin used a CDN as a black box; open it — anycast, TTL, ETag/conditional GET, stale-while-revalidate and cache-key design (authored anchor for the phase).

    Intermediate ~75mBuild
  2. Build a load balancer (HAProxy / NGINX style)

    Preview

    url-shortener named load balancing; now build it — L4/L7, health checks, power-of-two-choices and connection draining.

    Intermediate ~80mBuild
  3. Cache Stampede / Thundering Herd

    Preview

    With caching + TTL + hot-key internalized, confront the classic production failure: coordinated expiry and thundering herds.

    Intermediate ~45mDesign
  4. Negative Caching

    Preview

    A short companion to stampede — cache the absence of data behind a bloom filter to kill pointless lookups.

    Intermediate ~45mDesign
  5. Build a distributed cache (Memcached / Pelikan style)

    Preview

    Assemble consistent hashing + TTL + stampede defenses into a Memcached-style fleet; adds rendezvous hashing.

    Intermediate ~75mBuild
  6. Build Redis

    The caching capstone — single-threaded event loop, data-structure encodings, COW snapshots and replication/failover (needs CAP from build-wcs and leader election from pastebin, both now taught).

    Advanced ~75mBuild
  7. Cache Invalidation Across a Fleet

    The hard version of everything above — keeping a global cache fleet coherent with two-generals, leases, write-behind and consistency monitoring; all its needs (pub/sub, cdc, eventual consistency) are now taught.

    Advanced ~45mDesign

Async Backbone: Queues, Pub/Sub & Delivery

Decouple services with queues and get delivery semantics right — the exactly-once/idempotency/DLQ machinery every real-time, counter and transaction system downstream assumes — then apply it to reliable fan-out delivery.

0/6 done
  1. Build a Message Queue (RabbitMQ / SQS)

    Build the queue everything else assumes: at-least-once vs exactly-once, leases, dead-letter queues and backpressure (authored anchor).

    Intermediate ~80mBuild
  2. Build a pub/sub system (Google Pub/Sub / Redis Pub/Sub style)

    Preview

    Extend the queue to one-to-many — fan-out on write, push vs pull and delivery-semantics tradeoffs, seeding fan-out for feeds and chat.

    Intermediate ~70mBuild
  3. Webhook Delivery

    Preview

    First real use of the queue+idempotency stack — retries with backoff, DLQ and the outbox pattern for reliable external delivery.

    Intermediate ~45mDesign
  4. Distributed Job Scheduler

    Preview

    Run work exactly once across a fleet using the queue plus a lightweight leader — a gentle first taste of leader election and DAG scheduling.

    Intermediate ~45mDesign
  5. Notification System

    The canonical fan-out consumer — queues, pub/sub, DLQ, rate limiting, consumer-group rebalancing and provider bulkheads at scale.

    Intermediate ~60mDesign
  6. Push Notification Fanout (1 → 100M)

    Preview

    Push the fan-out to 100M (build-pubsub already taught fan-out on write) — amplification, dedup, backpressure and DLQ under extreme skew.

    Advanced ~45mDesign

Real-Time: Presence & Live Delivery

Push data to live clients over WebSockets. Everything here needs only websockets + pub/sub + caching, so it slots in before consensus and stays approachable; the authored online-indicator carries the load-bearing presence concept.

0/8 done
  1. Online Indicator

    Authored presence anchor that introduces WebSockets and sharded presence state at social-graph scale, establishing the primitives the rest of the phase reuses.

    Intermediate ~45mDesign
  2. Typing Indicators

    Preview

    The minimal ephemeral case now that presence is understood — TTL-based expiry and debounce with almost no failure modes.

    Beginner ~45mDesign
  3. Active-on-Document Avatars

    Preview

    Same presence primitives fanned out to a document's viewers with live cursor broadcast — the on-ramp to collaboration.

    Intermediate ~45mDesign
  4. Stock Ticker / Price Feed

    Preview

    First real backpressure problem — a firehose feed where slow clients must not stall fast ones; introduces causal ordering.

    Intermediate ~45mDesign
  5. Live Auction Bidding

    Preview

    Real-time plus correctness — optimistic concurrency, bid ordering, anti-sniping and fairness over WebSockets.

    Intermediate ~45mDesign
  6. Concurrent Hotel Viewers

    A live counter under extreme hot-key load — edge fan-in, request coalescing and HyperLogLog for approximate concurrency.

    Intermediate ~45mDesign
  7. Live Comments / Score Updates

    High-volume live fan-out introduces load shedding and fail-open-vs-closed under overload.

    Intermediate ~55mDesign
  8. Live Sports Scores to Millions

    Broadcast to millions — thundering-herd control on reconnect, last-event-id replay, anycast edge and tiered storage.

    Intermediate ~55mDesign

Counters, Feeds, Timelines & Chat

Master the fan-out spectrum (write/read/hybrid) and high-write aggregation. The iconic interview problems live here — twitter, instagram, slack, whatsapp — so a learner is interview-competent by mid-course, not after a capstone slog.

0/8 done
  1. Like Button at Scale

    The first hot-key write problem — sharded counters, CRDT counters and idempotent dedup on top of the queue + replication you now have.

    Intermediate ~45mDesign
  2. Read Receipts

    Preview

    Fan-out on write meets per-recipient state and exposes the write-amplification tradeoff in miniature.

    Intermediate ~45mDesign
  3. View Count on a Video/Post

    Introduces the lambda architecture — a speed layer (sharded counters + HLL) reconciled by a batch/stream layer; seeds stream processing for the whole data-intensive tier.

    Intermediate ~45mDesign
  4. Reddit / Hacker News

    Vote-driven ranking sits right on top of sharded counters, adding a bloom-filter seen-check and feed materialization.

    Intermediate ~45mDesign
  5. Twitter / X Timeline

    THE feed problem — fan-out on write vs read vs hybrid, push-vs-pull and the celebrity hot key, plus cursor pagination.

    Intermediate ~45mDesign
  6. Instagram News Feed

    Extends the timeline with two-stage retrieval, an online feature store and graceful degradation under deadline pressure.

    Intermediate ~45mDesign
  7. Slack / Discord

    Real-time chat combining presence (from Phase 5), per-channel fan-out and cell-based architecture — a top-5 interview problem placed the moment its prereqs are ready.

    Intermediate ~50mDesign
  8. WhatsApp / Messenger

    The messaging capstone — per-conversation FIFO, hybrid fan-out and multi-region active-active; needs fan-out on read, which twitter/instagram just taught.

    Advanced ~50mDesign

Consensus, Coordination & Service Infrastructure

Derive consensus once (Raft) with replication/WAL/failover already understood, then reuse it for every lock, leader, coordination service and CRDT — and build the RPC + mesh substrate that ties fleets together. The conceptual keystone of the course.

0/8 done
  1. Build Raft — consensus you can defend

    The heart of the course — with replication/WAL/failover in hand, derive consensus: leader election, replicated state machine and linearizability.

    Advanced ~100mBuild
  2. Distributed Lock

    Build a correct lock on quorum + leases + fencing tokens and see exactly why clock skew breaks the naive version.

    Advanced ~45mDesign
  3. Leader Election for a Singleton Job

    Preview

    Apply Raft to the smallest useful goal — one active worker — confronting split-brain, lease renewal and fencing grounded in the lock and Raft you just built.

    Advanced ~45mDesign
  4. Build a coordination service (ZooKeeper / etcd style)

    Preview

    Package Raft + locks + leases into a ZooKeeper/etcd — watches, ephemeral nodes and sessions that later systems depend on.

    Advanced ~90mBuild
  5. Build a CRDT library

    Preview

    The conflict-free path to convergence — join-semilattice merge and vector clocks — placed right before the collaborative products that consume CRDTs. (Justified exception: this is the canonical introduction of the vector clocks it nominally needs.)

    Advanced ~80mBuild
  6. Build a gRPC-style RPC framework

    Build the call layer every service rides — partial failure, deadline/cancellation propagation, streaming and retry budgets that prevent retry storms.

    Advanced ~85mBuild
  7. Build a Service Mesh (Envoy / Istio style)

    Wrap every RPC call — sidecars, control/data plane, circuit breakers, mTLS and traffic splitting for fleet-wide resilience and identity.

    Advanced ~90mBuild
  8. Distributed Cron — Mass Scheduled Email

    Distributed cron at scale — fencing + leader election + idempotency + batch reconciliation to fire millions of jobs exactly once, applying the whole coordination toolkit.

    Advanced ~75mDesign

Transactions, Consistency & Money

Money-grade correctness. CDC/outbox solve the dual-write problem, then sagas/2PC/durable-execution build up to distributed SQL and TrueTime-backed external consistency.

0/7 done
  1. Build a CDC pipeline (Debezium + outbox)

    Solve the dual-write problem at the source — CDC + outbox + schema registry stream DB changes reliably; the backbone of every saga ahead.

    Intermediate ~75mBuild
  2. Ticketmaster / Hotel Booking

    First multi-step transaction — reservation/escrow, distributed locks, sagas and 2PC on contended inventory.

    Intermediate ~55mDesign
  3. Payment / Wallet System

    Money-grade correctness — double-entry ledger, idempotent sagas, 2PC and batch reconciliation where mistakes cost real dollars.

    Advanced ~45mDesign
  4. Build a workflow engine (Temporal / Airflow / Cadence style)

    Generalize sagas into durable execution — deterministic replay, event sourcing, durable timers and workflow versioning (also teaches the event sourcing stock-exchange needs next).

    Advanced ~90mBuild
  5. Stock Exchange / Order Matching

    Determinism under contention — single-writer matching, deterministic replay and event sourcing with Raft-backed failover at microsecond scale.

    Advanced ~45mDesign
  6. Build a distributed SQL engine (CockroachDB-style)

    Preview

    Assemble Raft + MVCC + 2PC into distributed SQL — ranges, leaseholders, serializable (SSI) isolation and hybrid logical clocks.

    Advanced ~100mBuild
  7. Build a Spanner-style strongly consistent distributed database

    Preview

    Strong consistency with physics — TrueTime / bounded clock uncertainty for external consistency over 2PC + Raft.

    Advanced ~100mBuild

Data-Intensive: Streaming, Search & Analytics

Build the log, stream, columnar and observability engines, then the analytical and search products on top — everything here needs the storage engines from P2, the counter/stream primitives from P6, and consensus from P7.

0/16 done
  1. Build Kafka

    The messaging/log capstone — a replicated partitioned log with consumer offsets, leader election and exactly-once, unifying the whole async track (needs leader election, now taught).

    Advanced ~90mBuild
  2. Build a stream processor (Flink / Kafka Streams style)

    Preview

    Formalize the speed layer met in view-count — watermarks, windowing, keyed state and barrier-snapshot checkpointing.

    Advanced ~95mBuild
  3. Ad Click Aggregator

    Exactly-once analytics — event-time watermarks and lambda reconciliation for billing-grade click counts over the Kafka log.

    Advanced ~45mDesign
  4. Trending Topics

    Approximate top-k over a stream — count-min sketch, windowing, hysteresis and an abuse gate.

    Advanced ~75mDesign
  5. Live Viewer Count (YouTube/Twitch)

    Capstone counter combining HLL, stream processing, sampling and load-shedding to count millions of concurrent viewers.

    Advanced ~75mDesign
  6. Build a columnar OLAP store (ClickHouse / Druid style)

    Rotate storage 90 degrees — columnar layout, compression and vectorized execution for analytical scans (needs HLL from view-count).

    Advanced ~95mBuild
  7. Metrics / Monitoring System

    Preview

    A Prometheus-style product ties your TSDB to push-vs-pull scraping and downsampling end-to-end.

    Intermediate ~45mBuild
  8. Build a distributed logging stack (ELK / Loki)

    An ELK/Loki pipeline — backpressure, cardinality, tiered storage and sampling over the inverted index + object store.

    Intermediate ~80mBuild
  9. Build a distributed tracing system (Jaeger / Zipkin style)

    Preview

    Trace requests across services — context propagation and head-vs-tail sampling on a columnar store, closing the observability loop.

    Intermediate ~75mBuild
  10. Build a distributed search engine (Elasticsearch / OpenSearch style)

    Distribute the inverted index — sharded segments, scatter-gather and distributed top-k for full-text search.

    Beginner ~80mBuild
  11. Autocomplete / Typeahead

    Preview

    Prefix search product on the inverted index + a trie/FST with prefix sharding.

    Intermediate ~35mDesign
  12. Recent Searches / People Also Searched

    Preview

    Per-user history with TTL, privacy/retention and cold-start handling.

    Intermediate ~45mDesign
  13. Did You Mean / Spell Correction

    Preview

    Spell correction over the index — edit distance and n-gram indexing on top of the ranking Elasticsearch/autocomplete just taught.

    Intermediate ~45mDesign
  14. Build a vector database (Pinecone / Weaviate / pgvector style)

    Approximate nearest-neighbor search — HNSW, IVF and product quantization on a sharded index.

    Intermediate ~85mBuild
  15. Web Crawler

    A massive distributed graph traversal with bloom-filter dedup, politeness rate-limiting and frontier sharding — it produces the corpus the index consumes and seeds graph traversal for the capstones.

    Advanced ~45mDesign
  16. Recommendation System

    Preview

    Two-stage candidate-gen + ranking using embeddings + ANN (from build-vectordb) and a lambda feature pipeline.

    Advanced ~45mDesign

Grand Capstones: Whole Products & Frontier Systems

Compose everything into whole products and frontier systems — file sync, streaming video, graph/geo matching, collaboration and agentic AI. Nothing here is tractable until the earlier phases land; each opens on an authored capstone.

0/8 done
  1. Dropbox / Google Drive

    File-sync capstone on the S3 you built — content-addressable chunking, erasure coding and cursor-based delta sync (authored anchor for the phase).

    Advanced ~55mDesign
  2. YouTube / Netflix Streaming

    Media capstone — hot/cold tiering, adaptive bitrate, signed URLs and CDN at massive read scale.

    Advanced ~60mDesign
  3. Build a graph database (Neo4j / Dgraph-style)

    Index-free adjacency and graph partitioning — placed here so its distributed-transactions and pessimistic-locking needs (from Phase 8) and graph traversal (from web-crawler) are actually satisfied.

    Intermediate ~80mBuild
  4. Yelp / Nearby Places

    Preview

    Geo warm-up for the spatial cluster — geohashing/quadtree over the inverted index.

    Intermediate ~45mDesign
  5. Uber / Lyft — Match Drivers and Riders

    Geospatial matching capstone — spatial index, gossip dispatch, escrow matching and real-time dispatch in one design.

    Advanced ~60mDesign
  6. Google Maps / Routing

    Preview

    Routing capstone — contraction hierarchies over a road graph (graph traversal from web-crawler) plus map tiles and ETA.

    Advanced ~45mDesign
  7. Collaborative Editor (Google Docs)

    Real-time collaboration capstone — OT vs CRDT (from build-crdt), causal ordering and single-writer serialization with lease ownership.

    Advanced ~60mDesign
  8. AI Agent Platform

    The final capstone — durable execution + sandboxing + capability scoping + cost governance stitched over the entire stack.

    Advanced ~45mDesign

Stops marked Preview are brief today — the order still holds, and their concepts are covered by neighboring problems.