Build a vector database (Pinecone / Weaviate / pgvector style)
15 scenes · ~105 min · build the primitive

Build your own vector database (Pinecone / Weaviate / pgvector style)

Approximate nearest-neighbor over a billion 1536-dim vectors in 10 ms. Build the index from scratch (HNSW, IVF, PQ), pay the recall-vs-latency tax explicitly, support filtered + hybrid search, and feel why every LLM stack in 2026 has a vector store next to its KV store.

Scenes
15 interactive scenes
Time
about 105 minutes
Topic
Search, Indexing & Retrieval

What you are building, and why

You are building the simplest thing that could possibly answer one question: "find the items most similar to this one." Similar songs, similar documents, similar product photos, similar support tickets — all the same shape once each item is turned into an embedding, a list of numbers where "close together" means "similar in meaning." The query is a vector too; the answer is its nearest neighbors.

The naive plan is to compare the query against every stored vector and keep the closest. It is exact, and it is hopeless at scale. One 1536-dimension float32 vector is 6 KB; a billion of them is ~6 TB of raw data, and a single exact query is ~1.5 trillion multiply-adds that must stream all 6 TB past the CPU. There is no path from there to 10 ms — not with more cores, not with faster disks, not with an ordinary B-tree (closeness lives in all 1536 dimensions at once, and a B-tree orders one).

So you sign a deal: give up being exactly right to be fast enough. You return approximate nearest neighbors and arrange never to look at most of the data. The whole curriculum is one ladder of trades on a single recall-vs-latency-vs-memory chart — cluster the space (IVF), navigate a proximity graph (HNSW), compress the vectors (PQ), then face filters, hybrid retrieval, and sharding. Every mechanism moves one dot on that chart, and you can never max all three corners at once. You pick two.

Resist the urge to "describe Pinecone." Build each index yourself, pay each tax explicitly, and feel why — by 2026 — every LLM application keeps one of these next to its key-value store: RAG, semantic caching, recommendation, and dedup all reduce to nearest-neighbor over embeddings.

What you will be able to explain afterwards

  • vector embeddings as the unit of storage
  • exact NN is O(N·D) — why ANN is the only practical answer
  • HNSW — hierarchical navigable small worlds, ef_search tuning
  • IVF (inverted file) — coarse cluster + fine search
  • Product Quantization (PQ) — vector compression with bounded error loss
  • recall@k vs latency vs index size — pick two
  • filtered search (metadata + vector) — pre-filter vs post-filter vs fused
  • hybrid search: dense (embedding) + sparse (BM25) fused via reciprocal rank
  • index build cost vs query cost (HNSW is build-heavy)
  • distributed: shard by vector, replicate by shard, scatter-gather top-k
  1. 01
  2. 02
  3. 03
  4. 04
  5. 05
  6. 06
  7. 07
  8. 08
  9. 09
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15

Why vectors

Why a billion-vector exact scan can't hit 10 ms, and what 'closest' even means.

  1. 01
    Find the most similar thing — fast — brute-force vector search and the 10 ms wall
    Turn items into lists of numbers and 'most similar' becomes 'closest list' — but checking every one is exact and hopeless at a billion. That wall is why this whole system exists.
    ~7 min
  2. 02
    The vector and the distance metric
    'Closest' isn't one thing — L2 is the gap between arrowtips, cosine is the angle — and a metric that mismatches the embedding silently returns garbage. Normalize once and they agree.
    ~7 min

First index

Flat baseline, IVF cells, and the boundary miss that bites every junior.

  1. 03
    The Flat index — the 100%-recall baseline
    Wrap the brute-force scan as an index called Flat: recall 1.0 by definition. It's the yardstick every faster index is scored against — and it defines the two axes, recall@k and latency.
    ~7 min
  2. 04
    IVF — carve the space into cells
    k-means splits the space into Voronoi cells; at query time scan only the nprobe cells nearest the query — turning an all-N scan into a fraction of it. The first explicit recall-for-speed trade.
    ~7 min
  3. 05
    The boundary miss — one lonely point
    A cell wall is hard: a true neighbor a hair across it is invisible at low nprobe, even though it's closer than points you do return. Proximity in space ≠ membership in a probed cell.
    ~7 min

The graph

HNSW: hop a proximity graph; express-lane layers and the ef_search knee.

  1. 06
    HNSW — hop along a proximity graph
    Link each vector to its M nearest neighbors and search by greedy hops — no hard cell walls, so the lonely boundary point is just one more link away.
    ~7 min
  2. 07
    Express lanes — layers and ef_search
    Skip-list layers add express-lane nodes that cross huge distances in one hop, then descend for local refinement — roughly log search, with ef_search trading beam width for recall until it plateaus.
    ~7 min

Compress & pick

PQ shrinks 6 KB to 96 bytes, IVFPQ combines both, and the trilemma forces a choice.

  1. 08
    Product Quantization — 6 KB to 96 bytes
    Split each vector into subvectors and store the 1-byte id of each one's nearest prototype: 6 KB → ~96 bytes (~64×). The memory axis finally becomes a knob — at the cost of lossy distances.
    ~7 min
  2. 09
    IVFPQ — skip most cells, compress the rest
    Stack IVF (skip most cells) and PQ (compress the residual): the FAISS billion-scale workhorse — tiny memory and a fraction-of-N scan, paid for with compounded approximation.
    ~7 min
  3. 10
    The trilemma — pick two
    Recall, latency, memory: Flat maxes recall, HNSW speed-at-recall, IVFPQ memory — no index wins all three. Every deployment is a chosen point on the Pareto frontier.
    ~7 min

Production

Filters, the graph-disconnection trap, hybrid RRF, and sharded scatter-gather.

  1. 11
    Filtered search — pre vs post
    Real queries combine a metadata filter with similarity, and both naive orders break: post-filter can return fewer than k results; pre-filter sets up a subtler failure.
    ~7 min
  2. 12
    When the filter disconnects the graph — pre-filtering HNSW with extra intra-category edges
    Pre-filtering an HNSW search makes only matching nodes walkable — and if the path to a valid neighbor ran through a filtered-out node, the greedy walk dead-ends and recall craters.
    ~7 min
  3. 13
    Hybrid search — fuse dense and sparse with RRF
    Dense vectors capture meaning but miss exact tokens like SKUs and error codes; sparse keyword search nails them. Run both and fuse by rank with RRF — scale-free, no score calibration.
    ~7 min
  4. 14
    Distribute it — shards, scatter-gather, the LLM stack
    Shard vectors across nodes, scatter-gather every query, merge per-shard top-k — exact only if each shard over-fetches. This is the box that sits next to the KV store in every 2026 LLM app.
    ~7 min

Design canvas

Pick every knob for RAG, recommendation, semantic cache, or billion-on-a-budget.

  1. 15
    Design your vector database
    Capstone: pick the index, filter, hybrid, and sharding for RAG vs recommendation vs semantic cache vs billion-on-a-budget — each knob traceable to the scene that justified it.
    ~7 min

More in Search, Indexing & Retrieval

Finding a needle: inverted indexes, distributed search, vector and ANN retrieval, crawling, and the query box itself.

Prefer to design it yourself?

The same subject as a staged workspace: draw the architecture, and a simulator traces requests through the boxes you drew.

Open the Build a vector database (Pinecone / Weaviate / pgvector style) workspace