Arqly
Sign in
#10Build a vector database (Pinecone / Weaviate / pgvector style)
Approximate nearest-neighbor over a billion 1536-dim vectors in 10 ms. Build the index from scratch (HNSW, IVF, PQ), pay the recall-vs-latency tax explicitly, support filtered + hybrid search, and feel why every LLM stack in 2026 has a vector store next to its KV store.

You are building the simplest thing that could possibly answer one question: "find the items most similar to this one." Similar songs, similar documents, similar product photos, similar support tickets — all the same shape once each item is turned into an embedding, a list of numbers where "close together" means "similar in meaning." The query is a vector too; the answer is its nearest neighbors.

The naive plan is to compare the query against every stored vector and keep the closest. It is exact, and it is hopeless at scale. One 1536-dimension float32 vector is 6 KB; a billion of them is ~6 TB of raw data, and a single exact query is ~1.5 trillion multiply-adds that must stream all 6 TB past the CPU. There is no path from there to 10 ms — not with more cores, not with faster disks, not with an ordinary B-tree (closeness lives in all 1536 dimensions at once, and a B-tree orders one).

So you sign a deal: give up being exactly right to be fast enough. You return approximate nearest neighbors and arrange never to look at most of the data. The whole curriculum is one ladder of trades on a single recall-vs-latency-vs-memory chart — cluster the space (IVF), navigate a proximity graph (HNSW), compress the vectors (PQ), then face filters, hybrid retrieval, and sharding. Every mechanism moves one dot on that chart, and you can never max all three corners at once. You pick two.

Resist the urge to "describe Pinecone." Build each index yourself, pay each tax explicitly, and feel why — by 2026 — every LLM application keeps one of these next to its key-value store: RAG, semantic caching, recommendation, and dedup all reduce to nearest-neighbor over embeddings.

Reading: Malkov & Yashunin — Efficient and robust approximate nearest neighbor search using HNSW (TPAMI 2018) · Jegou, Douze, Schmid — Product quantization for nearest neighbor search (TPAMI 2011) · Pinecone — Engineering blog (HNSW + serverless architecture) · Weaviate — Architecture docs and HNSW implementation notes · pgvector README + IVF/HNSW implementation · Facebook FAISS — Library and design tutorial · ann-benchmarks.com — quantitative comparison of ANN indexes · Qdrant — Hybrid search & Reciprocal Rank Fusion (dense + sparse, why RRF beats score-addition) · Weaviate — Filtered vector search / ACORN (how pre-filtering disconnects an HNSW graph, and the two-hop fix) · Microsoft Research — Filtered-DiskANN (billion-scale ANN on disk with metadata filters)
vector embeddings as the unit of storage
exact NN is O(N·D) — why ANN is the only practical answer
HNSW — hierarchical navigable small worlds, ef_search tuning
IVF (inverted file) — coarse cluster + fine search
Product Quantization (PQ) — vector compression with bounded error loss
recall@k vs latency vs index size — pick two
filtered search (metadata + vector) — pre-filter vs post-filter vs fused
hybrid search: dense (embedding) + sparse (BM25) fused via reciprocal rank
index build cost vs query cost (HNSW is build-heavy)
distributed: shard by vector, replicate by shard, scatter-gather top-k