The trilemma — pick two

Recall, latency, and memory form a trilemma: Flat maxes recall, HNSW maxes speed-at-recall but eats RAM, IVFPQ maxes memory savings but sacrifices recall — no index wins all three, so every production decision is just choosing a point on this Pareto frontier.

Previously

Four indexes, three axes, one chart — and the picture is unambiguous: no index is best at recall AND latency AND memory at once. Choosing an index is choosing which axis to sacrifice. With the index-choice arc complete, we now leave the clean geometry behind and hit where production actually gets hard: real queries aren't just 'similar to X'.

Scene 10

The trilemma — pick two

Watch
Try it
Predict
Capture

Flat: perfect recall, huge & slow

IVFPQ: tiny & fast, lower recall

What to watch for

Every dot the arc dropped, finally on one chart. Flat sits top-left: recall 1.0 — but it's slow and its bubble is huge (it keeps every full vector). HNSW is fast and high-recall, but its bubble is just as big — it stores full vectors PLUS a graph of edges. IVFPQ has a tiny bubble (compressed) and is fast, but it sits lower — it gave up recall. IVF lands in the middle. Look across all three axes at once: no dot is high AND far-left AND small. The curve through the best dots is the frontier — the boundary of what any index can reach. Nothing lives above-and-left-of it.

The one chart the whole arc has been building.
Each dot is an index you've met: recall@10 goes UP the chart (higher = better), latency goes RIGHT (further right = slower), and the bubble's SIZE is memory (bigger = more RAM).
The soft curve through the best dots is the frontier — the edge of what's possible.
The filled, ringed dot is YOUR live config; drag the knobs and it slides along that edge but can never jump past it.

Implementation

Highlighted lines are the ones running in the diagram right now.

buyRecall (Flat / re-rank path)

exact distances on full vectors — pays in latency + memory

1def rank_exact(query, candidates):
2    scored = []
3    for v in candidates:           # full float32 vectors
4        d = l2(query, v.full)      # no approximation
5        scored.append((d, v.id))
6    return topk(scored)            # recall ceiling = Flat
7 
8# re-rank: rescue recall an approx index conceded
9approx = index.search(query, k * 10)
10return rank_exact(query, approx)   # exact, but slower + holds vectors

buySpeed (IVF probe / HNSW walk)

skip most of the data — pays in recall (boundary misses)

1def search_fast(query, nprobe, ef_search):
2    cells = nearest_centroids(query, nprobe)  # IVF: scan few cells
3    cand = scan(cells)
4    # OR HNSW greedy walk: enter top layer, hop down
5    cand = greedy_walk(query, ef_search)
6    return topk(cand)
7    # tighter nprobe / ef_search = faster, lossier:
8    # a true neighbor across an unprobed boundary is missed

buyMemory (Product Quantization)

store tiny codes, not vectors — pays in recall (lossy codes)

1def encode(v, m, codebooks):       # split into m subvectors
2    return bytes(
3        nearest_centroid_id(sub, codebooks[i])  # 1 byte each
4        for i, sub in enumerate(split(v, m))
5    )                              # 6144 bytes -> m bytes
6 
7def adc(query, code):              # query stays full-precision
8    return sum(dist_table[i][code[i]] for i in range(m))
9    # lossy: estimated distance, so recall sags vs Flat

Sources

Where this sits in Build a vector database (Pinecone / Weaviate / pgvector style)

Scene 10 of 15, in the Compress & pick act — PQ shrinks 6 KB to 96 bytes, IVFPQ combines both, and the trilemma forces a choice.. Recall, latency, memory: Flat maxes recall, HNSW speed-at-recall, IVFPQ memory — no index wins all three. Every deployment is a chosen point on the Pareto frontier.

Up next. Every query so far was pure similarity. But real users ask 'similar to this song AND released after 2020 AND under 3 minutes'. Bolting a metadata filter onto vector search sounds trivial — and it quietly breaks both obvious ways of doing it. Let's see how.

All 15 scenes in Build a vector database (Pinecone / Weaviate / pgvector style) · Every curriculum

Built with Arqly

Every scene in Build a vector database (Pinecone / Weaviate / pgvector style) builds on the one before it.All 15 Build a vector database (Pinecone / Weaviate / pgvector style) scenes