Index-free adjacency: follow pointers

Because every node holds direct pointers to its relationship records, a hop is a pointer dereference that costs the same whether the graph has ten nodes or ten billion — whereas a relational hop is an index seek that pays O(log N) per edge and gets slower as the data grows.

Previously

A first-class relationship is only useful if I can reach it without a lookup. Here's how: every node stores a direct pointer to its relationships, so a hop is a pointer dereference, not an index seek — that's index-free adjacency, the reason this whole category of database exists. But 'a pointer to its relationships' is vague. What does that record actually look like in memory?

Scene 03

Index-free adjacency: follow pointers

  1. Watch
  2. Try it
  3. Predict
  4. Capture
INDEX-FREEAliceBobCarolDaveErinFrankGraceHeidiIvanJudyThe RockThe MatrixInceptionGothamFRIENDRATEDLIVES_INFOLLOWSFollowing Alice's first relationship: node.firstRel …SPINElocal 0 · global 14one hop = one dereferenceRELATIONAL SEEK1 B-tree level deep10 nodes — O(log N) per hop
What to watch for

Forget the whole graph for a moment. Watch a single hop, Alice → Bob, happen in slow motion. Alice's node record holds the address of her first relationship. We follow that pointer to the relationship record — a join already computed and stored. The relationship record holds the address of its end node. We follow that to Bob. Three reads of addresses that were already written down. No index was searched. The faint structure on the side is what a relational store does on the very same hop: descend a B-tree to find the matching row.

Continue unlocks when the animation finishes.
Implementation

Highlighted lines are the ones running in the diagram right now.

Graph.expandOneHop
follow pointers — no index consulted
def expand_one_hop(node):
# follow 1: node record stores the address of its first rel
rel = deref(node.firstRel) # land ON the rel record
# follow 2: read the rel record (a precomputed join, baked in)
end_addr = rel.endNode # the address it points at
# follow 3: dereference that to land ON the neighbor node
neighbor = deref(end_addr) # 3 follows total, no index
return neighbor # O(1), independent of N
Relational.expandOneHop
seek an index for the matching row — O(log N) per edge
def expand_one_hop(user_id):
# no stored pointer — search the index for the join row
rows = btree_seek(USER_FRIEND, key=user_id)
# seek descends ceil(log_fanout(N)) levels — grows with the table
neighbors = [r.friend_id for r in rows]
return neighbors # O(log N), grows with size
Graph.kHopTraversal
k hops = k pointer chases — cost tracks the answer, not N
def k_hop(start, k=5):
frontier = {start}
for _ in range(k): # one level per hop
next = set()
for node in frontier:
# each expand is the 3-follow O(1) hop, no index
next |= expand_one_hop(node)
frontier = next
return frontier # cost ~ nodes reached, not N
Relational.kHopTraversal
k hops = k index seeks, each O(log N) — pays the depth k×
def k_hop(start_id, k=5):
frontier = {start_id}
for _ in range(k): # one self-join per hop
next = set()
for uid in frontier:
# no stored pointer — seek the join index each time
next |= btree_seek(USER_FRIEND, key=uid)
frontier = next
return frontier # k x O(log N) — grows as the table grows

Where this sits in Build a graph database (Neo4j / Dgraph-style)

Scene 03 of 16, in the Why graphs act — Friends-of-friends melts a join; edges go first-class.. Each node stores a direct pointer to its relationships, so following an edge is a pointer dereference — O(1) per hop, independent of total graph size — not a B-tree seek that gets slower as the database grows.

Up next. We keep saying 'the node points to its relationships' as if it's one pointer. It isn't — a node has many relationships, and they have to be laid out so you can walk them, add to them, and delete from them cheaply. Let's open up the actual store records and see the doubly-linked chain that index-free adjacency is physically made of.

All 16 scenes in Build a graph database (Neo4j / Dgraph-style) · Every curriculum

Built with Arqly
Every scene in Build a graph database (Neo4j / Dgraph-style) builds on the one before it.All 16 Build a graph database (Neo4j / Dgraph-style) scenes