Immutable objects: overwrite is a new version, not an edit

Build an S3-style distributed object store (12 scenes)

Scene 04 · Immutable objects: overwrite is a new version, not an edit

Bytes never change: overwrite = new version + atomic pointer flip; delete drops a marker the old bytes hide behind.

Previously

The index points at an object's current bytes — so we need to pin down what those bytes are: they turn out to be immutable, and 'overwrite' is not what you think.

Scene 04

Immutable objects: overwrite is a new version, not an edit

Watch

Diagram

One object key on the left holds a vertical stack of byte-blobs — newest on top. The index's single 'current' pointer lands on the top tile. A PUT to the same key drops a new blob on top and flips the pointer; older blobs grey out but stay. A DELETE drops a zero-byte tile (a delete marker) on top so a GET returns 404 while the old bytes live on behind it.

Sources

One key, one blob. Watch a PUT to the SAME key: it drops a brand-new blob on top, the index pointer flips up to it, and the old blob greys out but stays. Then watch an attempt to edit a byte in place — it's refused.

Implementation

Index.put

an overwrite writes new bytes — it never edits old ones

1def put(key, body):
2    # bytes are sealed at write time; allocate a fresh blob
3    versionId = newVersionId()
4    storage.writeImmutable(versionId, body)
5    if versioning.enabled(key):
6        # keep the old blob as a recoverable version
7        index[key].versions.prepend(versionId)
8    else:
9        old = index[key].current
10        index[key].versions = [versionId]
11        storage.free(old)  # old bytes discarded
12    index[key].current = versionId  # atomic pointer flip

Index.delete

a delete inserts a zero-byte marker — it erases nothing

1def delete(key):
2    if not versioning.enabled(key):
3        storage.free(index[key].current)
4        del index[key]
5        return
6    # versioning on: nothing is erased
7    marker = newDeleteMarker()  # 0 bytes, not billed
8    index[key].versions.prepend(marker)
9    index[key].current = marker  # GET will now 404
10    # prior versions still sit on disk, still billing

Index.get / reapVersion

read follows the pointer; only reap actually frees bytes

1def get(key):
2    cur = index[key].current
3    if cur.isDeleteMarker:
4        return 404  # marker is current; bytes live behind it
5    return storage.read(cur)  # newest sealed bytes
6 
7def reapVersion(key, versionId):
8    # the ONLY op that frees bytes and stops their bill
9    index[key].versions.remove(versionId)
10    storage.free(versionId)

Not sure what to ask? Tap a question — the staff engineer answers in the chat panel.

PreviousFolders are a lie — the flat keyspace and its index NextReplication vs erasure coding: same safety, a third the cost