Append, fsync, update, ack

Every put is four ordered steps — append at end-of-file, fsync per policy, update keydir in place, return ack — and the single open writer (enforced at open(), not at write time) makes that ordering the throughput ceiling of one Bitcask instance.

Previously

The split is the shape; the order is the contract. Append, then fsync, then keydir update, then ack — and one open writer at a time. That ordering is what makes the disk and the RAM agree.

Scene 02

Append, fsync, update, ack

Watch
Try it
Predict
Capture

What to watch for

A single producer issues PUT k1=v1. Watch the four steps light up in order — append at the tail, fsync, keydir row updated, ack to the client — and the throughput meter climb as more puts stream in.

Implementation

Highlighted lines are the ones running in the diagram right now.

Bitcask.put_record

the four ordered steps every put goes through

1def put_record(key, value):
2    rec = encode(crc, tstamp, ksz, vsz, key, value)
3    # 1) append at end-of-file on the active file
4    pos = active_file.size
5    active_file.write(rec)        # sequential append
6    # 2) fsync per sync_strategy (none | o_sync | interval)
7    if sync_strategy == 'o_sync':
8        active_file.fsync()
9    # 3) update keydir entry in place
10    keydir[key] = (active_file.id, vsz, pos, tstamp)
11    # 4) ack to the client
12    return ok

Bitcask.open_for_write

the writer lock is acquired once, at open() — not per write

1def open(dir, mode):
2    if mode == read_write:
3        # one writer per directory, enforced here
4        lock = try_flock(dir + '/bitcask.write.lock')
5        if lock is None:
6            return error('already open for writing')
7        active_file = open_or_create_active(dir)
8        keydir = scan_or_load_hint_files(dir)
9        return Handle(dir, active_file, keydir, lock)
10    # read-only handles share freely
11    return ReadOnlyHandle(dir)

Bitcask.maybe_rotate

active file crosses max_file_size: close it, open the next one

1def maybe_rotate(active_file):
2    if active_file.size < max_file_size:  # default 2 GB
3        return active_file
4    # 1) close the current active file — now immutable
5    active_file.close()
6    immutable_files.append(active_file)
7    # 2) open a fresh active file for subsequent appends
8    next_id = active_file.id + 1
9    return create_active(dir, next_id)
10    # rotation is not merge: closed bytes are unchanged here

Sources

Where this sits in Build a Bitcask-style KV store

Scene 02 of 9, in the Anatomy act — Log on disk, hash in RAM — and the four-step write.. Every put is four ordered steps on one open writer. Concurrent writers are rejected at open(), not at write time.

Up next. Writes funnel through one append + one fsync. Reads don't share that bottleneck — the keydir is random-access, so a get is bounded by the laziest part of the OS, not by the writer.

All 9 scenes in Build a Bitcask-style KV store · Every curriculum

Designs that use this

Built with Arqly

Every scene in Build a Bitcask-style KV store builds on the one before it.All 9 Build a Bitcask-style KV store scenes