Replication slot — the bookmark that fills disks

A replication slot is a server-side cursor identified by an LSN that pins WAL recycling at restart_lsn — so a dead or stalled connector means WAL accumulates until pg_wal/ fills the disk and the database goes read-only.

Previously

Debezium tails the log and emits one change event per row change — but if it crashes, the database needs to know where it left off so the WAL can be safely recycled. Postgres provides exactly this bookmark; it's called a replication slot, and the position it remembers is an LSN.

Scene 05

Replication slot — the bookmark that fills disks

Watch
Try it
Predict
Capture

What to watch for

Debezium needs to remember how far it's read so a restart doesn't lose its place. Postgres provides exactly this bookmark — a replication slot — pinned to an LSN. Watch restart_lsn slide forward in step with the write head while the connector is running. The disk stays green because Postgres can recycle WAL bytes behind the slot.

Implementation

Highlighted lines are the ones running in the diagram right now.

Postgres.advanceRestartLsn(slot, newLsn)

Slot's restart_lsn moves only when the consumer acks.

1def advance_restart_lsn(slot, new_lsn):
2    if not slot.active:
3        # consumer gone -> restart_lsn frozen
4        return
5    if new_lsn <= slot.restart_lsn:
6        return  # never moves backward
7    slot.restart_lsn = new_lsn
8    # only NOW may pg_wal/ GC bytes < restart_lsn
9    wake_checkpointer()

Postgres.maybeRecycleWal(currentLsn)

GC refuses to recycle bytes any slot still claims.

1def maybe_recycle_wal(current_lsn):
2    pin = min(s.restart_lsn for s in pg_replication_slots)
3    for seg in pg_wal_segments():
4        if seg.end_lsn < pin:
5            recycle(seg)  # safe: no slot needs it
6        else:
7            keep(seg)     # slot pins this WAL
8    if disk_used(pg_wal) >= READONLY_THRESHOLD:
9        # cannot durably commit without WAL space
10        set_database_read_only()
11        refuse_further_writes()

Connector.confirmFlushedLsn()

Consumer tells Postgres how far it has durably read.

1def confirm_flushed_lsn():
2    while connector.running:
3        flushed = kafka_producer.last_acked_lsn()
4        # standby status update message
5        replication_stream.send_status(
6            write_lsn=flushed,
7            flush_lsn=flushed,
8            apply_lsn=flushed,
9        )
10        sleep(status_interval_ms)

Sources

Where this sits in Build a CDC pipeline (Debezium + outbox)

Scene 05 of 12. A replication slot is a server-side cursor identified by an LSN that stops Postgres from recycling WAL the connector hasn't read — and an inactive slot is the #1 Debezium production failure.

Up next. The slot tracks where the connector left off in the WAL — but the WAL only goes back so far, so a connector attaching to a database with a billion existing rows cannot replay history from the log alone.

All 12 scenes in Build a CDC pipeline (Debezium + outbox) · Every curriculum

Built with Arqly

Every scene in Build a CDC pipeline (Debezium + outbox) builds on the one before it.All 12 Build a CDC pipeline (Debezium + outbox) scenes