Replay: re-run the code against the history — determinism and the non-determinism error

On restart the engine re-runs your workflow from the top and hands back each recorded result from history instead of re-executing the step, so it fast-forwards to exactly where the crash hit — but only if the code is deterministic and issues the same steps every time.

Previously

We have a durable history; replay re-runs the code against it, handing back recorded results so ORDER #1001 fast-forwards to step 2 without re-charging. But that only works if the code is deterministic — and charging a card, calling a payment API, sending an email are all non-deterministic side effects. Where do those live so replay doesn't repeat them?

Scene 03

Replay: re-run the code against the history

Watch
Try it
Predict
Capture

↓ replay re-runs the code from step 0

What to watch for

You restart after the crash with the durable history from last scene, but a record isn't a running program. So the engine does something clever: it re-runs your workflow function from the very top — and for every step that already has a recorded result, it hands your code that recorded value INSTEAD of doing the step again. So ChargeCard returns its recorded "ok" without touching the card a second time. The head fast-forwards step by step until it reaches the first step with no recorded result yet — ReserveInventory — which now runs for real. This re-run-against-history move is called replay: re-executing the code from the start, substituting each step's recorded result, to fast-forward back to exactly where the crash hit. Drag the scrub head across the strip and watch the gray 'replayed' steps fast-forward to the live one.

The strip is ORDER #1001's event history.
The scrub head drags through it: grayed steps are 'replayed' — the engine hands your code the recorded result instead of re-running the step (so ChargeCard is NOT re-charged).
The first un-recorded step glows 'executing for real'.
'Determinism' means the re-run must issue the exact same steps as last time; if you inject Math.random(), the replay branches differently, diverges from history, and the engine raises a 'non-determinism error' and freezes the workflow.

Implementation

Highlighted lines are the ones running in the diagram right now.

Engine.replay

re-run the workflow from step 0, folding over history

1def replay(workflow_fn, history):
2    cursor = 0  # position in recorded history
3    run(workflow_fn):
4        on command issued by code:
5            cmd = next command from code
6            apply(cmd, history, cursor)
7            cursor += 1
8    # history exhausted -> caught up; go live
9    resume_live_execution()

Engine.apply

for each step: hand back the recorded result, or run it live

1def apply(cmd, history, cursor):
2    if cursor < len(history):
3        event = history[cursor]
4        assert_matches(cmd, event)  # determinism gate
5        # NOT re-executed: ChargeCard is not re-charged
6        return event.recorded_result
7    else:
8        # first un-recorded step -> execute for real
9        result = execute_activity(cmd)
10        history.append(record(cmd, result))
11        return result

Engine.assert_matches

the determinism check: re-issued command must equal history

1def assert_matches(cmd, event):
2    # the re-run must issue the SAME command it did before
3    if cmd == event.command:
4        return  # reconciled, continue replay
5    # Math.random()/clock returned a different value,
6    # so the code branched off the recorded path
7    raise NonDeterminismError(
8        expected = event.command,
9        got = cmd,
10    )  # workflow freezes -- no safe merge

Sources

Where this sits in Build a workflow engine (Temporal / Airflow / Cadence style)

Scene 03 of 13, in the Replay act — Rebuild state by replay; quarantine in activities.. To resume, the engine re-runs your code from the top and hands back the recorded results instead of redoing them — which only works if the code is deterministic.

Up next. Replay must never re-charge the card. The only way to do that is to move every side effect out of the replayed workflow code and into a special boundary whose result gets recorded — so on replay the engine hands back the recorded result instead of doing the effect again. That boundary is the activity.

All 13 scenes in Build a workflow engine (Temporal / Airflow / Cadence style) · Every curriculum

Built with Arqly

Every scene in Build a workflow engine (Temporal / Airflow / Cadence style) builds on the one before it.All 13 Build a workflow engine (Temporal / Airflow / Cadence style) scenes