Build a workflow engine (Temporal / Airflow / Cadence style) (13 scenes)
Scene 01

The crash that charges you twice — durable execution and at-least-once delivery

A plain function keeps its progress only in RAM, so a crash after step one erases the fact that it ran — and a naive retry re-runs from the top and charges the card a second time.

Scene 01

The crash that charges you twice

  1. Watch
  2. Try it
  3. Predict
  4. Capture
ORDER #1001 — one plain function, four stepsred = activity (runs once, for real) · state lives in RAM → gone on crashPROCESSruns the four-step function top-to-bottomChargeCard$42✓ ran onceReserveInven…✓ ran onceShipPackage✓ ran onceSendConfirma…✓ ran onceRAMstate lives herecharged = falsestep = 0the only record of progressSTATEMENT · ORDER #1001$42charged once · cleanno retrythe run finished — nothing to retryat-least-once delivery — a call may run 0, 1, or many timesClean run: four steps, $42 once. But the only record of progress lives in RAM.
RAM: the live progress (charged=true, step=2). This is the ONLY record that step 1 ran.
What to watch for

Start with something that looks completely ordinary. ORDER #1001 is one plain function with four steps in order: ChargeCard($42), ReserveInventory, ShipPackage, SendConfirmationEmail. Run it once and it finishes clean — the statement reads $42, and the RAM panel ticks its progress as it goes: charged=true, then step=2, 3, 4. Now watch the same function the way real systems actually behave: the process can die at any moment. We'll drop a crash right after ChargeCard succeeds. The instant the skull hits, look at the RAM panel — it wipes blank. The only record that step 1 ever ran lived in that memory, and the crash erased it. The property this function is MISSING — progress that survives a crash, a reboot, a redeploy — is called durable execution. A plain in-memory call does not have it. Watch the clean run first, then the crash that wipes the proof.

Continue unlocks when the animation finishes.
Implementation

Highlighted lines are the ones running in the diagram right now.

fulfillOrder
a plain function — progress lives only in local variables
def fulfillOrder(order):
    charged = False        # in RAM, dies with the process
    chargeCard(order, $42)  # moves real money at the card network
    charged = True
    reserveInventory(order)
    shipPackage(order)
    sendConfirmationEmail(order)
cron.retry
the unfinished order is re-run from line one
# every minute: find orders that never finished
for order in db.ordersNotMarkedDone():
    # no memory of which steps already ran —
    # the only such record lived in RAM, now gone
    fulfillOrder(order)   # starts again at chargeCard
Sources

Where this sits in Build a workflow engine (Temporal / Airflow / Cadence style)

Scene 01 of 13, in the Durability act — Why RAM double-charges; the event history.. A plain function keeps its progress in RAM, so a crash after step one erases it — and a naive cron retry re-runs from the top and charges the card a second time.

Up next. The double-charge happened because the only record that step 1 ran lived in RAM, and the crash erased it. So the first fix is obvious: stop trusting memory — write down what happened, step by step, into a place a crash can't wipe.

All 13 scenes in Build a workflow engine (Temporal / Airflow / Cadence style) · Every curriculum

Built with Arqly
Every scene in Build a workflow engine (Temporal / Airflow / Cadence style) builds on the one before it.All 13 Build a workflow engine (Temporal / Airflow / Cadence style) scenes