Build a workflow engine (Temporal / Airflow / Cadence style) (13 scenes)
Scene 12

Design canvas: choose the right engine

Every workload picks a different orchestration tool — code-as-workflow replay for long-running stateful logic, a DAG scheduler for batch ETL, a state machine for serverless cloud-native flows — and the right choice is the one whose model fits the workload's shape, defended by the scene that taught each requirement.

Previously

You've earned every mechanism. The capstone isn't a new one — it's choosing the right tool. The same ORDER #1001 can be expressed as Temporal-style code, an Airflow DAG, or a Step Functions state machine. Each fits a different shape of workload, and seeing the 30-day wait and the cancel signal go awkward in a DAG is the cleanest proof of what durable execution is actually for.

Scene 12

Design canvas: choose the right engine

  1. Watch
  2. Try it
  3. Predict
  4. Capture
DESIGN CANVAS — match each workload to its orchestration modelrequirement weight:long-running-statefulAOrder fulfillment (30-day wait + canc…✓ 30-day wait✓ signals○ batch-DAGRECOMMENDED MODELTemporalcode-as-workflow repl…✓durable timer + signal — fits replayscene 7 / 8BNightly warehouse ETL refresh○ 30-day wait○ signals✓ batch-DAGRECOMMENDED MODELAirflowDAG scheduler✓short stateless tasks, no waitDAG homeCMulti-month subscription billing✓ 30-day wait○ signals○ batch-DAGRECOMMENDED MODELTemporalcode-as-workflow repl…✓durable state, ContinueAsNewscene 11DServerless approval flow (AWS-native)○ 30-day wait○ signals○ batch-DAGRECOMMENDED MODELStep Functionsstate machine✓declarative JSON, no code to replaycloud-nativeORDER #1001 · Order fulfillment (30-day…the same order rendered three waysTemporalcode-as-workflow replayrecommendedChargeCardReserveShipEmail⏱ durable timer · 30 days✉ cancel signal · nativewait + cancel are first-class — fits cleanlyAirflowDAG schedulerChargeCardReserveShipEmail✕ sensor idles a worker 30d✕ no actor to signal cancelno deterministic replay · no long in-process waitStep Functionsstate machineChargeCardReserveShipEmail{ "States": { "ChargeCard": { "Next": … } } }declarative JSON · no code to replay · cloud-nativeblue = workflow code (replayed, deterministic) · red = activity (runs once, recorded)verifier cites the scene behind every requirement; the model that fits the shape wins.
DAG scheduler = a graph of short stateless tasks, no long wait — the nightly ETL's home
What to watch for

No new machinery this time — just the payoff. Watch a workload land on the canvas: a nightly data-warehouse refresh (Workload B). It's a directed graph of short, stateless tasks — extract, transform, load — with no long wait and nothing to react to mid-run. That shape has a name you already half-know: a DAG scheduler (think Airflow) — a batch/ETL orchestrator that runs a graph of short tasks with task-level retry, and notably has NO deterministic replay and no long in-process wait. The verifier flashes green: nothing here needs the durability you spent eleven scenes building, so the DAG is its home. Now the same ORDER #1001 you've followed all arc — charge, reserve, ship, email — rendered three ways at a glance. As code-as-workflow replay (Temporal-style: normal-looking code made durable by event history + deterministic replay) the 30-day wait and the cancel signal are first-class. As a state-machine workflow (Step Functions: the flow is declarative JSON, serverless, no code to replay) it's tidy but cloud-bound. Drop it on the DAG, though, and watch the 30-day wait and the cancel go visibly awkward — a sensor idling a worker for a month, a cancel with no running actor to message. That awkwardness IS the proof of what durable execution is for.

Continue unlocks when the animation finishes.
Implementation

Highlighted lines are the ones running in the diagram right now.

chooseModel
maps a workload's shape to its orchestration model
def chooseModel(w):
    if w.needs_long_wait or w.needs_signals:
        # durable timer + signal must survive crashes
        return CODE_AS_WORKFLOW_REPLAY
    if w.is_batch_dag:
        # short stateless tasks, task-level retry
        return DAG_SCHEDULER
    return STATE_MACHINE  # declarative, serverless
OrderWorkflow.run (code-as-workflow replay)
the 30-day wait + cancel are first-class durable events
def run(order):
    charge_card(order)        # activity, runs once
    reserve(order)            # activity, runs once
    # durable timer: dormant, survives crashes
    fired = workflow.await(
        timer = sleep(days=30),
        signal = 'cancel',
    )
    if fired == 'cancel':
        return compensate(order)  # refund, release
    ship(order); email(order)
nightly_etl_dag (DAG scheduler)
short stateless tasks, no long wait, no signal
dag = DAG('nightly_etl', schedule='@daily')
extract  = task(extract,  retries=3)
transform = task(transform, retries=3)
load     = task(load,     retries=3)
extract >> transform >> load   # dependency edges
# no durable timer, no in-process wait, no signal,
# no deterministic replay — just retry a failed task
Sources

Where this sits in Build a workflow engine (Temporal / Airflow / Cadence style)

Scene 12 of 13, in the Evolve & ship act — Version safely, then choose the right engine.. Match each workload to its model — code-as-workflow replay, a DAG scheduler, or a state machine — and defend every choice with the scene that taught the requirement.

All 13 scenes in Build a workflow engine (Temporal / Airflow / Cadence style) · Every curriculum

Built with Arqly
Every scene in Build a workflow engine (Temporal / Airflow / Cadence style) builds on the one before it.All 13 Build a workflow engine (Temporal / Airflow / Cadence style) scenes