Build a Message Queue (RabbitMQ / SQS) (14 scenes)
Scene 01

The work that can't block the request — buffering work for an async worker pool

Some side effects are too slow for the HTTP request that triggered them — drop the work on a buffer and let a separate worker pool finish it asynchronously, so the request returns immediately.

Scene 01

The work that can't block the request

  1. Watch
  2. Try it
  3. Predict
  4. Capture
SYNCHRONOUSAPI blocks until the job is donebrowserGET /submitAPIhandlersend email · render PDF · transcodetimeout 5sno response — timeoutASYNCHRONOUSAPI drops on a buffer, returns 200 OKbrowserGET /submitAPI200 OK in 50ms200 OKbuffer~12s of workWORKERS · 3w1w2w3OK
API still blocked — past 5s timeout
What to watch for

Watch the top lane first: the API holds the connection open while a slow side effect runs, and the browser walks away at the timeout. Then the bottom lane runs the same submit — the API drops the work on a buffer, replies 200 OK in 50ms, and the workers pick it up afterward.

Continue unlocks when the animation finishes.

Where this sits in Build a Message Queue (RabbitMQ / SQS)

Scene 01 of 14. Some side effects (email, PDF, transcode) are too slow for the HTTP request — drop them on a buffer and let a worker pool finish them asynchronously.

Up next. We have a buffer between request and worker. But what shape is that buffer? Before we name it, we have to draw the line between the buffer we're about to build and the log-shaped thing Kafka gave us — same picture from a distance, opposite rules up close.

All 14 scenes in Build a Message Queue (RabbitMQ / SQS) · Every curriculum

Built with Arqly
Every scene in Build a Message Queue (RabbitMQ / SQS) builds on the one before it.All 14 Build a Message Queue (RabbitMQ / SQS) scenes