Flow control: a slow reader slows the writer

Build a gRPC-style RPC framework (14 scenes)

Scene 10 · Flow control: a slow reader slows the writer

The receiver advertises a window of credit; the sender may only send DATA up to it. A slow reader stops granting credit, so the producer pauses instead of OOMing.

Previously

Now that calls spread across backends, picture one server-streaming RPC firing `Greeting`s faster than the client can read them — without a brake the producer buffers until it runs out of memory, so HTTP/2 hands the reader a way to push back.

Scene 10

Flow control: a slow reader slows the writer

Watch

Diagram

A server-streaming RPC drawn as one pipe: the producer on the left fires `Greeting` DATA frames toward the client on the right. The gauge under the pipe is the receiver's FLOW-CONTROL WINDOW — its credit in octets; every DATA frame drains it, and each WINDOW_UPDATE the reader sends back refills it. When the gauge hits zero the producer shows a BLOCKED badge and pauses (that pause is BACKPRESSURE) rather than buffering. The ghost panel shows the no-flow-control world: a buffer growing toward an OOM skull. (HTTP/2 keeps a window at two levels — per-stream and per-connection; the diagram shows the per-stream one.)

Sources

A server-streaming RPC is running: the server keeps sending `Greeting` responses for one `greet("Ada")` call, each carried in a DATA frame (the frame type from scene 4 that holds the payload bytes). The new idea is the gauge under the pipe. The receiver advertises how many bytes it is willing to accept right now — a running balance of credit measured in octets. We call this credit balance the receiver's *flow-control window*. Every DATA frame the server sends spends credit and drains the gauge; as the client reads frames it sends a WINDOW_UPDATE frame back to grant more credit and refill the gauge. Watch the window drain and refill while the client keeps up — DATA flows steadily because credit keeps being replenished.

Implementation

Sender.sendData

spend credit per DATA frame; wait when it runs out

1def sendData(stream, payload):
2    frame = DataFrame(payload)  # only DATA is flow-controlled
3    # block until this stream has credit for the whole frame
4    while stream.window < frame.octets:
5        wait_for(WINDOW_UPDATE)  # the backpressure point
6    stream.window -= frame.octets
7    conn.window -= frame.octets   # per-connection level too
8    conn.write(frame)

Receiver.onData

reading frees credit and grants a WINDOW_UPDATE

1def onData(stream, frame):
2    stream.recvBuffer.append(frame)
3    stream.window -= frame.octets   # gauge drains
4 
5def onAppRead(stream, n_octets):
6    # the app consuming bytes is what frees credit
7    grant = WindowUpdate(stream.id, n_octets)
8    conn.write(grant)   # refills the sender's window

Sender.onWindowUpdate

credit returns; a parked sender wakes and resumes

1def onWindowUpdate(frame):
2    if frame.stream_id == 0:
3        conn.window += frame.increment   # connection level
4    else:
5        stream(frame.stream_id).window += frame.increment
6    # any sender parked in sendData's wait loop wakes here
7    wake_waiters()

Not sure what to ask? Tap a question — the staff engineer answers in the chat panel.

PreviousThe L4 pinning trap: balance requests, not connections NextHead-of-line blocking and the QUIC fix