Build a gRPC-style RPC framework (14 scenes)
Scene 10a

Head-of-line blocking and the QUIC fix

HTTP/2's many streams still ride one in-order TCP connection, so one lost packet stalls every stream until it's resent; QUIC moves streams onto UDP with per-stream loss recovery, so only the affected stream stalls.

Previously

Healthy flow-control windows on every stream don't help when the problem is one layer down: because all those streams share a single in-order TCP pipe, one dropped packet holds them all hostage until it's resent.

Scene 10a

Head-of-line blocking and the QUIC fix

  1. Watch
  2. Try it
  3. Predict
  4. Capture
sender10 streamsreceiverin-orderHTTP/2 over TCPone ordered pipe — in-order deliveryflowing#1#2#3#4#5#6#7#8#9#10sender10 streamsreceiverper-streamHTTP/3 over QUIC (UDP)independent per-stream loss recoveryflowing#1#2#3#4#5#6#7#8#9#10ten greet("Ada") streams on one ordered TCP pipe
ten greet("Ada") streams, one TCP pipe
What to watch for

Recall from the multiplexing scene that gRPC runs many independent streams down one HTTP/2 connection — here, ten concurrent greet("Ada") calls. We say multiplexing made the streams independent, and at the application layer it did: a slow response no longer blocks the others. But every one of those streams still travels inside a SINGLE TCP connection, and TCP hands the receiver one byte stream that it must deliver strictly in order. Watch what happens when one packet — a packet that happens to belong to stream #5 — is dropped. Because TCP refuses to deliver any later byte until the missing one is resent, all ten streams grey out and wait, even though nine of them never lost a thing. When one stalled item at the front holds up everything queued behind it, that's head-of-line blocking — and here it's happening at the transport layer, below the streams HTTP/2 tried to keep independent.

Continue unlocks when the animation finishes.
Implementation

Highlighted lines are the ones running in the diagram right now.

Sender.multiplex
ten gRPC streams interleave DATA frames down one connection
# one HTTP/2 stream per RPC, all on one connection
for frame in interleave(streams):   # round-robin
    packet = transport.packetize(frame)
    packet.seq = next_seq()  # one ordered seq space
    transport.send(packet)
    # window-bounded; only DATA frames are flow-controlled
    await peer.WINDOW_UPDATE if window == 0
TcpReceiver.deliver
one byte stream, handed up strictly in order
def on_packet(pkt):
    buffer[pkt.seq] = pkt
    # in-order gate: cannot skip a missing seq
    while buffer.has(next_expected):
        app.deliver(buffer.pop(next_expected))
        next_expected += 1
    if gap_at(next_expected):
        stall()  # every later stream waits here
        request_retransmit(next_expected)
QuicReceiver.deliver
per-stream reassembly over UDP — one gap, one stall
def on_packet(pkt):  # carries pkt.stream_id
    s = streams[pkt.stream_id]
    s.buffer[pkt.offset] = pkt
    # in-order only WITHIN this stream
    while s.buffer.has(s.next_offset):
        app.deliver(s.stream_id, s.buffer.pop(s.next_offset))
        s.next_offset += len(pkt)
    if gap_at(s.next_offset):
        s.stall()  # other streams keep flowing
        request_retransmit(s.stream_id, s.next_offset)

Where this sits in Build a gRPC-style RPC framework

Scene 10a of 14, in the Resilience act — Client-side balancing, backpressure, head-of-line blocking.. HTTP/2 fixed app-layer head-of-line blocking, but all streams share one in-order TCP pipe, so one lost packet stalls them all. QUIC moves streams below the loss boundary.

Up next. We can now move bytes fast, fairly, and resiliently — but we've never asked WHO is on the other end of the wire, and in a zero-trust fleet 'it's encrypted' is not the same as 'I know who's calling.'

All 14 scenes in Build a gRPC-style RPC framework · Every curriculum

Built with Arqly
Every scene in Build a gRPC-style RPC framework builds on the one before it.All 14 Build a gRPC-style RPC framework scenes