TCP gives you bytes, not messages

Build a gRPC-style RPC framework (14 scenes)

Scene 02 · TCP gives you bytes, not messages

TCP is a byte stream, not a message stream: two messages can arrive glued or split. A length prefix is what lets the receiver read exactly one message back.

Previously

We said greet("Ada") travels as bytes over the wire — but the wire is a TCP stream that doesn't respect your message boundaries, so before anything else the receiver has to be told where one message stops.

Scene 02

TCP gives you bytes, not messages

Watch

Diagram

On the left the client fires greet("Ada") and greet("Bob") back-to-back, each an opaque colored payload block. In the middle is the TCP byte stream — a flat run of bytes with no built-in boundaries between messages. On the right the server's recv() buffer shows how those bytes actually land: each recv() returns one arbitrary slice of the stream. FRAMING is any scheme that lets the receiver tell where one message ends and the next begins. LENGTH-PREFIX is the specific framing used here: write the message's length first, then its bytes; the reader reads the length, then reads exactly that many bytes — looping across reads — so the recovered message is independent of how TCP chopped the stream.

Sources

Here's the bug every first network program hits. A *TCP socket* (the connection your two machines talk over) is not a stream of messages — it's a *byte stream*: a single, continuous run of bytes with no markers between one message and the next. The client fires two calls back-to-back, greet("Ada") then greet("Bob"). Watch the server's recv() (the call that reads bytes off the socket): one read can return "Ada" plus *half* of "Bob" — the boundary you sent is simply gone. The receiver needs a way to recover where each message ends; the scheme that does that is called **framing** — any rule that marks message boundaries on the wire. The simplest framing is **length-prefix**: write each message's length first, then its bytes, so the reader can read the length and then read exactly that many bytes.

Implementation

Sender.frame

write the length ahead of each payload, then the bytes

1def send(sock, payload):
2    # 4-byte big-endian length, then the bytes
3    header = len(payload).to_bytes(4, 'big')
4    sock.sendall(header + payload)
5 
6send(sock, b'Ada')   # -> [00 00 00 03] Ada
7send(sock, b'Bob')   # -> [00 00 00 03] Bob

Receiver.recvOnce

the bug: treat one recv() as one message

1def handle(sock):
2    # one read, however TCP chopped the stream
3    data = sock.recv(1024)
4    return parse(data)  # whatever bytes landed

Receiver.readFrame

read the length, then read exactly that many bytes

1def readN(sock, n):
2    buf = b''
3    while len(buf) < n:        # loop across recv() calls
4        buf += sock.recv(n - len(buf))
5    return buf
6 
7def readFrame(sock):
8    header = readN(sock, 4)    # the length prefix
9    length = int.from_bytes(header, 'big')
10    return readN(sock, length) # exactly one message

Not sure what to ask? Tap a question — the staff engineer answers in the chat panel.

PreviousCalling a function on another machine NextThe schema: field numbers, not field names