Build a gRPC-style RPC framework

Every microservice talks over RPC, and the framework you ship determines half the system's failure modes. Build an RPC framework with codec, streams, deadlines, cancellation, retries, interceptors, and load-aware client-side balancing — and feel why gRPC ate the polyglot RPC market and why Thrift and JSON-over-HTTP linger.

SavedSaved on this device

The problem

Every microservice talks over RPC, and the framework you ship determines half the system's failure modes. Build an RPC framework with codec, streams, deadlines, cancellation, retries, interceptors, and load-aware client-side balancing — and feel why gRPC ate the polyglot RPC market and why Thrift and JSON-over-HTTP linger.

You have written a Flask, Express, or Spring service. You have called another service over HTTP with a client library, and you have used a TCP socket at least once. That is your starting point. By the end of this curriculum you will have built — conceptually, layer by layer — a gRPC-style RPC framework, and you will be able to design an RPC interface for a concrete workload: pick a codec, pick a transport, set a deadline and a retry policy, choose how the client balances across backends, choose an mTLS posture, and predict what breaks under packet loss or a brownout — and defend every choice with a named scene.

We start from the wire, not from the framework. Scene 1 is a single uncomfortable fact: a remote call is dressed up to look like greet("Ada"), but unlike a local call it can fail after the server already did the work — so the caller can be left not knowing whether it happened. Every later feature in the stack exists to manage one of the four ways that illusion leaks: latency, no shared memory, partial failure, and unordered concurrency. We name that frame once and call back to it the whole way up.

This is a curriculum about a specific density-of-vocabulary discipline: across 14 scenes there are exactly 26 named technical terms, and each scene introduces at most two. The diagram carries the load; the vocabulary follows. One running example — the call greet("Ada") — is threaded through every layer: you watch it become protobuf bytes, ride inside a DATA frame on one of many streams, travel under a shrinking deadline header, and finally prove its own identity over mTLS.

Resist the urge to "describe gRPC." Build each layer because the previous layer forced it, feel the failure mode each one prevents, and let the design canvas at the end push back on your choices.

What you will cover

the remote-call illusion: it looks like a local function but can fail after the work is done (partial failure)
framing on a TCP byte stream: recv() is not a message; length-prefix to find the boundaries
IDL + codec: protobuf field numbers (not names) as the schema-evolution superpower; never reuse a number
HTTP/2 as transport: one RPC = one stream; many streams multiplexed over one connection
unary, server-stream, client-stream, bidi — four shapes from one stream of DATA frames
deadlines, not timeouts: an absolute clock that PROPAGATES across an N-hop chain (the single most important feature)
cancellation as a first-class event (RST_STREAM); the Context object carries deadline + cancel together
retries: idempotency gates whether, a token-bucket budget + jittered backoff gate how much (retry storms / metastable failure)
interceptors / middleware: auth, metrics, tracing, retry composed as one onion around every call
client-side load balancing + discovery: why an L4 LB pins every stream to one backend (no central LB)
flow control / backpressure via the HTTP/2 window; a slow reader slows the writer
HTTP/2 fixed app-layer head-of-line blocking; TCP-layer HOL remains; QUIC/HTTP3 fixes it
TLS vs mTLS: channel encryption vs verified workload identity, propagated across the call chain

Reading

gRPC documentation — Core concepts (the four call types, metadata, channels): grpc.io/docs/what-is-grpc/core-concepts
gRPC blog — Deadlines ('always set a deadline'; the resource-exhaustion failure mode): grpc.io/blog/deadlines
gRPC guides — Retry policy (maxAttempts, retryable codes, backoff, throttling): grpc.io/docs/guides/retry
gRPC blog — Load balancing in gRPC (proxy vs client-side vs look-aside): grpc.io/blog/grpc-load-balancing
Kubernetes blog — gRPC load balancing on Kubernetes without tears (the L4 connection-pinning gotcha): kubernetes.io/blog/2018/11/07
Birrell & Nelson — Implementing Remote Procedure Calls (TOCS 1984): the origin — stubs, marshalling, binding
Waldo, Wyant, Wollrath, Kendall — A Note on Distributed Computing (Sun TR-94-29, 1994): why RPC can't be transparent
Protocol Buffers — Encoding (varint, tag = field<<3|wire_type, skipping unknown fields): protobuf.dev/programming-guides/encoding
Protocol Buffers — Proto3 language guide (field numbers, reserved, schema-evolution rules): protobuf.dev/programming-guides/proto3
Apache Thrift — Architecture / whitepaper (Slee, Agarwal, Kwiatkowski, Facebook 2007): the pluggable transport/protocol contrast
RFC 9113 — HTTP/2 (frames, streams, flow control, WINDOW_UPDATE, RST_STREAM): httpwg.org/specs/rfc9113
RFC 9000 — QUIC & RFC 9114 — HTTP/3 (per-stream loss recovery; the TCP head-of-line fix)
Marc Brooker / AWS Builders' Library — Timeouts, retries, and backoff with jitter (retry storms, metastable failure)
Robin Marx — Head-of-line blocking in QUIC and HTTP/3 (app-layer vs transport-layer HOL): github.com/rmarx/holblocking-blogpost
Cindy Sridharan — Distributed Tracing in Practice (context propagation across hops)

About Build a gRPC-style RPC framework

Difficulty: advanced
Time: about 85 minutes
Stages: 9
Topic: Caching, Proxies & the Edge

How this problem is worked

Nine stages, from what the thing is for to how it compares with the real implementations. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.

01Purpose & invariantsWhat is this for, and what must always be true of it?
02Workload characterizationWho writes, who reads, and in what shapes?
03Data model & on-disk formatWhat does the data look like at rest?
04Core algorithmsHow do the write path and the read path actually work?
05Distribution & replicationHow does this scale out and survive losing a machine?
06Consistency & correctnessUnder concurrency and failure, what is guaranteed?
07Failure modes & recoveryWhat actually happens when each part fails?
08Operational characteristicsCan a human run this at three in the morning?
09Trade-offs & comparisonWhere does this sit against the alternatives?

Primary sources for this problem

gRPC documentation — Core concepts (the four call types, metadata, channels): grpc.io/docs/what-is-grpc/core-concepts
gRPC blog — Deadlines ('always set a deadline'; the resource-exhaustion failure mode): grpc.io/blog/deadlines
gRPC guides — Retry policy (maxAttempts, retryable codes, backoff, throttling): grpc.io/docs/guides/retry
gRPC blog — Load balancing in gRPC (proxy vs client-side vs look-aside): grpc.io/blog/grpc-load-balancing
Kubernetes blog — gRPC load balancing on Kubernetes without tears (the L4 connection-pinning gotcha): kubernetes.io/blog/2018/11/07
Birrell & Nelson — Implementing Remote Procedure Calls (TOCS 1984): the origin — stubs, marshalling, binding
Waldo, Wyant, Wollrath, Kendall — A Note on Distributed Computing (Sun TR-94-29, 1994): why RPC can't be transparent
Protocol Buffers — Encoding (varint, tag = field<<3|wire_type, skipping unknown fields): protobuf.dev/programming-guides/encoding
Protocol Buffers — Proto3 language guide (field numbers, reserved, schema-evolution rules): protobuf.dev/programming-guides/proto3
Apache Thrift — Architecture / whitepaper (Slee, Agarwal, Kwiatkowski, Facebook 2007): the pluggable transport/protocol contrast
RFC 9113 — HTTP/2 (frames, streams, flow control, WINDOW_UPDATE, RST_STREAM): httpwg.org/specs/rfc9113
RFC 9000 — QUIC & RFC 9114 — HTTP/3 (per-stream loss recovery; the TCP head-of-line fix)
Marc Brooker / AWS Builders' Library — Timeouts, retries, and backoff with jitter (retry storms, metastable failure)
Robin Marx — Head-of-line blocking in QUIC and HTTP/3 (app-layer vs transport-layer HOL): github.com/rmarx/holblocking-blogpost
Cindy Sridharan — Distributed Tracing in Practice (context propagation across hops)

More in Caching, Proxies & the Edge

Everything between the client and the origin: in-memory caches, CDNs, load balancers and service proxies — and the three ways a cache betrays you.

Browse the full problem catalog, or see what the simulator does and does not model.