Distributed Rate Limiter

Enforce a per-key request limit across a fleet of enforcers — accurately, in under a millisecond, without becoming the outage.

SavedSaved on this device

The problem

Enforce a per-key request limit across a fleet of enforcers — accurately, in under a millisecond, without becoming the outage.

Build a distributed rate limiter: cap how many requests a given key — an API key, a user id, an IP — may make in a window, and enforce that cap across a whole fleet of enforcer instances, on the synchronous path of every request, in well under a millisecond, without ever becoming the outage you were trying to prevent.

The one-sentence version ("count requests in Redis, reject over the limit") is right and useless. The entire problem is the gap between a global invariant ("key X ≤ R req/s") and the local vantage points that must enforce it — 200 enforcer pods, each seeing only a slice of X's traffic. Close that gap naively and a client fans out across your fleet and gets 200× the limit; close it with a single shared counter and you've put a network round trip (and a single point of failure) on every request. This problem is a distributed-consistency problem wearing a counter's clothes.

We hold everything else fixed and go deep on that core: the token-bucket algorithm as an atomic Redis Lua script, a central-authoritative counter with a per-instance fail-open fallback, and the four tensions that define the space — accuracy vs latency vs availability, and what you do when the store dies.

Reading: Stripe — Scaling your API with rate limiters (token bucket on Redis, fail-open) · Cloudflare — How we built rate limiting to millions of domains (sliding-window counter) · GitHub — Sharded, replicated rate limiter in Redis (replica-expiry gotcha) · Envoy — Global rate limiting + Lyft `ratelimit` service (local + global) · Figma — An alternative approach to rate limiting (sliding-window counter, hot key) · Google SRE Book — Handling Overload (adaptive throttling, criticality) · IETF draft-ietf-httpapi-ratelimit-headers (RateLimit / RateLimit-Policy) · redis-cell — GCRA rate limiting as one command (CL.THROTTLE)

token bucket as an atomic Redis Lua script (the check IS a read-modify-write)
local vs global counters — fan-out over-admission (N× the limit)
central-authoritative counter + per-instance fail-open fallback
sliding-window counter (Cloudflare approximation) vs fixed-window 2× boundary burst
hot key = one hash slot = one shard = one core
Redis failover free-burst window (async replication + AOF)
fail-open vs fail-closed: the limiter must not become the DoS
429 + Retry-After + IETF RateLimit / RateLimit-Policy headers

01Clarifications

What would you ask before drawing a single box?

Ambiguity you would resolve with the interviewer: scope, scale, who uses it, what counts as done.

About Distributed Rate Limiter

Enforce a per-key request limit across a fleet of enforcers — accurately, in under a millisecond, without becoming the outage.

Difficulty: intermediate
Time: about 35 minutes
Stages: 10
Topic: System Design Fundamentals

How this problem is worked

Ten stages, from the questions you would ask an interviewer to the trade-offs you would defend. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.

01ClarificationsWhat would you ask before drawing a single box?
02Functional reqsWhat must this system actually do?
03Non-functionalWhat must it promise about speed, uptime and correctness?
04Capacity estimationHow much load and data does this have to hold?
05API designWhat does the outside world call, and what comes back?
06Data modelWhat gets stored, and what is it looked up by?
07Use-case breakdownHow does each requirement actually get served?
08High-level designWhich components handle a request, and in what order?
09Deep divesWhich part breaks first, and what do you do about it?
10Trade-offsWhat did this design cost, and what breaks at 10×?

Primary sources for this problem

Stripe — Scaling your API with rate limiters (token bucket on Redis, fail-open)
Cloudflare — How we built rate limiting to millions of domains (sliding-window counter)
GitHub — Sharded, replicated rate limiter in Redis (replica-expiry gotcha)
Envoy — Global rate limiting + Lyft `ratelimit` service (local + global)
Figma — An alternative approach to rate limiting (sliding-window counter, hot key)
Google SRE Book — Handling Overload (adaptive throttling, criticality)
IETF draft-ietf-httpapi-ratelimit-headers (RateLimit / RateLimit-Policy)
redis-cell — GCRA rate limiting as one command (CL.THROTTLE)

Build the primitives this design leans on

Each one is an animated curriculum that constructs the system from scratch.

Build Build RedisAn in-memory data-structure server: one thread, rich types, optional persistence, async replication. Internalize the cost of single-threaded simplicity and a dozen caching/HA decisions get easier.

More in System Design Fundamentals

The four primitives every later problem assumes — unique IDs, rate limits, caching a read-heavy endpoint, and making a retry safe.

Browse the full problem catalog, or see what the simulator does and does not model.