#04Distributed Rate Limiter
Enforce a per-key request limit across a fleet of enforcers — accurately, in under a millisecond, without becoming the outage.

Build a distributed rate limiter: cap how many requests a given key — an API key, a user id, an IP — may make in a window, and enforce that cap across a whole fleet of enforcer instances, on the synchronous path of every request, in well under a millisecond, without ever becoming the outage you were trying to prevent.

The one-sentence version ("count requests in Redis, reject over the limit") is right and useless. The entire problem is the gap between a global invariant ("key X ≤ R req/s") and the local vantage points that must enforce it — 200 enforcer pods, each seeing only a slice of X's traffic. Close that gap naively and a client fans out across your fleet and gets 200× the limit; close it with a single shared counter and you've put a network round trip (and a single point of failure) on every request. This problem is a distributed-consistency problem wearing a counter's clothes.

We hold everything else fixed and go deep on that core: the token-bucket algorithm as an atomic Redis Lua script, a central-authoritative counter with a per-instance fail-open fallback, and the four tensions that define the space — accuracy vs latency vs availability, and what you do when the store dies.

Reading: Stripe — Scaling your API with rate limiters (token bucket on Redis, fail-open) · Cloudflare — How we built rate limiting to millions of domains (sliding-window counter) · GitHub — Sharded, replicated rate limiter in Redis (replica-expiry gotcha) · Envoy — Global rate limiting + Lyft `ratelimit` service (local + global) · Figma — An alternative approach to rate limiting (sliding-window counter, hot key) · Google SRE Book — Handling Overload (adaptive throttling, criticality) · IETF draft-ietf-httpapi-ratelimit-headers (RateLimit / RateLimit-Policy) · redis-cell — GCRA rate limiting as one command (CL.THROTTLE)
token bucket as an atomic Redis Lua script (the check IS a read-modify-write)
local vs global counters — fan-out over-admission (N× the limit)
central-authoritative counter + per-instance fail-open fallback
sliding-window counter (Cloudflare approximation) vs fixed-window 2× boundary burst
hot key = one hash slot = one shard = one core
Redis failover free-burst window (async replication + AOF)
fail-open vs fail-closed: the limiter must not become the DoS
429 + Retry-After + IETF RateLimit / RateLimit-Policy headers