Circuit breaker — the state machine

1state = CLOSED
2error_rate_window = sliding(60s)
3open_started_at = 0
4 
5def on_request(req):
6    if state == CLOSED:
7        outcome = forward(req, timeout=T)
8        error_rate_window.record(outcome)
9        if error_rate_window.error_rate() > THRESHOLD:
10            state = OPEN
11            open_started_at = now()
12        return outcome
13    if state == OPEN:
14        if now() - open_started_at > COOLDOWN:
15            state = HALF_OPEN
16        else:
17            return fast_fail_503()    # ~1ms, no backend call
18    if state == HALF_OPEN:
19        outcome = forward(req, timeout=T)   # single PROBE
20        state = CLOSED if outcome.ok else OPEN
21        open_started_at = now()
22        return outcome

1# Without breaker: every caller waits the full timeout.
2#   N callers * T seconds = N*T thread-seconds blocked.
3#   Caller's thread pool fills with stuck requests.
4 
5# With breaker OPEN: every caller returns in ~1ms.
6#   Caller frees the resource and degrades gracefully.
7#   Traffic to OTHER (healthy) dependencies keeps flowing.

1# Envoy's `circuit_breakers` config is connection-pool
2# ceilings per cluster, not closed/open/half-open.
3# Per-replica state-machine semantics live in
4# outlier_detection (next scene).
5cluster:
6  circuit_breakers:
7    max_connections: 1024
8    max_pending_requests: 1024
9    max_requests: 1024
10    max_retries: 3