Scene 09

Origin shield — collapsing the herd

Without a shield, every POP independently misses on TTL expiry and stampedes origin in parallel; with a shield, lower-tier POPs ask one upper-tier POP first and origin sees one request instead of N.

Previously

Even with a perfect cache key, a popular object expiring across hundreds of POPs at once stampedes origin — unless we put one POP between them.

Scene 09

Origin shield — collapsing the herd

Watch
Try it
Predict
Capture

What to watch for

Shield is OFF. The popular URL is cached at every POP and a TTL countdown is ticking on every cell. When the countdown hits zero, every POP misses at the same instant — watch what happens to the origin gauge.

Implementation

Highlighted lines are the ones running in the diagram right now.

POP.on_miss (shield OFF)

every POP fans out to origin on its own miss

1def on_miss(req):
2    # cell expired or never populated here
3    upstream = origin
4    resp = upstream.fetch(req.url)
5    cache.put(
6        req.cache_key, resp,
7        ttl = resp.cache_control.max_age,
8    )
9    return resp
10    # at TTL boundary every POP runs this in parallel
11    # → origin sees N simultaneous fetches for one byte

POP.on_miss (shield ON)

lower tier asks the designated upper-tier POP first

1def on_miss(req):
2    # ask the shield POP, not origin
3    upstream = shield_pop_for(req.cache_key)
4    resp = upstream.fetch(req.url)  # +cross-region hop
5    cache.put(
6        req.cache_key, resp,
7        ttl = resp.cache_control.max_age,
8    )
9    return resp

ShieldPOP.fetch

request coalescing — N concurrent misses, 1 origin fetch

1inflight = {}  # cache_key → Future
2 
3def fetch(url):
4    key = cache_key(url)
5    if key in cache and not cache[key].stale:
6        return cache[key]
7    if key in inflight:
8        return inflight[key].await()  # piggyback
9    inflight[key] = spawn(origin.fetch(url))
10    resp = inflight[key].await()
11    cache.put(key, resp)
12    del inflight[key]
13    return resp

Sources

Where this sits in Build a CDN

Scene 09 of 13, in the Operating act — Shield, bypass routes, and the hit-ratio dashboard.. Without a shield, every POP independently misses on TTL expiry and stampedes origin in parallel; with a shield, origin sees one request instead of N.

Up next. Shield helps the cacheable traffic — but plenty of traffic is per-user and should never be cached at all; what does the CDN do for those routes?

All 13 scenes in Build a CDN · Every curriculum

Designs that use this

Built with Arqly

Every scene in Build a CDN builds on the one before it.All 13 Build a CDN scenes