Scene 03

POPs and anycast — one IP, many doors

Every POP advertises the same IP; the internet's BGP routing fabric — not a CDN dispatcher — delivers each user's packets to the topologically-nearest POP.

Previously

Edges only help if users actually land on a nearby one — so how does a user in Sydney find Sydney's edge with no load balancer in the middle?

Scene 03

POPs and anycast — one IP, many doors

Watch
Try it
Predict
Capture

Every POP advertises the SAME IP (104.16.0.1). The user's ISP — not the CDN — picks the one with the shortest BGP path. The chosen POP is ringed.

What to watch for

A user in Sydney fires a request to 104.16.0.1. Notice every POP on the map advertises the same IP. The user's ISP picks the shortest BGP path — that's the Sydney POP, ~15 ms away.

Implementation

Highlighted lines are the ones running in the diagram right now.

AuthoritativeDNS.resolve

every user gets the SAME IP — there is no per-region answer

1def resolve(name):
2    # cdn.example.com -> 104.16.0.1 for EVERYONE
3    # no geo-IP lookup, no per-region answer
4    return ANYCAST_VIP  # 104.16.0.1

ISPRouter.routePacket

BGP picks the shortest AS-path to the anycast prefix

1def routePacket(pkt):
2    # bgp_table[prefix] -> list of (nextHop, asPath)
3    routes = bgp_table[pkt.dstPrefix]  # 104.16.0.0/24
4    best = min(routes, key=lambda r: len(r.asPath))
5    forward(pkt, nextHop=best.nextHop)
6    # The CDN never saw this decision. Each ISP made
7    # it independently from its own BGP table.

POP.bgpLifecycle

a POP joins or leaves the anycast group by announcing or withdrawing the prefix

1def comeOnline():
2    # tell upstream peers 'I have a path to 104.16.0.0/24'
3    bgp.announce(ANYCAST_PREFIX, asPath=[self.asn])
4 
5def goOffline():
6    # withdraw the route — peers stop seeing us
7    bgp.withdraw(ANYCAST_PREFIX)
8    # ISPs recompute; users land on next-shortest POP.
9    # No client change. No DNS change. No CDN dispatcher.

Sources

Where this sits in Build a CDN

Scene 03 of 13, in the Edge basics act — The request journey, origin pain, an edge near each user, POPs and anycast.. Every POP advertises the same IP; the network's BGP routing fabric, not a CDN dispatcher, picks which edge each user reaches.

Up next. The user is now reaching a nearby POP — but how long does that POP hold the cached copy before checking origin again?

All 13 scenes in Build a CDN · Every curriculum

Designs that use this

Built with Arqly

Every scene in Build a CDN builds on the one before it.All 13 Build a CDN scenes