LESSON 22.3
Running in multiple regions
Why go multi-region, active-passive versus active-active, routing users to regions, keeping data consistent across them, and cells to limit blast radius.
6 min read
A region is a group of data centers in one geography. Losing one is rare but real, and some products need to keep working when it happens. Going multi-region is one of the most expensive decisions in system design, so be clear about why.
Edit disconnected replicas and compare the result after merging their counter components.
Why go multi-region
- Disaster recovery: survive the loss of a region.
- Latency: a round trip from Sydney to Virginia is around 200 ms. Serving users from a nearby region removes that from every request.
- Data residency: some data must stay in a jurisdiction, such as EU personal data.
Active-passive versus active-active
Active-passive
- Simple data model: one write region
- No write conflicts
- Standby capacity sits mostly idle
- Failover takes minutes and must be rehearsed
- Async replication loses a few seconds of writes
Pick it when: Disaster recovery is the main goal.
Active-active, homed data
- Every region serves traffic
- No conflicts: each user or tenant writes to a home region
- Cross-region requests for users away from home
- Rehoming during failover
Pick it when: Global products where data partitions by user or tenant.
Active-active, multi-writer
- Local writes everywhere
- Survives region loss without failover
- Conflicts to resolve
- Weaker consistency, or consensus with cross-region latency on writes
Pick it when: Data that tolerates merging, or globally consistent databases like Spanner.
Routing users
- Latency-based or geo DNS: simple, but clients cache DNS, so failover takes about the TTL plus stragglers.
- Anycast and global load balancers: one IP announced from many places; shifts traffic in seconds.
- Home-region routing: an edge layer looks up the user's home region and forwards there.
Data across regions
Stateless services are easy; the data is the hard part.
- Classify data. Some is global and read-mostly (catalog, config), replicated everywhere. Some belongs to a user or tenant and can be homed. Little truly needs global consistency.
- Replicate asynchronously by default; cross-region synchronous writes cost 50–200 ms each.
- Make writes idempotent and events replayable, so a region can be rebuilt or caught up after an outage.
- Decide what happens to un-replicated writes when a region dies: accept the loss (RPO), or require a cross-region quorum for the critical subset.
Cells and blast radius
Many large services split each region further into cells: complete, independent copies of the stack, each serving a fixed slice of users. A thin routing layer maps users to cells. A bad deploy, a poison request or a noisy tenant takes down one cell, not everyone, and deploys roll out cell by cell.
Try it
Multi-region Failover HLD
Design active-passive or active-active failover across regions.