LESSON 16.1
Replication models and lag
Leader-follower, multi-leader and leaderless replication, synchronous versus asynchronous acknowledgements, and the session anomalies caused by lag.
6 min read
Partitioning divides data; replication copies each partition for reads and recovery. When a write succeeds, which copies have it, and which may answer the next read?
Who accepts writes?
Leader-follower
- One write order per leader
- Straightforward replication path
- Leader capacity and failover matter
- Followers can lag
Pick it when: A common default for transactional records.
Multi-leader
- Local writes in several regions
- Can keep accepting writes during some disconnections
- Concurrent writes can conflict
- Resolution is application-dependent
Pick it when: Regional or offline writes where conflicts have defined semantics.
Leaderless
- Requests can contact several replicas directly
- Configurable acknowledgement thresholds
- Versions need reconciliation
- Repair and quorum rules are subtle
Pick it when: Dynamo-style key-value access with an explicit consistency contract.
In leader-follower replication, clients write through a leader that sends ordered changes to followers. Multi-leader systems allow more than one write authority, often one per region. Leaderless systems have a request coordinator contact replicas without a permanent write leader for the key. None of these labels alone specifies durability or the read contract.
When is a write acknowledged?
Asynchronous replication acknowledges before followers confirm the change. It keeps the write path short but risks losing recent acknowledged writes if the leader fails before copying them. Reading a follower may return an older value.
Synchronous replication waits for a specified set of replicas to acknowledge, ideally durable storage where required. Waiting for one remote follower is different from waiting for all of them. More required acknowledgements increase latency and can stop writes when those replicas are unreachable.
Ask what an acknowledgement means: received, written to a log, flushed to durable storage, or applied and queryable? Even a durably replicated write may not yet be applied by the replica chosen for a read. Failover must select an eligible replica and fence the old leader; automatic promotion alone doesn't establish safety.
Lag changes what users see
A user saves a profile, then loads it through a follower 500 ms behind. The old profile reappears. Refreshing may show the new one, followed by the old one again if the next request hits an even slower replica.
Two useful session guarantees address different problems:
- Read-your-writes: after a successful write, that session's reads include it. Route those reads to the leader, or carry a replication position and wait for a follower to reach it.
- Monotonic reads: once a session has seen version 12, it doesn't later see version 11. Track a minimum observed position, or use a replica with a continuity policy across failover.
Sticky routing helps normally, but doesn't handle promotion to an older copy. Position tokens must survive leadership changes, and waiting needs a deadline and fallback.
How much lag is acceptable?
Separate the product's paths. An analytics counter may tolerate seconds of lag; a transfer confirmation needs stronger guarantees. Serving a writer's own profile from the leader doesn't require sending every anonymous profile view there.
Monitor replication position gaps and apply delay, not just whether follower processes are alive. A follower can be reachable while hours behind. Remove excessively lagging replicas from sensitive read routes and investigate sustained lag before it becomes a recovery problem.
Try it
File Storage HLD
Choose replication and read-routing policies for metadata that users expect to see immediately after editing.