Replication
and Stale Reads
A successful write does not mean every copy has caught up. Follow one value through a small distributed system and discover the gap.
One write. Three perspectives.
Write → read a follower → advance twice → read again. A write queues two messages; only advancing time delivers them.
Leader: Hello · v0
Follower A: Hello · v0 · caught up
Follower B: Hello · v0 · caught up
No read yet. Predict the result, then read a follower.
In transit 0
FIFO per follower · delivery after two steps
No pending messages.
Event log
Most recent first · last 100 events
- Step 0 · All replicas begin at v0: “Hello”.
First, make a prediction
The three nodes start with Hello at version 0. Write Goodbye, then read follower A before advancing time. Which value should it return?
It returns Hello. The leader has accepted version 1, but the follower still holds version 0. Advance twice and read again: now A returns Goodbye. You have observed a stale read followed by convergence.
The gap is replication lag
Replication lag is the delay between a write being accepted at the leader and that write becoming available at a replica. Here, the delay is two explicit simulation steps. In a real system, transport, queues, disk work, and replica load can all contribute.
A follower answers from the state it has applied. It does not automatically know that the leader has something newer. In this model, “stale” means the version returned is older than the leader’s version at the moment of the read. The displayed last read remains a historical observation as time advances.
Break a connection
Reset. Block follower B, write Goodbye, and advance twice. A catches up; B does not. Read B, then restore its link. Restoration alone does not deliver anything: advance once more to drain its overdue updates.
Now try two writes while B is blocked. Its queue retains both versions. Once restored, they are applied in write order on the next step. The final value is the newest one. A disruption can make lag grow without preventing the leader from accepting writes in this model.
Which trade-off did we choose?
Acknowledging at the leader without waiting for followers keeps follower delay off the write acknowledgment path in this model. The cost is that a later follower read may return an older version—even for the client that just wrote it.
Reading the leader, waiting for a particular version at a replica, or providing session guarantees are possible design directions. Each introduces costs and failure cases. We do not simulate them here. Real database guarantees depend on the product, topology, acknowledgment policy, read routing, and configuration; “replicated” alone specifies very little.
Next: Read Your Writes compares an immediate follower read with a read that waits for your session’s last acknowledged write.
Model boundaries
- One leader and two followers; one key containing a string. All start at v0.
- A write is acknowledged at the leader immediately. This says nothing about crash durability.
- Versions are logical sequence numbers. Steps are teaching units, not milliseconds.
- Each write queues one message per follower, due two steps later. Only Advance one step changes time.
- Each step applies all due messages on connected links in FIFO order per follower. Intermediate values may be applied in the same step.
- Blocking affects only leader-to-follower replication. Messages are retained; reads remain available. This is a selective link disruption, not a full node outage.
- No elections, leader failure, retries, message loss, transactions, concurrent clients, or persistence. State resets on reload; the event log keeps the last 100 entries.
- This is an explanatory model, not a database benchmark or a universal consistency contract.
Take it to a design discussion
A profile update is confirmed, but a page refresh shows the old name. Before adding retries, ask: where did the write complete, where did the read go, and what guarantee connects them?
Sources and further reading
The explanations, diagram, and simulator on this page are original. Topic inspiration includes Martin Kleppmann, Designing Data-Intensive Applications (O’Reilly, 2017). This project is independent and is not affiliated with the author or publisher.
For concrete implementation differences, see the official documentation for PostgreSQL warm standby and asynchronous replication and MongoDB causal consistency. These describe their own systems, not the guarantees of this simulation.