← all lessons
System Design · Consistency & Replication ·

Read Scaling with Follower Reads: When Staleness Is a Feature, Not a Bug

Core concept
In a leader-follower replication setup (one node accepts writes; others copy them), reads don't have to go to the leader. You can route read traffic to followers, which replicate the leader asynchronously (with a small delay, not waiting for an acknowledgment). This means followers may lag behind by milliseconds to seconds — they're eventually consistent (will match the leader given enough time, but not instantly). The key insight is that for many read workloads, that lag is completely acceptable, and offloading reads to followers can multiply your read throughput almost linearly with the number of followers you add.

flowchart LR
    Client -->|write| Leader
    Leader -->|replicate async| F1[Follower 1]
    Leader -->|replicate async| F2[Follower 2]
    Client -->|read| F1
    Client -->|read| F2
    F1 -->|stale?| Warning[Replication lag]

Concrete real-world example
Imagine a social media feed. A user posts a photo; that write goes to the leader. Two seconds later, a different user (or even the same user on a second tab) reads their feed — and it's fine if that photo appears 2 seconds late. A product catalog on an e-commerce site is another classic case: the price of a TV doesn't need to be nanosecond-fresh on every page load. In both cases, reads are far more frequent than writes (often 10:1 or higher), so pushing reads to 5 followers effectively gives you 5× read throughput at low cost, while the leader handles only the write stream.

One trade-off / gotcha
Replication lag is not fixed — it's a variable tied to load. Under heavy write traffic, followers can fall hours behind, not just milliseconds. If your application logic assumes "a follower is at most 1 second stale" and then a deployment spike pushes lag to 90 seconds, reads can return shockingly outdated data. This is called replication lag amplification, and it bites hardest exactly when your system is under the most stress — the worst possible time. You must monitor follower lag actively (most databases expose a lag metric in bytes or seconds) and have a fallback policy, like routing to the leader if lag exceeds a threshold.

An interview-style question to ponder
You're designing a banking dashboard where users can view their account balance. The team wants to add read replicas to scale read traffic. The product manager says "slight staleness is fine — it's just a display." Should you trust that and route balance reads to followers? How would you decide?

Stuck? Show a hint

Think about what "slight staleness" actually means in financial context — specifically, what user action might immediately follow a balance read, and whether that action depends on the freshness of that read.

Show answer

No, you should not blindly route balance reads to followers without safeguards, even with PM approval.

  • The core risk isn't just "the number looks wrong" — it's that users make decisions based on the balance they see. If Alice sees a stale balance of $500 (actual: $50 after a recent debit), she may initiate a $400 transfer. Her next action is causally dependent on the read she just made, which means the read needs to be fresh enough to be safe.
  • The right framing is: distinguish display reads (show recent transactions, mostly cosmetic, staleness harmless) from decision reads (reads that gate a write). For decision reads, route to the leader or use a lease-based read (ask the leader to confirm it's still the authority before returning data) to guarantee freshness.
  • A practical middle path: add a query parameter or session flag — require_fresh=true — for balance reads that precede transfers, and let the application layer route those to the leader. Pure display reads (transaction history, graphs) can safely hit followers.
  • But why not just always read from the leader? Because the whole point was to scale reads — if every read goes to the leader, you've gained nothing. The discipline is categorizing reads by whether staleness creates a safety hazard vs. a cosmetic annoyance, and routing accordingly.
  • Watch out: the PM's intuition is right in aggregate (most reads are fine on followers) but wrong at the margin (the few reads that precede writes need fresh data) — and it's exactly those marginal reads that cause incidents.