Vector Clocks: Tracking Causality Without a Central Clock
Core concept
In a distributed system, there's no shared wall clock you can trust, so you can't simply use timestamps to determine which write "happened first." A vector clock solves this by giving each node a logical counter, and each message carries the full vector of all counters seen so far. When two events share a vector clock where one is strictly dominated by the other, you have a causal ordering. When neither dominates, you have a genuine concurrent conflict — and the system must decide how to resolve it rather than pretend one was simply "later."
Diagram
flowchart LR
A["Node A writes\nVC: A=1, B=0"] -->|sends to B| B["Node B receives\nVC: A=1, B=0"]
B -->|B increments| C["Node B writes\nVC: A=1, B=1"]
A -->|A increments| D["Node A writes\nVC: A=2, B=0"]
C -->|compare| E["Conflict Detector\nNeither dominates"]
D -->|compare| E
Concrete real-world example
Amazon's Dynamo (the internal key-value store, not DynamoDB) famously used vector clocks to manage shopping cart conflicts. If you add an item on your phone while offline and your partner adds a different item on a laptop, both writes get their own vector clock entries. When you sync, Dynamo detects the conflict because neither vector clock subsumes the other — and passes both versions to the application layer, which merges them (union of cart items). This is why your cart sometimes briefly shows duplicates: the merge is happening.
One trade-off / gotcha
Vector clocks grow unboundedly — one entry per node that ever touched the data. In a large cluster with many replicas or with client-side vector clocks (as Dynamo originally did), the clock can balloon into hundreds of entries. Amazon's fix was clock pruning: drop the oldest entries when the vector exceeds a size threshold. But pruning destroys causal information, turning some "detectable conflicts" back into silent last-write-wins merges — exactly the problem vector clocks were meant to avoid. Riak later switched to dotted version vectors to address this more cleanly.
An interview-style question to ponder
You're designing a collaborative text editor backed by a distributed key-value store. Two users simultaneously edit the same document key on different regional nodes. The store uses last-write-wins (LWW) with wall-clock timestamps. A senior engineer flags this as dangerous. Why, and what would you use instead?
Stuck? Show a hint
Two separate things make wall-clock LWW dangerous here. First, can clocks on different machines be trusted to the millisecond — and what does a clock that jumps backwards do to "latest timestamp wins"? Second, when two edits are genuinely simultaneous, is there even a correct winner — and what silently happens to the loser?
Show answer
LWW with wall clocks silently discards one user's entire edit, and you can't even detect that it happened.
- Wall clocks on different machines can skew by hundreds of milliseconds or more, and NTP corrections (NTP is the protocol that keeps machine clocks in sync) can move time backwards. A write from Node B timestamped 12:00:00.050 can be physically later than one from Node A at 12:00:00.100 after a clock adjustment — LWW would discard the later real-world write.
- Even with perfect clocks, two simultaneous edits are genuinely concurrent: there's no correct "winner." Silently picking one means the other user's change vanishes with no error, no warning, and no merge — a data loss event disguised as normal operation.
- The right tool here is vector clocks (or a CRDT like a logoot/LSEQ structure for text). Vector clocks let the system surface the conflict; the application can then merge both edits, show a diff, or ask the user. You preserve both writes until a human or merge function resolves them.
- But why not just use a single-leader setup with a serialization queue? You could — and for many editors, that's exactly right (Google Docs uses operational transformation (a technique where a central server rewrites concurrent edits so they apply in a consistent order) through a central server). But it sacrifices availability: if the leader is unreachable, nobody can edit. Vector clocks let multi-master, offline-capable systems remain available and reconcile later.
- Watch out: surfacing conflicts to users is only useful if your merge logic is correct — a naive "concatenate both versions" merge on structured JSON is just as destructive as LWW.