Definition
Eventual consistency breaks systems when teams assume immediate correctness and ignore staleness, retries, and conflict handling in application design.
Key Concepts
- Correctness assumptions leak from product logic into infrastructure.
- Retries amplify divergence and duplicate writes.
- Caches can outlive replica convergence windows.
- Observability must measure staleness explicitly.
How It Works
Failures emerge from interaction effects: async replication + optimistic clients + retries + caching + weak idempotency guarantees.
Comparison Table
| Failure mode | Root cause | Mitigation |
|---|---|---|
| Lost updates | LWW conflict policy | Idempotency + domain merges |
| User confusion | Read-your-writes gap | Session guarantees |
| Data drift | Repair lag | Anti-entropy/read repair tuning |
Production Implications
Dynamo and Cassandra deployments routinely succeed when teams pair them with idempotent APIs, staleness budgets, and repair workflows. Without those controls, seemingly minor anomalies become customer-visible incidents.
When to Use / Not Use
Use It When
- As a pre-mortem reference for eventually consistent architectures.
- For incident reviews and reliability design docs.
Avoid It When
- As justification to avoid eventual consistency entirely.
- As a substitute for workload-specific architecture analysis.
Key Takeaways
- Start with user-visible correctness requirements, then choose the weakest model that still preserves them.
- Instrument staleness and conflict rates; treat them as first-class production metrics.
- Design reconciliation and repair workflows before incidents force ad-hoc fixes.