Home / Blog / Post

March 2026 · Rich Robertson

What Is a Distributed Lock? (With Examples)

A distributed lock is a coordination mechanism, not safety by itself. This page is the overview, decision model, and navigation hub for distributed locks. It introduces core concepts and links to deeper specialist topics.

This article is part of a distributed coordination series covering locks, leases, leader election, and correctness tradeoffs in distributed systems.

Distributed Coordination Deep Dives

What Is a Distributed Lock, Exactly?

A distributed lock decides which process may perform a critical action at a point in time across machines and services. This is the distributed analogue of a mutex, but with failure and timing uncertainty introduced by crashes, partitions, and lease expiry.

The biggest design mistake is treating "only one process acquired the lock" as equivalent to end-to-end correctness. Correctness depends on what happens after acquisition: whether writes are still validated, whether stale owners can be rejected, and whether downstream systems enforce ownership semantics instead of trusting client intent (Kleppmann, 2016).

Efficiency vs. Correctness: the first design question

Before choosing Redis, ZooKeeper, etcd, or any lock API, classify the problem. Some locks are for efficiency: preventing duplicate cron execution, duplicate cache rebuilds, or repeated background work. If the lock fails, the result is mostly wasted compute and noisy logs.

Other locks are for correctness: preventing conflicting writes, invalid state transitions, or unsafe external side effects. If this lock fails, system state or real-world behavior can become wrong. In that case, lock acquisition is only one part of a correctness design; the write path still needs ownership validation and stale-writer rejection (Kleppmann, 2016).

Practical rule: if failure only causes extra work, simpler coordination is often acceptable. If failure can produce incorrect state, the lock and storage boundary must be engineered as a correctness mechanism, not just a scheduling hint.

Distributed Lock vs. Lease: what changes operationally

In production, most distributed locks are implemented as leases: ownership is valid for a bounded interval unless renewed. This improves crash recovery because dead owners eventually lose ownership without manual cleanup.

But a lease is time-scoped permission, not an execution kill switch. A paused client can resume after expiry and continue issuing writes unless downstream systems can prove it is stale. That is why lease-only designs often appear safe in tests but fail under pauses and partitions.

Where Distributed Locks Fail in Production

Process crash after acquisition: lock remains held until expiry; work is delayed or retried late. Missing property: timely ownership cleanup. Mitigation: short leases, heartbeat renewal, and recovery logic that is safe to re-run.

Long GC pause or runtime stall: owner stops renewing but later resumes code execution. Missing property: stale-owner detection at write time. Mitigation: fencing tokens checked by storage, not just lock expiry timers.

Network partition: owner may believe it still holds the lock while quorum elected a new owner. Missing property: single authoritative ownership view at the write boundary. Mitigation: quorum-backed lock services plus token-validated writes.

Lease expiry during slow work: critical section exceeds lease and ownership transfers mid-operation. Missing property: bounded critical-section duration or renewal safety. Mitigation: conservative lease sizing, progress heartbeats, and chunking long work into idempotent units.

Client resumes after lease loss: old process writes late and overwrites current owner output. Missing property: monotonic write authorization. Mitigation: fencing token/version checks in storage or side-effect gateway.

Accidental unlock of another owner: client deletes a lock key it no longer owns. Missing property: owner-bound release semantics. Mitigation: compare ownership token on unlock (for Redis, use scripted compare-and-delete pattern).

Lock service outage or degraded quorum: no owner can be acquired, or guarantees degrade under partial failure. Missing property: explicit degraded-mode behavior. Mitigation: fail closed for correctness-sensitive paths; use controlled fallback only for efficiency-oriented jobs.

Assuming stronger guarantees than provided: teams infer linearizable ownership from an eventually consistent primitive. Missing property: precise guarantee model. Mitigation: document exactly what is guaranteed by the lock service and verify it matches failure impact (Redis, n.d.; etcd, n.d.; Apache ZooKeeper, n.d.-a).

Fencing Tokens: why lock ownership is not enough

Fencing tokens solve the stale-owner problem directly. On each successful acquisition, the lock service issues a monotonically increasing token. Every write carries that token, and the storage layer accepts only tokens newer than what it has already accepted.

Concrete case: worker A acquires token 41 and then stalls. Worker B later acquires token 42 and continues legitimately. Storage accepts writes tagged 42 and rejects any later writes from token 41. This preserves correctness even if A resumes after losing lease ownership (Kleppmann, 2016).

This is one of the most important design boundaries in distributed locking: lease expiry alone does not protect downstream state; monotonic ownership validation does.

Redis vs. ZooKeeper vs. etcd: different tools, different guarantees

Redis: operationally common and low latency. Typical primitive is SET key value NX PX ttl with owner-checked release logic. Often a strong fit for efficiency-oriented coordination, but correctness claims depend on exact topology and failure assumptions (Redis, n.d.).

ZooKeeper: coordination-first model with ephemeral and sequential znodes plus watch-based workflows. Usually easier to reason about for strict ownership and election patterns, with higher operational overhead than single-node cache-style locking (Apache ZooKeeper, n.d.-a, n.d.-b).

etcd: consensus-backed key-value store with leases, transactions, and lock/mutex patterns built on linearizable operations. Strong fit when correctness-sensitive coordination and explicit ownership semantics are required, with predictable but non-trivial consensus costs (etcd, n.d.; etcd, 2024).

None of these is universally "best." The right choice is the one whose guarantees are easy to reason about under your failure model and business impact.

Choose this when:

Examples: when a distributed lock helps, and when it is insufficient

Scheduled job ownership (efficiency-focused): three schedulers compete for an hourly cache rebuild lock. Duplicate execution is wasteful but harmless. A simple lease-based lock is usually good enough.

Migration or one-off admin task: only one operator instance should run a schema backfill. Locking reduces accidental concurrent execution; additionally record progress checkpoints so restarts are safe.

Single-writer metadata update (correctness-focused): only one writer may advance a control-plane version. Lock alone is insufficient; storage must reject stale token writes.

Workflow side-effect coordination: sending partner notifications or payment captures requires exactly-once effect semantics. Use locking only with idempotency keys and boundary dedupe.

Leader election / maintenance ownership: one node owns compaction or shard rebalancing at a time. Use lease renewal plus explicit leader loss handling so work stops safely on ownership loss.

When not to use a distributed lock

Many systems are better served by stronger or simpler alternatives: idempotency keys, compare-and-swap updates, partition ownership, queue-based serialization, single-writer assignment, or boundary dedupe.

Mature design judgment is recognizing that the best lock decision is often not introducing a lock at all. If correctness can be guaranteed directly in data models and write contracts, coordination becomes simpler and more robust.

A simple decision rule

Why this breaks in real systems

Production systems have GC pauses, scheduler stalls, delayed packets, and imperfect clocks. Those conditions stretch lease assumptions and can create windows where two actors both believe they own work.

A process can pause longer than lease TTL, another owner can legitimately take over, and the paused process can resume and issue stale writes. Lease expiry does not stop execution. Correctness requires write-boundary validation.

Production story: in a maintenance workflow, owner A stalled during deployment while owner B acquired and completed an updated plan. A later resumed and wrote old state into the same row because token checks were absent. The lock service was healthy; the mutation boundary was not.

FAQ

Is a distributed lock the same as a lease?
Usually implemented as a lease, but conceptually a lock is ownership intent; lease adds expiration semantics.

Can Redis be used for distributed locks?
Yes, commonly for efficiency-oriented coordination. For correctness-critical paths, validate whether Redis deployment and release semantics match your failure model.

Why are fencing tokens needed?
They allow downstream systems to reject stale owners after lease loss, which expiry alone cannot guarantee.

What happens if the lock holder pauses?
Its lease may expire and another owner may proceed. If the paused holder resumes without token checks, it can corrupt state.

When should I avoid distributed locks?
When idempotency, CAS, partition ownership, queue serialization, or single-writer assignment can guarantee behavior more directly.

References

What to Read Next

For the broader context around correctness-oriented coordination tradeoffs, see Distributed Systems Engineering: Correctness, Coordination, Reliability.

If you want the deeper lease model and failure analysis, read Designing a Correct Distributed Lease Service: Tenure on Raft. For how these same correctness choices interact with overload behavior, see Backpressure in Distributed Systems: Stability, Correctness, and Graceful Degradation.