Core Code
Core Code Modern Dev Handbook
SYSTEM-DESIGN Contents
Constraints & Core Metrics
What Is System Design (Production-First) Functional vs Non-Functional Requirements Constraints, Assumptions, and Design Inputs Latency vs Throughput Tail Latency (p95/p99) and Why It Dominates UX SLI, SLO, SLA: The Reliability Contract Capacity Estimation Basics Bottlenecks and Queues (Little's Law Intuition)
Scaling & Traffic Management
Vertical vs Horizontal Scaling Load Balancing Fundamentals Caching Strategies Overview Cache Invalidation and Consistency Rate Limiting at Scale Backpressure: Preventing Overload Timeouts and Retries (Safe Defaults) Graceful Degradation and Traffic Shedding
Data & Consistency
Database Scaling Basics Replication: Read Replicas and Failover Consistency Models (Strong vs Eventual) Sharding Basics Partitioning and Hotspot Avoidance Idempotency and Deduplication Eventual Consistency and UX Schema Evolution and Backward Compatibility
Reliability Patterns
Timeouts: Bounding Damage Retry Strategies (Jitter, Caps, Budgets) Circuit Breaker Bulkhead Pattern Fallbacks and Graceful Degradation Load Shedding and Admission Control Backups and Restore Strategy Disaster Recovery (RPO/RTO)
Observability & Control
Golden Signals: Latency, Traffic, Errors, Saturation Metrics: SLIs and SLO Targets Logging at Scale (Structure and Cost) Correlation IDs and Context Propagation Distributed Tracing Overview Alerting and Runbooks Capacity Planning and Saturation Postmortems and Continuous Improvement
Case Studies
Case Study: URL Shortener Case Study: File Upload and Serving Case Study: Notification System Case Study: Chat System Overview Case Study: Rate Limiter Service Case Study: Social Feed (Fan-out Trade-offs) Case Study: Search Autocomplete Case Study: Analytics Ingestion Pipeline
Distributed Systems Foundations
CAP Theorem Consensus Basics Leader Election Distributed Locking Clock Skew Split Brain Quorum Reads and Writes Eventual vs Strong Consistency (Practical)
Service Communication Patterns
Synchronous vs Asynchronous Request-Response Pattern Message Queues Pub/Sub Pattern Event-Driven Architecture Saga Pattern CQRS Overview Outbox Pattern API Gateway Pattern Backend For Frontend (BFF)
Data at Scale Advanced
Index Design Deep Dive Query Planning Multi-Region Databases Read-After-Write Guarantees Global Databases Data Locality Large Table Strategies Background Jobs Design
Multi-Region Architecture
Multi-Region Basics Active-Active Active-Passive Geo DNS Cross-Region Replication Global Latency Optimization Regional Failover
Cost-Aware System Design
Cost vs Performance Storage Cost Trade-offs Cache Cost Analysis Overprovisioning vs Risk Cloud Pricing Mental Model Right-Sizing Instances
Security Architecture
Zero Trust Service-to-Service Auth Secrets Management Encryption at Rest Encryption in Transit DDoS Protection Multi-Tenant Data Isolation
System Design Interview
Design Twitter Design YouTube Design Uber Design Payment System Design E-commerce Platform Design Distributed Cache

Distributed Locking

Understand safe coordination across nodes with distributed locks.
On this page

What Distributed Locks Are For

Distributed locks coordinate work across multiple processes or machines: “only one worker should run this job”, “only one node should mutate this resource at a time”. They are tempting, but in production they are a frequent source of subtle outages because networks and clocks are unreliable.

Prefer Avoiding Locks

Production-first design tries to avoid distributed locks when possible by using safer patterns:

  • Idempotency: allow repeated execution safely
  • Single-writer per key: partition work so only one node owns a key range
  • Database constraints: unique indexes and transactions to enforce exclusivity
  • Leases with expiry: bounded ownership rather than indefinite locks

Why Locks Fail in Distributed Systems

  • Network partitions make lock ownership ambiguous
  • Clock skew breaks expiry assumptions
  • Process pauses (GC, CPU starvation) delay heartbeats
  • Lock services can become a single point of failure

Locks vs Leases

A lease is a lock with an expiry that must be renewed. Leases are generally safer because they are self-healing: if the holder dies or loses connectivity, ownership can eventually transfer. But leases require careful renewal and fencing.

Fencing Tokens Prevent Stale Owners

Even with leases, a slow process can believe it still owns the lock after another node has taken over. A fencing token is a monotonically increasing number that downstream systems can use to reject actions from stale owners.

Fencing concept:
- Lock acquisition returns token T (increasing)
- All writes include token T
- Storage rejects writes with token < latest token
This prevents stale lock holders from corrupting state.

When You Actually Need a Distributed Lock

  • One-time migrations where double execution is dangerous
  • Leader-only jobs where partitioning is not available
  • External side effects that cannot be made idempotent

Production-First Takeaway

Distributed locks are risky. Avoid them when possible via idempotency and partitioned ownership. If you must lock, use leases with renewal, add fencing tokens, and treat the lock service as critical infrastructure with monitoring and failure testing.

Clock Skew →