Backpressure and Flow Control
On this page
What Backpressure Solves
- Without flow control, queues grow, latency explodes, and systems collapse.
- Backpressure ensures producers slow down when consumers are saturated.
- Production rule: do not rely on infinite buffering, it only delays failure.
Signals of Saturation
- Queue depth and consumer lag.
- Thread pool or worker utilization.
- Latency percentile spikes and timeout rate increases.
- Memory pressure and GC pause growth.
Flow Control Techniques
- Concurrency limits: cap in flight requests per node.
- Token buckets: rate limit producers with burst control.
- Credit based: consumers grant credits to producers.
- Queue buffering: absorb bursts but enforce size limits.
Load Shedding
- Reject or degrade non critical requests under pressure.
- Return fast errors instead of slow timeouts.
- Prefer shedding at the edge to protect core services.
Bulkheads
- Isolate resources: separate pools for critical and non critical work.
- Prevent one noisy workload from starving others.
- Apply per tenant limits to avoid noisy neighbor issues.
Failure Modes
- Buffer bloat: large queues hide overload until latency becomes unusable.
- Thundering herd: recovery causes many workers to resume simultaneously.
- Queue collapse: memory exhaustion from unbounded buffering.
- Head of line blocking: one slow job blocks many behind it.
Operational Checklist
- Bound all queues and define drop or spill policies.
- Set concurrency limits and validate under load tests.
- Implement load shedding and return fast errors.
- Separate pools for critical paths and background work.
- Monitor saturation signals and automate scaling with caps.