Async Backpressure Basics

Apply basic backpressure in async Rust to avoid overload: bound queues, limit concurrency, and fail fast so production systems remain stable under traffic spikes.

On this page

Backpressure: Preventing Overload Cascades

In production, overload is normal: traffic spikes, dependencies slow down, and queues grow. Without backpressure, an async service can accept unlimited work, allocate unlimited memory, and collapse under load. Backpressure is the discipline of limiting how much work you accept so the system stays stable.

Production mindset:

Bound queues to cap memory growth
Limit concurrency to cap in-flight work
Fail fast when the system is saturated

Where Backpressure Matters in Async Rust

Async makes it easy to spawn tasks and push work into channels. That convenience is the risk: if you accept work faster than you can process it, memory and latency explode.

Common overload points:

HTTP request handlers spawning unbounded tasks
Unbounded channels between producers and consumers
Too many concurrent upstream calls
Background workers that cannot keep up with input

Bounded Channels: A Simple Backpressure Tool

Using a bounded channel creates a hard limit. When the channel is full, senders wait or fail depending on the API, applying natural backpressure.

Tokio provides bounded mpsc channels:

use tokio::sync::mpsc;

#[tokio::main]
async fn main() {
    let (tx, mut rx) = mpsc::channel::(100); // capacity 100

    let producer = tokio::spawn(async move {
        for i in 0..1000u64 {
            if tx.send(i).await.is_err() {
                break;
            }
        }
    });

    let consumer = tokio::spawn(async move {
        while let Some(v) = rx.recv().await {
            // process v
            let _ = v;
        }
    });

    let _ = producer.await;
    let _ = consumer.await;
}

Production note: capacity is a stability control. Pick a number that caps memory and latency, not a number that "never fills".

Try-Send: Fail Fast Instead of Waiting

Sometimes you do not want producers to wait. You want to reject work quickly when the queue is full (especially on HTTP boundaries).

use tokio::sync::mpsc;

fn try_enqueue(tx: &mpsc::Sender<u64>, v: u64) -> Result<(), String> {
    tx.try_send(v).map_err(|_| "queue full".to_string())
}

Production mapping: queue full often translates to HTTP 429 or 503 depending on your policy.

Limit Concurrency With Semaphore

Even with bounded queues, you can overload dependencies by running too many requests in parallel. A semaphore caps in-flight operations.

use std::sync::Arc;
use tokio::sync::Semaphore;

async fn call_upstream(_id: u64) -> Result<(), String> {
    Ok(())
}

#[tokio::main]
async fn main() {
    let sem = Arc::new(Semaphore::new(50)); // max 50 in-flight

    let mut handles = vec![];
    for i in 0..1000u64 {
        let s = Arc::clone(&sem);
        handles.push(tokio::spawn(async move {
            let _permit = s.acquire().await.unwrap();
            let _ = call_upstream(i).await;
        }));
    }

    for h in handles {
        let _ = h.await;
    }
}

Production rule: semaphores are a simple and effective overload control for upstream calls.

Avoid Unbounded Task Spawning

A common anti-pattern is spawning one task per request without limits. Under a traffic spike, you create millions of tasks. Even if each task is small, scheduling overhead and memory usage become a problem.

Production alternatives:

Bounded channels + fixed worker tasks
Semaphore-limited concurrency
Backpressure at the HTTP layer (rate limiting, 429/503)

Backpressure and Timeouts Work Together

Backpressure limits how much work you accept. Timeouts limit how long work can occupy resources. In production, you usually need both.

Minimal combination:

Semaphore limits concurrency
Timeouts bound dependency waits
Bounded queues cap buffering

Observability Signals

Backpressure should be visible. Track:

Queue depth (or capacity usage)
Semaphore permits in use
Rejected requests due to saturation
Latency increases under load

tracing::warn!(event = "backpressure", "queue full");

Production note: saturation signals often appear before full outages. They are early warnings.

Common Production Pitfalls

Using unbounded channels because it is easier
Spawning unbounded tasks under load
Setting queue capacity extremely high, defeating the purpose
No observability for saturation events
Backpressure applied too late, after memory growth already happened

Production Checklist

Bounded channels between producers and consumers
Concurrency caps (semaphore) around expensive operations
Fail fast when saturated where appropriate
Timeouts applied to keep in-flight work bounded
Saturation visible in logs/metrics

Backpressure is not a luxury feature. It is how production async services survive traffic spikes and dependency slowdowns without collapsing. Start with bounded queues and concurrency limits, then evolve as needed.

← Async Timeouts

Async Common Pitfalls →