Backpressure in Streaming (Don't OOM)

Production incident

A file download endpoint streams large content from an upstream storage service. Someone reads the full response into memory and then writes it to the client. Under a few concurrent downloads, memory spikes and pods get OOMKilled. Another incident: a background consumer reads from a message queue as fast as possible and buffers work in memory while the downstream DB is slow. Memory becomes an unbounded queue. Backpressure was missing.

Symptoms

• Memory grows with concurrency. OOMKills appear during spikes.
• GC time increases; throughput drops.
• Latency increases for all requests because the process is under memory pressure.
• Streaming endpoints buffer large payloads unexpectedly.

Root causes

• Reading full content into memory (ReadAsStringAsync/ReadAsByteArrayAsync) for large responses.
• Unbounded in-memory queues in background processing.
• No bounded channel / no flow control between producer and consumer.

Diagnosis

# Look for buffering calls
grep -R "ReadAsByteArrayAsync\|ReadAsStringAsync" -n .
grep -R "ToList\(" -n . | head -n 50

# Look for in-memory queues
grep -R "ConcurrentQueue\|BlockingCollection\|List<" -n .

Anti-pattern

// Buffers entire file in memory: OOM waiting to happen
var bytes = await resp.Content.ReadAsByteArrayAsync(ct);
return File(bytes, "application/octet-stream");

Correct pattern

Stream with ResponseHeadersRead and CopyToAsync. For background pipelines, use bounded channels and respect cancellation.

// Streaming download without buffering
var resp = await _http.SendAsync(req, HttpCompletionOption.ResponseHeadersRead, ct);
resp.EnsureSuccessStatusCode();

await using var stream = await resp.Content.ReadAsStreamAsync(ct);
Response.ContentType = "application/octet-stream";
await stream.CopyToAsync(Response.Body, ct);

// Bounded channel for producer/consumer backpressure
var channel = Channel.CreateBounded<WorkItem>(new BoundedChannelOptions(capacity: 500)
{
    FullMode = BoundedChannelFullMode.Wait
});

// Producer writes: will block when full (backpressure)
await channel.Writer.WriteAsync(item, ct);

// Consumer reads: controls throughput
await foreach (var work in channel.Reader.ReadAllAsync(ct))
{
    await ProcessAsync(work, ct);
}

Backpressure design rules

• Bound everything: queues, buffers, concurrency.
• Prefer streaming: do not materialize large payloads unless you must.
• Use time budgets: if the consumer is slow, you either shed load or slow the producer.
• Expose signals: queue length, channel fullness, drop/reject counts.

Security and performance impact

• Performance: prevents memory blowups and stabilizes throughput.
• Security: unbounded buffering is an easy DOS vector. Attackers can request large payloads and starve memory.

Operational notes

• Monitoring: memory, GC time, queue lengths, dropped/rejected work, streaming response sizes.
• Rollout: switch buffering endpoints to streaming with canary. Validate content integrity and client behavior.
• Rollback: if clients require buffering for specific features, keep it behind a limited, authenticated path with strict size limits.

Checklist

• No large payload buffering on hot paths.
• Streaming uses ResponseHeadersRead and CopyToAsync.
• Background pipelines use bounded channels or bounded queues.
• Queue length and memory pressure are monitored.
• Size limits exist to prevent abusive payloads.