Buffered IO, backpressure hints

Blocking I/O is simple but easy to misuse

Java classic I/O (InputStream, OutputStream) is blocking by default. This means read() blocks until data arrives. While simple, misuse can cause memory spikes, thread starvation, and slow downstream propagation.

Always use buffering intentionally

Unbuffered file or network I/O results in many small syscalls.

try (InputStream in = Files.newInputStream(path)) {
  int b;
  while ((b = in.read()) != -1) {
    process(b);
  }
}

This reads one byte at a time — inefficient.

Buffered pattern

try (
  InputStream in = new BufferedInputStream(Files.newInputStream(path));
  OutputStream out = new BufferedOutputStream(Files.newOutputStream(dest))
) {
  byte[] buffer = new byte[8192];
  int n;
  while ((n = in.read(buffer)) > 0) {
    out.write(buffer, 0, n);
  }
}

Use a reasonable buffer (8KB–64KB typical). Avoid tiny buffers.

Never read entire large file into memory blindly

Common production anti-pattern:

byte[] data = Files.readAllBytes(largeFile);

For multi-GB files, this can crash your JVM.

Prefer streaming processing

try (BufferedReader reader = Files.newBufferedReader(path)) {
  String line;
  while ((line = reader.readLine()) != null) {
    processLine(line);
  }
}

Process incrementally instead of loading everything.

Stream copy utility

For copying streams safely:

public static long copy(InputStream in, OutputStream out) throws IOException {
  byte[] buffer = new byte[8192];
  long total = 0;
  int n;
  while ((n = in.read(buffer)) > 0) {
    out.write(buffer, 0, n);
    total += n;
  }
  return total;
}

This avoids intermediate accumulation.

Backpressure in blocking I/O

In blocking I/O, backpressure happens naturally: if the consumer (writer) is slow, write() blocks, slowing the producer (reader). However, this only works if:

• You do not buffer unboundedly in memory.
• You do not decouple producer/consumer with unbounded queues.

Anti-pattern: unbounded buffering between threads

BlockingQueue<byte[]> queue = new LinkedBlockingQueue<>();

If producer is faster than consumer, memory grows until OOM.

Better: bounded queue

BlockingQueue<byte[]> queue = new ArrayBlockingQueue<>(100);

Now producer blocks when queue is full → natural backpressure.

Network stream timeouts

Blocking read() can hang forever if remote peer stops responding.

For sockets, configure read timeouts explicitly.

Cancellation awareness

Blocking I/O does not respond automatically to thread interruption in all cases. For graceful shutdown:

• Close the underlying stream.
• Handle InterruptedIOException.

Chunk size considerations

• Too small → syscall overhead.
• Too large → cache inefficiency.
• 8KB–32KB usually reasonable baseline.

Streaming JSON or CSV

Prefer streaming parsers instead of loading entire document:

• Use streaming APIs when parsing large JSON.
• Avoid accumulating full file in memory.

Flushing strategy

Do not call flush() after every write unless required. Flushing frequently defeats buffering benefits and increases syscall overhead.

Production failure scenario

A service reads uploaded files using readAllBytes and stores them in memory before processing. Under concurrent uploads, heap usage spikes and OOM occurs. Fix: stream processing + size limits.

File size limits

Always validate expected input size if accepting uploads or external streams. Unbounded input size is a denial-of-service vector.

Checklist

• Use BufferedInputStream / BufferedOutputStream.
• Prefer chunked streaming over readAllBytes for large data.
• Implement bounded queues when decoupling threads.
• Configure socket read timeouts.
• Avoid excessive flush calls.
• Validate input size limits.

Final principle

I/O is about flow control. If you buffer without bounds, you remove backpressure and move the bottleneck into memory — which will eventually fail under production load.