IO-bound vs CPU-bound (Choosing the Right Tool)

Classify the Work

• I/O-bound: waiting on network/disk dominates
• CPU-bound: computation dominates
• Mixed: needs separation (offload CPU work)

Decision Table

• I/O-bound: threads or asyncio
• CPU-bound: multiprocessing or native extensions
• Mixed: async for I/O + process pool for CPU tasks

Mixed Example: Offload CPU

import asyncio
from concurrent.futures import ProcessPoolExecutor

def cpu(x: int) -> int:
    return x * x

async def main():
    loop = asyncio.get_running_loop()
    with ProcessPoolExecutor(max_workers=4) as ex:
        result = await loop.run_in_executor(ex, cpu, 10)
        return result

print(asyncio.run(main()))

Operational Checklist

• Measure under realistic load before committing to a concurrency model.
• Bound concurrency and add timeouts around external calls.
• Design shutdown behavior (cancellation, draining queues) explicitly.

Failure Modes

• Complexity tax: mixing models without need increases incident risk.
• Overload: fan-out without limits saturates CPU, memory, or dependencies.