concurrent.futures (ThreadPool/ProcessPool)

High-Level Pools

concurrent.futures provides a simple abstraction for parallel execution with futures, timeouts, and cancellation hooks.

ThreadPool Example

from concurrent.futures import ThreadPoolExecutor, as_completed

def fetch(url: str) -> str:
    return url

urls = ["a", "b", "c"]
with ThreadPoolExecutor(max_workers=10) as ex:
    futures = [ex.submit(fetch, u) for u in urls]
    for f in as_completed(futures, timeout=5):
        print(f.result())

ProcessPool Example

from concurrent.futures import ProcessPoolExecutor

def cpu(x: int) -> int:
    return x * x

with ProcessPoolExecutor(max_workers=4) as ex:
    print(list(ex.map(cpu, range(10))))

Operational Checklist

• Bound concurrency with max_workers.
• Use timeouts to prevent indefinite waits.
• Handle exceptions per-future; do not assume success.

Failure Modes

• Unbounded fan-out: too many tasks overwhelm downstream services.
• Silent failures: exceptions hidden until result() is called.