Multiprocessing Basics (CPU Parallelism)

Processes for CPU Parallelism

Multiprocessing bypasses the GIL by using multiple processes. It is effective for CPU-heavy tasks but has higher overhead.

Basic Pool

from multiprocessing import Pool

def work(x: int) -> int:
    return x * x

with Pool(processes=4) as pool:
    results = pool.map(work, [1, 2, 3, 4])

Operational Checklist

• Limit process count (often to CPU cores, then measure).
• Avoid passing huge objects between processes; serialize costs matter.
• Be explicit about start method when needed (platform differences).

Failure Modes

• Serialization overhead: pickling dominates runtime.
• Fork pitfalls: inherited state (open sockets, locks) can cause weird bugs.