Study notes · 11% of the exam

Concurrency, the GIL and asyncio

Pick the right tool for the workload: processes for CPU-bound Python, threads for blocking I/O, asyncio for many concurrent waits. Then know exactly how errors, cancellation and shared state behave in each.

Key points

1
The GIL lets one thread run Python bytecode at a time; blocking I/O and many C extensions (hashlib, zlib, NumPy) release it, so threads still help for I/O and native work.
2
The GIL does not make your code thread-safe: x += 1, check-then-act and other read-modify-write sequences need a threading.Lock.
3
multiprocessing pickles functions and arguments: use top-level functions, guard the entry point with if __name__ == "__main__": (required with spawn, the default on Windows and macOS), and batch tiny tasks with chunksize.
4
In asyncio a coroutine only yields at an await that really suspends; time.sleep or a synchronous driver blocks the whole loop, so offload with asyncio.to_thread.
5
gather keeps argument order and does not cancel siblings on error; TaskGroup (3.11+) cancels siblings and raises an ExceptionGroup handled with except*.
6
Cancellation is a CancelledError (a BaseException since 3.8) raised at the await; always re-raise it, or timeouts and TaskGroups stop working.
7
Keep references to tasks from create_task (or use a TaskGroup), and always consume futures with result(), or exceptions are silently lost.

Common traps

Calling a coroutine function without awaiting it does nothing except warn "coroutine was never awaited".
cache.setdefault(key, load(key)) still calls load every time, because arguments are evaluated first.
A thread-pool task that waits on another task submitted to the same full pool deadlocks.

Read the source

Test yourself on Concurrency, the GIL and asyncio

Ten questions, with the answer and explanation after each one.