Assessing Python Thread Safety
CPython documents thread safety per-module/per-API using four levels (as of docs.python.org/3.16/library/threadsafety.html):
- Thread-safe — safe to call from multiple threads concurrently, no external locking needed.
- Conditionally thread-safe — safe under specific conditions (e.g., each thread uses its own instance, or a documented restriction is followed).
- Not thread-safe — requires external synchronization (locks) if shared across threads.
- GIL-only safe / undocumented — safety, if any, is an accidental artifact of the GIL, not a documented guarantee; do not rely on it, especially under free-threaded (no-GIL) builds.
When reviewing code: identify every stdlib object shared across threads, look up its documented level, and add locking or redesign if it's not "Thread-safe."
Progress:
- Step 1: List all stdlib modules/objects touched by multiple threads in the code under review.
- Step 2: For each, check the official thread-safety classification (module docs or the threadsafety.html table).
- Step 3: Classify each usage into one of the four levels above.
- Step 4: For "Conditionally thread-safe," verify the stated condition actually holds in this codebase.
- Step 5: For "Not thread-safe" or "GIL-only," flag and prescribe a fix (lock, thread-local storage, immutable/copy-on-read pattern, or queue-based handoff).
- Step 6: Re-check under free-threaded CPython (PEP 703 builds) assumptions — GIL-only safety disappears there.
- Step 7: Document findings inline or in a review comment with the level and citation.
Example 1:
Input: Multiple threads append to the same list and read from it without locks.
Output: list.append/list.pop on a single list are individually atomic under the GIL (documented as thread-safe for these ops), but compound operations (check-then-act, e.g., "if item in list: list.remove(item)") are not thread-safe as a sequence. Recommend a threading.Lock around the compound operation or switch to queue.Queue.
Example 2:
Input: A dict is read by many threads and occasionally written by one.
Output: Single dict[key] = value and dict[key] reads are atomic (thread-safe for individual ops) even under free-threading. But iteration (for k in d) concurrent with mutation is not thread-safe — can raise RuntimeError: dictionary changed size during iteration. Recommend snapshotting with list(d.items()) before iterating, or a lock around the mutate+iterate region.
Example 3:
Input: Code relies on sys.modules caching behavior during concurrent imports.
Output: The import system is documented as thread-safe (uses an internal per-module lock since Python 3.3+), so concurrent import of the same module from multiple threads is safe without extra locking.
Example 4:
Input: Shared collections.OrderedDict mutated by two threads with no lock, code "seems fine" because tests pass under GIL.
Output: Flag as GIL-only safe / undocumented for compound patterns (e.g., move_to_end + iterate). Passing tests under the GIL is not a guarantee — under a free-threaded build this can race. Add explicit locking.
- Never infer thread safety from "it works in testing" — the GIL historically masked races; free-threaded builds won't.
- Distinguish atomicity of a single bytecode-level op from safety of a sequence of ops — the former being safe doesn't make compound logic safe.
- Prefer higher-level concurrency primitives (
queue.Queue,concurrent.futures,threading.Lock/RLock) over relying on implicit atomicity. - When a module's docs don't explicitly state a thread-safety level, treat it as not thread-safe by default (conservative assumption).
- Re-audit any "GIL reliance" code before deploying on Python builds with
--disable-gil/ free-threading enabled. - Cite the specific documented level and source when flagging an issue, so reviewers can verify.
- Assuming an entire class/module is thread-safe because one method is documented as such — check per-operation, not just per-object.
- Confusing "atomic" (won't corrupt memory) with "correct" (won't produce logically wrong results under interleaving).
- Treating GIL-era safe patterns as portable guarantees going forward — they are not, once free-threaded CPython is in play.
- Overlooking mutable default arguments or module-level globals shared implicitly across threads.
- Forgetting that C-extension modules may have their own (possibly undocumented) thread-safety characteristics independent of the GIL.