Assessing Bytearray Thread Safety
Under free-threaded CPython (PEP 703, no-GIL builds), individual bytearray method calls are atomic (thread-safe against corruption), but sequences of operations are not atomic. Any code that reads-then-writes, or relies on invariants holding across multiple statements, needs an explicit threading.Lock.
Pythonimport threading buf = bytearray(1024) lock = threading.Lock() # UNSAFE under free-threading: check-then-act race if len(buf) > 0: buf[0] = 42 # buf could have been resized/cleared by another thread # SAFE: single atomic call buf.append(1) # atomic, no lock needed # SAFE: compound operation needs a lock with lock: if len(buf) > 0: buf[0] = 42
Progress:
- Identify every
bytearrayinstance shared across threads - Classify each access as single-operation vs. compound (multi-statement)
- For single operations (single method call, single subscript get/set,
append,extend, slice assignment as one statement), no lock needed — CPython guarantees the operation itself won't corrupt internal state - For compound operations (read-modify-write, length-check-then-index, iterate-while-mutating, buffer resize interacting with a
memoryview), add athreading.Lockorthreading.RLockaround the whole sequence - Pay special attention to
memoryviewobjects backed by abytearray— resizing the bytearray while a memoryview is exported is unsafe and can raiseBufferErroror, if not guarded, race - Verify on the standard GIL build too — locks are correct there and add negligible overhead, so code should be written lock-safe regardless of build
- Test with
PYTHON_GIL=0(or a free-threaded build) using stress loops (many threads hammering the same bytearray) to surface races that only appear without the GIL
Example 1: Input:
Pythoncounter_buf = bytearray(8) def increment(): val = int.from_bytes(counter_buf, "little") val += 1 counter_buf[:] = val.to_bytes(8, "little")
Output: Unsafe — this is a classic read-modify-write race. Two threads can read the same val before either writes back, losing an increment. Fix:
Pythonlock = threading.Lock() def increment(): with lock: val = int.from_bytes(counter_buf, "little") val += 1 counter_buf[:] = val.to_bytes(8, "little")
Example 2: Input:
Pythonlog_buf = bytearray() def append_entry(data: bytes): log_buf.extend(data)
Output: Safe as-is. extend is a single atomic method call; free-threaded CPython guarantees the internal buffer state stays consistent even with concurrent callers. No lock required (though ordering of entries across threads is still nondeterministic — add a lock only if ordering matters, not for safety).
Example 3: Input:
Pythondata = bytearray(range(256)) mv = memoryview(data) def resize(): data.extend(b"\x00" * 100) # thread A def read(): print(mv[0]) # thread B
Output: Unsafe. Resizing a bytearray while a memoryview exports it is undefined/racy territory even beyond normal thread-safety — CPython may raise BufferError on the resize if the memoryview is still exported, and concurrent access patterns are unreliable. Guard both the resize and any memoryview-based access with the same lock, or avoid holding long-lived memoryviews over bytearrays that are concurrently resized.
- Treat "thread-safe against corruption" and "correct under concurrency" as different things — an atomic op prevents crashes/corrupted memory, not logical races.
- Default to wrapping any multi-step interaction with a shared
bytearrayin a lock; only skip the lock when you're confident the operation is a single C-level call. - Prefer
bytes(immutable) overbytearraywhen the data doesn't need in-place mutation — it sidesteps thread-safety concerns entirely. - When using
memoryviewon abytearray, keep the memoryview's lifetime short and lock around any resize (extend,+=,del buf[:], slice assignment that changes length) that happens concurrently with the memoryview's use. - Write lock-based code even if you only target the GIL build today — it future-proofs against free-threaded builds and costs little.
- Assuming "the docs say bytearray operations are thread-safe" means the whole program logic is thread-safe — it only covers individual operations, not sequences.
- Forgetting that slice assignment with a size change (
buf[2:5] = b"longer_replacement") can trigger a resize, which interacts badly with concurrently held memoryviews. - Using a plain
Lockfor read-heavy workloads when anRLockor athreading.Condition-based readers/writer pattern would avoid unnecessary serialization — pick the right primitive for the access pattern, not just "any lock." - Testing only on the standard GIL-enabled interpreter and assuming that guarantees correctness on free-threaded builds — the GIL currently masks many races that free-threading exposes.