Ensuring Memoryview Thread Safety

Python
import threading

# Underlying mutable buffer shared across threads
buf = bytearray(1024)
mv = memoryview(buf)

lock = threading.Lock()

def worker(start, end, value):
    with lock:
        mv[start:end] = bytes([value]) * (end - start)

threads = [threading.Thread(target=worker, args=(i*100, (i+1)*100, i)) for i in range(10)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Key fact: on the standard GIL-enabled CPython build, individual memoryview operations (single-item get/set, slicing) are atomic at the bytecode level, but compound operations (read-modify-write, multi-step slice assignment) are not atomic and require explicit locking. On free-threaded builds (--disable-gil / PEP 703), even individual operations can race unless the memoryview's internal locking guarantees are understood — treat both builds the same way: assume nothing is atomic across multiple opcodes.

Progress:

• Step 1: Identify every memoryview/buffer-protocol object shared across threads
• Step 2: Classify each access pattern as read-only, single-write, or read-modify-write
• Step 3: Determine if the underlying object is mutable (bytearray, array.array, numpy array) or immutable (bytes)
• Step 4: Add synchronization (Lock/RLock) around any compound or read-modify-write access
• Step 5: Verify behavior under both the default GIL build and the free-threaded build if targeting 3.13+
• Step 6: Check for use-after-free hazards — releasing a buffer while a memoryview into it is still live on another thread

Step 1 — Inventory shared objects Grep for memoryview(, .cast(, buffer(, and any object passed to threading.Thread args that supports the buffer protocol.

Step 2 — Classify access

• Read-only across threads → generally safe, no lock needed.
• Single atomic write (mv[i] = x) → safe under GIL builds for single-element assignment; still verify on free-threaded builds.
• Read-modify-write (mv[i] += 1, slice assignment computed from current contents) → always needs a lock.

Step 3 — Mutability of backing store

• bytes objects backing a memoryview are immutable — no data race possible, but the memoryview object itself (its state: shape, released flag) can still race.
• bytearray, array.array, mmap, numpy buffers — mutable, so concurrent writes need locks.

Step 4 — Add synchronization Wrap compound operations in threading.Lock. Prefer one lock per logical buffer, not a global lock, to avoid contention.

Step 5 — Verify across builds Free-threaded CPython removes the GIL, so operations that were "accidentally atomic" under GIL (e.g., single bytecode ops) may no longer be. Test with python3.13t or the --disable-gil build if supporting it.

Step 6 — Lifetime hazards Calling .release() on a memoryview or letting the exporting object be garbage collected while another thread holds a view into it causes undefined behavior or crashes. Ensure the exporting object outlives all views, and don't release a shared memoryview from one thread while another is using it — protect release with the same lock.

Example 1: Input:

Python
counter_buf = bytearray(8)
mv = memoryview(counter_buf).cast('q')  # int64 view

def increment():
    mv[0] += 1  # read-modify-write, NOT atomic

Output (fixed):

Python
counter_buf = bytearray(8)
mv = memoryview(counter_buf).cast('q')
lock = threading.Lock()

def increment():
    with lock:
        mv[0] += 1

Reasoning: mv[0] += 1 compiles to separate LOAD/ADD/STORE steps — a race condition on any build, GIL or not.

Example 2: Input:

Python
data = bytearray(range(256))
mv = memoryview(data)

def reader():
    return bytes(mv)  # full read

def releaser():
    mv.release()

Output (fixed): Guard release with the same lock used for reads, and only release once no other thread holds a reference:

Python
lock = threading.Lock()

def reader():
    with lock:
        return bytes(mv)

def releaser():
    with lock:
        mv.release()

Reasoning: releasing a memoryview while another thread is mid-read causes a BufferError or crash; synchronize lifetime operations, not just data mutation.