fleche.storage.thread_safe

Thread-safety mixins for storage classes.

These mixins subclass KeyManagement and override _operation_context to inject a threading lock around every storage operation. Because every concrete storage class (ValueMemory, Sql, …) ultimately inherits from KeyManagement, these mixins compose with any backend via Python’s MRO:

@dataclass(frozen=True)
class ThreadSafeValueMemory(SerializingMixin, ValueMemory): ...

Choose the mixin based on the access pattern you need:

• SerializingMixin — single global RLock. Every operation waits for the same lock, so the storage is touched by at most one thread at a time. Use when the backing store is not thread-safe and per-key parallelism is not needed.

• PerKeyLockMixin — a striped lock table, one RLock per key. Operations on different keys proceed in parallel; operations on the same key are serialized. Use when contention is dominated by hot keys and the backing store supports concurrent access on disjoint keys.

Both mixins use reentrant locks so nested acquisitions (e.g. expand being called inside load) do not deadlock.

Attributes

_per_instance_locks
_instances_lock

Classes

_PicklableLock	A threading.Lock wrapper that survives pickle round-trips.
_PicklableRLock	A threading.RLock wrapper that survives pickle round-trips.
SerializingMixin	Mixin that serializes all storage operations behind a single reentrant lock.
PerKeyLockMixin	Mixin that locks per-key so concurrent ops on different keys proceed in parallel.

Module Contents

class fleche.storage.thread_safe._PicklableLock

A threading.Lock wrapper that survives pickle round-trips.

The lock is re-initialised fresh on unpickle — its acquired/released state is not preserved. This is intentionally an in-process pickling aid (e.g. for multiprocessing spawn or joblib), not an inter-process synchronisation primitive: each process gets its own independent lock that shares no state with locks in other processes.

_factory

_lock

__reduce__()

__enter__()

__exit__(*args)

class fleche.storage.thread_safe._PicklableRLock

Bases: _PicklableLock

A threading.RLock wrapper that survives pickle round-trips.

Same in-process-only semantics as _PicklableLock; reentrant so that nested acquisitions (e.g. expand inside load) do not deadlock.

_factory

class fleche.storage.thread_safe.SerializingMixin

Bases: fleche.storage.base.KeyManagement

Mixin that serializes all storage operations behind a single reentrant lock.

Place before the concrete storage class in the MRO:

@dataclass(frozen=True)
class SerializingValueMemory(SerializingMixin, ValueMemory): ...

_lock: _PicklableRLock

_operation_context(key)

Context manager entered around every operation on key.

The base implementation is a no-op. Override in a mixin to inject any resource scoped to the operation — a threading lock, a SQLAlchemy session, an open file handle, a decompression stream, etc.

Receiving key lets implementations choose between a single global resource (ignore the key) or per-key resources (e.g. a striped lock table or a key-specific file handle).

Composing multiple mixins: use super() to chain so that every mixin in the MRO gets to wrap the operation:

@contextlib.contextmanager
def _operation_context(self, key):
    with self._lock:                   # this mixin's resource
        with super()._operation_context(key):
            yield

fleche.storage.thread_safe._per_instance_locks: weakref.WeakKeyDictionary[PerKeyLockMixin, tuple[weakref.WeakValueDictionary[fleche.digest.Digest | str, threading.RLock], _PicklableLock]]

fleche.storage.thread_safe._instances_lock: threading.Lock

class fleche.storage.thread_safe.PerKeyLockMixin

Bases: fleche.storage.base.KeyManagement

Mixin that locks per-key so concurrent ops on different keys proceed in parallel.

A lightweight threading.Lock guards the lock-table itself; once the per-key RLock is obtained the table lock is released, so two threads operating on different keys never block each other. Operations on the same key are serialized by the per-key lock, which is reentrant to allow nested calls (e.g. expand inside load).

Instances must be hashable. Place before the concrete storage class in the MRO:

@dataclass(frozen=True)
class PerKeyValuePickle(PerKeyLockMixin, ValuePickleFile): ...

_get_key_lock(key: fleche.digest.Digest | str) → threading.RLock

_operation_context(key)

Context manager entered around every operation on key.

The base implementation is a no-op. Override in a mixin to inject any resource scoped to the operation — a threading lock, a SQLAlchemy session, an open file handle, a decompression stream, etc.

Receiving key lets implementations choose between a single global resource (ignore the key) or per-key resources (e.g. a striped lock table or a key-specific file handle).

Composing multiple mixins: use super() to chain so that every mixin in the MRO gets to wrap the operation:

@contextlib.contextmanager
def _operation_context(self, key):
    with self._lock:                   # this mixin's resource
        with super()._operation_context(key):
            yield