fleche.storage.thread_safe ========================== .. py:module:: fleche.storage.thread_safe .. autoapi-nested-parse:: Thread-safety mixins for storage classes. These mixins subclass :class:`~fleche.storage.base.KeyManagement` and override ``_operation_context`` to inject a threading lock around every storage operation. Because every concrete storage class (``ValueMemory``, ``Sql``, …) ultimately inherits from ``KeyManagement``, these mixins compose with any backend via Python's MRO:: @dataclass(frozen=True) class ThreadSafeValueMemory(SerializingMixin, ValueMemory): ... Choose the mixin based on the access pattern you need: * :class:`SerializingMixin` — single global ``RLock``. Every operation waits for the same lock, so the storage is touched by at most one thread at a time. Use when the backing store is not thread-safe and per-key parallelism is not needed. * :class:`PerKeyLockMixin` — a striped lock table, one ``RLock`` per key. Operations on *different* keys proceed in parallel; operations on the *same* key are serialized. Use when contention is dominated by hot keys and the backing store supports concurrent access on disjoint keys. Both mixins use reentrant locks so nested acquisitions (e.g. ``expand`` being called inside ``load``) do not deadlock. Classes ------- .. autoapisummary:: fleche.storage.thread_safe.SerializingMixin fleche.storage.thread_safe.PerKeyLockMixin Module Contents --------------- .. py:class:: SerializingMixin Bases: :py:obj:`fleche.storage.base.KeyManagement` Mixin that serializes all storage operations behind a single reentrant lock. Place before the concrete storage class in the MRO:: @dataclass(frozen=True) class SerializingValueMemory(SerializingMixin, ValueMemory): ... .. py:attribute:: _lock :type: threading.RLock .. py:method:: _operation_context(key) Context manager entered around every operation on ``key``. The base implementation is a no-op. Override in a mixin to inject any resource scoped to the operation — a threading lock, a SQLAlchemy session, an open file handle, a decompression stream, etc. Receiving ``key`` lets implementations choose between a single global resource (ignore the key) or per-key resources (e.g. a striped lock table or a key-specific file handle). **Composing multiple mixins**: use ``super()`` to chain so that every mixin in the MRO gets to wrap the operation:: @contextlib.contextmanager def _operation_context(self, key): with self._lock: # this mixin's resource with super()._operation_context(key): yield .. py:class:: PerKeyLockMixin Bases: :py:obj:`fleche.storage.base.KeyManagement` Mixin that locks per-key so concurrent ops on different keys proceed in parallel. A lightweight ``threading.Lock`` guards the lock-table itself; once the per-key ``RLock`` is obtained the table lock is released, so two threads operating on *different* keys never block each other. Operations on the *same* key are serialized by the per-key lock, which is reentrant to allow nested calls (e.g. ``expand`` inside ``load``). Place before the concrete storage class in the MRO:: @dataclass(frozen=True) class PerKeyValueMemory(PerKeyLockMixin, ValueMemory): ... .. py:attribute:: _key_locks :type: weakref.WeakValueDictionary[fleche.digest.Digest | str, threading.RLock] .. py:attribute:: _meta_lock :type: threading.Lock .. py:method:: _get_key_lock(key: fleche.digest.Digest | str) -> threading.RLock .. py:method:: _operation_context(key) Context manager entered around every operation on ``key``. The base implementation is a no-op. Override in a mixin to inject any resource scoped to the operation — a threading lock, a SQLAlchemy session, an open file handle, a decompression stream, etc. Receiving ``key`` lets implementations choose between a single global resource (ignore the key) or per-key resources (e.g. a striped lock table or a key-specific file handle). **Composing multiple mixins**: use ``super()`` to chain so that every mixin in the MRO gets to wrap the operation:: @contextlib.contextmanager def _operation_context(self, key): with self._lock: # this mixin's resource with super()._operation_context(key): yield