fleche.storage.pickle_file

Attributes

logger

Classes

PickleFileBackend	Store values as files on the filesystem using a serialization module.
ValuePickleFile	Mixin that locks per-key so concurrent ops on different keys proceed in parallel.
CallPickleFile	Mixin that locks per-key so concurrent ops on different keys proceed in parallel.

Module Contents

fleche.storage.pickle_file.logger

class fleche.storage.pickle_file.PickleFileBackend

Bases: fleche.storage.file.FileStorage

Store values as files on the filesystem using a serialization module.

secret_key: tuple[bytes, Ellipsis] = ()

dumps: Callable

loads: Callable

compress: bool = False

__post_init__()

classmethod with_pickle(*args, **kwargs)

Construct a PickleFileBackend using the standard pickle module.

classmethod with_cloudpickle(*args, **kwargs)

Construct a PickleFileBackend using the cloudpickle module.

classmethod with_dill(*args, **kwargs)

Construct a PickleFileBackend using the dill module.

_to_file(value: Any, path: pathlib.Path) → None

_from_file(path: pathlib.Path) → Any

compress_all() → None

Rewrite all stored files in gzip-compressed form.

decompress_all() → None

Rewrite all stored files in uncompressed form.

class fleche.storage.pickle_file.ValuePickleFile

Bases: fleche.storage.thread_safe.PerKeyLockMixin, fleche.storage.base.ValueMixin, fleche.storage.destructuring.DestructuringMixin, PickleFileBackend

Mixin that locks per-key so concurrent ops on different keys proceed in parallel.

A lightweight threading.Lock guards the lock-table itself; once the per-key RLock is obtained the table lock is released, so two threads operating on different keys never block each other. Operations on the same key are serialized by the per-key lock, which is reentrant to allow nested calls (e.g. expand inside load).

Instances must be hashable. Place before the concrete storage class in the MRO:

@dataclass(frozen=True)
class PerKeyValuePickle(PerKeyLockMixin, ValuePickleFile): ...

class fleche.storage.pickle_file.CallPickleFile

Bases: fleche.storage.thread_safe.PerKeyLockMixin, fleche.storage.base.CallMixin, PickleFileBackend

Mixin that locks per-key so concurrent ops on different keys proceed in parallel.

A lightweight threading.Lock guards the lock-table itself; once the per-key RLock is obtained the table lock is released, so two threads operating on different keys never block each other. Operations on the same key are serialized by the per-key lock, which is reentrant to allow nested calls (e.g. expand inside load).

Instances must be hashable. Place before the concrete storage class in the MRO:

@dataclass(frozen=True)
class PerKeyValuePickle(PerKeyLockMixin, ValuePickleFile): ...