fleche.storage

Storage subpackage public API.

This module re-exports the primary storage interfaces and implementations for backward compatibility with from fleche.storage import … imports.

Submodules

• fleche.storage.bagofholding_file
• fleche.storage.base
• fleche.storage.file
• fleche.storage.memory
• fleche.storage.pickle_file
• fleche.storage.sql
• fleche.storage.void

Exceptions

SaveError	Common base class for all non-exit exceptions.
AmbiguousDigestError	Inappropriate argument value (of correct type).

Classes

Storage	Abstract base class for defining storage mechanisms.
CallStorage	Special storage for saving Call instances.
CallStorageAdapter	Implement a CallStorage from a generic Storage.
DestructuringMixin	Mixin that recursively destructures collections on save/load.
DestructuringStorage	Storage wrapper that recursively destructures collections.
Memory	A concrete implementation of Storage that stores values in an in-memory dictionary.
Void	A concrete implementation of Storage that does not store anything.
FileStorage	File-based storage backend using pickle.
PickleFile	Store values as files on the filesystem using a serialization module.
BagOfHoldingH5File	File-based storage backend using pickle.
Sql	SQLAlchemy-backed CallStorage with JSON metadata and DB-backed expand().

Package Contents

exception fleche.storage.SaveError

Bases: Exception

Common base class for all non-exit exceptions.

exception fleche.storage.AmbiguousDigestError

Bases: ValueError

Inappropriate argument value (of correct type).

class fleche.storage.Storage

Bases: StorageBase

Abstract base class for defining storage mechanisms.

save(value: Any, key: fleche.digest.Digest | None = None) → fleche.digest.Digest

abstractmethod _save(value: Any, key: fleche.digest.Digest) → fleche.digest.Digest

load(key: fleche.digest.Digest | str) → Any

abstractmethod _load(key: fleche.digest.Digest) → Any

_contains(key: fleche.digest.Digest) → bool

class fleche.storage.CallStorage

Bases: StorageBase

Special storage for saving Call instances.

save(call: fleche.call.Call) → fleche.digest.Digest

abstractmethod _save(call: fleche.call.Call) → fleche.digest.Digest

load(key: str | fleche.digest.Digest) → fleche.call.Call

abstractmethod _load(key: fleche.digest.Digest) → fleche.call.Call

transform(func: Callable[[fleche.call.Call], fleche.call.Call] | None = None) → None

Applies a transformation function to all Call objects in the storage.

Parameters:

func (Callable[[Call], Call] | None) – A function that takes a Call and returns a transformed Call. If None, the identity function is used (useful for re-calculating keys).

query(template: fleche.call.QueryCall) → Iterable[fleche.call.Call]

Find cached calls that ‘match’ the template.

Returns all calls where the given arguments, results or metadata match exactly the stored ones. Values may be given either as they are or as Digest.

Parameters:

template (Call) – specification for calls to return; use None as wildcard.

Returns:

an iterable over all matching call objects

Return type:

Iterable[Call]

_contains(key: fleche.digest.Digest) → bool

class fleche.storage.CallStorageAdapter

Bases: CallStorage

Implement a CallStorage from a generic Storage.

storage: Storage

_save(call: fleche.call.Call) → fleche.digest.Digest

_load(key: fleche.digest.Digest) → fleche.call.Call

_contains(key: fleche.digest.Digest) → bool

_evict(key: fleche.digest.Digest) → None

list() → Iterable[fleche.digest.Digest]

class fleche.storage.DestructuringMixin

Bases: Storage

Mixin that recursively destructures collections on save/load.

Place before a concrete Storage in the MRO to add destructuring behavior. Lists, tuples, and dicts are broken apart so each element is stored independently; on load the original structure is reassembled.

Example:

class DestructuringMemory(DestructuringMixin, Memory):
    pass

# ``storage`` here is the backing dict required by ``Memory``, not a
# Storage instance.
dm = DestructuringMemory(storage={})
key = dm.save([1, [2, 3]])
assert dm.load(key) == [1, [2, 3]]

remaining_depth: int = 0

_intern_rec(value: Any) → tuple[Any, int | float]

Post-order traversal: recurse to leaves, decide inline-vs-store on the way back up.

Returns (result, depth) where result is the plain value when depth < remaining_depth (the element is inlined in its parent’s Digested wrapper) or a Digest when the element was written to storage separately. Every node in the structure is visited exactly once (O(n)), unlike a separate depth-counting pass.

_save(value: Any, key: fleche.digest.Digest) → fleche.digest.Digest

_load(key: fleche.digest.Digest | Any) → Any

class fleche.storage.DestructuringStorage

Bases: DestructuringMixin, _DelegatingStorage

Storage wrapper that recursively destructures collections.

This is a convenience class combining DestructuringMixin with delegation to a wrapped storage. Prefer using DestructuringMixin directly as a mixin with a concrete storage class when possible.

remaining_depth: int = 0

__post_init__()

class fleche.storage.Memory

Bases: fleche.storage.base.Storage

A concrete implementation of Storage that stores values in an in-memory dictionary.

storage: dict[fleche.digest.Digest, Any]

_save(value: Any, key: fleche.digest.Digest) → fleche.digest.Digest

_load(key: fleche.digest.Digest) → Any

_contains(key: fleche.digest.Digest) → bool

list() → Iterable[fleche.digest.Digest]

_evict(key: fleche.digest.Digest) → None

class fleche.storage.Void

Bases: fleche.storage.base.Storage

A concrete implementation of Storage that does not store anything.

_save(value: Any, key: fleche.digest.Digest) → fleche.digest.Digest

_load(key: fleche.digest.Digest) → Any

list() → Iterable[fleche.digest.Digest]

_evict(key: fleche.digest.Digest) → None

_contains(key: fleche.digest.Digest) → bool

class fleche.storage.FileStorage

Bases: fleche.storage.base.Storage

File-based storage backend using pickle.

Stores objects on the filesystem.

root: pathlib.Path

lock_timeout: float = 1.0

lock_wait_start: float = 0.001

__post_init__() → None

_path(key: str) → pathlib.Path

list() → Iterable[fleche.digest.Digest]

_evict(key: fleche.digest.Digest) → None

_save(value: Any, key: fleche.digest.Digest) → fleche.digest.Digest

_load(key: fleche.digest.Digest) → Any

abstractmethod _to_file(value: Any, path: pathlib.Path) → None

abstractmethod _from_file(path: pathlib.Path) → Any

_contains(key: fleche.digest.Digest) → bool

class fleche.storage.PickleFile

Bases: fleche.storage.file.FileStorage

Store values as files on the filesystem using a serialization module.

secret_key: list[bytes] = []

serializer: Any

compress: bool = False

__post_init__()

classmethod with_pickle(*args, **kwargs)

Construct a PickleFile using the standard pickle module.

classmethod with_cloudpickle(*args, **kwargs)

Construct a PickleFile using the cloudpickle module.

classmethod with_dill(*args, **kwargs)

Construct a PickleFile using the dill module.

__getstate__()

__setstate__(state)

_to_file(value: Any, path: pathlib.Path) → None

_from_file(path: pathlib.Path) → Any

class fleche.storage.BagOfHoldingH5File

Bases: fleche.storage.file.FileStorage

File-based storage backend using pickle.

Stores objects on the filesystem.

__post_init__()

_to_file(value: Any, path: pathlib.Path) → None

_from_file(path: pathlib.Path) → Any

class fleche.storage.Sql

Bases: fleche.storage.base.CallStorage

SQLAlchemy-backed CallStorage with JSON metadata and DB-backed expand().

url: str | None = None

echo: bool = False

engine: Any

session: Any

__post_init__() → None

__getstate__()

__setstate__(state)

_save(call: fleche.call.Call) → fleche.digest.Digest

_load(key: fleche.digest.Digest) → fleche.call.Call

_contains(key: fleche.digest.Digest) → bool

list() → Iterable[fleche.digest.Digest]

expand(key: fleche.digest.Digest | str) → fleche.digest.Digest

Expands a short-hand digest to the full length one.

_evict(key: fleche.digest.Digest) → None

_normalize_value(v: Any) → str

Return the stored form used in SQL for argument/result matching.

We must match the generic CallStorage.query semantics which compare digest(template_value) == digest(stored_call_value). In this backend, stored argument/result values are hex-digest strings, and digest(Digest(x)) == x. Therefore we should always compare Arg.value/CallModel.result to str(digest(template_value)).

_build_call_conditions(template: fleche.call.QueryCall) → List[Any]

_apply_argument_filters(stmt: Any, arguments: dict[str, Any] | None) → Any

_apply_metadata_filters(stmt: Any, meta_specs: dict[str, dict[str, Any]] | None) → Any

query(template: fleche.call.QueryCall) → Iterable[fleche.call.Call]

Find cached calls matching a template using SQL-side filtering.

Semantics match CallStorage.query: - Fields set to None are wildcards. - Arguments and result are compared by digest(template_value) == digest(stored_value). - Metadata can be filtered by providing template.metadata as a mapping of

metadata name -> dict of key/value filters. An empty dict for a given name means “presence of that metadata name”. Filters with simple types (str, bool, int, float) are pushed down to SQL via JSON-extract expressions; other types (e.g., lists) or None values fall back to client-side checks after loading.

This method builds a SELECT over calls, joining the arguments table and metadata table as needed to reduce candidate rows, then loads the resulting calls and performs any remaining client-side validation.

Parameters:

template – A Call used as a template. None-valued fields are wildcards.

Yields:

Call – Matching calls including their decoded metadata.