fleche.config

Configuration system for fleche.

Storage type names

The type key in a storage config dict is case-sensitive and uses the following lowercase identifiers:

"memory": In-memory dict (ValueMemory / CallMemory). No required keys. Optional (value backend): remaining_depth (int, default 0).
"void": No-op — discards all data (ValueVoid / CallVoid). No required keys.
"pickle": Filesystem backend serialised with the standard pickle module (ValuePickleFile / CallPickleFile). Required: root (path to storage directory). Optional: compress (bool, default False) — gzip-compress files. Optional: lock_timeout (float, default 1.0) — file-lock acquisition timeout (s). Optional: secret_key (list of hex strings) — HMAC-SHA256 signing keys; each element is a hex-encoded byte string (same format as FLECHE_SECRET_KEY). If omitted, falls back to the FLECHE_SECRET_KEY environment variable. Optional (value backend): remaining_depth (int, default 0).
"cloudpickle": Filesystem backend serialised with cloudpickle — handles more complex Python objects than pickle. Required: root. Optional: compress (bool, default False) — gzip-compress files. Optional: lock_timeout (float, default 1.0) — file-lock acquisition timeout (s). Optional: secret_key (list of hex strings) — same as "pickle". Optional (value backend): remaining_depth (int, default 0).
"dill": Filesystem backend serialised with dill. Required: root. Optional: compress (bool, default False) — gzip-compress files. Optional: lock_timeout (float, default 1.0) — file-lock acquisition timeout (s). Optional: secret_key (list of hex strings) — same as "pickle". Optional (value backend): remaining_depth (int, default 0).
"bagofholding_hdf": HDF5-backed storage via the bagofholding library (ValueBagOfHoldingH5File / CallBagOfHoldingH5File). Required: root. Optional: lock_timeout (float, default 1.0) — file-lock acquisition timeout (s). Optional: version_validator (str, default omitted) — version validation strategy passed to bagofholding.h5.bag.H5Bag.load(). One of "exact", "semantic-minor", "semantic-major", "none". When omitted, bagofholding’s default applies. Optional (value backend): remaining_depth (int, default 0).
"sql": SQL database via SQLAlchemy (Sql). Call storage only. Required: url (SQLAlchemy connection URL, e.g. "sqlite:///~/.fleche/calls.db"). Optional: echo (bool, default False) — log SQL statements.

Example fleche.toml

[default]
cache = "persistent"
metadata = ["Runtime"]

[persistent]
values.type = "cloudpickle"
values.root = "~/.fleche/values"
calls.type = "cloudpickle"
calls.root = "~/.fleche/calls"

[fast]
values.type = "memory"
calls.type = "memory"

[with_sql_calls]
values.type = "cloudpickle"
values.root = "~/.fleche/values"
calls.type = "sql"
calls.url = "sqlite:///~/.fleche/calls.db"

# SizeLimitedCache — evicts oldest entries once 100 entries are stored
[limited]
values.type = "memory"
calls.type = "memory"
max_size = 100

# ReadOnlyCache — loads from storage but never writes new results
[readonly]
values.type = "cloudpickle"
values.root = "~/.fleche/values"
calls.type = "cloudpickle"
calls.root = "~/.fleche/calls"
read_only = true

# CacheStack — TOML array-of-tables; saves to the bottom layer,
# loads top-down and back-fills hits to the bottom
[[mystack]]
values.type = "memory"
calls.type = "memory"

[[mystack]]
values.type = "cloudpickle"
values.root = "~/.fleche/values"
calls.type = "cloudpickle"
calls.root = "~/.fleche/calls"

# SshCache — share results with another machine over SSH.  The remote
# runs `python -m fleche remote --serve` and proxies into its own
# configured cache.  Compose with a local cache by stacking two
# entries (saves go to the first entry; reads fall back to the SSH
# remote and back-fill hits into the local layer).
[[shared]]
values.type = "cloudpickle"
values.root = "~/.fleche/values"
calls.type = "cloudpickle"
calls.root = "~/.fleche/calls"

[[shared]]
type = "ssh"
host = "user@bigpc.example.com"
cache_name = "shared"               # optional: named cache on remote
python = "python3"                  # optional: remote python interpreter
ssh_options = ["-o", "ControlMaster=auto",
               "-o", "ControlPath=~/.ssh/cm-%r@%h:%p",
               "-o", "ControlPersist=10m"]
setup_commands = ["module load python/3.11",  # optional: shell snippets
                  "source ~/.venv/bin/activate"]  # run before the server
workdir = "~/project"               # optional: cd here before launching
                                    # the server, so the remote can import
                                    # the project's local modules

Config file discovery

When the active cache or default metadata is loaded, fleche walks from the current working directory upward, picking up every fleche.toml it encounters. The walk stops at $HOME (inclusive) or at the filesystem root, whichever comes first. $XDG_CONFIG_HOME/fleche/cache.toml (defaulting to ~/.config/fleche/cache.toml per the XDG base directory spec when XDG_CONFIG_HOME is unset or empty) is appended as a final lowest-priority layer.

All discovered files are shallow-merged at the top level: files closer to the CWD win, and a closer file’s top-level table fully replaces the same key in a farther file (tables are not recursively merged).

Attributes

`logger`
`_live_caches`
`_STORAGE_NAME_MAPPING`
`_STORAGE_CLASS_TO_NAME`

Functions

`_load_config`(→ dict[str, Any])
`_collect_config_paths`(→ list[pathlib.Path])	Return config paths in priority order (closest first, lowest last).
`_load_merged_config`(→ dict[str, Any])	Load and shallow-merge all config files on the walk path.
`load_default_metadata`()	Load the default metadata from the merged configuration files.
`storage_from_config`(…)	Construct a `StorageBackend` from a config dict.
`_asdict_init_only`(→ dict[str, Any])	Like `dataclasses.asdict()` but restricted to `init=True` fields.
`storage_to_config`(→ dict[str, Any])	Convert a Storage instance to a config dict (inverse of `storage_from_config`).
`cache_from_config`(→ fleche.caches.BaseCache)	Construct a `BaseCache` from a config dict or list.
`cache_to_config`(→ dict[str, Any] \| list[dict[str, Any]])	Convert a `BaseCache` to a config dict or list.
`_default_memory_cache`(→ fleche.caches.Cache)	Return (and intern) a fresh in-memory cache, optionally logging the fallback reason.
`load_cache_config`(→ fleche.caches.BaseCache)	Load a cache from the configuration file.

Module Contents

fleche.config.logger[source]

fleche.config._live_caches: dict[str | None, fleche.caches.BaseCache][source]

fleche.config._load_config(path: pathlib.Path) → dict[str, Any][source]

fleche.config._collect_config_paths() → list[pathlib.Path][source]

Return config paths in priority order (closest first, lowest last).

Walks from the current working directory up to $HOME (inclusive), collecting any fleche.toml files encountered. If the walk reaches the filesystem root without crossing $HOME (or $HOME is unset), it stops at the root. Finally, $XDG_CONFIG_HOME/fleche/cache.toml is appended as the lowest-priority fallback. Per the XDG base directory spec, an unset or empty XDG_CONFIG_HOME defaults to $HOME/.config.

fleche.config._load_merged_config() → dict[str, Any][source]

Load and shallow-merge all config files on the walk path.

Files closer to the CWD override files farther away. Top-level keys from the closest file fully replace the same key from any farther file (no recursive table merging).

fleche.config.load_default_metadata()[source]: Load the default metadata from the merged configuration files.

fleche.config._STORAGE_NAME_MAPPING[source]

fleche.config._STORAGE_CLASS_TO_NAME: dict[type, str][source]

fleche.config.storage_from_config(d: dict[str, Any], type: Literal['call']) → fleche.storage.CallStorage[source]

fleche.config.storage_from_config(d: dict[str, Any], type: Literal['value']) → fleche.storage.ValueStorage

Construct a StorageBackend from a config dict.

The dict must contain a "type" key (case-sensitive, lowercase) and any additional parameters required by that storage backend. The input dict is not mutated.

Supported type values and their parameters:

{"type": "memory"}
{"type": "void"}
{"type": "pickle", "root": "<path>"} — optional: compress, lock_timeout, secret_key (list of hex strings), remaining_depth (value only)
{"type": "cloudpickle", "root": "<path>"} — same optional keys as "pickle"
{"type": "dill", "root": "<path>"} — same optional keys as "pickle"
{"type": "bagofholding_hdf", "root": "<path>"} — optional: lock_timeout, version_validator, remaining_depth (value only)
{"type": "sql", "url": "<sqlalchemy-url>"} (call storage only) — optional: echo

See the module docstring for full descriptions of each key.

fleche.config._asdict_init_only(obj) → dict[str, Any][source]

Like dataclasses.asdict() but restricted to init=True fields.

init=False fields are internal state (locks, caches) that must not appear in serialised config.

fleche.config.storage_to_config(s: fleche.storage.ValueStorage | fleche.storage.CallStorage) → dict[str, Any][source]

Convert a Storage instance to a config dict (inverse of storage_from_config).

The returned dict contains a "type" key and any additional parameters needed to reconstruct the storage via storage_from_config().

fleche.config.cache_from_config(d: dict[str, Any] | list[dict[str, Any]]) → fleche.caches.BaseCache[source]

Construct a BaseCache from a config dict or list.

The cache type is determined implicitly from the shape of the input:

A list of dicts is treated as a CacheStack, with each element processed recursively.
A dict containing a max_size key creates a SizeLimitedCache.
A dict containing read_only: true wraps the resulting cache in a ReadOnlyCache.
Otherwise a plain Cache is created.

The input dict is not mutated.

Examples

>>> c = cache_from_config({"values": {"type": "memory"}, "calls": {"type": "memory"}})
>>> type(c).__name__
'Cache'

>>> c = cache_from_config({"values": {"type": "memory"}, "calls": {"type": "memory"}, "max_size": 100})
>>> isinstance(c, caches.SizeLimitedCache)
True

>>> c = cache_from_config({"values": {"type": "memory"}, "calls": {"type": "memory"}, "read_only": True})
>>> isinstance(c, caches.ReadOnlyCache)
True

>>> c = cache_from_config([{"values": {"type": "memory"}, "calls": {"type": "memory"}}, {"values": {"type": "void"}, "calls": {"type": "void"}}])
>>> isinstance(c, caches.CacheStack)
True

fleche.config.cache_to_config(c: fleche.caches.BaseCache) → dict[str, Any] | list[dict[str, Any]][source]

Convert a BaseCache to a config dict or list.

This is the inverse of cache_from_config(). The output can be round-tripped back via cache_from_config(cache_to_config(cache)).

Cache → dict with "values" and "calls"
SizeLimitedCache → same dict plus "max_size"
ReadOnlyCache wrapping a Cache or SizeLimitedCache → inner cache dict with "read_only": True
CacheStack → list of dicts

Raises:: ValueError – for unsupported cache types or unsupported ReadOnlyCache inner types.

fleche.config._default_memory_cache(name: str | None, reason: str | None = None) → fleche.caches.Cache[source]: Return (and intern) a fresh in-memory cache, optionally logging the fallback reason.

fleche.config.load_cache_config(name: str | None = None) → fleche.caches.BaseCache[source]

Load a cache from the configuration file.

If name is None, the default cache is loaded. The names ‘memory’, ‘void’, and ‘default’ are special-cased: ‘memory’ and ‘void’ return transient backends; ‘default’ resolves to whichever cache the config file designates as the default (equivalent to calling this function with name=None).

Note: The Tags metadata cannot be configured from the config file.