fleche.remote

SSH-connected cache for sharing fleche results across machines.

SshCache is a BaseCache that forwards every operation to a remote python -m fleche remote --serve process over a single persistent SSH subprocess. The remote process loads its own fleche.toml and proxies operations into whichever cache it has configured, so the remote side keeps full freedom to use any backend (file / SQL / HDF5 / stack).

Typical configuration in fleche.toml — local cache first, remote cache second, composed automatically into a CacheStack:

[default]
cache = "shared"

[[shared]]                          # local layer (saves go here)
values.type = "cloudpickle"
values.root = "~/.fleche/values"
calls.type = "sql"
calls.url = "sqlite:///~/.fleche/calls.db"

[[shared]]                          # remote layer (read-through)
type = "ssh"
host = "marvin@bigpc.example.com"
cache_name = "shared"               # optional: named cache on remote
python = "python3"                  # optional
ssh_options = ["-o", "ControlMaster=auto",
               "-o", "ControlPath=~/.ssh/cm-%r@%h:%p",
               "-o", "ControlPersist=10m"]
setup_commands = ["module load python/3.11",
                  "source ~/.venv/bin/activate"]    # optional
workdir = "~/project"               # optional: cd before launching server

ControlMaster + ControlPersist in ~/.ssh/config (or via ssh_options) mean 2FA is prompted once and then re-used across multiple fleche sessions within the persist window. The SSH subprocess is spawned lazily on the first cache operation and reused for the lifetime of the Python process, so within a single run only one authentication round-trip is required.

Exceptions

RemoteConnectionError

Raised when the SSH subprocess cannot be reached or has died.

Classes

SshCache

A cache that forwards every operation to a remote fleche over SSH.

Functions

serve(→ None)

Run the remote cache request loop until input_stream reaches EOF.

Module Contents

fleche.remote.serve(input_stream, output_stream, cache: fleche.caches.BaseCache, *, cache_name: str | None = None) → None

Run the remote cache request loop until input_stream reaches EOF.

Reads RPC frames from input_stream, dispatches them against cache, and writes responses to output_stream. Exceptions from the cache are propagated to the client as ("err", exception) frames; if an exception cannot be cloudpickled it is replaced with a RuntimeError carrying its repr.

Parameters:

• input_stream – binary readable stream (e.g. sys.stdin.buffer).

• output_stream – binary writable stream (e.g. sys.stdout.buffer).

• cache – the cache to serve.

• cache_name – Optional name the server was launched with — echoed back in info responses for debugging.

exception fleche.remote.RemoteConnectionError

Bases: RuntimeError

Raised when the SSH subprocess cannot be reached or has died.

class fleche.remote.SshCache

Bases: fleche.caches.BaseCache

A cache that forwards every operation to a remote fleche over SSH.

The remote side runs python -m fleche remote --serve; its active cache is determined by its own fleche.toml (optionally overridden by cache_name — looked up via fleche.config.load_cache_config()).

The SSH subprocess is spawned lazily on the first cache operation and reused for the lifetime of the Python process. Set up ControlMaster / ControlPersist in ~/.ssh/config or via ssh_options to share the underlying connection across multiple fleche runs.

Parameters:

• host – SSH target, e.g. "user@host" or any alias from ~/.ssh/config.

• cache_name – Optional named cache on the remote. None (default) uses the remote’s default cache.

• python – Remote python executable. Defaults to "python3".

• ssh_options – Extra command-line arguments inserted between ssh and host, e.g. ("-o", "ControlMaster=auto").

• setup_commands – Shell snippets run on the remote before the server process starts, joined with && so any failure aborts the launch. Typical uses are HPC environment setup — ("module load python/3.11", "source ~/.venv/bin/activate"). Each snippet is passed to the remote shell verbatim; quote any user-provided values yourself.

• workdir – Optional remote directory to cd into before launching the server. Because the server starts the cache via python -m, the working directory lands on sys.path, so setting it lets the remote import the local modules referenced by unpickled calls (the “fudge imports” use case). The cd runs ahead of setup_commands.

host: str

cache_name: str | None = None

python: str = 'python3'

ssh_options: tuple[str, Ellipsis] = ()

setup_commands: tuple[str, Ellipsis] = ()

workdir: str | None = None

_conn: _Connection

_info_cache: dict[str, Any] | None = None

__post_init__() → None

_ensure_handshake() → None

Trigger the lazy version handshake (idempotent).

Called at the top of every BaseCache method so the first RPC of a session implicitly fetches the server’s info dict, which both populates the read-only short-circuit cache and runs the fleche/cloudpickle version-skew check via _warn_on_version_skew(). Subsequent ops are zero-cost.

save(call: fleche.call.Call) → str

load(key: str) → fleche.call.LazyCall

load_value(key: str) → Any

evict(key: str | fleche.digest.Digest) → None

contains(key: str) → bool

expand(key: fleche.digest.Digest | str) → fleche.digest.Digest

Expand a short digest prefix to its full-length digest.

Parameters:

key (str or Digest) – the short digest prefix to expand

Returns:

the full-length digest

Return type:

Digest

Raises:

• KeyError – if the key is not found

• AmbiguousDigestError – if the prefix matches more than one entry

_shrink(*keys: fleche.digest.Digest | str) → tuple[Digest, ...]

Partition and shrink all keys; always returns a same-length tuple of short digests.

_query(call: fleche.call.QueryCall) → Iterable[fleche.call.LazyCall]

reconnect() → None

Drop the current SSH subprocess; the next operation reconnects.

Useful if the underlying transport hangs or the remote needs to be re-authenticated. Not invoked automatically — auto-reconnect would silently re-trigger 2FA prompts.

close() → None

Close the SSH subprocess if it is currently open.

property read_only: bool

Whether the remote cache will reject every save / evict.

Read from the cached server info (one info RPC on first access, then reused for the lifetime of the connection). Used to short-circuit save and evict locally so the client doesn’t pay a round-trip just to receive Rejected.

info(*, refresh: bool = True) → dict[str, Any]

Return a snapshot of the remote server’s view of itself.

Keys: cache (the served cache serialised via fleche.config.cache_to_config() — a structured dict that round-trips through cache_from_config(), with credential fields like secret_key and URL passwords redacted server-side), cache_name (the --cache argument the server was launched with, if any), read_only (whether saves/evicts will be rejected), fleche_version, cloudpickle_version, cwd, hostname, python, pid.

Primary use: any “the remote isn’t doing what I expected” question (wrong cache, wrong cwd, surprising read_only, surprising Python). The first lazy fetch also drives the version handshake — see _cached_info().

Parameters:

refresh – When True (default), make a fresh info RPC and update the local cache. When False, return the cached copy if one exists (fetching on first call).

_cached_info() → dict[str, Any]

Internal accessor: fetch info once, then reuse it.

The first fetch doubles as a version handshake — see _warn_on_version_skew(). Any RPC method that short-circuits on the cached info (currently save / evict via the read_only flag) therefore implicitly triggers the handshake on its first call.