fleche.remote ============= .. py:module:: fleche.remote .. autoapi-nested-parse:: SSH-connected cache for sharing fleche results across machines. :class:`SshCache` is a :class:`~fleche.caches.BaseCache` that forwards every operation to a remote ``python -m fleche remote --serve`` process over a single persistent SSH subprocess. The remote process loads its own ``fleche.toml`` and proxies operations into whichever cache it has configured, so the remote side keeps full freedom to use any backend (file / SQL / HDF5 / stack). Typical configuration in ``fleche.toml`` — local cache first, remote cache second, composed automatically into a :class:`~fleche.caches.CacheStack`:: [default] cache = "shared" [[shared]] # local layer (saves go here) values.type = "cloudpickle" values.root = "~/.fleche/values" calls.type = "sql" calls.url = "sqlite:///~/.fleche/calls.db" [[shared]] # remote layer (read-through) type = "ssh" host = "marvin@bigpc.example.com" cache_name = "shared" # optional: named cache on remote python = "python3" # optional ssh_options = ["-o", "ControlMaster=auto", "-o", "ControlPath=~/.ssh/cm-%r@%h:%p", "-o", "ControlPersist=10m"] setup_commands = ["module load python/3.11", "source ~/.venv/bin/activate"] # optional workdir = "~/project" # optional: cd before launching server ControlMaster + ControlPersist in ``~/.ssh/config`` (or via ``ssh_options``) mean 2FA is prompted once and then re-used across multiple fleche sessions within the persist window. The SSH subprocess is spawned lazily on the first cache operation and reused for the lifetime of the Python process, so within a single run only one authentication round-trip is required. Exceptions ---------- .. autoapisummary:: fleche.remote.RemoteConnectionError Classes ------- .. autoapisummary:: fleche.remote.SshCache Functions --------- .. autoapisummary:: fleche.remote.serve Module Contents --------------- .. py:function:: serve(input_stream, output_stream, cache: fleche.caches.BaseCache, *, cache_name: str | None = None) -> None Run the remote cache request loop until *input_stream* reaches EOF. Reads RPC frames from *input_stream*, dispatches them against *cache*, and writes responses to *output_stream*. Exceptions from the cache are propagated to the client as ``("err", exception)`` frames; if an exception cannot be cloudpickled it is replaced with a :class:`RuntimeError` carrying its repr. :param input_stream: binary readable stream (e.g. ``sys.stdin.buffer``). :param output_stream: binary writable stream (e.g. ``sys.stdout.buffer``). :param cache: the cache to serve. :param cache_name: Optional name the server was launched with — echoed back in ``info`` responses for debugging. .. py:exception:: RemoteConnectionError Bases: :py:obj:`RuntimeError` Raised when the SSH subprocess cannot be reached or has died. .. py:class:: SshCache Bases: :py:obj:`fleche.caches.BaseCache` A cache that forwards every operation to a remote fleche over SSH. The remote side runs ``python -m fleche remote --serve``; its active cache is determined by its own ``fleche.toml`` (optionally overridden by *cache_name* — looked up via :func:`fleche.config.load_cache_config`). The SSH subprocess is spawned lazily on the first cache operation and reused for the lifetime of the Python process. Set up ControlMaster / ControlPersist in ``~/.ssh/config`` or via *ssh_options* to share the underlying connection across multiple fleche runs. :param host: SSH target, e.g. ``"user@host"`` or any alias from ``~/.ssh/config``. :param cache_name: Optional named cache on the remote. ``None`` (default) uses the remote's default cache. :param python: Remote python executable. Defaults to ``"python3"``. :param ssh_options: Extra command-line arguments inserted between ``ssh`` and *host*, e.g. ``("-o", "ControlMaster=auto")``. :param setup_commands: Shell snippets run on the remote *before* the server process starts, joined with ``&&`` so any failure aborts the launch. Typical uses are HPC environment setup — ``("module load python/3.11", "source ~/.venv/bin/activate")``. Each snippet is passed to the remote shell verbatim; quote any user-provided values yourself. :param workdir: Optional remote directory to ``cd`` into before launching the server. Because the server starts the cache via ``python -m``, the working directory lands on ``sys.path``, so setting it lets the remote import the local modules referenced by unpickled calls (the "fudge imports" use case). The ``cd`` runs ahead of *setup_commands*. .. py:attribute:: host :type: str .. py:attribute:: cache_name :type: str | None :value: None .. py:attribute:: python :type: str :value: 'python3' .. py:attribute:: ssh_options :type: tuple[str, Ellipsis] :value: () .. py:attribute:: setup_commands :type: tuple[str, Ellipsis] :value: () .. py:attribute:: workdir :type: str | None :value: None .. py:attribute:: _conn :type: _Connection .. py:attribute:: _info_cache :type: dict[str, Any] | None :value: None .. py:method:: __post_init__() -> None .. py:method:: _ensure_handshake() -> None Trigger the lazy version handshake (idempotent). Called at the top of every BaseCache method so the first RPC of a session implicitly fetches the server's info dict, which both populates the read-only short-circuit cache and runs the fleche/cloudpickle version-skew check via :func:`_warn_on_version_skew`. Subsequent ops are zero-cost. .. py:method:: save(call: fleche.call.Call) -> str .. py:method:: load(key: str) -> fleche.call.LazyCall .. py:method:: load_value(key: str) -> Any .. py:method:: evict(key: str | fleche.digest.Digest) -> None .. py:method:: contains(key: str) -> bool .. py:method:: expand(key: fleche.digest.Digest | str) -> fleche.digest.Digest Expand a short digest prefix to its full-length digest. :param key: the short digest prefix to expand :type key: str or :class:`Digest` :returns: the full-length digest :rtype: :class:`Digest` :raises KeyError: if the key is not found :raises AmbiguousDigestError: if the prefix matches more than one entry .. py:method:: _shrink(*keys: fleche.digest.Digest | str) -> tuple[Digest, ...] Partition and shrink all keys; always returns a same-length tuple of short digests. .. py:method:: _query(call: fleche.call.QueryCall) -> Iterable[fleche.call.LazyCall] .. py:method:: reconnect() -> None Drop the current SSH subprocess; the next operation reconnects. Useful if the underlying transport hangs or the remote needs to be re-authenticated. Not invoked automatically — auto-reconnect would silently re-trigger 2FA prompts. .. py:method:: close() -> None Close the SSH subprocess if it is currently open. .. py:property:: read_only :type: bool Whether the remote cache will reject every ``save`` / ``evict``. Read from the cached server info (one ``info`` RPC on first access, then reused for the lifetime of the connection). Used to short-circuit ``save`` and ``evict`` locally so the client doesn't pay a round-trip just to receive :class:`~fleche.caches.Rejected`. .. py:method:: info(*, refresh: bool = True) -> dict[str, Any] Return a snapshot of the remote server's view of itself. Keys: ``cache`` (the served cache serialised via :func:`fleche.config.cache_to_config` — a structured dict that round-trips through :func:`~fleche.config.cache_from_config`, with credential fields like ``secret_key`` and URL passwords redacted server-side), ``cache_name`` (the ``--cache`` argument the server was launched with, if any), ``read_only`` (whether saves/evicts will be rejected), ``fleche_version``, ``cloudpickle_version``, ``cwd``, ``hostname``, ``python``, ``pid``. Primary use: any "the remote isn't doing what I expected" question (wrong cache, wrong cwd, surprising read_only, surprising Python). The first lazy fetch also drives the version handshake — see :meth:`_cached_info`. :param refresh: When True (default), make a fresh ``info`` RPC and update the local cache. When False, return the cached copy if one exists (fetching on first call). .. py:method:: _cached_info() -> dict[str, Any] Internal accessor: fetch info once, then reuse it. The first fetch doubles as a version handshake — see :func:`_warn_on_version_skew`. Any RPC method that short-circuits on the cached info (currently ``save`` / ``evict`` via the ``read_only`` flag) therefore implicitly triggers the handshake on its first call.