fleche.executor
Executor wrapper that intercepts submit for fleche-decorated functions.
Motivation: users passing a fleche.fleche()-decorated function to
executor.submit(...) have to remember to call BoundWrapper.bind()
themselves to carry the active cache/metadata state into the worker. They also
pay the submit/serialisation cost even when the result is already cached. The
same problem affects plain (non-fleche) callables that merely call a
fleche-decorated function somewhere in their body: unless they are bound too,
the active cache/metadata state never reaches the worker process.
wrap_executor() patches the submit method of an executor instance
(we cannot subclass, since callers pass us instances of third-party executors)
so that:
non-fleche callables are bound via
BoundWrapper.bind()and submitted to the originalsubmitunchanged otherwise, so that any fleche calls nested inside them still see the active cache/metadata state (a callable that is already aBoundWrapperis submitted as-is, since it is already bound),fleche callables whose result is already cached are returned via an already-completed
Futurewithout touching the executor, andotherwise the call is bound via
BoundWrapper.bind()and submitted to the originalsubmit.
Executors that declare their own keyword-only parameters on submit
(e.g. resources= or resource_dict=) have those split off from the
caller’s **kwargs and forwarded to the underlying submit while the
remaining keyword arguments are bound as part of the function payload.
Functions
|
Monkey-patch |
Module Contents
- fleche.executor.wrap_executor(executor)[source]
Monkey-patch
executor.submitto intercept fleche-wrapped functions.Calling
wrap_executor()on an executor that is already wrapped is a no-op: the patch is not stacked, and the originalsubmitcontinues to refer to the pre-wrap method.- Parameters:
executor – any object with a
submit(func, *args, **kwargs)method (e.g.concurrent.futures.Executorsubclass instances).- Returns:
The same
executorinstance, with a replacedsubmitattribute.