{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "!rm .fleche -rf" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Getting Started with Fleche" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook demonstrates the main features of the `fleche` library, a caching library for Python." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Long-running calculation" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "No config file found. Using default memory cache.\n" ] } ], "source": [ "import time\n", "from fleche import fleche, cache, tags, project\n", "from fleche.digest import Digest" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "@fleche\n", "def long_running_calculation(x):\n", " print(f'Running calculation for {x}...')\n", " time.sleep(2)\n", " return x * x" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Running calculation for 2...\n", "First call took 2.00 seconds.\n" ] } ], "source": [ "start = time.time()\n", "long_running_calculation(2)\n", "end = time.time()\n", "print(f'First call took {end - start:.2f} seconds.')" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Second call took 0.00 seconds.\n" ] } ], "source": [ "start = time.time()\n", "long_running_calculation(2)\n", "end = time.time()\n", "print(f'Second call took {end - start:.2f} seconds.')" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Running calculation for 100...\n", "Second call took 2.00 seconds.\n" ] } ], "source": [ "start = time.time()\n", "long_running_calculation(100)\n", "end = time.time()\n", "print(f'Second call took {end - start:.2f} seconds.')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, the second call returns almost instantly, because the result was cached.\n", "As soon as the argument changes, fleche runs the original fucntion again." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Recursive function" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "@fleche\n", "def fib(n):\n", " if n < 2:\n", " return n\n", " return fib(n-1) + fib(n-2)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fib(20) took 0.0067 seconds with caching.\n" ] } ], "source": [ "start = time.time()\n", "fib(20)\n", "end = time.time()\n", "print(f'fib(20) took {end - start:.4f} seconds with caching.')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Without caching, this would be much slower as each call to `fib` would be recomputed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Passing Digests as Arguments\n", "\n", "`fleche` supports passing `Digest` objects directly to cached functions. When a function receives a `Digest`, `fleche` automatically expands it to its actual value from the cache before executing the function. You can use the convenience wrapper `D` to mark a string as a digest." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Running calculation for 10...\n", "Value Digest: 60079f7901a9295349d1796c037afc132e81286f785ddeeb763104ef02363102\n", "Short digest: 60079f79\n", "Doubling 100...\n", "Result: 200\n" ] } ], "source": [ "from fleche import D\n", "from fleche.digest import digest as value_digest\n", "\n", "@fleche\n", "def double(x):\n", " print(f\"Doubling {x}...\")\n", " return x * 2\n", "\n", "# 1. Run the calculation to ensure it is cached\n", "long_running_calculation(10)\n", "\n", "# 2. Compute the value digest for 100 (the cached result)\n", "v = long_running_calculation(10)\n", "val_dig = value_digest(v)\n", "print(f\"Value Digest: {val_dig}\")\n", "\n", "# 3. Pass a short digest prefix of the value to double(); it will expand to 100.\n", "short = str(val_dig)[:8]\n", "print(f\"Short digest: {short}\")\n", "print(f\"Result: {double(D(short))}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Querying Cached Calls " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### via Function Wrapper\n", "\n", "You can retrieve previously cached calls that match some of your function's arguments and metadata using the function wrapper's `query` method. Any field left as `None` is treated as a wildcard. Arguments and result are compared by digest internally, but the wrapper decodes them back to Python objects when returning matches.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Query returns an iterator object, that also defines additional utilities, e.g. to create a table of queried calls, do" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduletimestarttimestopwalltime
4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953long_running_calculation__main__1.774227e+091.774227e+092.000136
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__1.774227e+091.774227e+092.000227
a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8long_running_calculation__main__1.774227e+091.774227e+092.000230
\n", "
" ], "text/plain": [ " name \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n", "\n", " module timestart \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 1.774227e+09 \n", "\n", " timestop walltime \n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "long_running_calculation.query().table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The table includes the call digest as an index and the metadata associated with the call.\n", "\n", "Arguments can be selectively included in the table via the `arguments`. You only pay the loading cost for the specified arguments, not for arguments that are not included in the table." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemodulextimestarttimestopwalltime
4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953long_running_calculation__main__21.774227e+091.774227e+092.000136
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__1001.774227e+091.774227e+092.000227
a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8long_running_calculation__main__101.774227e+091.774227e+092.000230
\n", "
" ], "text/plain": [ " name \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n", "\n", " module x \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 2 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 100 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 10 \n", "\n", " timestart \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 \n", "\n", " timestop walltime \n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 " ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "long_running_calculation.query().table(arguments=['x'])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduleresultxtimestarttimestopwalltime
4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953long_running_calculation__main__421.774227e+091.774227e+092.000136
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__100001001.774227e+091.774227e+092.000227
a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8long_running_calculation__main__100101.774227e+091.774227e+092.000230
\n", "
" ], "text/plain": [ " name \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n", "\n", " module result x \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 4 2 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 10000 100 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 100 10 \n", "\n", " timestart \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 \n", "\n", " timestop walltime \n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 " ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "long_running_calculation.query().table(arguments=['x'], results=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### via Cache" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "from fleche.call import QueryCall" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduletimestarttimestopwalltime
4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953long_running_calculation__main__1.774227e+091.774227e+092.000136
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__1.774227e+091.774227e+092.000227
698e29f05ba00ee23503848cd166215f62cd976d54431594036979d8d56f254ffib__main__1.774227e+091.774227e+090.000004
405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4edeafb1c6f236fc80fib__main__1.774227e+091.774227e+090.000002
24231d9bc47f7abc0ea485d178fc8457dce8082790f8c948b936ebe906352225fib__main__1.774227e+091.774227e+090.000838
\n", "
" ], "text/plain": [ " name \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... fib \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... fib \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... fib \n", "\n", " module timestart \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... __main__ 1.774227e+09 \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... __main__ 1.774227e+09 \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... __main__ 1.774227e+09 \n", "\n", " timestop walltime \n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 0.000004 \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 0.000002 \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 0.000838 " ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache().query(QueryCall(module=\"__main__\")).table().head()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduletimestarttimestopwalltime
698e29f05ba00ee23503848cd166215f62cd976d54431594036979d8d56f254ffib__main__1.774227e+091.774227e+090.000004
405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4edeafb1c6f236fc80fib__main__1.774227e+091.774227e+090.000002
24231d9bc47f7abc0ea485d178fc8457dce8082790f8c948b936ebe906352225fib__main__1.774227e+091.774227e+090.000838
dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2b9a284dc1bb743d79fib__main__1.774227e+091.774227e+090.001454
e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6faeefc15b4231178912fib__main__1.774227e+091.774227e+090.001772
\n", "
" ], "text/plain": [ " name module \\\n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... fib __main__ \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... fib __main__ \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... fib __main__ \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... fib __main__ \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... fib __main__ \n", "\n", " timestart \\\n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... 1.774227e+09 \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... 1.774227e+09 \n", "\n", " timestop walltime \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 0.000004 \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 0.000002 \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 0.000838 \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... 1.774227e+09 0.001454 \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... 1.774227e+09 0.001772 " ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache().query(QueryCall(name=\"fib\")).table().head()" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduletimestarttimestopwalltime
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__1.774227e+091.774227e+092.000227
3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748de84aad8f244a2d8fdouble__main__1.774227e+091.774227e+090.000019
\n", "
" ], "text/plain": [ " name \\\n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... double \n", "\n", " module timestart \\\n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... __main__ 1.774227e+09 \n", "\n", " timestop walltime \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... 1.774227e+09 0.000019 " ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache().query(QueryCall(arguments={\"x\": 100})).table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Metadata" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`fleche` allows you to add metadata to your cached functions using the `tags` context manager. This can be useful for organizing and querying your results." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "@fleche\n", "def another_calculation(a, b):\n", " return a + b" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "with tags(project='my_project', category='testing'):\n", " another_calculation(1, 2)\n", " another_calculation(3, 4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This metadata is stored alongside the cached result. This metadata can be used to query the cache as well." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "another_calculation LazyArguments({'a': 'da217d50752f3371d9f8b62a3e72409592bd34b74e14fdac43e2b137bd59f21f', 'b': '92a9b214d814a4a7b5f9ba52e2248c6d16ec0196d8cd7798b345471118bf0c67'}) {'project': 'my_project', 'category': 'testing'}\n", "LazyArguments({'a': '65d52a82c5a72f12ca0499522dc9274a0e6822e1038630ba68f94400b3e4c98f', 'b': '83ada2198553b88cb3d0882f7fca8c4e9531049b978df3e9e3b5d6301c6c0bfa'}) 7\n" ] } ], "source": [ "# Query by metadata presence (tags) and a specific key-value filter\n", "for call in another_calculation.query(1, 2, metadata={\"tags\": {}}):\n", " # presence-only: any call with 'tags'\n", " print(call.name, call.arguments, call.metadata.get(\"tags\"))\n", "\n", "for call in another_calculation.query(3, 4, metadata={\"tags\": {\"project\": \"my_project\"}}):\n", " # equality filter on metadata\n", " assert call.metadata[\"tags\"][\"project\"] == \"my_project\"\n", " # arguments and result are decoded if they were stored as digests\n", " print(call.arguments, call.result)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Caching Methods of User-defined Types\n", "\n", "`fleche` can also cache methods of classes. For this to work, the class must be \"digest-compatible\". You can make a class digest-compatible by implementing a `__digest__` method or by using a `dataclass`." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "class MyClass:\n", " def __init__(self, val):\n", " self.val = val\n", " \n", " def __digest__(self):\n", " # The digest defines how the instance is identified in the cache\n", " return Digest(str(self.val))\n", "\n", " @fleche\n", " def compute(self, x):\n", " print(f\"Computing {self.val} + {x}...\")\n", " time.sleep(1)\n", " return self.val + x" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computing 10 + 5...\n", "Result: 15\n", "First call took 1.00 seconds.\n", "Result: 15\n", "Second call (same instance) took 0.00 seconds.\n" ] } ], "source": [ "obj = MyClass(10)\n", "\n", "start = time.time()\n", "print(f\"Result: {obj.compute(5)}\")\n", "print(f\"First call took {time.time() - start:.2f} seconds.\")\n", "\n", "start = time.time()\n", "print(f\"Result: {obj.compute(5)}\")\n", "print(f\"Second call (same instance) took {time.time() - start:.2f} seconds.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you mutate the instance such that its digest changes, the cache will be missed." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computing 20 + 5...\n", "Result: 25\n", "Call after mutation took 1.00 seconds.\n" ] } ], "source": [ "obj.val = 20\n", "start = time.time()\n", "print(f\"Result: {obj.compute(5)}\")\n", "print(f\"Call after mutation took {time.time() - start:.2f} seconds.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Passing Digests as Arguments\n", "\n", "`fleche` supports passing `Digest` objects directly to cached functions. When a function receives a `Digest`, `fleche` automatically expands it to its actual value from the cache before executing the function. You can use the convenience wrapper `D` to mark a string as a digest." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Value Digest: 60079f7901a9295349d1796c037afc132e81286f785ddeeb763104ef02363102\n", "Short digest: 60079f79\n", "Result: 200\n" ] } ], "source": [ "from fleche import D\n", "from fleche.digest import digest as value_digest\n", "\n", "@fleche\n", "def double(x):\n", " print(f\"Doubling {x}...\")\n", " return x * 2\n", "\n", "# 1. Run the calculation to ensure it is cached\n", "long_running_calculation(10)\n", "\n", "# 2. Compute the value digest for 100 (the cached result)\n", "v = long_running_calculation(10)\n", "val_dig = value_digest(v)\n", "print(f\"Value Digest: {val_dig}\")\n", "\n", "# 3. Pass a short digest prefix of the value to double(); it will expand to 100.\n", "short = str(val_dig)[:8]\n", "print(f\"Short digest: {short}\")\n", "print(f\"Result: {double(D(short))}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Metadata" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`fleche` allows you to add metadata to your cached functions using the `tags` context manager. This can be useful for organizing and querying your results." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "@fleche\n", "def another_calculation(a, b):\n", " return a + b" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "with tags(project='my_project', category='testing'):\n", " another_calculation(1, 2)\n", " another_calculation(3, 4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This metadata is stored alongside the cached result. You can then use the `metadata.table` method to view the metadata for all cached results." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduletimestarttimestopwalltimeprojectcategory
4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953long_running_calculation__main__1.774227e+091.774227e+092.000136NaNNaN
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__1.774227e+091.774227e+092.000227NaNNaN
698e29f05ba00ee23503848cd166215f62cd976d54431594036979d8d56f254ffib__main__1.774227e+091.774227e+090.000004NaNNaN
405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4edeafb1c6f236fc80fib__main__1.774227e+091.774227e+090.000002NaNNaN
24231d9bc47f7abc0ea485d178fc8457dce8082790f8c948b936ebe906352225fib__main__1.774227e+091.774227e+090.000838NaNNaN
dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2b9a284dc1bb743d79fib__main__1.774227e+091.774227e+090.001454NaNNaN
e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6faeefc15b4231178912fib__main__1.774227e+091.774227e+090.001772NaNNaN
981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f4fcd3190b6187b0acfib__main__1.774227e+091.774227e+090.002040NaNNaN
fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d199b8606eed60579d91fib__main__1.774227e+091.774227e+090.002305NaNNaN
db6b5632c77a01a692e13c486c54536a0c46907972d93900a60a1eab59110b93fib__main__1.774227e+091.774227e+090.002581NaNNaN
cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f18b57abc5e07b6457fib__main__1.774227e+091.774227e+090.002940NaNNaN
f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78dad4feb055b56fd47defib__main__1.774227e+091.774227e+090.003206NaNNaN
bc7f8b67d293d9fbdec343f49c44d46e655516537cc492ba3a990edb7f86f176fib__main__1.774227e+091.774227e+090.003480NaNNaN
6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f01322946f5e53e7cffib__main__1.774227e+091.774227e+090.003752NaNNaN
f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1bd078db3377cca4c3fib__main__1.774227e+091.774227e+090.004020NaNNaN
2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c35567b81514b90c053f2fib__main__1.774227e+091.774227e+090.004293NaNNaN
60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7a18a0001fd0b65cf9fib__main__1.774227e+091.774227e+090.004573NaNNaN
9b23a905e639149d35169118ffefadec03a181cbd290006620e671655b1499c3fib__main__1.774227e+091.774227e+090.004852NaNNaN
18e77dab7a4a0385a66512547336e7a02ee62803634fd801e6dd49c6e5444e2ffib__main__1.774227e+091.774227e+090.005127NaNNaN
e1a78503325bcbb94116e03febed590b14bdd7f955c4f15b8649e613fbd6bf5cfib__main__1.774227e+091.774227e+090.005413NaNNaN
34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c4342b29231a8213eafib__main__1.774227e+091.774227e+090.005702NaNNaN
9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d1fd7ef30aa2c75873fib__main__1.774227e+091.774227e+090.005992NaNNaN
c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf19505cc660b50e6d9fib__main__1.774227e+091.774227e+090.006301NaNNaN
a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8long_running_calculation__main__1.774227e+091.774227e+092.000230NaNNaN
3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748de84aad8f244a2d8fdouble__main__1.774227e+091.774227e+090.000019NaNNaN
a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae7a2e91aa1fbac8c80another_calculation__main__1.774227e+091.774227e+090.000011my_projecttesting
1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0feca453b301b40156aanother_calculation__main__1.774227e+091.774227e+090.000025my_projecttesting
81d4df70836ded7d8a0dd70348976c431210ff824d870acb2932b984e7936e14compute__main__1.774227e+091.774227e+091.000169NaNNaN
e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf7ba47c5bf2fa219aecompute__main__1.774227e+091.774227e+091.000242NaNNaN
\n", "
" ], "text/plain": [ " name \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... fib \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... fib \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... fib \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... fib \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... fib \n", "981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... fib \n", "fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... fib \n", "db6b5632c77a01a692e13c486c54536a0c46907972d9390... fib \n", "cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... fib \n", "f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... fib \n", "bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... fib \n", "6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... fib \n", "f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... fib \n", "2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... fib \n", "60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... fib \n", "9b23a905e639149d35169118ffefadec03a181cbd290006... fib \n", "18e77dab7a4a0385a66512547336e7a02ee62803634fd80... fib \n", "e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... fib \n", "34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... fib \n", "9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... fib \n", "c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... fib \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... double \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... another_calculation \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... another_calculation \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... compute \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... compute \n", "\n", " module timestart \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... __main__ 1.774227e+09 \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... __main__ 1.774227e+09 \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... __main__ 1.774227e+09 \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... __main__ 1.774227e+09 \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... __main__ 1.774227e+09 \n", "981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... __main__ 1.774227e+09 \n", "fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... __main__ 1.774227e+09 \n", "db6b5632c77a01a692e13c486c54536a0c46907972d9390... __main__ 1.774227e+09 \n", "cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... __main__ 1.774227e+09 \n", "f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... __main__ 1.774227e+09 \n", "bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... __main__ 1.774227e+09 \n", "6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... __main__ 1.774227e+09 \n", "f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... __main__ 1.774227e+09 \n", "2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... __main__ 1.774227e+09 \n", "60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... __main__ 1.774227e+09 \n", "9b23a905e639149d35169118ffefadec03a181cbd290006... __main__ 1.774227e+09 \n", "18e77dab7a4a0385a66512547336e7a02ee62803634fd80... __main__ 1.774227e+09 \n", "e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... __main__ 1.774227e+09 \n", "34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... __main__ 1.774227e+09 \n", "9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... __main__ 1.774227e+09 \n", "c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... __main__ 1.774227e+09 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 1.774227e+09 \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... __main__ 1.774227e+09 \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... __main__ 1.774227e+09 \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... __main__ 1.774227e+09 \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... __main__ 1.774227e+09 \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... __main__ 1.774227e+09 \n", "\n", " timestop walltime \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 0.000004 \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 0.000002 \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 0.000838 \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... 1.774227e+09 0.001454 \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... 1.774227e+09 0.001772 \n", "981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... 1.774227e+09 0.002040 \n", "fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... 1.774227e+09 0.002305 \n", "db6b5632c77a01a692e13c486c54536a0c46907972d9390... 1.774227e+09 0.002581 \n", "cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... 1.774227e+09 0.002940 \n", "f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... 1.774227e+09 0.003206 \n", "bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... 1.774227e+09 0.003480 \n", "6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... 1.774227e+09 0.003752 \n", "f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... 1.774227e+09 0.004020 \n", "2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... 1.774227e+09 0.004293 \n", "60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... 1.774227e+09 0.004573 \n", "9b23a905e639149d35169118ffefadec03a181cbd290006... 1.774227e+09 0.004852 \n", "18e77dab7a4a0385a66512547336e7a02ee62803634fd80... 1.774227e+09 0.005127 \n", "e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... 1.774227e+09 0.005413 \n", "34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... 1.774227e+09 0.005702 \n", "9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... 1.774227e+09 0.005992 \n", "c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... 1.774227e+09 0.006301 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... 1.774227e+09 0.000019 \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... 1.774227e+09 0.000011 \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... 1.774227e+09 0.000025 \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... 1.774227e+09 1.000169 \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... 1.774227e+09 1.000242 \n", "\n", " project category \n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... NaN NaN \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... NaN NaN \n", "698e29f05ba00ee23503848cd166215f62cd976d5443159... NaN NaN \n", "405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... NaN NaN \n", "24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... NaN NaN \n", "dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... NaN NaN \n", "e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... NaN NaN \n", "981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... NaN NaN \n", "fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... NaN NaN \n", "db6b5632c77a01a692e13c486c54536a0c46907972d9390... NaN NaN \n", "cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... NaN NaN \n", "f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... NaN NaN \n", "bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... NaN NaN \n", "6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... NaN NaN \n", "f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... NaN NaN \n", "2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... NaN NaN \n", "60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... NaN NaN \n", "9b23a905e639149d35169118ffefadec03a181cbd290006... NaN NaN \n", "18e77dab7a4a0385a66512547336e7a02ee62803634fd80... NaN NaN \n", "e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... NaN NaN \n", "34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... NaN NaN \n", "9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... NaN NaN \n", "c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... NaN NaN \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... NaN NaN \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... NaN NaN \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... my_project testing \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... my_project testing \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... NaN NaN \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... NaN NaN " ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache().table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Filtering\n", "\n", "The metadata table is just pandas so you can query and filter as you like." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namemoduletimestarttimestopwalltimeprojectcategory
4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953long_running_calculation__main__1.774227e+091.774227e+092.000136NaNNaN
cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9elong_running_calculation__main__1.774227e+091.774227e+092.000227NaNNaN
a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8long_running_calculation__main__1.774227e+091.774227e+092.000230NaNNaN
3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748de84aad8f244a2d8fdouble__main__1.774227e+091.774227e+090.000019NaNNaN
a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae7a2e91aa1fbac8c80another_calculation__main__1.774227e+091.774227e+090.000011my_projecttesting
1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0feca453b301b40156aanother_calculation__main__1.774227e+091.774227e+090.000025my_projecttesting
81d4df70836ded7d8a0dd70348976c431210ff824d870acb2932b984e7936e14compute__main__1.774227e+091.774227e+091.000169NaNNaN
e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf7ba47c5bf2fa219aecompute__main__1.774227e+091.774227e+091.000242NaNNaN
\n", "
" ], "text/plain": [ " name \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... double \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... another_calculation \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... another_calculation \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... compute \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... compute \n", "\n", " module timestart \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 1.774227e+09 \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... __main__ 1.774227e+09 \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... __main__ 1.774227e+09 \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... __main__ 1.774227e+09 \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... __main__ 1.774227e+09 \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... __main__ 1.774227e+09 \n", "\n", " timestop walltime \\\n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... 1.774227e+09 0.000019 \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... 1.774227e+09 0.000011 \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... 1.774227e+09 0.000025 \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... 1.774227e+09 1.000169 \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... 1.774227e+09 1.000242 \n", "\n", " project category \n", "4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... NaN NaN \n", "cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... NaN NaN \n", "a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... NaN NaN \n", "3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... NaN NaN \n", "a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... my_project testing \n", "1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... my_project testing \n", "81d4df70836ded7d8a0dd70348976c431210ff824d870ac... NaN NaN \n", "e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... NaN NaN " ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cache().table().query('name!=\"fib\"')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Querying Cached Calls via Function Wrapper\n", "\n", "You can retrieve previously cached calls that match some of your function's arguments and metadata using the function wrapper's `query` method. Any field left as `None` is treated as a wildcard. Arguments and result are compared by digest internally, but the wrapper decodes them back to Python objects when returning matches.\n", "\n", "Example using the `another_calculation` wrapper we created above:\n" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "another_calculation LazyArguments({'a': 'da217d50752f3371d9f8b62a3e72409592bd34b74e14fdac43e2b137bd59f21f', 'b': '92a9b214d814a4a7b5f9ba52e2248c6d16ec0196d8cd7798b345471118bf0c67'}) {'project': 'my_project', 'category': 'testing'}\n", "LazyArguments({'a': '65d52a82c5a72f12ca0499522dc9274a0e6822e1038630ba68f94400b3e4c98f', 'b': '83ada2198553b88cb3d0882f7fca8c4e9531049b978df3e9e3b5d6301c6c0bfa'}) 7\n" ] } ], "source": [ "# Query by metadata presence (tags) and a specific key-value filter\n", "for call in another_calculation.query(1, 2, metadata={\"tags\": {}}):\n", " # presence-only: any call with 'tags'\n", " print(call.name, call.arguments, call.metadata.get(\"tags\"))\n", "\n", "for call in another_calculation.query(3, 4, metadata={\"tags\": {\"project\": \"my_project\"}}):\n", " # equality filter on metadata\n", " assert call.metadata[\"tags\"][\"project\"] == \"my_project\"\n", " # arguments and result are decoded if they were stored as digests\n", " print(call.arguments, call.result)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Lazy Loading\n", "\n", "When you load a call from the cache, fleche returns it as a `LazyCall` by default. Arguments and results are only fetched from storage when you actually access them \u2014 so iterating over a large cache or inspecting metadata stays fast even when individual results are huge.\n" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Obtained LazyCall for fib(20)\n", "Accessing result now (triggers load)...\n", "Result: 6765\n" ] } ], "source": [ "# Default load: returns a LazyCall \u2014 no deserialization yet\n", "key = fib.digest(20)\n", "lazy_call = cache().load(key)\n", "print(f\"Got a {type(lazy_call).__name__} for {lazy_call.name}(20)\")\n", "\n", "# Accessing .result triggers the actual load from storage\n", "print(f\"Result: {lazy_call.result}\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To load everything upfront instead, pass `lazy=False` or call `.fetch()` on a lazy call you already have.\n" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Both of these produce a fully-loaded Call:\n", "cache().load(key, lazy=False) == lazy_call.fetch()\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.11" } }, "nbformat": 4, "nbformat_minor": 4 }