{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"!rm .fleche -rf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Getting Started with Fleche"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook demonstrates the main features of the `fleche` library, a caching library for Python."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Long-running calculation"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"No config file found. Using default memory cache.\n"
]
}
],
"source": [
"import time\n",
"from fleche import fleche, cache, tags, project\n",
"from fleche.digest import Digest"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"@fleche\n",
"def long_running_calculation(x):\n",
" print(f'Running calculation for {x}...')\n",
" time.sleep(2)\n",
" return x * x"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running calculation for 2...\n",
"First call took 2.00 seconds.\n"
]
}
],
"source": [
"start = time.time()\n",
"long_running_calculation(2)\n",
"end = time.time()\n",
"print(f'First call took {end - start:.2f} seconds.')"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Second call took 0.00 seconds.\n"
]
}
],
"source": [
"start = time.time()\n",
"long_running_calculation(2)\n",
"end = time.time()\n",
"print(f'Second call took {end - start:.2f} seconds.')"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running calculation for 100...\n",
"Second call took 2.00 seconds.\n"
]
}
],
"source": [
"start = time.time()\n",
"long_running_calculation(100)\n",
"end = time.time()\n",
"print(f'Second call took {end - start:.2f} seconds.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, the second call returns almost instantly, because the result was cached.\n",
"As soon as the argument changes, fleche runs the original fucntion again."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Recursive function"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"@fleche\n",
"def fib(n):\n",
" if n < 2:\n",
" return n\n",
" return fib(n-1) + fib(n-2)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"fib(20) took 0.0067 seconds with caching.\n"
]
}
],
"source": [
"start = time.time()\n",
"fib(20)\n",
"end = time.time()\n",
"print(f'fib(20) took {end - start:.4f} seconds with caching.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Without caching, this would be much slower as each call to `fib` would be recomputed."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Passing Digests as Arguments\n",
"\n",
"`fleche` supports passing `Digest` objects directly to cached functions. When a function receives a `Digest`, `fleche` automatically expands it to its actual value from the cache before executing the function. You can use the convenience wrapper `D` to mark a string as a digest."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running calculation for 10...\n",
"Value Digest: 60079f7901a9295349d1796c037afc132e81286f785ddeeb763104ef02363102\n",
"Short digest: 60079f79\n",
"Doubling 100...\n",
"Result: 200\n"
]
}
],
"source": [
"from fleche import D\n",
"from fleche.digest import digest as value_digest\n",
"\n",
"@fleche\n",
"def double(x):\n",
" print(f\"Doubling {x}...\")\n",
" return x * 2\n",
"\n",
"# 1. Run the calculation to ensure it is cached\n",
"long_running_calculation(10)\n",
"\n",
"# 2. Compute the value digest for 100 (the cached result)\n",
"v = long_running_calculation(10)\n",
"val_dig = value_digest(v)\n",
"print(f\"Value Digest: {val_dig}\")\n",
"\n",
"# 3. Pass a short digest prefix of the value to double(); it will expand to 100.\n",
"short = str(val_dig)[:8]\n",
"print(f\"Short digest: {short}\")\n",
"print(f\"Result: {double(D(short))}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Querying Cached Calls "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### via Function Wrapper\n",
"\n",
"You can retrieve previously cached calls that match some of your function's arguments and metadata using the function wrapper's `query` method. Any field left as `None` is treated as a wildcard. Arguments and result are compared by digest internally, but the wrapper decodes them back to Python objects when returning matches.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Query returns an iterator object, that also defines additional utilities, e.g. to create a table of queried calls, do"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000136 | \n",
"
\n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
"
\n",
" \n",
" | a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000230 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n",
"\n",
" module timestart \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 1.774227e+09 \n",
"\n",
" timestop walltime \n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 "
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"long_running_calculation.query().table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The table includes the call digest as an index and the metadata associated with the call.\n",
"\n",
"Arguments can be selectively included in the table via the `arguments`. You only pay the loading cost for the specified arguments, not for arguments that are not included in the table."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" x | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 2 | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000136 | \n",
"
\n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 100 | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
"
\n",
" \n",
" | a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 10 | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000230 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n",
"\n",
" module x \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 2 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 100 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 10 \n",
"\n",
" timestart \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 \n",
"\n",
" timestop walltime \n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 "
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"long_running_calculation.query().table(arguments=['x'])"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" result | \n",
" x | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 4 | \n",
" 2 | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000136 | \n",
"
\n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 10000 | \n",
" 100 | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
"
\n",
" \n",
" | a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 100 | \n",
" 10 | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000230 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n",
"\n",
" module result x \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 4 2 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 10000 100 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 100 10 \n",
"\n",
" timestart \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 \n",
"\n",
" timestop walltime \n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 "
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"long_running_calculation.query().table(arguments=['x'], results=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### via Cache"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"from fleche.call import QueryCall"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000136 | \n",
"
\n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
"
\n",
" \n",
" | 698e29f05ba00ee23503848cd166215f62cd976d54431594036979d8d56f254f | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000004 | \n",
"
\n",
" \n",
" | 405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4edeafb1c6f236fc80 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000002 | \n",
"
\n",
" \n",
" | 24231d9bc47f7abc0ea485d178fc8457dce8082790f8c948b936ebe906352225 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000838 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... fib \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... fib \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... fib \n",
"\n",
" module timestart \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... __main__ 1.774227e+09 \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... __main__ 1.774227e+09 \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... __main__ 1.774227e+09 \n",
"\n",
" timestop walltime \n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 0.000004 \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 0.000002 \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 0.000838 "
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cache().query(QueryCall(module=\"__main__\")).table().head()"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
"
\n",
" \n",
" \n",
" \n",
" | 698e29f05ba00ee23503848cd166215f62cd976d54431594036979d8d56f254f | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000004 | \n",
"
\n",
" \n",
" | 405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4edeafb1c6f236fc80 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000002 | \n",
"
\n",
" \n",
" | 24231d9bc47f7abc0ea485d178fc8457dce8082790f8c948b936ebe906352225 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000838 | \n",
"
\n",
" \n",
" | dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2b9a284dc1bb743d79 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.001454 | \n",
"
\n",
" \n",
" | e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6faeefc15b4231178912 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.001772 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name module \\\n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... fib __main__ \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... fib __main__ \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... fib __main__ \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... fib __main__ \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... fib __main__ \n",
"\n",
" timestart \\\n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... 1.774227e+09 \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... 1.774227e+09 \n",
"\n",
" timestop walltime \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 0.000004 \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 0.000002 \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 0.000838 \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... 1.774227e+09 0.001454 \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... 1.774227e+09 0.001772 "
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cache().query(QueryCall(name=\"fib\")).table().head()"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
"
\n",
" \n",
" \n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
"
\n",
" \n",
" | 3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748de84aad8f244a2d8f | \n",
" double | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000019 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... double \n",
"\n",
" module timestart \\\n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... __main__ 1.774227e+09 \n",
"\n",
" timestop walltime \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... 1.774227e+09 0.000019 "
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cache().query(QueryCall(arguments={\"x\": 100})).table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Metadata"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`fleche` allows you to add metadata to your cached functions using the `tags` context manager. This can be useful for organizing and querying your results."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"@fleche\n",
"def another_calculation(a, b):\n",
" return a + b"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"with tags(project='my_project', category='testing'):\n",
" another_calculation(1, 2)\n",
" another_calculation(3, 4)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This metadata is stored alongside the cached result. This metadata can be used to query the cache as well."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"another_calculation LazyArguments({'a': 'da217d50752f3371d9f8b62a3e72409592bd34b74e14fdac43e2b137bd59f21f', 'b': '92a9b214d814a4a7b5f9ba52e2248c6d16ec0196d8cd7798b345471118bf0c67'}) {'project': 'my_project', 'category': 'testing'}\n",
"LazyArguments({'a': '65d52a82c5a72f12ca0499522dc9274a0e6822e1038630ba68f94400b3e4c98f', 'b': '83ada2198553b88cb3d0882f7fca8c4e9531049b978df3e9e3b5d6301c6c0bfa'}) 7\n"
]
}
],
"source": [
"# Query by metadata presence (tags) and a specific key-value filter\n",
"for call in another_calculation.query(1, 2, metadata={\"tags\": {}}):\n",
" # presence-only: any call with 'tags'\n",
" print(call.name, call.arguments, call.metadata.get(\"tags\"))\n",
"\n",
"for call in another_calculation.query(3, 4, metadata={\"tags\": {\"project\": \"my_project\"}}):\n",
" # equality filter on metadata\n",
" assert call.metadata[\"tags\"][\"project\"] == \"my_project\"\n",
" # arguments and result are decoded if they were stored as digests\n",
" print(call.arguments, call.result)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Caching Methods of User-defined Types\n",
"\n",
"`fleche` can also cache methods of classes. For this to work, the class must be \"digest-compatible\". You can make a class digest-compatible by implementing a `__digest__` method or by using a `dataclass`."
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"class MyClass:\n",
" def __init__(self, val):\n",
" self.val = val\n",
" \n",
" def __digest__(self):\n",
" # The digest defines how the instance is identified in the cache\n",
" return Digest(str(self.val))\n",
"\n",
" @fleche\n",
" def compute(self, x):\n",
" print(f\"Computing {self.val} + {x}...\")\n",
" time.sleep(1)\n",
" return self.val + x"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Computing 10 + 5...\n",
"Result: 15\n",
"First call took 1.00 seconds.\n",
"Result: 15\n",
"Second call (same instance) took 0.00 seconds.\n"
]
}
],
"source": [
"obj = MyClass(10)\n",
"\n",
"start = time.time()\n",
"print(f\"Result: {obj.compute(5)}\")\n",
"print(f\"First call took {time.time() - start:.2f} seconds.\")\n",
"\n",
"start = time.time()\n",
"print(f\"Result: {obj.compute(5)}\")\n",
"print(f\"Second call (same instance) took {time.time() - start:.2f} seconds.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you mutate the instance such that its digest changes, the cache will be missed."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Computing 20 + 5...\n",
"Result: 25\n",
"Call after mutation took 1.00 seconds.\n"
]
}
],
"source": [
"obj.val = 20\n",
"start = time.time()\n",
"print(f\"Result: {obj.compute(5)}\")\n",
"print(f\"Call after mutation took {time.time() - start:.2f} seconds.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Passing Digests as Arguments\n",
"\n",
"`fleche` supports passing `Digest` objects directly to cached functions. When a function receives a `Digest`, `fleche` automatically expands it to its actual value from the cache before executing the function. You can use the convenience wrapper `D` to mark a string as a digest."
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Value Digest: 60079f7901a9295349d1796c037afc132e81286f785ddeeb763104ef02363102\n",
"Short digest: 60079f79\n",
"Result: 200\n"
]
}
],
"source": [
"from fleche import D\n",
"from fleche.digest import digest as value_digest\n",
"\n",
"@fleche\n",
"def double(x):\n",
" print(f\"Doubling {x}...\")\n",
" return x * 2\n",
"\n",
"# 1. Run the calculation to ensure it is cached\n",
"long_running_calculation(10)\n",
"\n",
"# 2. Compute the value digest for 100 (the cached result)\n",
"v = long_running_calculation(10)\n",
"val_dig = value_digest(v)\n",
"print(f\"Value Digest: {val_dig}\")\n",
"\n",
"# 3. Pass a short digest prefix of the value to double(); it will expand to 100.\n",
"short = str(val_dig)[:8]\n",
"print(f\"Short digest: {short}\")\n",
"print(f\"Result: {double(D(short))}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Metadata"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`fleche` allows you to add metadata to your cached functions using the `tags` context manager. This can be useful for organizing and querying your results."
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"@fleche\n",
"def another_calculation(a, b):\n",
" return a + b"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"with tags(project='my_project', category='testing'):\n",
" another_calculation(1, 2)\n",
" another_calculation(3, 4)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This metadata is stored alongside the cached result. You can then use the `metadata.table` method to view the metadata for all cached results."
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
" project | \n",
" category | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000136 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 698e29f05ba00ee23503848cd166215f62cd976d54431594036979d8d56f254f | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000004 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4edeafb1c6f236fc80 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000002 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 24231d9bc47f7abc0ea485d178fc8457dce8082790f8c948b936ebe906352225 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000838 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2b9a284dc1bb743d79 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.001454 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6faeefc15b4231178912 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.001772 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f4fcd3190b6187b0ac | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.002040 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d199b8606eed60579d91 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.002305 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | db6b5632c77a01a692e13c486c54536a0c46907972d93900a60a1eab59110b93 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.002581 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f18b57abc5e07b6457 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.002940 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78dad4feb055b56fd47de | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.003206 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | bc7f8b67d293d9fbdec343f49c44d46e655516537cc492ba3a990edb7f86f176 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.003480 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f01322946f5e53e7cf | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.003752 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1bd078db3377cca4c3 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.004020 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c35567b81514b90c053f2 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.004293 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7a18a0001fd0b65cf9 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.004573 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 9b23a905e639149d35169118ffefadec03a181cbd290006620e671655b1499c3 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.004852 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 18e77dab7a4a0385a66512547336e7a02ee62803634fd801e6dd49c6e5444e2f | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.005127 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | e1a78503325bcbb94116e03febed590b14bdd7f955c4f15b8649e613fbd6bf5c | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.005413 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c4342b29231a8213ea | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.005702 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d1fd7ef30aa2c75873 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.005992 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf19505cc660b50e6d9 | \n",
" fib | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.006301 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000230 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748de84aad8f244a2d8f | \n",
" double | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000019 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae7a2e91aa1fbac8c80 | \n",
" another_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000011 | \n",
" my_project | \n",
" testing | \n",
"
\n",
" \n",
" | 1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0feca453b301b40156a | \n",
" another_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000025 | \n",
" my_project | \n",
" testing | \n",
"
\n",
" \n",
" | 81d4df70836ded7d8a0dd70348976c431210ff824d870acb2932b984e7936e14 | \n",
" compute | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 1.000169 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf7ba47c5bf2fa219ae | \n",
" compute | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 1.000242 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... fib \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... fib \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... fib \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... fib \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... fib \n",
"981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... fib \n",
"fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... fib \n",
"db6b5632c77a01a692e13c486c54536a0c46907972d9390... fib \n",
"cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... fib \n",
"f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... fib \n",
"bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... fib \n",
"6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... fib \n",
"f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... fib \n",
"2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... fib \n",
"60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... fib \n",
"9b23a905e639149d35169118ffefadec03a181cbd290006... fib \n",
"18e77dab7a4a0385a66512547336e7a02ee62803634fd80... fib \n",
"e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... fib \n",
"34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... fib \n",
"9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... fib \n",
"c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... fib \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... double \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... another_calculation \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... another_calculation \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... compute \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... compute \n",
"\n",
" module timestart \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... __main__ 1.774227e+09 \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... __main__ 1.774227e+09 \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... __main__ 1.774227e+09 \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... __main__ 1.774227e+09 \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... __main__ 1.774227e+09 \n",
"981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... __main__ 1.774227e+09 \n",
"fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... __main__ 1.774227e+09 \n",
"db6b5632c77a01a692e13c486c54536a0c46907972d9390... __main__ 1.774227e+09 \n",
"cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... __main__ 1.774227e+09 \n",
"f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... __main__ 1.774227e+09 \n",
"bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... __main__ 1.774227e+09 \n",
"6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... __main__ 1.774227e+09 \n",
"f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... __main__ 1.774227e+09 \n",
"2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... __main__ 1.774227e+09 \n",
"60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... __main__ 1.774227e+09 \n",
"9b23a905e639149d35169118ffefadec03a181cbd290006... __main__ 1.774227e+09 \n",
"18e77dab7a4a0385a66512547336e7a02ee62803634fd80... __main__ 1.774227e+09 \n",
"e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... __main__ 1.774227e+09 \n",
"34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... __main__ 1.774227e+09 \n",
"9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... __main__ 1.774227e+09 \n",
"c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... __main__ 1.774227e+09 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 1.774227e+09 \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... __main__ 1.774227e+09 \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... __main__ 1.774227e+09 \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... __main__ 1.774227e+09 \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... __main__ 1.774227e+09 \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... __main__ 1.774227e+09 \n",
"\n",
" timestop walltime \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... 1.774227e+09 0.000004 \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... 1.774227e+09 0.000002 \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... 1.774227e+09 0.000838 \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... 1.774227e+09 0.001454 \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... 1.774227e+09 0.001772 \n",
"981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... 1.774227e+09 0.002040 \n",
"fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... 1.774227e+09 0.002305 \n",
"db6b5632c77a01a692e13c486c54536a0c46907972d9390... 1.774227e+09 0.002581 \n",
"cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... 1.774227e+09 0.002940 \n",
"f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... 1.774227e+09 0.003206 \n",
"bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... 1.774227e+09 0.003480 \n",
"6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... 1.774227e+09 0.003752 \n",
"f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... 1.774227e+09 0.004020 \n",
"2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... 1.774227e+09 0.004293 \n",
"60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... 1.774227e+09 0.004573 \n",
"9b23a905e639149d35169118ffefadec03a181cbd290006... 1.774227e+09 0.004852 \n",
"18e77dab7a4a0385a66512547336e7a02ee62803634fd80... 1.774227e+09 0.005127 \n",
"e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... 1.774227e+09 0.005413 \n",
"34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... 1.774227e+09 0.005702 \n",
"9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... 1.774227e+09 0.005992 \n",
"c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... 1.774227e+09 0.006301 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... 1.774227e+09 0.000019 \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... 1.774227e+09 0.000011 \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... 1.774227e+09 0.000025 \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... 1.774227e+09 1.000169 \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... 1.774227e+09 1.000242 \n",
"\n",
" project category \n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... NaN NaN \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... NaN NaN \n",
"698e29f05ba00ee23503848cd166215f62cd976d5443159... NaN NaN \n",
"405dfbadf453a9d5bbe3482fbb05251a6fc18b90045f8b4... NaN NaN \n",
"24231d9bc47f7abc0ea485d178fc8457dce8082790f8c94... NaN NaN \n",
"dabebddba19859bdb1a075e422b1842dde433c7d6ca51c2... NaN NaN \n",
"e2b3a4eaf9036f83560677b9bad64cbb8263cf3496ba6fa... NaN NaN \n",
"981a54293b3027931a47d21c275955d6dab2fe6d8fa4c2f... NaN NaN \n",
"fdb0a9b8b1df91293924ad8fc03bcf041f3e50f924e9d19... NaN NaN \n",
"db6b5632c77a01a692e13c486c54536a0c46907972d9390... NaN NaN \n",
"cd94bda61e484ca82c470b6ab50c0f1b98055e6a905d82f... NaN NaN \n",
"f4efb0e68400d195d8d57051088964d98b9ae4f7dbb78da... NaN NaN \n",
"bc7f8b67d293d9fbdec343f49c44d46e655516537cc492b... NaN NaN \n",
"6fbe26ceafe80276c7714b2aae14c25bd38e24683d6624f... NaN NaN \n",
"f3cefd42fd07119792c38084ecbb33f1955ccd83d8021e1... NaN NaN \n",
"2b7c9e8c5e0a73afddb28a49d47415462b3c84adc05c355... NaN NaN \n",
"60bf26e7cdec62d1a8f9ebfbebb336850cf588d547badc7... NaN NaN \n",
"9b23a905e639149d35169118ffefadec03a181cbd290006... NaN NaN \n",
"18e77dab7a4a0385a66512547336e7a02ee62803634fd80... NaN NaN \n",
"e1a78503325bcbb94116e03febed590b14bdd7f955c4f15... NaN NaN \n",
"34814b3262b7920c7cef43eaf8a1e86e849611ed0d0317c... NaN NaN \n",
"9214caae9e9819230882f7e9f0f1970fd5f2fc354c3404d... NaN NaN \n",
"c5a5bb6cf5b69c7eb5e244ed540af44391dff12a77fcecf... NaN NaN \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... NaN NaN \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... NaN NaN \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... my_project testing \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... my_project testing \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... NaN NaN \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... NaN NaN "
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cache().table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Filtering\n",
"\n",
"The metadata table is just pandas so you can query and filter as you like."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" name | \n",
" module | \n",
" timestart | \n",
" timestop | \n",
" walltime | \n",
" project | \n",
" category | \n",
"
\n",
" \n",
" \n",
" \n",
" | 4435e2927c194eff1bb90270ab3f353f0db1a8b50ec04932de88f7a4e5745953 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000136 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4cfb21baa5874c2b9e | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000227 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1b9e6318e4a97368e8 | \n",
" long_running_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 2.000230 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | 3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748de84aad8f244a2d8f | \n",
" double | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000019 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae7a2e91aa1fbac8c80 | \n",
" another_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000011 | \n",
" my_project | \n",
" testing | \n",
"
\n",
" \n",
" | 1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0feca453b301b40156a | \n",
" another_calculation | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 0.000025 | \n",
" my_project | \n",
" testing | \n",
"
\n",
" \n",
" | 81d4df70836ded7d8a0dd70348976c431210ff824d870acb2932b984e7936e14 | \n",
" compute | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 1.000169 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
" | e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf7ba47c5bf2fa219ae | \n",
" compute | \n",
" __main__ | \n",
" 1.774227e+09 | \n",
" 1.774227e+09 | \n",
" 1.000242 | \n",
" NaN | \n",
" NaN | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" name \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... long_running_calculation \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... long_running_calculation \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... long_running_calculation \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... double \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... another_calculation \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... another_calculation \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... compute \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... compute \n",
"\n",
" module timestart \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... __main__ 1.774227e+09 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... __main__ 1.774227e+09 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... __main__ 1.774227e+09 \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... __main__ 1.774227e+09 \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... __main__ 1.774227e+09 \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... __main__ 1.774227e+09 \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... __main__ 1.774227e+09 \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... __main__ 1.774227e+09 \n",
"\n",
" timestop walltime \\\n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... 1.774227e+09 2.000136 \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... 1.774227e+09 2.000227 \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... 1.774227e+09 2.000230 \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... 1.774227e+09 0.000019 \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... 1.774227e+09 0.000011 \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... 1.774227e+09 0.000025 \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... 1.774227e+09 1.000169 \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... 1.774227e+09 1.000242 \n",
"\n",
" project category \n",
"4435e2927c194eff1bb90270ab3f353f0db1a8b50ec0493... NaN NaN \n",
"cb9db168b7a8eb99bda637a54241ad5d4e2726b5cd67ad4... NaN NaN \n",
"a7ce6824785adc406ca3561dcf98b3c64ddf1539d2467e1... NaN NaN \n",
"3fb9bd0ba9c388530e4a0f5fbb789aaf7b5c598c1fae748... NaN NaN \n",
"a8a3061653183cff08e5c414f9a4087550ab5e33cc186ae... my_project testing \n",
"1e352b538b9219d4e5fcaf429882a4b3587f3bc6358ef0f... my_project testing \n",
"81d4df70836ded7d8a0dd70348976c431210ff824d870ac... NaN NaN \n",
"e9f9ca077d0566f335e54df480d202815d7b5b5ac5e7ccf... NaN NaN "
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cache().table().query('name!=\"fib\"')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Querying Cached Calls via Function Wrapper\n",
"\n",
"You can retrieve previously cached calls that match some of your function's arguments and metadata using the function wrapper's `query` method. Any field left as `None` is treated as a wildcard. Arguments and result are compared by digest internally, but the wrapper decodes them back to Python objects when returning matches.\n",
"\n",
"Example using the `another_calculation` wrapper we created above:\n"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"another_calculation LazyArguments({'a': 'da217d50752f3371d9f8b62a3e72409592bd34b74e14fdac43e2b137bd59f21f', 'b': '92a9b214d814a4a7b5f9ba52e2248c6d16ec0196d8cd7798b345471118bf0c67'}) {'project': 'my_project', 'category': 'testing'}\n",
"LazyArguments({'a': '65d52a82c5a72f12ca0499522dc9274a0e6822e1038630ba68f94400b3e4c98f', 'b': '83ada2198553b88cb3d0882f7fca8c4e9531049b978df3e9e3b5d6301c6c0bfa'}) 7\n"
]
}
],
"source": [
"# Query by metadata presence (tags) and a specific key-value filter\n",
"for call in another_calculation.query(1, 2, metadata={\"tags\": {}}):\n",
" # presence-only: any call with 'tags'\n",
" print(call.name, call.arguments, call.metadata.get(\"tags\"))\n",
"\n",
"for call in another_calculation.query(3, 4, metadata={\"tags\": {\"project\": \"my_project\"}}):\n",
" # equality filter on metadata\n",
" assert call.metadata[\"tags\"][\"project\"] == \"my_project\"\n",
" # arguments and result are decoded if they were stored as digests\n",
" print(call.arguments, call.result)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Lazy Loading\n",
"\n",
"When you load a call from the cache, fleche returns it as a `LazyCall` by default. Arguments and results are only fetched from storage when you actually access them \u2014 so iterating over a large cache or inspecting metadata stays fast even when individual results are huge.\n"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Obtained LazyCall for fib(20)\n",
"Accessing result now (triggers load)...\n",
"Result: 6765\n"
]
}
],
"source": [
"# Default load: returns a LazyCall \u2014 no deserialization yet\n",
"key = fib.digest(20)\n",
"lazy_call = cache().load(key)\n",
"print(f\"Got a {type(lazy_call).__name__} for {lazy_call.name}(20)\")\n",
"\n",
"# Accessing .result triggers the actual load from storage\n",
"print(f\"Result: {lazy_call.result}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To load everything upfront instead, pass `lazy=False` or call `.fetch()` on a lazy call you already have.\n"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Both of these produce a fully-loaded Call:\n",
"cache().load(key, lazy=False) == lazy_call.fetch()\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}