You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add a functools.cache variant for methods to avoid keeping instances alive #102618
functools.cache, when applied to a method, caches the self argument. This can also keep the instance alive long after it is no longer needed. See for example this discussion.
Following the same API as singledispatch and singledispatchmethod, we should consider adding a cachemethod decorator that either avoids caching self, or uses a weakref for it.
Reacted by Nick Koukoufilippas, Aaron, Kaleb Barrett, nmoreaud, Nick Rogers, Grant Culp, Aaron Zuspan, Xiaoyan Wang, Stuart Axon, Colton Loftus and 2 more
To me the fact that lru_cache on methods passes the self parameter is obvious as a regular user of the language. The issue in my mind is that the exact caching mechanism is not mentioned and the problem where this causes a "leak" is due to an implementation detail that isn't explicitly mentioned. The documentation could hint at the potential pitfall by stating that important implementation detail explicitly.
Something along the lines of...
All arguments passed to the cached function are retained by the cache with strong references until its eviction. This means there may be cases where the cache is the only thing with strong references to the argument objects.
I encountered this behavior too, which results in a memory leak.
I think that there should be a warning in the documentation, because the cache seems to work on method whereas actually it leaks memory: https://docs.python.org/3/library/functools.html#functools.lru_cache
The best would be, of course, to have a cached_method decorator which doesn't leak.
A debug assertion would have saved me a lot of time, but it is probably not possible to implement it because of the additional performance cost of such a test and because there is an official example here: https://docs.python.org/3/faq/programming.html#how-do-i-cache-method-calls
@nmoreaud kids.cache is not mantained and I don't see such a feature. And cachetools as mentioned is more manual.
@dsal3389 Ideally something could be made to work just like cached_property, where it rewrites the attribute on the instance to be a cached method like cached_property rewrites the attribute to be the value that was computed.
EDIT 1: Support type(obj).attr(inst) by implementing __call__.
EDIT 2: Use __set_name__ for setting the object name to support attr = cached_method(lambda: stuff(123))
EDIT 3:
In __get__ first try to look up the cached function call from the object since some builtin functions do some very funny lookups.
For example, len(a) does not call the equivalent of getattr(a, '__len__')(), it looks in the class first. And it doesn't do the equivalent of getattr(type(a), '__len__')(a) either, __get__ is always invoked with the instance object. So without this trial getattr, the implementation would recreate the cache and function on every call of len(a).
Has anyone filed a report against linting tools to warn when functools.cache and functools.lru_cache are used on methods? This is assuming that this will never be fixed in the language proper.
For those like me who landed on this issue, here's a cachemethod that solves this.
Notably, it:
works on self types without a __dict__;
works on self types that are not hashable;
should be reasonably efficient as it avoids re-hashing the runtime arguments;
does not rely on descriptors, so it will cache even when called via SomeClass.some_cached_method(self, *args, **kwargs), not just self.some_cached_method(*args, **kwargs);
is static typing compatible.
importfunctoolsasftimportinspectimportweakreffromcollections.abcimportCallablefromtypingimportTYPE_CHECKING, Concatenate, ParamSpec, TypeVar_Self=TypeVar("_Self")
_Params=ParamSpec("_Params")
_Return=TypeVar("_Return")
# Like `weakref.WeakKeyDictionary`, but uses identity-based hashing and equality.class_WeakIdDict:
__slots__= ("_dict",)
def__init__(self):
self._dict= {}
def__getitem__(self, key):
item, _=self._dict[id(key)]
returnitemdef__setitem__(self, key, value):
id_key=id(key)
ref=weakref.ref(key, lambda_: self._dict.pop(id_key))
self._dict[id_key] = (value, ref)
# Used to compute an object's hash just once.class_CacheKey:
__slots__= ("hashvalue", "value")
def__init__(self, value):
self.hashvalue=hash(value)
self.value=valuedef__hash__(self) ->int:
returnself.hashvaluedef__eq__(self, other) ->bool:
# Assume `type(other) is _Key`returnself.value==other.valuedefcachemethod(fn: Callable[Concatenate[_Self, _Params], _Return]) ->Callable[Concatenate[_Self, _Params], _Return]:
"""Like `functools.cache`, except that it only holds a weak reference to its first argument. Note that `functools.cached_property` (which also uses a weak reference) can often be used for similar purposes. The differences are that (a) `cached_property` will be pickled whilst `cachemethod` will not, and (b) `cachemethod` can be used on functions with additional arguments, and (c) `cachemethod` requires brackets to call, helping to visually emphasise that computationl work may be being performed. """cache1=_WeakIdDict()
sig=inspect.signature(fn)
parameters=list(sig.parameters.values())
iflen(parameters) ==0:
raiseValueError("Cannot use `cachemethod` on a function without a `self` argument.")
ifparameters[0].kindnotin {inspect.Parameter.POSITIONAL_ONLY, inspect.Parameter.POSITIONAL_OR_KEYWORD}:
raiseValueError("Cannot use `cachemethod` on a function without a positional argument.")
parameters=parameters[1:]
sig=sig.replace(parameters=parameters)
@ft.wraps(fn)deffn_wrapped(self: _Self, *args: _Params.args, **kwargs: _Params.kwargs) ->_Return:
# Standardise arguments to a single form to encourage cache hits.# Not binding `self` (and instead doing the signature-adjustment above) in order to avoid keeping a strong# reference to `self` via `argkey`.bound=sig.bind(*args, **kwargs)
delargs, kwargsargkey=_CacheKey((bound.args, tuple(bound.kwargs.items())))
try:
out=cache1[self][argkey]
exceptKeyError:
try:
cache2=cache1[self]
exceptKeyError:
cache2=cache1[self] = {}
out=cache2[argkey] =fn(self, *bound.args, **bound.kwargs)
returnoutreturnfn_wrapped
For those like me who landed on this issue, here's a cachemethod that solves this.
Notably, it:
works on self types without a __dict__;
works on self types that are not hashable;
should be reasonably efficient as it avoids re-hashing the runtime arguments;
does not rely on descriptors, so it will cache even when called via SomeClass.some_cached_method(self, *args, **kwargs), not just self.some_cached_method(*args, **kwargs);
is static typing compatible.
import functools as ft
import inspect
import weakref
from collections.abc import Callable
from typing import TYPE_CHECKING, Concatenate, ParamSpec, TypeVar
def cachemethod(fn: Callable[Concatenate[_Self, _Params], _Return]) -> Callable[Concatenate[_Self, _Params], _Return]:
"""Like functools.cache, except that it only holds a weak reference to its first argument.
Note that `functools.cached_property` (which also uses a weak reference) can often be used for similar purposes.
The differences are that (a) `cached_property` will be pickled whilst `cachemethod` will not, and (b) `cachemethod`
can be used on functions with additional arguments, and (c) `cachemethod` requires brackets to call, helping to
visually emphasise that computationl work may be being performed.
"""
cache1 = _WeakIdDict()
sig = inspect.signature(fn)
parameters = list(sig.parameters.values())
if len(parameters) == 0:
raise ValueError("Cannot use `cachemethod` on a function without a `self` argument.")
if parameters[0].kind not in {inspect.Parameter.POSITIONAL_ONLY, inspect.Parameter.POSITIONAL_OR_KEYWORD}:
raise ValueError("Cannot use `cachemethod` on a function without a positional argument.")
parameters = parameters[1:]
sig = sig.replace(parameters=parameters)
@ft.wraps(fn)
def fn_wrapped(self: _Self, *args: _Params.args, **kwargs: _Params.kwargs) -> _Return:
# Standardise arguments to a single form to encourage cache hits.
# Not binding `self` (and instead doing the signature-adjustment above) in order to avoid keeping a strong
# reference to `self` via `argkey`.
bound = sig.bind(*args, **kwargs)
del args, kwargs
argkey = _CacheKey((bound.args, tuple(bound.kwargs.items())))
try:
out = cache1[self][argkey]
except KeyError:
try:
cache2 = cache1[self]
except KeyError:
cache2 = cache1[self] = {}
out = cache2[argkey] = fn(self, *bound.args, **bound.kwargs)
return out
return fn_wrapped
you might not keep references to the instance so it can be garbage collected, but you use the builtin id function which is just returning the data position in memory (if I remember correctly) and what the default __hash__ implementation also uses behind the scenes
Two objects with non-overlapping lifetimes may have the same id() value.
this will cause false cache hits if 2 different objects are allocated on the same memory location
this is also the reason the default cache methods keep reference to the data, so it won't be garbage collected thus it will be impossible for other instances to be allocated on the same memory location
the only solution is to bind the cache to the running instance (as a private property or something) and make the cache not hold references to self, so when the instance is deleted the cache will also be deleted
Is it possible for an object to GC'd, then a new one to be reallocated, before weakref.ref(..., callback=...) runs?
From the weakref.ref documentation, it states that the callback is evaluated when the wrapped object is about to be finalized. The wording of this suggests to me that the callback is evaluated immediately before the old object is GC'd, and therefore necessarily a new one cannot yet be allocated in the same memory location.
totally missed that lambda callback while glancing over the code ref = weakref.ref(key, lambda _: self._dict.pop(id_key))
this looks like a good approach, I wonder why the stdlib doesn't really use it like u suggested, maybe a good question / suggestion for the python discourse?
@ktbarrett but if you don't cache the instance, how can you distinguish between cache hits that are coming from different instances? most of the time you don't want to mix them
here is a very simple example (theory) how it can hit you (assume self is not used in the cache key)
fromfunctoolsimportcacheclassFoo:
def__init__(self, n: int) ->None:
self._n=n@cachedeffoo(self) ->str:
ifself._n%5==0:
return"yes"else:
return"no"x=Foo(5)
y=Foo(7)
print(x.foo()) # will print "yes"print(y.foo()) # will print also "yes", expected to be "no"
You place the cache on the instance rather than keeping it in the class. This is how cached_property works (well it's a "cache" of 1).
This has the disadvantage of requiring that the instance be mutable in this way, which is the kind of sneakiness that can lead to unexpected behavior!
For example, functools.cached_property will actually be pickled. This is unlike regular property - and is actually the reason that prompted me to write the cachemethod above.
Neither issues bothered the core devs when @cached_property was merged. stdlib isn't some random library that can change frequently and attempt to handle everyone's case.
Activity
ktbarrett commentedon Jun 11, 2024
To me the fact that
lru_cacheon methods passes theselfparameter is obvious as a regular user of the language. The issue in my mind is that the exact caching mechanism is not mentioned and the problem where this causes a "leak" is due to an implementation detail that isn't explicitly mentioned. The documentation could hint at the potential pitfall by stating that important implementation detail explicitly.Something along the lines of...
nmoreaud commentedon Jun 27, 2024
I encountered this behavior too, which results in a memory leak.
I think that there should be a warning in the documentation, because the cache seems to work on method whereas actually it leaks memory: https://docs.python.org/3/library/functools.html#functools.lru_cache
The best would be, of course, to have a
cached_methoddecorator which doesn't leak.A debug assertion would have saved me a lot of time, but it is probably not possible to implement it because of the additional performance cost of such a test and because there is an official example here: https://docs.python.org/3/faq/programming.html#how-do-i-cache-method-calls
dsal3389 commentedon Aug 13, 2024
encountered it too, I wrote a module for it until python supports it officially, PR's are more then welcome
https://github.com/dsal3389/cachemethod
nmoreaud commentedon Aug 13, 2024
There are existing implementations in kids.cache and cachetools (more manual) packages
ktbarrett commentedon Aug 14, 2024
@nmoreaud kids.cache is not mantained and I don't see such a feature. And cachetools as mentioned is more manual.
@dsal3389 Ideally something could be made to work just like
cached_property, where it rewrites the attribute on the instance to be a cached method likecached_propertyrewrites the attribute to be the value that was computed.EDIT 1: Support
type(obj).attr(inst)by implementing__call__.EDIT 2: Use
__set_name__for setting the object name to supportattr = cached_method(lambda: stuff(123))EDIT 3:
In
__get__first try to look up the cached function call from the object since some builtin functions do some very funny lookups.For example,
len(a)does not call the equivalent ofgetattr(a, '__len__')(), it looks in the class first. And it doesn't do the equivalent ofgetattr(type(a), '__len__')(a)either,__get__is always invoked with the instance object. So without this trial getattr, the implementation would recreate the cache and function on every call oflen(a).ktbarrett commentedon Aug 14, 2024
Has anyone filed a report against linting tools to warn when
functools.cacheandfunctools.lru_cacheare used on methods? This is assuming that this will never be fixed in the language proper.AlexWaygood commentedon Aug 14, 2024
@ktbarrett, both flake8-bugbear and ruff already have this rule. The rule is B019 in both tools.
patrick-kidger commentedon Apr 29, 2025
For those like me who landed on this issue, here's a
cachemethodthat solves this.Notably, it:
selftypes without a__dict__;selftypes that are not hashable;SomeClass.some_cached_method(self, *args, **kwargs), not justself.some_cached_method(*args, **kwargs);dsal3389 commentedon Apr 29, 2025
@patrick-kidger
you might not keep references to the instance so it can be garbage collected, but you use the builtin
idfunction which is just returning the data position in memory (if I remember correctly) and what the default__hash__implementation also uses behind the sceneshttps://docs.python.org/3/library/functions.html#id
this will cause false cache hits if 2 different objects are allocated on the same memory location
this is also the reason the default
cachemethods keep reference to the data, so it won't be garbage collected thus it will be impossible for other instances to be allocated on the same memory locationI wrote a quick an dirty POC for this
the only solution is to bind the cache to the running instance (as a private property or something) and make the cache not hold references to
self, so when the instance is deleted the cache will also be deletedpatrick-kidger commentedon Apr 29, 2025
Is it possible for an object to GC'd, then a new one to be reallocated, before
weakref.ref(..., callback=...)runs?From the
weakref.refdocumentation, it states that the callback is evaluated when the wrapped object is about to be finalized. The wording of this suggests to me that the callback is evaluated immediately before the old object is GC'd, and therefore necessarily a new one cannot yet be allocated in the same memory location.dsal3389 commentedon Apr 29, 2025
totally missed that lambda callback while glancing over the code
ref = weakref.ref(key, lambda _: self._dict.pop(id_key))this looks like a good approach, I wonder why the stdlib doesn't really use it like u suggested, maybe a good question / suggestion for the python discourse?
ktbarrett commentedon Jun 4, 2025
@dsal3389 I think caching the self object, weakref or not, is unnecessary. It's doing more than it needs to.
dsal3389 commentedon Jun 4, 2025
@ktbarrett but if you don't cache the instance, how can you distinguish between cache hits that are coming from different instances? most of the time you don't want to mix them
here is a very simple example (theory) how it can hit you (assume
selfis not used in the cache key)ktbarrett commentedon Jun 4, 2025
You place the cache on the instance rather than keeping it in the class. This is how
cached_propertyworks (well it's a "cache" of 1).patrick-kidger commentedon Jun 4, 2025
This has the disadvantage of requiring that the instance be mutable in this way, which is the kind of sneakiness that can lead to unexpected behavior!
For example,
functools.cached_propertywill actually be pickled. This is unlike regularproperty- and is actually the reason that prompted me to write thecachemethodabove.ktbarrett commentedon Jun 4, 2025
Neither issues bothered the core devs when
@cached_propertywas merged. stdlib isn't some random library that can change frequently and attempt to handle everyone's case.Add a newsfile for pythongh-102618
Introduce a `functools.cached_method` decorator
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods
Replace functools.cache by wrapt.lru_cache
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods
Replace use of functools.cache/lru_cache by wrapt.lru_cache on methods (