Repository navigation
gh-102618: Introduce a functools.cached_method decorator
#150002
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sirosen
wants to merge
22
commits into
python:main
Choose a base branch
from
sirosen:cached-method
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+255
−7
Open
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
a5b4b0d
Introduce a `functools.cached_method` decorator
sirosen 1425819
Refine the implementation of cached_method
sirosen 8ff6b31
Minor tweaks to cached_method docs
sirosen 30f839b
Remove cached_method locking logic
sirosen 9d109fa
Fix signature line in docs for cached_method
sirosen d26358a
Fix use of a latin abbreviation
sirosen 0686932
Update Doc/library/functools.rst
sirosen 262a53e
Update functools to 'lazy import weakref'
sirosen fffc5d3
Add `functools.cached_method` to What's New
sirosen 57cc653
Merge branch 'main' into cached-method
sirosen 06c2b18
Apply suggestions from code review
sirosen 365c484
Apply suggestion from @StanFromIreland
sirosen 981013b
Move `cached_method` above `cached_property`
sirosen 8086aa5
Check type of 'maxsize' in cached_method
sirosen a7f206c
Fix race condition in cached_method
sirosen 128ea42
Move the docstring for cached_method to public func
sirosen 126cfa5
Update Doc/whatsnew/3.16.rst
sirosen 4901223
Clarify `cached_method` behaviors per review
sirosen 017bb4b
Apply batched suggestions from code review
sirosen fd18b9e
Fix cached_method with a bound method type
sirosen bd51b09
Enhance docs for cached_method
sirosen 3a98f0f
Merge branch 'main' into cached-method
sirosen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 2 additions & 0 deletions
2
Misc/NEWS.d/next/Library/2026-05-18-11-34-16.gh-issue-102618.4y-SEm.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Added a new ``functools.cached_method`` decorator, which uses weakrefs and | ||
| avoids putting ``self`` into the cache. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about static methods, class methods, and properties?
I prefer that we explicitly rule these out in the docs or document how
cached_methodcomposes with@classmethod,@staticmethod, and@property, including which decorator orders are supported.A few points to consider:
Class methods with
@cached_methodvs. class methods with@lru_cache:This creates a separate cache for each class, whereas
@classmethod+@lru_cachealso works, but uses a shared cache whose keys includecls. In the latter caseclsmust be hashable, and refs toclsare kept in cache entries (which is mostly fine, as normally classes are long-lived and hashable).One limitation of the current composition is that
C.method.cache_info()andC.method.cache_clear()are unavailable.Could we clarify whether this combination is supported and decide how users should inspect or clear its caches in this case?
@staticmethod+@cached_methodproblems:Here, the identity of
xselects a separate cache, there is noCinstance involved. Ifxsupports weak references, this can appear to work, despite having different semantics from ordinary function caching.It also changes the calling convention, i.e.
C.method(x=obj)fails because the wrapper does not preserve the original function's first parameter name. A static method with no arguments fails too, as does one whose first argument cannot be weakly referenced, such as an integer or string.Nowadays you can do:
Could the documentation explicitly recommend this pattern? or
@cache, and discourage@staticmethodabove@cached_methodas unsuitable for general static methods? The inner decorator cannot generally detect the surroundingstaticmethodat decoration time.@property+@cached_method:(This example uses explicit slots because
dataclass(slots=True, weakref_slot=True)currently fails with this composition).This is now an alternative to
cached_propertywith different semantics. It works without an instance__dict__, whereascached_propertyrequires a mutable instance dictionary. It also preserves ordinary property setter behavior. On otherwise mutable instances, a getter-only property prevents assignment, whilecached_propertypermits overwriting the cached value. It would be nice to document (or rule out) this composition.Finally, putting
cached_methodaboveclassmethod,staticmethod, orpropertydoes not currently provide the corresponding descriptor behavior. It would help to document these unsupported orders, and consider clearer errors where the decorator can identify them.Explicit support for an outer
cached_methodcould be a separate API choice (which has its advantages, e.g. makingcached_methodidiomatic/intrinsic for methods, enforce only one way to make cached properties).Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My inclination is to say that
cached_methoddoes not support static methods or classmethods. I don't think there's any strong reason that someone would want it for those cases, if they're also aware of thecache/lru_cachedecorators. I'm also not sure how it could work even if we did want support, at least withcached_methodas the inner decorator. I think a documentation addition is sufficient.For
property, supporting that composition was intentional -- @jaraco suggested it early in the process for this feature. To me it makes sense, since the methods decorated by properties (all of getters, setters, and deleters) behave like ordinary instance methods.Although I like the idea of superseding
cached_property, as you note they have different APIs. I'm personally against any deprecation in the near term.I can work a bit on dedicated detection and error messages where possible. I think it should be possible to catch these all.
I'm open to it in the future, but if we can agree to start without it, I'd be happier.
I think
cached_methodmakes more sense as the inner decorator forproperty, and as the outer decorator forclassmethodandstaticmethod. But that is likely to confuse end users. And, as above, I'm not sure there's much compelling need forcached_method(classmethod(...)).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm trying hard to keep the docs short and easy to read, but I added a new paragraph and note-directive to address composition with
property,classmethod, andstaticmethod.I'm aware that it technically works in some of the classmethod and staticmethod cases, so I'm simply documenting it as "unsupported", and intentionally not specifying what that means exactly. I hope that's satisfactory. I think trying to detect and raise a TypeError isn't worthwhile, given that it can only be done in a subset of cases.