You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
LazyImportType.resolve() does not replace the module global and can import twice #152298
Calling resolve() on a lazy import proxy forces the import and returns the resolved object, but it does not replace the original lazy object in the module globals. A later normal global access reifies the same lazy object again.
Hmm, is this a bug? I thought that since we have a cache in sys.modules, repeated imports are (intended to be) fast, and so resolve() doing a re-import is not a problem.
In the same way, it's not a problem that with lazy from module import a, b, c we can eventually import module three times (unlike the non-lazy variant). Is it?
@encukou I did a little digging to try and quantify if addressing this change actually provides value and it looks like the case where it helps the most is if resolve() is called repeatedly (which may be a scenario that is most likely hit during a stress test).
benchmark numbers
Benchmark
Base Average
PR Average
Verdict
reify_import — first access of 2,000 lazy import bindings
0.795 ms
0.773 ms
within noise (+0.1–0.3% instructions)
reify_from — first access of 2,000 lazy from bindings
0.270 ms
0.271 ms
within noise (+0.1–0.2% instructions)
steady_global — 1M reads of a resolved lazy global
17.654 ms
17.374 ms
within noise
resolve_repeat — 100k proxy.resolve() calls on one proxy
plausible-ish scenario where the improvement could come into play
importannotationliblazyimportjson# A handler signature where one annotation names a type that only exists for type checkers.defhandler(request: TypeCheckingOnly, decoder: json.JSONDecoder) ->None: ...
# e.g. a web framework, CLI library or DI container inspecting handlers repeatedlyfor_inrange(1000):
annotationlib.get_annotations(handler, format=annotationlib.Format.FORWARDREF)
Without the PR, we materialize json via resolve 1000x and with the change, it happens once and is cached.
Granted for this codepath to even trigger, we need a function with mix of annotations that can and cannot be accessed at runtime to trigger the ForwardRef codepath, we need json to not be loaded prior to being used as an annotation, and we need the codepath (handler) in this case to be hit again and again...and even in that case, the raw time savings isn't huge...it just feels "more correct" to load it once than on every single hit.
With all that said, I'm glad to drop the PR + we can close the issue if it seems mostly moot and just added complexity for not enough benefit. Just wanted to take a holistic pass on making sense around if this is worth addressing
I read that as “it's possible to deliberately bypass caches”. Adding a third one on top of sys.modules & _PyLazyImport_Reify/_PyDict_ReplaceItemIf still seems unnecessary to me.
In your scenario, the get_annotations caller should add a cache. The function's docs describe an expensive uncached operation, even if they aren't explicit about it.
Activity
1sgtpepper commentedon Jun 26, 2026
I'm working on a fix for this.
pythongh-152298: Regenerate lazy import test cases
pythongh-152298: Cover module attribute lazy reification
pythongh-152298: Clarify lazy resolve owner cleanup
pythongh-152298: Bind lazy owners after global stores
pythongh-152298: Bind lazy import owner before storing
pythongh-152298: Restore post-store lazy owner binding
pythongh-152298: Protect pending lazy owner bindings
pythongh-152298: Finalize pending lazy owners on commit
pythongh-152298: Keep pending lazy owners unfinalized
pythongh-152298: Cover reentrant lazy owner finalization
pythongh-152298: Snapshot lazy proxy in reentrant test
pythongh-152298: Require owner claim for explicit lazy resolve
21 remaining items
pythongh-152298: Rerun CI
pythongh-152298: Preserve lazy submodule sentinel errors
pythongh-152298: Preserve lazy import global bindings
pythongh-152298: Rerun CI
pythongh-152298: Guard lazy resolve writeback reentry
Reify each lazy import at most once
brittanyrey commentedon Sep 18, 2026
Putting up a new PR to try to address this. #157730
encukou commentedon Sep 28, 2026
Hmm, is this a bug? I thought that since we have a cache in
sys.modules, repeated imports are (intended to be) fast, and soresolve()doing a re-import is not a problem.In the same way, it's not a problem that with
lazy from module import a, b, cwe can eventually importmodulethree times (unlike the non-lazy variant). Is it?brittanyrey commentedon Oct 7, 2026
@encukou I did a little digging to try and quantify if addressing this change actually provides value and it looks like the case where it helps the most is if
resolve()is called repeatedly (which may be a scenario that is most likely hit during a stress test).benchmark numbers
reify_import— first access of 2,000lazy importbindingsreify_from— first access of 2,000lazy frombindingssteady_global— 1M reads of a resolved lazy globalresolve_repeat— 100kproxy.resolve()calls on one proxystartup_argparse—import argparse+ parse (subprocess)plausible-ish scenario where the improvement could come into play
Without the PR, we materialize
jsonviaresolve1000x and with the change, it happens once and is cached.Granted for this codepath to even trigger, we need a function with mix of annotations that can and cannot be accessed at runtime to trigger the
ForwardRefcodepath, we need json to not be loaded prior to being used as an annotation, and we need the codepath (handler) in this case to be hit again and again...and even in that case, the raw time savings isn't huge...it just feels "more correct" to load it once than on every single hit.With all that said, I'm glad to drop the PR + we can close the issue if it seems mostly moot and just added complexity for not enough benefit. Just wanted to take a holistic pass on making sense around if this is worth addressing
encukou commentedon Oct 8, 2026
Thank you!
I read that as “it's possible to deliberately bypass caches”. Adding a third one on top of
sys.modules&_PyLazyImport_Reify/_PyDict_ReplaceItemIfstill seems unnecessary to me.In your scenario, the
get_annotationscaller should add a cache. The function's docs describe an expensive uncached operation, even if they aren't explicit about it.brittanyrey commentedon Oct 9, 2026
That makes sense, we can handle this corner case at the application level.
Want to close the issue + PR?