You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
JIT doesn't trace exec()-generated functions called from hot loops #149564
When a hot loop calls functions generated via exec(), the loop body gets traced but the callees don't. Each callee is individually cold (called ~60 times), but the call site is hot (5,000+ calls total across different callees).
Concrete case: tinygrad generates ~5,000 pattern matching functions at startup via exec(). They're called from a tight loop:
for_, match, _inpats:
if (ret:=match(uop, ctx)) isnotNone: returnret
PYTHON_JIT=1 on 3.14.4 produces no improvement. The specializer works (LOAD_ATTR_SLOT fires correctly), but the JIT never compiles the callees.
The aggregate hotness idea makes sense. If the call site is hot, treat that as a sign to compile whatever passes through it. My main worry is memory. You end up compiling thousands of functions that barely run and that will waste quite some memory for very little gain.
Will throw up a PR anyways, lets see what the core devs think.
Wanted to get some input on the approach before resubmitting.
The core idea was that when _COLD_EXIT fires and the exit target is a CALL instruction, set the callee's RESUME_CHECK_JIT counter to trigger so it gets traced on next execution. This handles the case where distinct exec'd functions keep failing _CHECK_FUNCTION_VERSION but never get their own traces.
The concern is whether adding a _PyOpcode_Deopt check on every cold exit is worth it. Is this the right approach, or is there a better way to propagate hotness to callees here?
cc @markshannon@brandtbucher@Fidget-Spinner
Tested on 3.15 beta 1's JIT via uv. The issue still reproduces. None of the exec'd callees get traced (TIER2_RESUME_THRESHOLD = 8192, each callee only gets ~60 calls). JIT is ~8% slower than interpreter due to _CHECK_FUNCTION_VERSION side exits.
Activity
anujbharambe commentedon May 8, 2026
The aggregate hotness idea makes sense. If the call site is hot, treat that as a sign to compile whatever passes through it. My main worry is memory. You end up compiling thousands of functions that barely run and that will waste quite some memory for very little gain.
Will throw up a PR anyways, lets see what the core devs think.
pythongh-149564: Add _COLD_EXIT callee RESUME boost
pythongh-149564: Fix _COLD_EXIT boost - rename oparg, drop CALL_KW
anujbharambe commentedon May 9, 2026
Wanted to get some input on the approach before resubmitting.
The core idea was that when
_COLD_EXITfires and the exit target is aCALLinstruction, set the callee'sRESUME_CHECK_JITcounter to trigger so it gets traced on next execution. This handles the case where distinct exec'd functions keep failing_CHECK_FUNCTION_VERSIONbut never get their own traces.The concern is whether adding a
_PyOpcode_Deoptcheck on every cold exit is worth it. Is this the right approach, or is there a better way to propagate hotness to callees here?cc @markshannon @brandtbucher @Fidget-Spinner
Fidget-Spinner commentedon May 9, 2026
I think Python 3.15 JIT should fix this? Can you try out 3.15 beta 1's JIT from uv?
anujbharambe commentedon May 11, 2026
Tested on 3.15 beta 1's JIT via uv. The issue still reproduces. None of the exec'd callees get traced (
TIER2_RESUME_THRESHOLD= 8192, each callee only gets ~60 calls). JIT is ~8% slower than interpreter due to_CHECK_FUNCTION_VERSIONside exits.cocolato commentedon Jul 2, 2026
IMO, the hotness propagation should be handled in
_COLD_EXIT, which lets us handle bothCALLandCALL_KW.pythongh-149564: Make polymorphic JIT calls opaque