Skip to content

JIT doesn't trace exec()-generated functions called from hot loops #149564

Open

Description

@kimjune01

When a hot loop calls functions generated via exec(), the loop body gets traced but the callees don't. Each callee is individually cold (called ~60 times), but the call site is hot (5,000+ calls total across different callees).

Concrete case: tinygrad generates ~5,000 pattern matching functions at startup via exec(). They're called from a tight loop:

for _, match, _ in pats:
    if (ret := match(uop, ctx)) is not None: return ret

PYTHON_JIT=1 on 3.14.4 produces no improvement. The specializer works (LOAD_ATTR_SLOT fires correctly), but the JIT never compiles the callees.

Related: faster-cpython/ideas#738, #118093 (tier 2 entry at function calls).

It would be useful to either trace through exec()-generated callees at hot call sites, or detect aggregate hotness across callees sharing a call site.

Linked PRs

Activity

  1. anujbharambe commented on May 8, 2026

    @anujbharambe
    Contributor

    The aggregate hotness idea makes sense. If the call site is hot, treat that as a sign to compile whatever passes through it. My main worry is memory. You end up compiling thousands of functions that barely run and that will waste quite some memory for very little gain.
    Will throw up a PR anyways, lets see what the core devs think.

  2. added 2 commits that reference this issue on May 9, 2026
  3. anujbharambe commented on May 9, 2026

    @anujbharambe
    Contributor

    Wanted to get some input on the approach before resubmitting.
    The core idea was that when _COLD_EXIT fires and the exit target is a CALL instruction, set the callee's RESUME_CHECK_JIT counter to trigger so it gets traced on next execution. This handles the case where distinct exec'd functions keep failing _CHECK_FUNCTION_VERSION but never get their own traces.
    The concern is whether adding a _PyOpcode_Deopt check on every cold exit is worth it. Is this the right approach, or is there a better way to propagate hotness to callees here?
    cc @markshannon @brandtbucher @Fidget-Spinner

  4. Fidget-Spinner commented on May 9, 2026

    @Fidget-Spinner
    Member

    I think Python 3.15 JIT should fix this? Can you try out 3.15 beta 1's JIT from uv?

  5. anujbharambe commented on May 11, 2026

    @anujbharambe
    Contributor

    Tested on 3.15 beta 1's JIT via uv. The issue still reproduces. None of the exec'd callees get traced (TIER2_RESUME_THRESHOLD = 8192, each callee only gets ~60 calls). JIT is ~8% slower than interpreter due to _CHECK_FUNCTION_VERSION side exits.

  6. cocolato commented on Jul 2, 2026

    @cocolato
    Member

    IMO, the hotness propagation should be handled in _COLD_EXIT, which lets us handle both CALL and CALL_KW.

  7. added a commit that references this issue on Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-JITtype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions