Skip to content

Fix bug with recursive calls not being counted correctly - #444

Merged
Erotemic merged 6 commits into
pyutils:mainfrom
Theelx:fix-recursive-line-timing
Oct 17, 2026
Merged

Erotemic merged 6 commits into
pyutils:mainfrom
Theelx:fix-recursive-line-timing

Conversation

@Theelx

@Theelx Theelx commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator

Hey everyone! Sorry for the long absence from working on line profiler. When Python 3.15.0 released a few days ago, I remembered that they have a new profiling module, and that reminded me that line-profiler was a thing that I'd been meaning to speed up for a while. I have some speedups in store that will massively reduce the overhead from line-profiler. On 3.12, the code goes from running ~14.7x slower when being profiled to ~2.6x slower (geomean) across like 5 different scenarios, and there are similar speedups from 3.10 to 3.15. The public API in this optimized version is also fully compatible with the current API. I'm excited to share it, however it'll be a massive PR (currently +2023 lines and -850), so I want to split it up into a few smaller ones that are more targeted.

This PR contains the only bug fix that the speedups rely on. @notEvil found this bug and mentioned it in #230, and I chose to fix it here. Once this is merged, I can start sending in some PRs for the massive speedups!

@Theelx
Theelx requested review from Erotemic and TTsangSC and removed request for TTsangSC October 11, 2026 01:46
@Theelx
Theelx requested a review from TTsangSC October 11, 2026 01:50
@codecov

codecov Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.12%. Comparing base (b5ca752) to head (0c932ab).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #444      +/-   ##
==========================================
+ Coverage   84.95%   85.12%   +0.17%     
==========================================
  Files          21       21              
  Lines        2412     2413       +1     
  Branches      376      376              
==========================================
+ Hits         2049     2054       +5     
+ Misses        263      259       -4     
  Partials      100      100              

see 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 984ad72...0c932ab. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TTsangSC

TTsangSC commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Hi, welcome back! Just wanted to say that this looks exciting. But unfortunately I won't be able to do a proper review until a week later (~20th); currently in the middle of a messy relocation.1 Please see if this is fine with your timeline.

EDIT: speaking of 3.15, do you happen to know if the current Cython code works as-is for it? Or should we be changing something?

Footnotes

  1. hence the lack of progress on my recent PRs (Rehabilitating #431 (1/2?): threading fix + patching infrastructure #442 etc.) ↩

@Theelx

Theelx commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

Hey TTsang, if you can't do a proper review for a week then that's fine; this can wait. However, it may be useful if Erotemic can review this and if it's okay then merge it, and then the larger next PR can be reviewed by both of you. This PR is relatively short and sweet. For 3.15, the current Cython code seems to work as-is. I was testing 3.15.0 in my local test matrix earlier and it profiled fine after doing the requisite magic with tracing and such.

@Theelx

Theelx commented Oct 11, 2026

Copy link
Copy Markdown
Collaborator Author

Update on the speedup numbers I shared in the initial PR description:

Benchmark Type Scenarios 3.10 3.13 3.15 All versions
Sync Python tight_loop, many_calls, calls_unprofiled, recursive, generators 2.02x 6.58x 6.34x 4.38x
Async async_calls, async_tasks 1.67x 6.52x 6.58x 4.15x
Cython cy_tight_loop, cy_many_calls, cy_recursive 4.22x 11.55x 11.39x 8.22x
Overall all 10 above 2.42x 7.77x 7.61x 5.23x

Each of those numbers is a geometric mean reduction of overhead. 3.12 was really the turning point where the speedups began to be massive (I didn't measure 3.12 because Cython is broken on it). The sys.monitoring code that applies to 3.12 and above seems to have been designed without much thought put into reducing overhead, so there was tons of low-hanging fruit there that I picked. I'm excited to send in the speedup PRs once this is merged!

Once the speedups are done, I plan to work on making sure line_profiler behaves under the free-threaded build of Python. This will be a major challenge (so it will take a while), but I'll figure it out!

@TTsangSC

Copy link
Copy Markdown
Collaborator

without much thought put into reducing overhead

Happen to know the guy who did that part and can tell you he didn't ;)
... yeah, should've focused a bit a lot more on performance looking back at it. Lesson learned.

free-threaded build

In principle it (at least the Cython core) seems that it should be alright, since each thread should only interact with its own data via threading.local and dicts1 (until the .get_stats()), but we'll see. Have a feeling that the hard part will be figuring out what cases would be problematic and thus needs testing and fixing.

That said, I remember that our Cython core is currently incompatible with multiple interpreters (e.g. concurrent.futures.InterpreterPoolExecutor), after unsuccessfully taking a crack at that as a part of #431. Perhaps your fix will end up encountering the same issues and eventually ironing them out...?

Footnotes

  1. Which however calls into question whether the dicts' intrinsic locks will end up negating the potential perf gain. ↩

@Theelx

Theelx commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

Hey @Erotemic, do you happen to have time to take a look at this? Totally fine if not, I was just hoping to get to the overhead reductions and speedups sometime soon so that other people can benefit, and this is a bit of a blocker. Also, looking at your activity graph, I think it could be useful to give TTsang commit and merge permissions so that you don't have to spend as much time on this project. TTsang has been consistently active here for a long time and seems dedicated to the success of line_profiler, and it could help reduce any burden on you.

@Erotemic

Copy link
Copy Markdown
Member

@Theelx This looks good to merge. I don't see anything obviously incorrect, and the motivation makes sense to me. We need to know the time at the start and end of a particular code line, and currently we use the hash of its code (modulo details) to do that. But if a line ends up making a recursive call to itself, the inner scope can record its own measurement of that line before the outer scope has finished, and because both lines have the same code, they have the same key, and thus the inner scope clobbers the outer scope. So instead we are using the address of the Python frame (which contains the code object), so that keys on both the code and the context it was executed in. Then later we can correctly aggregate across multiple frames that measure the same code.

I had GPT 6 in chat mode review the PR and I've had it distill the discussion into a list of suggestions to improve the PR quality. Your choice if you take any of these on. I think points 1, 2, and 4 look like they are worth it. 6 is a nitpick that doesn't bother me, but its probably right. Point 3 I think indulges the LLM tendency to over-test things, but its not a terrible idea. Point 5 doesn't seem worth it if it costs additional complexity, and the proposed solution might cause memory churn, so it might not be the right way to address the issue.

Details

🤖

1. Avoid redundant hash-map lookups (performance)

Currently, inner_trace_callback does:

if deref(last_map).count(frame_key):
    old = deref(last_map)[frame_key]

That performs two lookups. The exit path similarly checks count() before calling last_erase_if_present(), which searches again.

Consider finding the entry once and reusing the iterator:

cdef LastTimeMap.iterator it
it = deref(last_map).find(frame_key)

has_last = it != deref(last_map).end()
if has_last:
    old = deref(it).second
    has_last = old.block_hash == block_hash

# On exit, reuse the same iterator:
if not is_line_event and it != deref(last_map).end():
    deref(last_map).erase(it)

The line-event update could similarly use the iterator for an existing entry, avoiding another lookup through operator[]. This reduces overhead in the profiler's hot path.

2. Clarify what frame_key represents

This variable actually contains two kinds of identifiers:

if py_frame != NULL:
    frame_key = <int64><intptr_t>py_frame
elif is_cython:
    frame_key = <int64>block_hash
else:
    frame_key = <int64><intptr_t>get_current_frame_key()

Consider renaming it to last_time_key and documenting that it represents frame identity for Python but code identity for Cython under sys.monitoring.

Technically, a frame address and code hash could have the same integer value, causing their timing records to collide. Separate maps or tagged keys would eliminate that possibility, but I wouldn't introduce extra hot-path complexity unless the risk warrants it.

3. Test frame lifetimes beyond ordinary recursion

The existing regression test covers nested recursive calls. I'd also check generators/coroutines that suspend and resume, since multiple executions of the same code can be interleaved.

For example:

prof = LineProfiler()

@prof
def gen():
    yield 1
    yield 2

with prof:
    a, b = gen(), gen()
    assert next(a) == 1
    assert next(b) == 1
    assert next(a) == 2
    assert next(b) == 2

The test should verify the resulting line hit counts and timings, not just successful execution.

Other worthwhile cases:

  • Exceptions that propagate out of a frame.
  • Generator closure and coroutine cancellation.
  • Repeated calls where frame addresses may be reused.
  • Both legacy tracing and sys.monitoring.

The concern is whether a stale record could survive and later be attributed to a different invocation, especially if an address is reused for the same code.

4. Avoid converting all threads' data in c_last_time

Currently:

records = (<dict>self._c_last_time)[
    PyThread_get_thread_ident()
]

This converts the entire outer C++ map to Python dictionaries before selecting the current thread.

Instead, consider looking up the thread first:

thread_it = self._c_last_time.find(
    PyThread_get_thread_ident()
)

if thread_it == self._c_last_time.end():
    raise KeyError(...)

# Iterate the current thread's LastTimeMap directly
# and build only the required Python dictionary.

This would require typed C++ iterator handling in Cython, rather than another <dict> conversion.

Also, the new documentation says c_last_time selects the "innermost" recursive frame, but the code actually selects the record with the latest timestamp:

if current is None or record['time'] > current['time']:
    c_last_time[block_hash] = record

Those aren't necessarily equivalent for suspended/interleaved executions. I'd either clarify the documentation or explicitly establish the intended selection semantics.

5. Consider retained memory after deep recursion

With frame-based keys, LastTimeMap can now grow proportionally to the number of active recursive invocations.

Although entries are erased and clear() is called on disable, std::unordered_map generally retains its allocated bucket capacity.

If a long-lived profiler once observes deep recursion, it may retain that memory for the rest of its lifetime.

One possible approach is to conditionally release large bucket arrays after profiling stops. Conceptually, in C++:

if (last_map.empty() &&
    last_map.bucket_count() > threshold) {
    LastTimeMap empty;
    last_map.swap(empty);
}

The old allocation is released when empty is destroyed. I'd only add this if memory measurements show it matters; retaining capacity may be beneficial for subsequent profiling sessions.

6. Clarify the changelog

The recursion fix applies to normal Python functions, but recursive Cython calls under sys.monitoring still use code-block keys.

I'd change the entry to something like:

* FIX: Corrected line timing for recursive Python function calls
  by tracking timing state per execution frame. (#444)

Overall, my highest-priority suggestions are #1 (hot-path efficiency), #2 (clear semantics), and #3 (lifecycle tests). The others are lower priority and could reasonably be deferred.

I don't see an obvious new use-after-free from using frame addresses as integer keys, because the stored addresses are not dereferenced. The primary remaining concern is ensuring timing records are cleaned up correctly throughout the frame lifecycle.

Also, because you are thinking of free-threading, in the discussion it flagged:

cdef unordered_map[int64, LastTimeMap] _c_last_time
cdef unordered_map[int64, LineTimeMap] _c_code_map

As not synchronized and shared across multiple threads.

It also flagged that get_current_frame_key is using PyThreadState_Get()->current_frame, which is not part of the stable public API, but its also not the first place in the code where we rely on the non-public API, so just something to note (which also reminds me we don't have 3.15 in the test matrix, which I need to fix).

It also raised a pre-existing problem, which I've opened a separate issue for: #445

@Erotemic Erotemic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider if you want to implement any of the GPT suggested improvements, here. But this can be merged as is. So make the changes or let me know if you want to defer.

@Theelx

Theelx commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator Author

Hey @Erotemic, thanks for the suggestions! Suggestions 1 and 4 are already handled by my speedup branches that will come shortly after this is merged I'll implement 2, 3, and 6 now.

@Erotemic
Erotemic merged commit e7e93af into pyutils:main Oct 17, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants