Skip to content

Further speed improvements - #450

Open
Theelx wants to merge 5 commits into
pyutils:mainfrom
Theelx:faster-callback-wrapping
Open

Theelx wants to merge 5 commits into
pyutils:mainfrom
Theelx:faster-callback-wrapping

Conversation

@Theelx

@Theelx Theelx commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

This PR contains the last batch of speedups that I have so far. Benchmarks are below. The main improvements come from using structs and arrays rather than hashmaps, with some smaller Linux-specific improvements from speeding up the clock reading when possible. Avoiding hashmaps does come with an increase in complexity, but it's something like 70% of the total overhead reduction from this PR. Also, by removing hashmaps, it scales with core count far far better on free-threading builds! Currently, the free-threaded builds don't fully work with line_profiler due to races that cause certain line timings to be lost and occasional crashes (though free-threaded builds have never worked with line_profiler AFAIK), however I have ideas to fix those that I will work on in parallel to TTsang's work on supporting child processes.

Default settings (wrap_trace=False) (overhead reduction vs. main)

Type Scenarios 3.10 3.13 3.15 All versions
Sync Python tight_loop, many_calls, calls_unprofiled, recursive, generators 1.78x 1.80x 1.88x 1.82x
Async async_calls, async_tasks 1.42x 1.52x 1.57x 1.50x
Cython cy_tight_loop, cy_many_calls, cy_recursive 2.56x 2.36x 2.44x 2.45x
Overall all 10 above 1.90x 1.89x 1.96x 1.92x

Wrapping another tool's callbacks (wrap_trace=True) (overhead reduction vs. main)

Type Scenarios 3.10 3.13 3.15 All versions
Sync Python tight_loop, many_calls, calls_unprofiled, recursive, generators 1.30x 5.59x 5.71x 3.46x
Async async_calls, async_tasks 1.19x 5.68x 5.76x 3.39x
Cython cy_tight_loop, cy_many_calls, cy_recursive 1.30x 6.35x 6.37x 3.75x
Overall all 10 above 1.28x 5.83x 5.91x 3.53x

Default settings (wrap_trace=False) (overhead reduction vs. no profiling at all)

Type Scenarios 3.10 3.13 3.15 All versions
Sync Python tight_loop, many_calls, calls_unprofiled, recursive, generators 6.22x → 3.60x 5.84x → 3.21x 5.72x → 3.08x 5.93x → 3.29x
Async async_calls, async_tasks 4.51x → 3.13x 2.86x → 1.89x 2.97x → 1.87x 3.37x → 2.23x
Cython cy_tight_loop, cy_many_calls, cy_recursive 57.71x → 22.28x 36.35x → 15.54x 36.67x → 14.92x 42.53x → 17.28x
Overall all 10 above 11.38x → 6.05x 8.76x → 4.63x 8.76x → 4.48x 9.56x → 5.01x

As you can see, the overall performance overhead from using line_profiler is made significantly smaller by this change. I'd hazard a guess that it's the most efficient non-sampling profiler for Python with these changes!

@TTsangSC Could you review this sometime in the next week or so? Once I address the review comments and get this merged, I'd love to work with you with the child process thing and with free-threading support.

@TTsangSC

Copy link
Copy Markdown
Collaborator

Exciting stuff! I'll try to review this ASAP (probably over the weekend – no promises tho), and am looking forward to your working your optimizing magic on the child-process feature.

(Probably a good idea to get these landed during the honeymoon period at the new job, before expectations and duties ramp up and I get too swamped ;))

@Theelx

Theelx commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Sounds good! I may actually be able to get the free-threading working before you get to review this (depends on how bad it ends up being). If that happens, and if the free-threading changes aren't massive, then I'll just push the free-threading changes here into this PR so that you don't have to review multiple PRs spread out of a lot of time.

@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.40%. Comparing base (dbc38f3) to head (81e3551).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #450   +/-   ##
=======================================
  Coverage   81.40%   81.40%           
=======================================
  Files          21       21           
  Lines        2484     2485    +1     
  Branches      410      410           
=======================================
+ Hits         2022     2023    +1     
  Misses        347      347           
  Partials      115      115           
Files with missing lines Coverage Δ
line_profiler/_diagnostics.py 73.91% <100.00%> (+1.18%) ⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 958d4a8...81e3551. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Theelx

Theelx commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Update: the free threading changes are looking like they'll be pretty big (I see a net addition of roughly 1000 lines being likely) so I suspect that I'll need to make a new PR on top of this one. Hopefully I'll be able to get the changes done by Sunday :)

@Erotemic

Copy link
Copy Markdown
Member

Extremely impressive results. Avoiding hashmaps in favor of structs and arrays as an optimization makes sense.

As usually I'm throwing the PR at GPT 6.1-Sol in an initial pass. It's findings do raise some concerns, particularly how there seems to be an introduced dependence between the amount of memory allocated and the physical line number in the file. Although, I do wonder how much it matters as how often will line numbers in a file get so large to the point where it becomes an issue? Still, I think its useful to flag.

Details

🤖

Code review: pyutils/line_profiler PR #450

Open

Author: Theelx · 15 files · +950 / −377 lines

This is a substantial optimization of the profiling engine, rather than a small performance patch. The main architectural change replaces several hash-map-based data structures with C++ structs, arrays, and cached pointers. It also changes timer handling and optimizes callback dispatch.

The author reports the following aggregate improvements compared with main:

Workload Reported speedup
Normal profiling (wrap_trace=False) 1.92×
Wrapping other tracing callbacks (wrap_trace=True) 3.53×
Cython workloads 2.45×
Async workloads 1.50×

These are author-supplied benchmark results, rather than measurements I have independently reproduced.

The important review question is whether the reduced profiling overhead justifies the additional native-state complexity without compromising the accuracy and safety of profiling results.

Findings

P1

Dense line storage can allocate memory proportional to the source file's absolute line number

_profile_data.hpp:57–74 · _profiler_base.pyx:307–308, 338–340

The new ProfileEntry::register_line() stores line timings in a vector indexed relative to first_line. However, registration includes the -1 sentinel returned by PyCode_Addr2Line() for bytecode with no associated source line.

This is a real case for functions containing exception handling and generators. I verified on CPython 3.13 that an ordinary try/except function returns line numbers including -1.

If such a function is defined near line 100,000 of a file, its vector must span approximately 100,000 elements, even if the function itself contains only five lines.

Illustrative allocation for one function with a −1 sentinel

0 MiB7 MiB14 MiB21 MiB28 MiBLine 1,000Line 10,000Line 100,000Line 1,000,000

Assumes 24 bytes per LineData on a typical 64-bit platform; excludes vector and allocator overhead.

The old hash-map representation did not allocate entries for all intervening line numbers.

Requested change: Exclude non-executable -1 sentinel locations from the dense representation, or handle them separately. Add regression tests using a short function placed far down in a generated source file, including exception handling and generators.

Source: registration and vector resizing

P1

Legacy tracing loses events when profiling is enabled within an existing frame

_line_profiler.pyx:1191–1198 · c_trace_callbacks.c:275–280

On PyTrace_CALL, the new implementation sets frame->f_trace_lines = 0 when the function is not currently profiled.

Suppose function outer() begins while another function is being profiled. Halfway through outer(), a helper registers outer() with the active profiler. The remaining lines of that frame will not generate trace events because its f_trace_lines flag has already been cleared.

I reproduced this underlying CPython behavior using sys.settrace:

Trace behavior Later line events
Previous behavior, with line events left enabled Observed
New behavior, with f_trace_lines=0 Missing

Requested change: Preserve the ability to register functions during execution, or explicitly handle restoring line events in active frames. Add a legacy-mode regression test for add_function() during execution.

Source: new legacy trace suppression

3. Permanent growth of the global block registry

Priority: P2 — memory lifecycle

The new _blocks registry in _line_profiler.pyx stores heap-allocated Block objects. The implementation explicitly retains them permanently so cached CodeCache.block pointers remain valid.

LineProfiler.__dealloc__() deletes the associated ProfileEntry objects but leaves their Block objects allocated.

That means repeated profiling of newly generated code can grow native memory even after the relevant profiler instances are destroyed.

This is separate from the dense-array issue above.

I would ask for either a reclaimable registry design with cache invalidation or a documented, measured bound on retained memory for realistic dynamic-code workloads.

Source: global block registry

4. Native behavior needs targeted regression coverage

Priority: P2 — testing

The only changed test file adds a helper docstring, a section heading, and a documentation correction. It does not introduce tests for the new native storage, timer selection, or legacy frame optimization.

The existing CI is encouraging: the BinPyCI workflow completed successfully across Linux, macOS, Windows, and multiple Python versions. Codecov reports 81.40% project coverage, but this figure does not establish coverage of the new C++ implementation.

I would add focused tests for:

  • Dense storage with high source line numbers, exception handlers, and generators.
  • Legacy tracing with dynamic function registration and recursive calls.
  • Profiler creation/destruction with many different code blocks, measuring native memory.
  • LINE_PROFILER_TSC=0 versus enabled timing, with numerical accuracy comparisons and cross-core execution.
  • Multiple active profilers sharing code, including enabling and disabling them independently.

What I like about the implementation

There is substantial work worth preserving here. The main performance improvements have a coherent technical basis:

  • Removing hashing from the common line-event path should reduce per-event overhead.
  • Cached code-object metadata avoids repeated bytecode inspection.
  • The C++ ProfileEntry design provides efficient access for common single-threaded workloads.
  • Avoiding unnecessary line-number lookups on exit events is a sensible optimization.
  • TSC selection is guarded by both CPUID information and the Linux clocksource, with a configuration switch to disable it.
  • The build configuration includes the new C++ header dependency.

I would not request a wholesale rewrite. The dense representation is useful once the sentinel handling and memory behavior are corrected.

Merge recommendation

Request changes before merging.

The two P1 issues are concrete regressions worth fixing. The persistent block registry should also be addressed or bounded before the optimization is released for long-running applications.

The author has already acknowledged that free-threaded builds can lose timing data and crash, and is planning a separate PR for that work. I would keep that effort separate rather than expand this PR substantially.

The performance gains are worth pursuing. My preferred path is to fix the two correctness/resource issues, add targeted tests, rerun benchmarks, and merge the optimization without coupling it to the free-threading implementation.

I have not submitted any comments or review decisions to GitHub.

I did ask it about potential directions to keep the struct / array improvements while avoiding this line-number dependence and have its response:

Details

🤖

The struct/array optimization looks like a worthwhile improvement, and I'd like to preserve the fast path it introduces. I do have one concern about the dense line-number indexing in ProfileEntry.

Currently, register_line() allocates LineData entries for the entire range between the smallest and largest registered line numbers. In particular, PyCode_Addr2Line() can return -1 for bytecode without an associated source line. Registering that alongside a function at line 5,000 could allocate roughly 5,000 entries for a function with only a handful of executable lines.

Filtering out -1 before registration should resolve the immediate problem. However, large gaps between valid source line numbers can still cause disproportionate allocations.

I think an elegant solution would be to separate line-number lookup from timing-data storage:

  • Keep timing records in a compact std::vector<LineData>, with stable slot indices.
  • Introduce a small LineIndex abstraction that maps source line numbers to timing slots.
  • Use direct offset indexing for ordinary functions with dense line ranges, preserving the current fast path.
  • Fall back to a sparse representation (such as a sorted vector with binary search) when the line-number span is disproportionately large.
  • Select the representation during registration, keeping the additional complexity out of the tracing callback.

Stable timing-slot indices would also remove the need to adjust LastLine.line_index when registering a line before the existing range.

I wouldn't optimize specifically for million-line source files, which are unlikely to be representative. The concern is that fairly ordinary modules can have functions with noncontiguous source locations, and the memory allocation shouldn't depend heavily on their absolute line numbers.

This seems achievable without sacrificing the performance benefits of the PR. I'd suggest starting with the -1 fix, then benchmarking a bounded dense/sparse LineIndex against the current implementation. If the sparse fallback adds negligible cost to ordinary profiling, I think it would make the new representation substantially more robust.

@Theelx

Theelx commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review! I've now pushed another two commits to address the issues it raised. Can you check the current state again to see if it looks reasonable?

@Erotemic

Copy link
Copy Markdown
Member

GPT 6 re-review:

Details

🤖

Re-review: line_profiler PR #450
Substantially improved
Reviewed head: 9e055ca · November 14, 2026

The author has pushed two additional commits specifically addressing our review. The changes follow our proposed design closely, including the adaptive LineIndex, stable timing slots, reclaimable block storage, and regression tests.

My assessment is considerably more favorable now. The original memory-allocation objection is resolved architecturally. The remaining question is whether the new lifetime and trace-restoration logic introduces edge cases.
What changed since our first review
Previous finding New implementation Assessment
Dense arrays expand because of -1 line numbers Separates negative line numbers from timing records Resolved
Large gaps between source lines waste memory Adaptive dense/sparse LineIndex Resolved
Existing frames lose tracing after registration Restores line events and handles resumed generators Largely resolved
Global Block objects never freed Removes unused blocks and invalidates cached pointers Resolved for GIL builds
Missing regression tests Adds 273 lines of focused tests Improved

The changes are concentrated in seven files, adding approximately 514 lines and removing 54 relative to the previous head.

  1. The adaptive indexing design is sound
    The new _profile_data.hpp introduces LineIndex, which selects between direct array indexing and binary search based on the line-number distribution.

It also maintains stable slot identifiers. This eliminates the previous need to shift LastLine.line_index values when registering an earlier source line.

The design now has the property we wanted: storage is bounded by the number of registered lines, with a fixed allowance for gaps, rather than growing with the absolute source line number.

The author also preserves the -1 line registration separately for compatibility without letting it distort the dense index.

I would accept this design.

View the revised LineIndex implementation
2. The new test coverage addresses the original concerns
The added tests exercise both tracing backends and include:

  • A function near source line 1,000,000, with a separate 20,000-line gap.
  • Exception handling and generators.
  • Registering a function while recursive calls are already executing.
  • Registering a suspended generator before resuming it.
  • Enabling a profiler while the target frame is already active.
  • Multiple profilers sharing the same function.
  • Deallocating profilers and checking that registered blocks are reclaimed.
  • TSC enabled/disabled timing comparisons, including CPU-affinity changes where supported.
    I particularly like the registry reclamation test: it creates 200 dynamically generated functions, destroys their profilers, checks the block count returns to baseline, and then profiles a function again to exercise cache invalidation.
    Latest CI run
    30/30
    Jobs successful
    Updated regression suite
    273
    Lines added

The new tests are substantially more relevant to the native changes than the original PR's test diff. They passed across the CI environments, although I haven't run an independent local benchmark.
3. One concern remains in the frame-restoration logic
P2 — performance and tracing semantics
Source: _line_profiler.pyx, restore_line_events()

The new implementation restores line events when:
not frame.f_trace_lines and get_block(frame.f_code) is not None

But get_block() only establishes that some profiler has registered that code block.

It does not establish that any profiler for that block is currently active on the relevant thread.

Consider two profilers:

  • Profiler A is active and tracing an unrelated function.
  • Profiler B has registered worker() but is currently disabled.
    When worker() starts, the legacy trace callback correctly suppresses its line events because no active profiler needs them.

Later, registering another function can trigger restore_line_events(). Because worker() is present in the global block registry, its line events can be reenabled even though profiler B remains disabled.

The rest of that invocation can generate line callbacks that simply discover nobody wants them. For a long-running loop, this defeats part of the optimization.

There is also a narrower semantic concern: this restoration can overwrite an explicit frame.f_trace_lines=False setting because the implementation doesn't record which component originally disabled the events.

I would prefer restoring only frames whose code is actually profiled by an active profiler on their thread. Ideally, restoration should also distinguish flags disabled by line_profiler from those disabled externally.

This is a code-path finding, not an independently reproduced failing test. I would request a focused test with two profilers, one active and one inactive, before calling it a confirmed regression.
4. A performance question worth checking
The PR description still contains the original benchmark numbers, despite the data-structure changes.

I would like to see the benchmark suite rerun against the updated implementation. In particular, the new index retains a sorted vector even in dense mode, and LastLine still uses linear searches through active frame records. The latter could become expensive with deep recursion, even though ordinary workloads should remain fast.

Neither is a demonstrated regression. A recursion-depth sweep and refreshed benchmarks would establish their practical cost.
Revised merge recommendation
Previous recommendation

Request changes

Current recommendation

Close to approval

I would no longer block this PR over the dense-array memory issue, the block registry lifetime, or the original active-frame tracing problem. The author has supplied appropriate implementations and regression tests for those concerns.

My remaining requests would be comparatively small: verify that restore_line_events() only reenables frames that actually need profiling, and rerun the performance benchmarks after the new indexing implementation.

I would also keep free-threading support separate, as the author proposed. It remains a known limitation of the current profiling engine, and adding concurrency changes now would complicate the review substantially.

Bottom line: The author has responded constructively and implemented essentially the architecture we recommended. I think the core changes are now reasonable to merge after resolving or ruling out the frame-restoration concern.

Looks like much of the issues were addressed. It's pointing out 2 more minor points. The frame restoration issue it mentions looks like a correct find. It also probably makes sense for @TTsangSC (if you have time) to take a pass at it before I actually dive into my final review and read the code.

@Theelx

Theelx commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

Ok, I've implemented a fix for the first issue. The second issue is just the AI asking for updated benchmarks; I re-ran them locally after this latest commit and found no difference above the noise floor of ~1.5% against the last time that the numbers were reported.

Comment thread line_profiler/timers.c Outdated
char clocksource[64] = {0};
FILE *f;
if (!__get_cpuid(0x80000007, &eax, &ebx, &ecx, &edx)
|| !(edx & (1u << 8)))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is maybe a bit too magical and may need more doc – or I'm just too noob when it comes to real close-to-metal stuff (among other things). Looking at Wikipedia, bit index 8 on edx is supposed to be for "AVX-512 vector intersection instructions on 32/64-bit integers", which doesn't quite seem to relate to the situation at hand.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops didn't mean to comment this soon. Anyway the rest of the review will come later today.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit 8 of edx only means AVX-512 under leaf 7, but with leaf 0x80000007 it means invariant TSC. That being said, I agree that it was a bit too magic, so I've pushed a commit that should make it more clear.

@TTsangSC

Copy link
Copy Markdown
Collaborator

Sorry but the read has been going slower than I expected., but I hope to have all my comments ready by Thursday.

@TTsangSC TTsangSC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing stuff, really appreciate the various savings you're making here.

As with the last time, these are maybe less review-y than they are discussions:

Comment on lines +52 to +74
void insert(int lineno, int slot)
{
auto pair = std::make_pair(lineno, slot);
sorted.insert(
std::lower_bound(sorted.begin(), sorted.end(), pair), pair);
int first = sorted.front().first, last = sorted.back().first;
size_t span = (size_t)last - (size_t)first + 1;
if (span > MAX_SPAN_PER_LINE * sorted.size() + MIN_DENSE_SPAN) {
// Too sparse for a table
dense = false;
std::vector<int>().swap(table);
} else if (dense && first == first_line) {
if (span > table.size()) table.resize(span, -1);
table[lineno - first_line] = slot;
} else { // (Re)build the table
dense = true;
first_line = first;
table.assign(span, -1);
for (const auto &registered : sorted) {
table[registered.first - first] = registered.second;
}
}
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can explicitly trigger this resizing behavior in tests? Or is that impossible due to how each LineIndex is only associated with one ProfileEntry, and thus one Block which is only scanned once?

# event: stop it from firing here at all
note_events_disabled()
return _DISABLE
return on_line_event(self, code, lineno)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I follow why on_line_event() and on_exit_event() are spun off into their own functions, but I suppose it has to do with our previous discussions on def-ing instead of cpdef-ing these methods I guess? Like we still get one extra dispatch this way, but since it will be python-level to C-level it's worth it?

# their times)
for i in range(self._entries.size()):
self._entries[i].clear_last_lines(ident)
self._disabled_threads.add(ident)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm reading this correctly, it seems that this attribute is only ever used by .c_last_time, as an indication that we've encountered prior profiling data on the thread, and thus we have something which my reasonably be called "last time"? In that case it would seem to me that _disabled_threads isn't terribly indicative of a name – one would have thought that it's an actively curated collection of thread ids on which the profiler is currently disabled.

That and maybe we can just get rid of c_last_time altogether, but maybe that's a discussion to be left for 6.0 (which this PR may well become).

self.own_callbacks = [handle_line, handle_return, handle_yield,
handle_raise, handle_reraise]
for i in range(_NUM_LINE_TRACING_EVENTS):
self.callbacks[i] = _register_callback(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just asking: in practice, how much of a difference does it make to switch to essentially manual zip()-ing of fixed-length sequences, vs the original (dictionary lookups)?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants