Skip to content

Massively reduce profiling overhead (~3x) and split cython into parallel compilation units - #447

Merged
Erotemic merged 4 commits into
pyutils:mainfrom
Theelx:sub-optimizations
Nov 13, 2026
Merged

Erotemic merged 4 commits into
pyutils:mainfrom
Theelx:sub-optimizations

Conversation

@Theelx

@Theelx Theelx commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

Hey! This PR massively reduces overhead from sys.monitoring, which is used on python 3.12+. It has a geomean reduction in overhead of ~4x on versions 3.12+, and an even higher reduction of overhead on code that isn't even being profiled (~10x). I also split the massive _line_profiler.pyx file up into some smaller units so that the project can build faster. It was taking ~30s to build the C-extension before because the huge .pyx generated a ton of C++ that then took g++ a while to compile. By splitting it up into smaller files, it helps with maintainability but also lets g++ parallelize its compilation across multiple cores, bringing build time down to ~10s on my Linux machine. There are some issues with MSVC not being able to handle the multiple compilation units though, so the builds aren't any faster on Windows.

Type Scenarios 3.10 3.13 3.15 All versions
Sync Python tight_loop, many_calls, calls_unprofiled, recursive, generators 1.30x 3.67x 3.66x 2.59x
Async async_calls, async_tasks 1.25x 4.40x 4.35x 2.88x
Cython cy_tight_loop, cy_many_calls, cy_recursive 1.65x 5.12x 5.07x 3.50x
Overall all 10 above 1.39x 4.21x 4.18x 2.90x

@Theelx
Theelx requested review from Erotemic and TTsangSC October 18, 2026 17:49
@Theelx

Theelx commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator Author

I have 2 more speedup PRs after this that, together, will reduce overhead even more (another ~2x)!

@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.40%. Comparing base (0c932ab) to head (dbc38f3).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #447      +/-   ##
==========================================
- Coverage   85.12%   81.40%   -3.73%     
==========================================
  Files          21       21              
  Lines        2413     2484      +71     
  Branches      376      410      +34     
==========================================
- Hits         2054     2022      -32     
- Misses        259      347      +88     
- Partials      100      115      +15     

see 9 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3400d7...dbc38f3. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Theelx Theelx mentioned this pull request Oct 18, 2026
@Theelx

Theelx commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator Author

For coverage concerns: I was planning to work on coverage once all the speedups are merged but before they're released, so I can work on coverage before 5.2.0 is released.

@Erotemic

Erotemic commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Don't worry too much about coverage. The change threshold is probably too strict. As long as it is above 75-80%, that's probably good. What's more important is looking at what is and isn't covered. If there are new uncovered lines with important logic, then that's a target to fix. I'm looking now and I see a lot of red is due to if TYPE_CHECKING, and that isn't a big deal. Modern Python type annotations take care of a lot of what was historically important about Python coverage - checking that you didn't pass around incompatible objects at runtime.

The numbers do indeed look exciting, and faster compilation and better code organization is always nice. Even though MSVC builds aren't faster, you would expect runtime to benefit though?

I have not looked at this PR in detail, and I'll need to do that before merging, but I did throw it at an LLM for review, and the results look relevant and actionable.

🤖 GPT 6 says:

Details

Reviewed revision 8d3c806 · October 18, 2026

Preliminary verdict: Request changes. The performance optimizations look worthwhile, and the CI matrix is green, but I found a correctness issue in the new event-disabling mechanism that should be addressed before merging.

1. Disabled monitoring events survive profiler shutdown on Python 3.12–3.13

P2 · Confirmed API behavior

Location: [_line_profiler.pyx:590–595](https://github.com/pyutils/line_profiler/blob/8d3c8062a114dfd43a2cb5df50ab1eb8dc8fa3b9/line_profiler/_line_profiler.pyx?utm_source=chatgpt.com#L590-L595)

The new optimization returns sys.monitoring.DISABLE for unprofiled locations. However, on shutdown, the profiler restarts those events only if another tool already occupied its monitoring tool ID:

if self.mon_state.name is not None:    restart_disabled_events()self.mon_state.deregister()

On Python 3.12 and 3.13, sys.monitoring.free_tool_id() does not clear previously disabled code locations. Consequently, a different profiler that subsequently acquires the same tool ID can inherit suppressed events and miss callbacks.

I reproduced that underlying behavior on Python 3.13.5: tool A disabled line events, released its ID, and tool B acquired the same ID. Tool B received no line callbacks for the previously disabled locations.

Suggested fix: Restart the events before deregistration whenever this profiler has disabled any, regardless of whether another tool currently owns the ID. Add a regression test that exercises tool-ID reuse after profiler shutdown.

This matters for interoperability with other profilers, even though line_profiler itself already restarts disabled events when it is re-enabled.

2. Changing wrap_trace while profiling is active does not restore disabled events

P2 · Reproduced against the built PR

Locations: [_line_profiler.pyx:462–478](https://github.com/pyutils/line_profiler/blob/8d3c8062a114dfd43a2cb5df50ab1eb8dc8fa3b9/line_profiler/_line_profiler.pyx?utm_source=chatgpt.com#L462-L478) and [861–869](https://github.com/pyutils/line_profiler/blob/8d3c8062a114dfd43a2cb5df50ab1eb8dc8fa3b9/line_profiler/_line_profiler.pyx?utm_source=chatgpt.com#L861-L869)

When wrap_trace=False, the profiler disables line events for unprofiled functions. Changing wrap_trace=True during the same profiling session changes the manager flag but never restarts those events.

I tested this with another monitoring callback registered under PROFILER_ID.

Action Lines observed by other tool
Call unprofiled function with wrap_trace=False 0
Set prof.wrap_trace = True, call again 0
Call sys.monitoring.restart_events(), then call again 3

The newly enabled wrapping therefore fails to forward events for locations the profiler previously disabled.

Suggested fix: When wrap_trace transitions from False to True during an active profiling session, call restart_disabled_events(). Add a test covering that transition with another tool's callback registered.

Additional review observations

Build and CI: All 31 reported jobs passed. I also compiled the PR's source distribution locally on CPython 3.13.

Core timing loop: The iterator reuse, cached code hashes, and removal of repeated map lookups appear consistent with the previous logic. I did not identify another definite correctness regression in those changes.

Coverage: Codecov reports overall coverage falling from 85.12% to 81.40%. Some of that may reflect the module split, but the additional uncovered paths deserve inspection.

I have not independently reproduced the claimed performance improvements against the base revision; the review's two confirmed failures concern correctness rather than speed.

Recommendation

Request changes before merging, with two required regression tests: tool-ID reuse after shutdown and runtime wrap_trace changes.

Both findings arise from the same optimization and should be straightforward to address without abandoning it. I would keep the caching and Cython module split.

Once these are fixed and tested, I would be comfortable moving toward merge, subject to a final look at the revised event lifecycle.

@Theelx

Theelx commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator Author

Even though MSVC builds aren't faster, you would expect runtime to benefit though?

Could you clarify what you mean by "expect runtime to benefit"? The compilation speedups are at build-time, not runtime. AFAIK, they won't speed up the runtime at all (because they're build-time).

Thanks to you and GPT-6 for the preliminary review! I'm addressing them now and will be able to push some changes soon.

@Erotemic

Copy link
Copy Markdown
Member

Could you clarify what you mean by "expect runtime to benefit"?

I was under the impression that this sped up build time and the cost of profiling at runtime. Rereading the description - I'm no stranger to making stupid reading mistakes - it still reads that way:

massively reduces overhead from sys.monitoring

Your table also appears to be a runtime table. I assume that's from the Linux box?

@Theelx

Theelx commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator Author

Ah yeah. This PR does both the compilation/build-time speedups and the runtime overhead speedups. I was just confused because when you were talking about MSVC and runtime in the same sentence, I thought you thought that the compilation stuff would speed up runtime too. Glad to hear I was wrong!

The table I sent is a runtime table from my linux box, yeah.

@TTsangSC

TTsangSC commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Hi, just wanted to let you know that I've finally gotten to a place where I can get back into coding and stuff – was very swamped by life stuff (moving and furniture) the last couple weeks. And to say thanks for tuning up the sluggish part, which I mostly did with feature completeness in mind but again neglected performance.1 Will write a review (and hopefully work on my own PRs) over the weekend.

Apropos, do you have your benchmarking scenarios somewhere? I remember @Erotemic suggesting and beginning the work on (#381) keeping track of performance more systematically with tests/dev tools, but that did not end up in the codebase. Maybe your scenarios would be a good fresh start for that purpose?

Footnotes

  1. Which of course you have already begun to mitigate in Speed Up Worst-Case Overhead #376. ↩

@Theelx

Theelx commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

Sounds good, and if you don't have any time over the weekend then that's fine too! I can push my benchmarking scenarios to a different repo, will add a link here shortly. I made the benchmarking scenarios with Claude though, so I can't promise that they are error-free (though a quick skim suggests that they're fine).

@Theelx

Theelx commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

Benchmarking code: https://github.com/Theelx/line_profiler/tree/faster-callback-wrapping/bench

@Theelx
Theelx force-pushed the sub-optimizations branch from 3716914 to face801 Compare October 23, 2026 16:01

@TTsangSC TTsangSC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. Here are some points that have caught my notice; most (but not all) of them are intended to be discussions and not so much direct suggestions.

Comment thread line_profiler/_line_profiler.pyx Resolved
Comment thread line_profiler/_profiler_base.pyx Resolved
Comment thread line_profiler/_line_profiler.pyx Resolved
Comment thread line_profiler/_line_profiler.pyx Resolved
Comment thread line_profiler/_line_profiler.pyx Resolved
Comment thread tests/test_line_profiler.py Resolved
Comment thread tests/test_line_profiler.py Resolved
Comment thread line_profiler/_profiler_base.pyx Resolved
Comment thread line_profiler/_sys_monitoring.pyx Resolved
Comment thread tests/test_line_profiler.py Resolved
Comment thread line_profiler/_line_profiler.pyx Resolved
@Theelx

Theelx commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

@TTsangSC Just wanted to check: do you have any more comments before you think it's ready for @Erotemic to review?

@TTsangSC

Copy link
Copy Markdown
Collaborator

Looks great, I think it's ready.

Comment thread line_profiler/_line_profiler.pyx Resolved
@Theelx

Theelx commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

@Erotemic Hey Jon, do you think you could take a look at this sometime in the next week or two?

@Erotemic Erotemic left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so I believe I have a good understanding of the changes and mechanisms for the speedup.

  • PEP 523 gives us a way to attach metadata to code objects, and because the bytecode on these objects is immutable, so this should be safe, and we can hash the bytecode once per parent code object.

  • You swapped set[LineProfiler] with a C++ vector, which makes sense.

  • We return sys.monitoring.DISABLE from the handle_line_event callback to avoid events for functions that don't need them, and then monitoring is reenabled later in restart_disabled_events.

  • Moving handle_line_event from cpdef -> def should be fine because sys.monitoring using python based callbacks anyway, so we never benefited from the c-callable. (And @cython.binding(False) likely helps a bit)

  • We are reusing iterators being more careful with duplicate lookups in inner_trace_callback.

  • Removing _base_callback and more carefully calling what you need where you need it (I'm less positive on this change because it does increase the complexity of the code, but I think its fine if it has a measurable impact on performance)

  • Moving code into pyx / pxd is what is giving some of the compiler speedups. Less joint code to work over, and parallel compilation as you said.

# Index of the per-code-object scratch slot (see PEP 523) in which
# get_block_info() caches a BlockInfo; negative if none is available
cdef Py_ssize_t _BLOCK_INFO_EXTRA_INDEX = (
PyUnstable_Eval_RequestCodeExtraIndex(free_block_info))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we are requesting a slot index so we can stuff extra metadata into code objects (which appears to be exactly what this sort of API is intended for, neat).

But I'm a little concerned about it being done at the global scope at import time. The docs claim that this is done once per interpreter, but will this play nice with subinterpreters? Reading up on it, I think its fine? Each subinterpreter should get its own instance of the module.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As TTsang pointed out, unfortunately the entire system of our Cython profiling is incompatible with subinterpreters, so it isn't just this change. Specifically, subinterpreters keep their module globals in C statics, so Cython refuses to load them into more than one interpreter per process. That rules out sharing _BLOCK_INFO_EXTRA_INDEX too. If we ever add subinterpreter support, the index will need to move into per-module state along with everything else, because the index (and its free function) is registered per interpreter, however that's a thing for far into the future I suspect.

@Erotemic

Copy link
Copy Markdown
Member

@Theelx reviewed, thanks for the patience. And thank you @TTsangSC for taking on the brunt of the review. It made my job a lot easier. The only outstanding items are:

  1. Your thoughts on sub-interpreters and having _BLOCK_INFO_EXTRA_INDEX at the global scope and
  2. Is the speedup from removing _base_callback worth the maintenance overhead in terms of the speed improvement? Giving its in the critical section, I'm guessing it will be.

Based on this feedback, let me know if you want to make changes / measurements, or if you want me to merge. I think its good to go as it is.

WRT to the benchmarking, I might revisit it in the future, but its fallen off of my priority list.

@TTsangSC

Copy link
Copy Markdown
Collaborator

You're welcome: :)

One note regarding sub-interpreters: AFAIK the package already didn't work in them before this. I tried playing around with those when doing #431 because one of the concurrent.futures backends uses them, but can't seem to get the Cython module to set up and import correctly in them no matter what I tried. May need something deeper on the CPython level for managing module state and setup...

But maybe all these changes have shaken up the structure of the Cython code enough that we can make it work with a bit more elbow grease IDK.

@Theelx

Theelx commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

For _base_callback, it actually wasn't removed only for speed (though speed was definitely part of the reason). The handlers now need to know whether the location was profiled, so they can return DISABLE, and _base_callback returned void. Regarding its speed, its callers also built the (lineno,) / (offset,) / (obj,) argument tuples and looked up sys.monitoring.events.* on every event, even though those are only needed when wrap_trace is on (off by default). This could have been avoided with a check for if wrap_trace was on/off (which probably would have been cheap), but given that I already had to use a different function to return DISABLE it just made more sense to replace _base_callback entirely.

Edit: I think it's ready to merge as-is, unless you have more questions about _base_callback. The next PR in this series will also make it support free-threading along with a further ~2x overhead reduction, so I don't think subinterpreters will be necessary at that point.

Edit 2: I just finished a benchmark of adding back _base_callback, and with it added back the overhead is roughly 20% more averaged all scenarios, so I think we ought to keep it removed.
image

@Erotemic

Copy link
Copy Markdown
Member

Thanks @Theelx

@Erotemic
Erotemic merged commit 958d4a8 into pyutils:main Nov 13, 2026
49 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants