Repository navigation
Massively reduce profiling overhead (~3x) and split cython into parallel compilation units - #447
Conversation
|
I have 2 more speedup PRs after this that, together, will reduce overhead even more (another ~2x)! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #447 +/- ##
==========================================
- Coverage 85.12% 81.40% -3.73%
==========================================
Files 21 21
Lines 2413 2484 +71
Branches 376 410 +34
==========================================
- Hits 2054 2022 -32
- Misses 259 347 +88
- Partials 100 115 +15 see 9 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
For coverage concerns: I was planning to work on coverage once all the speedups are merged but before they're released, so I can work on coverage before 5.2.0 is released. |
|
Don't worry too much about coverage. The change threshold is probably too strict. As long as it is above 75-80%, that's probably good. What's more important is looking at what is and isn't covered. If there are new uncovered lines with important logic, then that's a target to fix. I'm looking now and I see a lot of red is due to The numbers do indeed look exciting, and faster compilation and better code organization is always nice. Even though MSVC builds aren't faster, you would expect runtime to benefit though? I have not looked at this PR in detail, and I'll need to do that before merging, but I did throw it at an LLM for review, and the results look relevant and actionable. 🤖 GPT 6 says: DetailsReviewed revision Preliminary verdict: Request changes. The performance optimizations look worthwhile, and the CI matrix is green, but I found a correctness issue in the new event-disabling mechanism that should be addressed before merging. 1. Disabled monitoring events survive profiler shutdown on Python 3.12–3.13P2 · Confirmed API behavior The new optimization returns On Python 3.12 and 3.13, I reproduced that underlying behavior on Python 3.13.5: tool A disabled line events, released its ID, and tool B acquired the same ID. Tool B received no line callbacks for the previously disabled locations. Suggested fix: Restart the events before deregistration whenever this profiler has disabled any, regardless of whether another tool currently owns the ID. Add a regression test that exercises tool-ID reuse after profiler shutdown. This matters for interoperability with other profilers, even though 2. Changing
|
| Action | Lines observed by other tool |
|---|---|
Call unprofiled function with wrap_trace=False |
0 |
Set prof.wrap_trace = True, call again |
0 |
Call sys.monitoring.restart_events(), then call again |
3 |
The newly enabled wrapping therefore fails to forward events for locations the profiler previously disabled.
Suggested fix: When wrap_trace transitions from False to True during an active profiling session, call restart_disabled_events(). Add a test covering that transition with another tool's callback registered.
Additional review observations
Build and CI: All 31 reported jobs passed. I also compiled the PR's source distribution locally on CPython 3.13.
Core timing loop: The iterator reuse, cached code hashes, and removal of repeated map lookups appear consistent with the previous logic. I did not identify another definite correctness regression in those changes.
Coverage: Codecov reports overall coverage falling from 85.12% to 81.40%. Some of that may reflect the module split, but the additional uncovered paths deserve inspection.
I have not independently reproduced the claimed performance improvements against the base revision; the review's two confirmed failures concern correctness rather than speed.
Recommendation
Request changes before merging, with two required regression tests: tool-ID reuse after shutdown and runtime wrap_trace changes.
Both findings arise from the same optimization and should be straightforward to address without abandoning it. I would keep the caching and Cython module split.
Once these are fixed and tested, I would be comfortable moving toward merge, subject to a final look at the revised event lifecycle.
Could you clarify what you mean by "expect runtime to benefit"? The compilation speedups are at build-time, not runtime. AFAIK, they won't speed up the runtime at all (because they're build-time). Thanks to you and GPT-6 for the preliminary review! I'm addressing them now and will be able to push some changes soon. |
I was under the impression that this sped up build time and the cost of profiling at runtime. Rereading the description - I'm no stranger to making stupid reading mistakes - it still reads that way:
Your table also appears to be a runtime table. I assume that's from the Linux box? |
|
Ah yeah. This PR does both the compilation/build-time speedups and the runtime overhead speedups. I was just confused because when you were talking about MSVC and runtime in the same sentence, I thought you thought that the compilation stuff would speed up runtime too. Glad to hear I was wrong! The table I sent is a runtime table from my linux box, yeah. |
|
Hi, just wanted to let you know that I've finally gotten to a place where I can get back into coding and stuff – was very swamped by life stuff (moving and furniture) the last couple weeks. And to say thanks for tuning up the sluggish part, which I mostly did with feature completeness in mind but again neglected performance.1 Will write a review (and hopefully work on my own PRs) over the weekend. Apropos, do you have your benchmarking scenarios somewhere? I remember @Erotemic suggesting and beginning the work on (#381) keeping track of performance more systematically with tests/dev tools, but that did not end up in the codebase. Maybe your scenarios would be a good fresh start for that purpose? Footnotes
|
|
Sounds good, and if you don't have any time over the weekend then that's fine too! I can push my benchmarking scenarios to a different repo, will add a link here shortly. I made the benchmarking scenarios with Claude though, so I can't promise that they are error-free (though a quick skim suggests that they're fine). |
|
Benchmarking code: https://github.com/Theelx/line_profiler/tree/faster-callback-wrapping/bench |
3716914 to
face801
Compare
TTsangSC
left a comment
There was a problem hiding this comment.
Sorry for the delay. Here are some points that have caught my notice; most (but not all) of them are intended to be discussions and not so much direct suggestions.
|
Looks great, I think it's ready. |
|
@Erotemic Hey Jon, do you think you could take a look at this sometime in the next week or two? |
There was a problem hiding this comment.
Ok, so I believe I have a good understanding of the changes and mechanisms for the speedup.
-
PEP 523 gives us a way to attach metadata to code objects, and because the bytecode on these objects is immutable, so this should be safe, and we can hash the bytecode once per parent code object.
-
You swapped
set[LineProfiler]with a C++ vector, which makes sense. -
We return sys.monitoring.DISABLE from the handle_line_event callback to avoid events for functions that don't need them, and then monitoring is reenabled later in restart_disabled_events.
-
Moving handle_line_event from cpdef -> def should be fine because sys.monitoring using python based callbacks anyway, so we never benefited from the c-callable. (And
@cython.binding(False)likely helps a bit) -
We are reusing iterators being more careful with duplicate lookups in inner_trace_callback.
-
Removing _base_callback and more carefully calling what you need where you need it (I'm less positive on this change because it does increase the complexity of the code, but I think its fine if it has a measurable impact on performance)
-
Moving code into pyx / pxd is what is giving some of the compiler speedups. Less joint code to work over, and parallel compilation as you said.
| # Index of the per-code-object scratch slot (see PEP 523) in which | ||
| # get_block_info() caches a BlockInfo; negative if none is available | ||
| cdef Py_ssize_t _BLOCK_INFO_EXTRA_INDEX = ( | ||
| PyUnstable_Eval_RequestCodeExtraIndex(free_block_info)) |
There was a problem hiding this comment.
So we are requesting a slot index so we can stuff extra metadata into code objects (which appears to be exactly what this sort of API is intended for, neat).
But I'm a little concerned about it being done at the global scope at import time. The docs claim that this is done once per interpreter, but will this play nice with subinterpreters? Reading up on it, I think its fine? Each subinterpreter should get its own instance of the module.
There was a problem hiding this comment.
As TTsang pointed out, unfortunately the entire system of our Cython profiling is incompatible with subinterpreters, so it isn't just this change. Specifically, subinterpreters keep their module globals in C statics, so Cython refuses to load them into more than one interpreter per process. That rules out sharing _BLOCK_INFO_EXTRA_INDEX too. If we ever add subinterpreter support, the index will need to move into per-module state along with everything else, because the index (and its free function) is registered per interpreter, however that's a thing for far into the future I suspect.
|
@Theelx reviewed, thanks for the patience. And thank you @TTsangSC for taking on the brunt of the review. It made my job a lot easier. The only outstanding items are:
Based on this feedback, let me know if you want to make changes / measurements, or if you want me to merge. I think its good to go as it is. WRT to the benchmarking, I might revisit it in the future, but its fallen off of my priority list. |
|
You're welcome: :) One note regarding sub-interpreters: AFAIK the package already didn't work in them before this. I tried playing around with those when doing #431 because one of the But maybe all these changes have shaken up the structure of the Cython code enough that we can make it work with a bit more elbow grease IDK. |
|
Thanks @Theelx |

Hey! This PR massively reduces overhead from sys.monitoring, which is used on python 3.12+. It has a geomean reduction in overhead of ~4x on versions 3.12+, and an even higher reduction of overhead on code that isn't even being profiled (~10x). I also split the massive _line_profiler.pyx file up into some smaller units so that the project can build faster. It was taking ~30s to build the C-extension before because the huge .pyx generated a ton of C++ that then took g++ a while to compile. By splitting it up into smaller files, it helps with maintainability but also lets g++ parallelize its compilation across multiple cores, bringing build time down to ~10s on my Linux machine. There are some issues with MSVC not being able to handle the multiple compilation units though, so the builds aren't any faster on Windows.