Repository navigation
Conversation
|
Exciting stuff! I'll try to review this ASAP (probably over the weekend – no promises tho), and am looking forward to your working your optimizing magic on the child-process feature. (Probably a good idea to get these landed during the honeymoon period at the new job, before expectations and duties ramp up and I get too swamped ;)) |
|
Sounds good! I may actually be able to get the free-threading working before you get to review this (depends on how bad it ends up being). If that happens, and if the free-threading changes aren't massive, then I'll just push the free-threading changes here into this PR so that you don't have to review multiple PRs spread out of a lot of time. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #450 +/- ##
=======================================
Coverage 81.40% 81.40%
=======================================
Files 21 21
Lines 2484 2485 +1
Branches 410 410
=======================================
+ Hits 2022 2023 +1
Misses 347 347
Partials 115 115
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
Update: the free threading changes are looking like they'll be pretty big (I see a net addition of roughly 1000 lines being likely) so I suspect that I'll need to make a new PR on top of this one. Hopefully I'll be able to get the changes done by Sunday :) |
|
Extremely impressive results. Avoiding hashmaps in favor of structs and arrays as an optimization makes sense. As usually I'm throwing the PR at GPT 6.1-Sol in an initial pass. It's findings do raise some concerns, particularly how there seems to be an introduced dependence between the amount of memory allocated and the physical line number in the file. Although, I do wonder how much it matters as how often will line numbers in a file get so large to the point where it becomes an issue? Still, I think its useful to flag. Details🤖 Code review: pyutils/line_profiler PR #450Open Author: Theelx · 15 files · +950 / −377 lines This is a substantial optimization of the profiling engine, rather than a small performance patch. The main architectural change replaces several hash-map-based data structures with C++ structs, arrays, and cached pointers. It also changes timer handling and optimizes callback dispatch. The author reports the following aggregate improvements compared with
These are author-supplied benchmark results, rather than measurements I have independently reproduced. The important review question is whether the reduced profiling overhead justifies the additional native-state complexity without compromising the accuracy and safety of profiling results. FindingsP1 Dense line storage can allocate memory proportional to the source file's absolute line number
The new This is a real case for functions containing exception handling and generators. I verified on CPython 3.13 that an ordinary If such a function is defined near line 100,000 of a file, its vector must span approximately 100,000 elements, even if the function itself contains only five lines. Illustrative allocation for one function with a −1 sentinel 0 MiB7 MiB14 MiB21 MiB28 MiBLine 1,000Line 10,000Line 100,000Line 1,000,000 Assumes 24 bytes per LineData on a typical 64-bit platform; excludes vector and allocator overhead. The old hash-map representation did not allocate entries for all intervening line numbers. Requested change: Exclude non-executable Source: registration and vector resizing P1 Legacy tracing loses events when profiling is enabled within an existing frame
On Suppose function I reproduced this underlying CPython behavior using
Requested change: Preserve the ability to register functions during execution, or explicitly handle restoring line events in active frames. Add a legacy-mode regression test for Source: new legacy trace suppression 3. Permanent growth of the global block registryPriority: P2 — memory lifecycle The new
That means repeated profiling of newly generated code can grow native memory even after the relevant profiler instances are destroyed. This is separate from the dense-array issue above. I would ask for either a reclaimable registry design with cache invalidation or a documented, measured bound on retained memory for realistic dynamic-code workloads. 4. Native behavior needs targeted regression coveragePriority: P2 — testing The only changed test file adds a helper docstring, a section heading, and a documentation correction. It does not introduce tests for the new native storage, timer selection, or legacy frame optimization. The existing CI is encouraging: the I would add focused tests for:
What I like about the implementationThere is substantial work worth preserving here. The main performance improvements have a coherent technical basis:
I would not request a wholesale rewrite. The dense representation is useful once the sentinel handling and memory behavior are corrected. Merge recommendationRequest changes before merging. The two P1 issues are concrete regressions worth fixing. The persistent block registry should also be addressed or bounded before the optimization is released for long-running applications. The author has already acknowledged that free-threaded builds can lose timing data and crash, and is planning a separate PR for that work. I would keep that effort separate rather than expand this PR substantially. The performance gains are worth pursuing. My preferred path is to fix the two correctness/resource issues, add targeted tests, rerun benchmarks, and merge the optimization without coupling it to the free-threading implementation. I have not submitted any comments or review decisions to GitHub. I did ask it about potential directions to keep the struct / array improvements while avoiding this line-number dependence and have its response: Details🤖 The struct/array optimization looks like a worthwhile improvement, and I'd like to preserve the fast path it introduces. I do have one concern about the dense line-number indexing in Currently, Filtering out I think an elegant solution would be to separate line-number lookup from timing-data storage:
Stable timing-slot indices would also remove the need to adjust I wouldn't optimize specifically for million-line source files, which are unlikely to be representative. The concern is that fairly ordinary modules can have functions with noncontiguous source locations, and the memory allocation shouldn't depend heavily on their absolute line numbers. This seems achievable without sacrificing the performance benefits of the PR. I'd suggest starting with the |
|
Thanks for the review! I've now pushed another two commits to address the issues it raised. Can you check the current state again to see if it looks reasonable? |
|
GPT 6 re-review: Details🤖 Re-review: line_profiler PR #450 The author has pushed two additional commits specifically addressing our review. The changes follow our proposed design closely, including the adaptive LineIndex, stable timing slots, reclaimable block storage, and regression tests. My assessment is considerably more favorable now. The original memory-allocation objection is resolved architecturally. The remaining question is whether the new lifetime and trace-restoration logic introduces edge cases. The changes are concentrated in seven files, adding approximately 514 lines and removing 54 relative to the previous head.
It also maintains stable slot identifiers. This eliminates the previous need to shift LastLine.line_index values when registering an earlier source line. The design now has the property we wanted: storage is bounded by the number of registered lines, with a fixed allowance for gaps, rather than growing with the absolute source line number. The author also preserves the -1 line registration separately for compatibility without letting it distort the dense index. I would accept this design. View the revised LineIndex implementation
The new tests are substantially more relevant to the native changes than the original PR's test diff. They passed across the CI environments, although I haven't run an independent local benchmark. The new implementation restores line events when: But get_block() only establishes that some profiler has registered that code block. It does not establish that any profiler for that block is currently active on the relevant thread. Consider two profilers:
Later, registering another function can trigger restore_line_events(). Because worker() is present in the global block registry, its line events can be reenabled even though profiler B remains disabled. The rest of that invocation can generate line callbacks that simply discover nobody wants them. For a long-running loop, this defeats part of the optimization. There is also a narrower semantic concern: this restoration can overwrite an explicit frame.f_trace_lines=False setting because the implementation doesn't record which component originally disabled the events. I would prefer restoring only frames whose code is actually profiled by an active profiler on their thread. Ideally, restoration should also distinguish flags disabled by line_profiler from those disabled externally. This is a code-path finding, not an independently reproduced failing test. I would request a focused test with two profilers, one active and one inactive, before calling it a confirmed regression. I would like to see the benchmark suite rerun against the updated implementation. In particular, the new index retains a sorted vector even in dense mode, and LastLine still uses linear searches through active frame records. The latter could become expensive with deep recursion, even though ordinary workloads should remain fast. Neither is a demonstrated regression. A recursion-depth sweep and refreshed benchmarks would establish their practical cost. Request changes Current recommendation Close to approval I would no longer block this PR over the dense-array memory issue, the block registry lifetime, or the original active-frame tracing problem. The author has supplied appropriate implementations and regression tests for those concerns. My remaining requests would be comparatively small: verify that restore_line_events() only reenables frames that actually need profiling, and rerun the performance benchmarks after the new indexing implementation. I would also keep free-threading support separate, as the author proposed. It remains a known limitation of the current profiling engine, and adding concurrency changes now would complicate the review substantially. Bottom line: The author has responded constructively and implemented essentially the architecture we recommended. I think the core changes are now reasonable to merge after resolving or ruling out the frame-restoration concern. Looks like much of the issues were addressed. It's pointing out 2 more minor points. The frame restoration issue it mentions looks like a correct find. It also probably makes sense for @TTsangSC (if you have time) to take a pass at it before I actually dive into my final review and read the code. |
|
Ok, I've implemented a fix for the first issue. The second issue is just the AI asking for updated benchmarks; I re-ran them locally after this latest commit and found no difference above the noise floor of ~1.5% against the last time that the numbers were reported. |
| char clocksource[64] = {0}; | ||
| FILE *f; | ||
| if (!__get_cpuid(0x80000007, &eax, &ebx, &ecx, &edx) | ||
| || !(edx & (1u << 8))) |
There was a problem hiding this comment.
This is maybe a bit too magical and may need more doc – or I'm just too noob when it comes to real close-to-metal stuff (among other things). Looking at Wikipedia, bit index 8 on edx is supposed to be for "AVX-512 vector intersection instructions on 32/64-bit integers", which doesn't quite seem to relate to the situation at hand.
There was a problem hiding this comment.
Whoops didn't mean to comment this soon. Anyway the rest of the review will come later today.
There was a problem hiding this comment.
Bit 8 of edx only means AVX-512 under leaf 7, but with leaf 0x80000007 it means invariant TSC. That being said, I agree that it was a bit too magic, so I've pushed a commit that should make it more clear.
|
Sorry but the read has been going slower than I expected., but I hope to have all my comments ready by Thursday. |
TTsangSC
left a comment
There was a problem hiding this comment.
Amazing stuff, really appreciate the various savings you're making here.
As with the last time, these are maybe less review-y than they are discussions:
| void insert(int lineno, int slot) | ||
| { | ||
| auto pair = std::make_pair(lineno, slot); | ||
| sorted.insert( | ||
| std::lower_bound(sorted.begin(), sorted.end(), pair), pair); | ||
| int first = sorted.front().first, last = sorted.back().first; | ||
| size_t span = (size_t)last - (size_t)first + 1; | ||
| if (span > MAX_SPAN_PER_LINE * sorted.size() + MIN_DENSE_SPAN) { | ||
| // Too sparse for a table | ||
| dense = false; | ||
| std::vector<int>().swap(table); | ||
| } else if (dense && first == first_line) { | ||
| if (span > table.size()) table.resize(span, -1); | ||
| table[lineno - first_line] = slot; | ||
| } else { // (Re)build the table | ||
| dense = true; | ||
| first_line = first; | ||
| table.assign(span, -1); | ||
| for (const auto ®istered : sorted) { | ||
| table[registered.first - first] = registered.second; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
I wonder if we can explicitly trigger this resizing behavior in tests? Or is that impossible due to how each LineIndex is only associated with one ProfileEntry, and thus one Block which is only scanned once?
| # event: stop it from firing here at all | ||
| note_events_disabled() | ||
| return _DISABLE | ||
| return on_line_event(self, code, lineno) |
There was a problem hiding this comment.
Not sure if I follow why on_line_event() and on_exit_event() are spun off into their own functions, but I suppose it has to do with our previous discussions on def-ing instead of cpdef-ing these methods I guess? Like we still get one extra dispatch this way, but since it will be python-level to C-level it's worth it?
| # their times) | ||
| for i in range(self._entries.size()): | ||
| self._entries[i].clear_last_lines(ident) | ||
| self._disabled_threads.add(ident) |
There was a problem hiding this comment.
If I'm reading this correctly, it seems that this attribute is only ever used by .c_last_time, as an indication that we've encountered prior profiling data on the thread, and thus we have something which my reasonably be called "last time"? In that case it would seem to me that _disabled_threads isn't terribly indicative of a name – one would have thought that it's an actively curated collection of thread ids on which the profiler is currently disabled.
That and maybe we can just get rid of c_last_time altogether, but maybe that's a discussion to be left for 6.0 (which this PR may well become).
| self.own_callbacks = [handle_line, handle_return, handle_yield, | ||
| handle_raise, handle_reraise] | ||
| for i in range(_NUM_LINE_TRACING_EVENTS): | ||
| self.callbacks[i] = _register_callback( |
There was a problem hiding this comment.
Just asking: in practice, how much of a difference does it make to switch to essentially manual zip()-ing of fixed-length sequences, vs the original (dictionary lookups)?
This PR contains the last batch of speedups that I have so far. Benchmarks are below. The main improvements come from using structs and arrays rather than hashmaps, with some smaller Linux-specific improvements from speeding up the clock reading when possible. Avoiding hashmaps does come with an increase in complexity, but it's something like 70% of the total overhead reduction from this PR. Also, by removing hashmaps, it scales with core count far far better on free-threading builds! Currently, the free-threaded builds don't fully work with line_profiler due to races that cause certain line timings to be lost and occasional crashes (though free-threaded builds have never worked with line_profiler AFAIK), however I have ideas to fix those that I will work on in parallel to TTsang's work on supporting child processes.
Default settings (
wrap_trace=False) (overhead reduction vs. main)Wrapping another tool's callbacks (
wrap_trace=True) (overhead reduction vs. main)Default settings (
wrap_trace=False) (overhead reduction vs. no profiling at all)As you can see, the overall performance overhead from using line_profiler is made significantly smaller by this change. I'd hazard a guess that it's the most efficient non-sampling profiler for Python with these changes!
@TTsangSC Could you review this sometime in the next week or so? Once I address the review comments and get this merged, I'd love to work with you with the child process thing and with free-threading support.