Repository navigation
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
|
Validated against clean main For disk mode, MO cache stays warm while OS data-file pages start cold before each query. The harness evicts only the task clone's original immutable SHARED files, checks zero residency using fincore metadata, then starts query measurements. All 528 cold/warm disk boundaries passed. Eviction/metadata time is outside query latency. Memory-mode evidence is reused from the same unchanged sources/binaries/configuration.
The native Q18 diagnostic confirms early qualification:624 ORDERS rows and623 CUSTOMER rows produce624 dimension-join rows, with one LINEITEM scan and ordinary32-way qualifier parallelism. The failed earlier prototype materialized15M dimension-join rows before qualification; that implementation is superseded. No executor/cache algorithm was changed. Timing variation is disclosed, not subtracted from wall scores. SF100 Q17 raw medians are+11.39%, but three paired suite positions are nearly unchanged/faster; slow episodes also occur in main. Main five wall samples:5.0210,3.5677,3.5972,3.5991,3.5937s; candidate:4.9698,4.0068,3.6074,4.8195,3.5991s. Physical reads remain9.38–9.48 GB and CPU work remains about 33 CPU-seconds; estimated non-GC instructions differ+0.63%. The rule does not rewrite Q17. These captures do not show a stable increase in Q17 work; the raw slower median remains visible below. SF10 memory Q6's+26.61% wall/+27.64% raw instruction difference coincides with GC, while estimated non-GC instructions differ+0.06%. Before controlling OS pages, unchanged main varied79.46 s→71.12 s with stable MO cache hits/requested bytes but different physical I/O. Those captures are retained as diagnostics and are superseded for disk latency acceptance; they are not pooled with these tables. sf10-memory: all22 queries, Total, CPU and allocations
Allocation counters include common process/scrape and asynchronous engine work. No globally exact copy-byte counter is available. GC CPU is measured process GC CPU, not a wall-time subtraction. | Query | Main Go MB | Candidate Go MB | Go bytes Δ | Main Go objects | Candidate Go objects | Main GC CPU s | Candidate GC CPU s | Native allocation attribution limit: the default allocator publishes counters after a 100 ms window; MPool publishes after a one-second window. Their query-boundary increments are retained in raw JSON, but can belong to neighboring queries. They are not exact per-query allocation evidence and are not used to declare a per-query allocation pass. IO/cache counters overlap the default allocator and must not be added again. Long-window native totals also have endpoint lag; exact per-query native costs need separate operator/component evidence. External process cores: main 0.042, candidate 0.029; interrupt cores separately: 0.337, 0.334. Period-weighted instruction samples estimate GC attribution. These are sampling estimates, not exact foreground counters, GC-subtracted wall times, or independent-process confidence intervals. Raw hardware counters are above.
sf10-disk: all22 queries, Total, CPU and allocations
Allocation counters include common process/scrape and asynchronous engine work. No globally exact copy-byte counter is available. GC CPU is measured process GC CPU, not a wall-time subtraction. | Query | Main Go MB | Candidate Go MB | Go bytes Δ | Main Go objects | Candidate Go objects | Main GC CPU s | Candidate GC CPU s | Native allocation attribution limit: the default allocator publishes counters after a 100 ms window; MPool publishes after a one-second window. Their query-boundary increments are retained in raw JSON, but can belong to neighboring queries. They are not exact per-query allocation evidence and are not used to declare a per-query allocation pass. IO/cache counters overlap the default allocator and must not be added again. Long-window native totals also have endpoint lag; exact per-query native costs need separate operator/component evidence. External process cores: main 0.096, candidate 0.047; interrupt cores separately: 0.324, 0.305. Period-weighted instruction samples estimate GC attribution. These are sampling estimates, not exact foreground counters, GC-subtracted wall times, or independent-process confidence intervals. Raw hardware counters are above.
sf100-disk: all22 queries, Total, CPU and allocations
Allocation counters include common process/scrape and asynchronous engine work. No globally exact copy-byte counter is available. GC CPU is measured process GC CPU, not a wall-time subtraction. | Query | Main Go MB | Candidate Go MB | Go bytes Δ | Main Go objects | Candidate Go objects | Main GC CPU s | Candidate GC CPU s | Native allocation attribution limit: the default allocator publishes counters after a 100 ms window; MPool publishes after a one-second window. Their query-boundary increments are retained in raw JSON, but can belong to neighboring queries. They are not exact per-query allocation evidence and are not used to declare a per-query allocation pass. IO/cache counters overlap the default allocator and must not be added again. Long-window native totals also have endpoint lag; exact per-query native costs need separate operator/component evidence. External process cores: main 0.000, candidate 0.116; interrupt cores separately: 0.548, 0.552. Period-weighted instruction samples estimate GC attribution. These are sampling estimates, not exact foreground counters, GC-subtracted wall times, or independent-process confidence intervals. Raw hardware counters are above.
|
|
Queue requested — the merge queue status continues in this comment ↓. |
⏳ Waiting for queue conditionsEnters the queue as soon as one of these 7 queue rules has nothing left. Queue rule
Queue rule
Queue rule
4 other queue rulesQueue rule
Queue rule
Queue rule
Queue rule
Queue rule
|
What type of PR is this?
Which issue(s) this PR fixes:
Related to #30857
What this PR does / why we need it:
Reuse a qualifying grouped SUM when a later aggregate computes the same exact measure over the same fact rows through key-preserving dimension joins. Carry the existing measure through those joins, remove the redundant fact scan and connector, and replace the repeated aggregate with a projection.
Keep qualification as SEMI through normal join ordering and pushdown. Apply reuse only at a lowered edge whose probe is a direct surviving dimension scan, so qualification filters the probe before the other dimensions materialize its rows. Preserve the dimension predicates and existing runtime-filter planning.
Require matching input bags, snapshots, filters, key and argument types, total exact SUM arithmetic, and catalog-proven dimension uniqueness. Retain the original plan for unsupported or unprofitable cases, including disabled SEMI pushdown. Preserve NULLs, missing dimensions, output metadata, HAVING, ordering and limits.
Design
Reviewed design.
Validation
Focused rewrite tests and the owning planner package pass. The full-planner test verifies that qualification remains below the surviving customer join. Negative tests cover join-spanning and retired-fact probes, disabled pushdown, incompatible bags/types and missing uniqueness proofs.
The changed SQL case passes normal mo-tester comparison: 19 statements, twice on the same task-owned instance, with schema teardown checked after each run. Owning-package static checks and convention checks pass. Loaded-schema operator diagnostics confirm early qualification, one fact scan and normal qualifier parallelism.
The local all22 comparison covers SF10 memory/disk and SF100 disk, with excluded cold suites and five warm suites per process. Disk comparisons control OS data-page residency while retaining MO's configured cache. Results and timing/metric limitations are posted in a separate comment. The full repository pre-push static gate passes with the repository-pinned tools in the default CI build mode. Functional tests and benchmark builds use SIMD/v3. The optional whole-repository amd64 SIMD lint mode also reports two unsafe-allowlist failures on unchanged main in vector-distance files; this PR does not modify them.