Repository navigation
Conversation
lfittl
force-pushed
the
instrument-usage-stack-v18
branch
from
October 3, 2026 17:01
cbcf577 to
8ef30a4
Compare
lfittl
force-pushed
the
instrument-usage-stack-v18
branch
7 times, most recently
from
October 22, 2026 20:52
407c2b7 to
03cd5a0
Compare
lfittl
force-pushed
the
instrument-usage-stack-v18
branch
2 times, most recently
from
October 30, 2026 21:52
34d3785 to
ca7b618
Compare
This replaces different repeated code blocks that read pgBufferUsage / pgWalUsage, and may have also been running a timer to measure activity, with the Instrumentation struct and associated helpers. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion: https://postgr.es/m/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mail.gmail.com
…ith INSTR_* macros This encapsulates the ownership of these globals better, and will allow a subsequent refactoring. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CAP53PkzZ3UotnRrrnXWAv%3DF4avRq9MQ8zU%2BbxoN9tpovEu6fGQ%40mail.gmail.com#fc7140e8af21e07a90a09d7e76b300c4
lfittl
force-pushed
the
instrument-usage-stack-v18
branch
from
October 30, 2026 23:08
ca7b618 to
2dc556e
Compare
This adds regression tests that cover some of the expected behaviour around the buffer statistics reported in EXPLAIN ANALYZE, specifically how they behave in parallel query, nested function calls and abort situations. Testing this is challenging because there can be different sources of buffer activity, so we rely on temporary tables where we can to prove that activity was captured and not lost. This supports a future commit that will rework some of the instrumentation logic that could cause areas covered by these tests to fail. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion: https://postgr.es/m/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mail.gmail.com
Previously, in order to determine the buffer/WAL usage of a given code section, we utilized continuously incrementing global counters that get updated when the actual activity (e.g. shared block read) occurred, and then calculated a diff when the code section ended. This resulted in a bottleneck for executor node instrumentation specifically, with profiles showing BufferUsageAccumDiff in EXPLAIN (ANALYZE, BUFFERS) runs. Instead, introduce a stack-based mechanism, where the actual activity writes into the current stack entry. In the case of executor nodes, this means that each node gets its own stack entry that is pushed at InstrStartNode, and popped at InstrStopNode. Stack entries are zero initialized (avoiding the diff mechanism) and get added to their parent entry when they are finalized, i.e. no more modifications can occur. To correctly handle abort situations, any use of instrumentation stacks must involve either a top-level QueryInstrumentation struct, and its associated InstrQueryStart/InstrQueryStop helpers (which use resource owners to handle aborts), or the Instrumentation struct itself with dedicated PG_TRY/PG_FINALLY calls that ensure the stack is in a consistent state after an abort. In tests, the stack-based instrumentation mechanism reduces the overhead of EXPLAIN (ANALYZE, BUFFERS ON, TIMING OFF) for a large COUNT(*) query from about 35% to about 20% on top of the actual runtime. Callers of the plain Instrumentation API (InstrAlloc/InstrInitOptions with InstrStart/InstrStop) keep measuring by diffing the global pgBufferUsage/pgWalUsage counters for now. The diff-mode handling lives only in InstrStart/InstrStop themselves: node instrumentation never measures by diffing, so InstrStartNode/InstrStopNode go through common helpers that handle just the timer and the stack, keeping the diff-mode branches out of the per-tuple ExecProcNodeInstr path (which would otherwise regress EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF) slightly relative to master). Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> (in an earlier version) Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> (in an earlier version) Discussion: https://postgr.es/m/CAP53PkxrmpECzVFpeeEEHDGe6u625s%2BYkmVv5-gw3L_NDSfbiA%40mail.gmail.com#cb583a08e8e096aa1f093bb178906173
The stack-based instrumentation left the plain Instrumentation API (InstrAlloc/InstrInitOptions with InstrStart/InstrStop) measuring by diffing the global pgBufferUsage/pgWalUsage counters, and kept pgBufferUsage updated alongside the stack for that purpose. Convert the remaining users of that API to the stack: VACUUM and ANALYZE logging as well as the planning-time measurement of EXPLAIN and EXECUTE use QueryInstrumentation, which handles aborts through the resource owner, while pg_stat_statements wraps its planner and utility statement timing in PG_TRY/PG_FINALLY and the EXPLAIN (SERIALIZE) destination receiver registers its entry as a child of the query instrumentation. With that, Instrumentation entries always track WAL/buffer usage through the stack, so remove the diff mode along with the global pgBufferUsage and BufferUsageAccumDiff, and update comments that referred to it. Extensions that read pgBufferUsage directly should use InstrStart and InstrStop around the code section they want to measure instead. The related global pgWalUsage is kept for now due to its use in pgstat to track aggregate WAL activity and heap_page_prune_and_freeze for measuring FPIs. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion: https://postgr.es/m/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mail.gmail.com
For most queries, the bulk of the overhead of EXPLAIN ANALYZE happens in ExecProcNodeInstr when starting/stopping instrumentation for that node. Previously each ExecProcNodeInstr would check which instrumentation options are active in the InstrStartNode/InstrStopNode calls, and do the corresponding work (timers, instrumentation stack, etc.). These conditionals being checked for each tuple being emitted add up, and cause a non-optimal set of instructions to be generated by the compiler. Because we already have a mechanism to specify a function pointer when instrumentation is enabled, we can instead create specialized functions that are tailored to the instrumentation options enabled, and avoid conditionals on subsequent ExecProcNodeInstr calls. In a stress test with a large COUNT(*) that does many ExecProcNode calls, this roughly halves the overhead of EXPLAIN (ANALYZE, TIMING OFF, BUFFERS OFF), from ~13% on top of actual runtime to ~6%. When using BUFFERS ON the same query goes from ~20% to ~15% overhead. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mail.gmail.com
This sets up a separate instrumentation stack that is used while an Index Scan fetches heap tuples, or an Index Only Scan checks the visibility map and fetches heap tuples for pages that are are not all-visible. EXPLAIN ANALYZE will now show "Table Buffers" that represent such activity. The activity is also included in regular "Buffers" together with index activity and that of any child nodes. Author: Lukas Fittl <lukas@fittl.com> Suggested-by: Andres Freund <andres@anarazel.de> (in an earlier version) Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> (in an earlier version) Reviewed-by: Tomas Vondra <tomas@vondra.me> (in an earlier version) Discussion: https://postgr.es/m/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mail.gmail.com
This is intended for testing instrumentation related logic as it pertains to the top level stack that is maintained as a running total. There is currently no in-core user that utilizes the top-level values in this manner, and especially during abort situations this helps ensure we don't lose activity due to incorrect handling of unfinalized node stacks.
pg_stat_database's blk_read_time and blk_write_time were accumulated in the backend-local counters pgStatBlockReadTime and pgStatBlockWriteTime, which pgstat_count_io_op_time() incremented alongside the equivalent fields of the current instrumentation stack entry. With the stack-based instrumentation the session-level totals are available in instr_top, so the separate counters are redundant. Remove the counters and the pgstat_count_buffer_read_time() and pgstat_count_buffer_write_time() macros, and instead have pgstat_update_dbstats() compute the increase of the shared and local block read/write times in instr_top since the previous report. This covers the same activity as before: relation and temp relation reads, writes and extends, but not temporary file I/O or WAL I/O. instr_top is only updated when an instrumentation stack entry is finalized, but pgstat_report_stat() is only called between transactions, when a backend is idle or at process exit. The stack is empty at the first two, but a FATAL error exits without unwinding the C stack and leaves entries behind, so finalize anything still on the stack before the final report. Parallel workers report their own activity themselves, like before, which requires each worker's instr_top to be complete. The switch to stack-based instrumentation left a gap, where per-node instrumentation does not accumulate upwards on the worker, to avoid double-counting in the leader import. Resolve by adding an additional walker that skips over the parent nodes and the query level instrumentation, directly accumulating to the top instrumentation in parallel workers. The leader additionally imports the workers' usage into its stack for query-level consumers such as EXPLAIN and pg_stat_statements, and thus into its instr_top. To count each activity exactly once, track everything imported from workers in the new instr_from_workers, updated at the same points as the import, and subtract it from instr_top when reporting. Since the amounts are now computed as a difference of the session totals, these totals must never be reset. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion: https://postgr.es/m/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mail.gmail.com
lfittl
force-pushed
the
instrument-usage-stack-v18
branch
from
October 31, 2026 14:32
2dc556e to
a5cc07e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.