Repository navigation
Conversation
Utility statements share many things with regular queries, but thus far there was no utility-statement equivalent of QueryDesc to hold common fields that matter for execution. Instead this information was passed down via an ever-growing list of arguments to ProcessUtility / ProcessUtility_hook. To enable a future change that will introduce another field, and to reduce future breakage of the hook API, introduce a dedicated utility statement descriptor (UtilityStmtDesc), and make it the single argument passed to ProcessUtility. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion:
Historically, parallel workers updated the global instrumentation (pgWalUsage/pgBufferUsage) in two places: the worker's own copy and through reporting via shared memory, and explicit accumulation, the leader's copy. This double reporting in the globals of multiple processes was acceptable when the globals were only used for statement-level reporting (and the worker globals effectively ignored), but that stopped being true after commit 0146924 started using pgWalUsage for reporting to the cumulative statistics system. This caused a bug with parallel workers double reporting their WAL in pg_stat_wal, once via the leader, and once via the worker, leading to incorrect data for utility statements like parallel vacuum, and queries that update hint bits, e.g. in a parallel seq scan. To fix, never accumulate worker activity into the leader's globals. Instead, introduce a consistent statement-level instrumentation we can accumulate into. For queries this exists with QueryDesc->query_instr, but for utility statements this needs to be invented. Introduce a new instrumentation ("instr") field on UtilityStmtDesc, and make it accessible to utility statement logic by introducing a new global ActiveUtilityStmtDesc that points to the currently active utility statement. Anyone interested in instrumentation must request it through instr_options, matching the existing logic for queries. Adjust pg_stat_statements to capture its statistics for utility statements from this instrumentation, instead of reading the globals. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion:
Previously pg_stat_database's blk_read_time and blk_write_time counters were accumulated separately via the backend-local pgStatBlockReadTime and pgStatBlockWriteTime, despite the same information already being tracked in pgBufferUsage. Remove the separate counters and instead diff pgBufferUsage on the pre-existing flush interval to capture the I/O timings. This reporting happens separately on the leader and each worker in a parallel query, benefiting from the recent fix that ensured that pgBufferUsage on the leader does not include worker activity. In passing, update the ANALYZE command to also derive its I/O timing log output from pgBufferUsage (via Instrumentation helpers), avoiding a dependency on the now removed pgStatBlock(Read|Write)Time globals. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.