Repository navigation
Conversation
Each parallel worker tracks its own activity in its copy of the global instrumentation counters (pgWalUsage/pgBufferUsage), reports the delta through shared memory when it finishes, and the leader then accumulates that into its copy of the globals (InstrAccumParallelQuery). Having the same activity in the globals of two processes was acceptable when the globals were only used for statement-level reporting, where the worker globals are effectively ignored. That stopped being true when commit 0146924 started diffing pgWalUsage to report WAL activity to the cumulative statistics system. This caused a bug with parallel workers double reporting their WAL in pg_stat_wal, once via the leader and once via the worker, leading to incorrect data for utility statements like parallel vacuum, and queries that cause pruning or update hint bits, e.g. in a parallel sequential scan. The same bug applies to the per-backend cumulative statistics accessible via pg_stat_get_backend_wal(). To fix, remember the cumulative total of WAL/buffer statistics that we have imported from the workers into the leader, and subtract it when reporting to the cumulative statistics system. That way each worker reports its activity itself, exactly once. Author: Lukas Fittl <lukas@fittl.com> Reviewed-by: Discussion:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each parallel worker tracks its own activity in its copy of the global instrumentation counters (pgWalUsage/pgBufferUsage), reports the delta through shared memory when it finishes, and the leader then accumulates that into its copy of the globals (InstrAccumParallelQuery).
Having the same activity in the globals of two processes was acceptable when the globals were only used for statement-level reporting, where the worker globals are effectively ignored. That stopped being true when commit 0146924 started diffing pgWalUsage to report WAL activity to the cumulative statistics system. This caused a bug with parallel workers double reporting their WAL in pg_stat_wal, once via the leader and once via the worker, leading to incorrect data for utility statements like parallel vacuum, and queries that cause pruning or update hint bits, e.g. in a parallel sequential scan. The same bug applies to the per-backend cumulative statistics accessible via pg_stat_get_backend_wal().
To fix, remember the cumulative total of WAL/buffer statistics that we have imported from the workers into the leader, and subtract it when reporting to the cumulative statistics system. That way each worker reports its activity itself, exactly once.