Repository navigation
GPU: smaller thread groups for the merger track fit and the tracking data on Metal - #16059
Merged
Merged
Conversation
The kernels, the memory clear, GPUMemCpy and WriteToConstantMemory ignored the events they were given, which is how the chain orders the work of its lanes across streams. Wait for the event list before and record the event after, as the CUDA backend does.
Consecutive command buffers of one Metal queue are only ordered as far as the hazard tracking of the bound buffers sees, and the kernels reach most memory through addresses stored in constant memory. A kernel could thus start before the one before it on the same stream had finished: on a 5-event pp time frame the cluster finder found no clusters, and between 12,125 and 12,142 instead of 19,009 with every kernel synchronised. Make each command buffer of a stream wait on a per-stream event that the previous one signals, which gives a CUDA stream's in-order semantics without blocking the host.
The multiprocessor count, of which the automatically sized grids launch minBlocks blocks each, was never set and stayed 1, so those kernels ran one or a few threadgroups on a 32-core M1 Max. Metal does not report the core count; read gpu-core-count from the GPU's IORegistry entry, as CUDA uses multiProcessorCount, and link IOKit for that.
Every kernel, memory clear and copy was a command buffer of its own, 2,689 per event on a 5-event pp time frame, each committed at once and, since they run in order, each waiting for the previous one's event. Keep encoding into the stream's open buffer instead, ordered by a per-stream MTLFence between encoders, and commit it only where something has to wait for it: draining the stream, recording an event, waiting for events. This makes 535 command buffers per event, and the time per event about 3 to 6% less on an M1 Max.
GPUCA_METAL_NO_FAST_MATH_FLAGS is a string of two flags, appended to METAL_FLAGS as one list element, so the Metal compiler got the single argument '-fno-fast-math -ffp-contract=off' and rejected it: with GPUCA_DETERMINISTIC_MODE set to NO_FAST_MATH or OPTO2 the Metal kernels did not build. Split the string first.
…etal A tuning of the Metal column of GPUParameters.csv only, which changes nothing for the other backends. The column was empty, so the kernel ran with the default 256 threads and 1 block per GPU core, 32 blocks on an M1 Max. With 8 blocks per core it takes 50 instead of 109 ms per event of GPU time on a 30-collision Pb-Pb time frame, and the whole event 1.339 instead of 1.389 s (--runs 10, 3 interleaved rounds). 2 and 4 blocks are slower, 16 no faster. The other kernels with an automatic grid do not get faster with more blocks.
…data on Metal A tuning of the Metal column of GPUParameters.csv only, which changes nothing for the other backends. On an M1 Max with a 30-collision Pb-Pb time frame, GPU time per event: GPUTPCGMMergerTrackFit 256: 158.5 ms 128 x 2: 141.6 64 x 4: 120.9 32 x 8: 120.3 GPUTPCCreateTrackingData 256: 44.0 ms 128: 27.3 64: 17.9 32: 19.8 and the whole event 1.331 s before, 1.278 s with [64, 4] and 64 (--runs 10, 3 interleaved rounds, spread under 15 ms). Smaller groups did not help GPUTPCTrackletConstructor (219 ms at 256, 218 at 128, 223 at 64, slower at 192, 512 and 1024), GPUTPCGMMergerSectorRefit (131), GPUTPCExtrapolationTracking (67 to 64) or GPUTPCGMMergerFollowLoopers (34 to 36), which keep the defaults.
Member
Author
|
All changes in the Metal folder / Metal parameters. Merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A tuning of the Metal column of GPUParameters.csv only, which changes
nothing for the other backends. On an M1 Max with a 30-collision Pb-Pb
time frame, GPU time per event:
GPUTPCGMMergerTrackFit 256: 158.5 ms 128 x 2: 141.6 64 x 4: 120.9 32 x 8: 120.3
GPUTPCCreateTrackingData 256: 44.0 ms 128: 27.3 64: 17.9 32: 19.8
and the whole event 1.331 s before, 1.278 s with [64, 4] and 64 (--runs
10, 3 interleaved rounds, spread under 15 ms). Smaller groups did not
help GPUTPCTrackletConstructor (219 ms at 256, 218 at 128, 223 at 64,
slower at 192, 512 and 1024), GPUTPCGMMergerSectorRefit (131),
GPUTPCExtrapolationTracking (67 to 64) or GPUTPCGMMergerFollowLoopers
(34 to 36), which keep the defaults.