Skip to content

Commit 9e0c15e

Browse files
committed
GPU: make Metal's cluster widths round as on the CPU
Needs review. On a 30-collision Pb-Pb time frame, 7 of 4,416,506 clusters differed between the CPU and the Metal backend, each by one step of a packed width, sigmaTime or sigmaPad. Those widths were all within one float ulp below a quantisation step: sigma * 32 was 11.4999994, 14.4999992, 29.4999996 or 31.4999988, where the packing rounds at n + 0.5, so any difference in the last bit decides between n and n + 1. The CPU build compiles this with -ffast-math. Its arm64 code for finalize() computes 1 / Q once, the means as products with it, the variance as one fused multiply-add, fma(SS, 1 / Q, -mean * mean), and an IEEE sqrt. Metal's fast-math division and sqrt are not correctly rounded. Do the same operations on Metal explicitly, with precise::divide, fma and precise::sqrt: the pbpb clusters then match the CPU bit for bit, with no measurable cost; the rest of the Metal code keeps fast math. This matches the CPU build that the compiler produces on arm64 with clang. An x86 build, or another compiler, may choose a different sequence under -ffast-math. Writing the sequence explicitly for all backends would make the widths independent of those choices, but can change the CPU results on some platforms.
1 parent c166e26 commit 9e0c15e

1 file changed

Lines changed: 13 additions & 0 deletions

File tree

‎GPU/GPUTracking/TPCClusterFinder/ClusterAccumulator.cxx‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,13 +62,26 @@ GPUd() void ClusterAccumulator::finalize(const CfChargePos& pos, const Charge q,
6262
{
6363
mQtot += q;
6464

65+
#ifdef __METAL__
66+
// Metal's fast-math division and sqrt are not correctly rounded, which moves
67+
// widths that sit within an ulp of a packing step. Do the operations the
68+
// arm64 -ffast-math CPU build does here: reciprocal, products, fused variance.
69+
const float invQ = metal::precise::divide(1.f, mQtot);
70+
mPadMean *= invQ;
71+
mTimeMean *= invQ;
72+
const float padMean2 = mPadMean * mPadMean;
73+
const float timeMean2 = mTimeMean * mTimeMean;
74+
mPadSigma = metal::precise::sqrt(metal::fma(mPadSigma, invQ, -padMean2));
75+
mTimeSigma = metal::precise::sqrt(metal::fma(mTimeSigma, invQ, -timeMean2));
76+
#else
6577
mPadMean /= mQtot;
6678
mTimeMean /= mQtot;
6779
mPadSigma /= mQtot;
6880
mTimeSigma /= mQtot;
6981

7082
mPadSigma = CAMath::Sqrt(mPadSigma - mPadMean * mPadMean);
7183
mTimeSigma = CAMath::Sqrt(mTimeSigma - mTimeMean * mTimeMean);
84+
#endif
7285

7386
Pad pad = pos.pad();
7487
mPadMean += pad;

0 commit comments

Comments
 (0)