Skip to content

Use Triton GroupNorm on H100 - #106

Merged
PatrickRMiles merged 1 commit into
LBANN:mainfrom
michaelmckinsey1:michaelmckinsey1-patch-14
Sep 22, 2026
Merged

PatrickRMiles merged 1 commit into
LBANN:mainfrom
michaelmckinsey1:michaelmckinsey1-patch-14

Conversation

@michaelmckinsey1

@michaelmckinsey1 michaelmckinsey1 commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator

This is only because it is faster than the PyTorch compiled contiguous groupnorm (~2x). It's likely due to avoiding the channels-last conversion, not anything to do with the MI300A specific optimization.

@PatrickRMiles
PatrickRMiles merged commit a4cd661 into LBANN:main Sep 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants