Repository navigation
perftest: add a ROCm build to CI - #418
Merged
Merged
Conversation
sbates130272
added a commit
to sbates130272/perftest
that referenced
this pull request
Nov 4, 2026
CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green. Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way, as a matrix over gfx950 (Instinct MI355X, CDNA4) and gfx1250 (Instinct MI455X, CDNA5). Those two were picked to cover consecutive Instinct generations: HIP sources that compile correctly for one generation may have microarchitecture-specific issues on the next, and testing both catches that. fail-fast is off: a failure is far more likely to be specific to one generation than common to both, and cancelling the other leg hides exactly the comparison that would say so. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc, the HIP and HSA headers, and device support for both matrix targets -- llc -mcpu=help lists gfx950 and gfx1250, and both compile. 7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API. Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite. Verified by replaying these steps in the container across both matrix architectures: configure accepts every option, and make and make install succeed warning-free. Link: linux-rdma#418 Signed-off-by: Stephen Bates <sbates@raithlin.com>
sbates130272
added a commit
to sbates130272/perftest
that referenced
this pull request
Nov 4, 2026
CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green. Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way, as a matrix over gfx950 (Instinct MI355X, CDNA4) and gfx1250 (Instinct MI455X, CDNA5). Those two were picked to cover consecutive Instinct generations: HIP sources that compile correctly for one generation may have microarchitecture-specific issues on the next, and testing both catches that. fail-fast is off: a failure is far more likely to be specific to one generation than common to both, and cancelling the other leg hides exactly the comparison that would say so. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc, the HIP and HSA headers, and device support for both matrix targets -- llc -mcpu=help lists gfx950 and gfx1250, and both compile. 7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API. Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite. Verified by running this workflow in a fork of the repository across both matrix architectures. Both legs passed: configure accepts every option, and make and make install succeed warning-free. Link: linux-rdma#418 Signed-off-by: Stephen Bates <sbates@raithlin.com>
sbates130272
added a commit
to sbates130272/perftest
that referenced
this pull request
Nov 4, 2026
CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green. Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way, as a matrix over gfx950 (Instinct MI355X, CDNA4) and gfx1250 (Instinct MI455X, CDNA5). Those two were picked to cover consecutive Instinct generations: HIP sources that compile correctly for one generation may have microarchitecture-specific issues on the next, and testing both catches that. fail-fast is off: a failure is far more likely to be specific to one generation than common to both, and cancelling the other leg hides exactly the comparison that would say so. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc, the HIP and HSA headers, and device support for both matrix targets -- llc -mcpu=help lists gfx950 and gfx1250, and both compile. 7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API. Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite. Verified by running this workflow in a fork of the repository across both matrix architectures. Both legs passed: configure accepts every option, and make and make install succeed warning-free. Link: linux-rdma#418 Signed-off-by: Stephen Bates <sbates@raithlin.com>
sbates130272
added a commit
to sbates130272/perftest
that referenced
this pull request
Nov 4, 2026
CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green. Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way, as a matrix over gfx950 (Instinct MI355X, CDNA4) and gfx1250 (Instinct MI455X, CDNA5). Those two were picked to cover consecutive Instinct generations: HIP sources that compile correctly for one generation may have microarchitecture-specific issues on the next, and testing both catches that. fail-fast is off: a failure is far more likely to be specific to one generation than common to both, and cancelling the other leg hides exactly the comparison that would say so. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc, the HIP and HSA headers, and device support for both matrix targets -- llc -mcpu=help lists gfx950 and gfx1250, and both compile. 7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API. Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite. Verified by running this workflow in a fork of the repository across both matrix architectures. Both legs passed: configure accepts every option, and make and make install succeed warning-free. Link: linux-rdma#418 Signed-off-by: Stephen Bates <sbates@raithlin.com>
CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green. Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc and the HIP and HSA headers needed to compile the ROCm backend. 7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API. This is a single host-only build; there are no HIP kernel sources in the project yet, so there is nothing to compile for a specific device target. When device code and --with-rocm-arch support are introduced this job can gain an architecture matrix at that point. Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite. Verified by running this workflow in a fork of the repository. The job passed: configure accepts every option, and make and make install succeed warning-free. Link: linux-rdma#418 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Stephen Bates <sbates@raithlin.com>
sbates130272
force-pushed
the
feat/rocm-ci-build
branch
from
November 6, 2026 18:49
bf8dcc6 to
f94d2ea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds a CI job that builds the ROCm configuration, alongside the two existing CUDA jobs.
One new file,
.github/workflows/ubuntu24_04_rocm7_2.yaml. No source changes.Why
The two existing workflows pass no
--enable-rocm, sosrc/rocm_memory.cand everything behindHAVE_ROCMandHAVE_ROCM_DMABUFis never compiled on a pull request. A change that breaks the ROCm backend goes green today.Details worth reviewing
It's a matrix over
gfx950andgfx1250, withfail-fast: false. Those two sit on opposite sides of the differences HIP code has to get right — CDNA against RDNA, and a 64-lane wavefront against a 32-lane one — so building both says more than building either twice.fail-fastis off because a failure is far more likely to be specific to one target's code generation than common to both, and cancelling the other leg hides exactly the comparison that would tell you which.It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The
-fulland-completetags, and an apt install of theamdrocm-coremetapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen, BLAS and FFT. A compile check touches none of it.rocm/dev-ubuntu-24.04:7.2.4(used here)rocm/dev-ubuntu-26.04:10.1.0-fullamdrocm-core-sdk10.1ROCm 7.2.4 rather than 10.1 is deliberate, not an oversight. AMD publishes slim tags only through 7.2.x; every 10.x tag is
-full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, and that image provides it —llc -mcpu=helplists bothgfx950andgfx1250, and both compile. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API.Using the image also keeps repository plumbing out of the workflow. ROCm 10.1 is no longer served from the
repo.radeon.comlayout older releases use — it moved tostable.repo.amd.comwith deb822 sources andamdrocm-*package names — and a workflow carrying that recipe would need revisiting the next time it changes.Two small things that look defensive but aren't.
The ROCm prefix is resolved at runtime rather than assumed. The image ships
/opt/rocmas a symlink to a versioned prefix, andconfigureneeds the include path, so the job resolves whichever spelling is present.--with-rocm-archis probed for rather than passed. It selects the GPU that HIP kernels are compiled for, so it only exists once there are HIP sources to compile.configuretreats an unknown--with-*as a warning rather than an error, so passing it blind would silently do nothing while appearing to work. Probing means this workflow is correct both before and after such sources exist, without needing to be amended when they do.Testing
Replayed these exact steps in the container, both matrix architectures, against this tree:
configureaccepts every option,makeandmake installsucceed, zero warnings.Also replayed against a branch that does carry HIP sources, to confirm the probe does what it claims:
configurereports the matrix architecture as its target, andllvm-objdump --offloadingshows the resulting object contains a bundle for it. So the matrix selects what it says it does rather than merely accepting a flag.Scope
Builds and installs only, like the CUDA jobs. There is no RDMA device in a GitHub runner and the project has no test suite, so this catches compile and link breakage and nothing more.