This repository provides a Spack environment for installing and managing THAPI.
The easiest way to install THAPI is to use spack. This guide assumes Spack 1.0
or newer is installed and available in your PATH. If it is not, see How to Install Spack.
First, add THAPI-spack as a Spack repo (Spack clones it into its cache). Then, install THAPI:
spack repo add https://github.com/argonne-lcf/THAPI-spack.git
spack install thapiTo work from a local clone instead (e.g. to try a branch), point Spack at the repo inside it:
git clone https://github.com/argonne-lcf/THAPI-spack
spack repo add ./THAPI-spack/spack_repo/thapiOnce installed, you can load THAPI as a Spack package by doing:
spack load thapispack external find can be used to find existing packages known to Spack on the system. This way
you can avoid building them to save time when installing THAPI. Use the following command before
spack install thapi to find external packages available on the system. Make sure to module load
the packages (or set other environment variables like PATH) before running spack external find.
spack external find --all --exclude llvmWe always exclude llvm: the entry found by spack external find cannot satisfy h2yaml's LLVM
dependency. Declare llvm with the helper instead -- see Reusing a system LLVM.
In addition to LLVM, some packages found using spack external find are known to cause build failures.
If you run into such cases, use spack external find --exclude <pkg> so that Spack will build them
instead. For example, to exclude bzip2, xz and curl along with llvm, you can do:
spack external find --all --exclude bzip2 --exclude xz --exclude curl --exclude llvmthapi@0.0.14: requires h2yaml, which needs llvm@18:+clang+python. +python means that the LLVM
install must contain clang's Python bindings (clang/cindex.py), not just libclang.so. Building
LLVM from source is by far the most time-consuming step of a THAPI install, so it is worth reusing
a system LLVM install whenever one is available.
Point the helper script (scripts/gen-llvm-external.py in a clone of this repo) at any LLVM 18+ on
the system (module avail llvm, /usr/lib/llvm-*, /opt/llvm*, or your site software tree) and apply
what it prints. PYVER is the Python version Spack will run h2yaml with, so the overlay is laid out
to match.
PYVER=$(spack spec -j h2yaml | grep -v '^==>' \
| jq -r '[.spec.nodes[] | select(.name == "python") | .version | split(".")[:2] | join(".")]
| unique | .[0]')
python3 scripts/gen-llvm-external.py /path/to/system/llvm --python-version $PYVER > llvm-external.yaml
spack config add -f llvm-external.yamlThe script needs only Python 3, no dependencies. If the system LLVM already has Python bindings, the
script points the external straight at it. Otherwise -- a site LLVM is usually read-only, so they cannot
just be dropped in next to it -- it builds an overlay in --overlay-dir (default ~/.spack): a tree of
symlinks to the real install, plus the bindings, downloaded from the matching LLVM release if the
system has none. The external points into that directory, so keep it.
If there is no LLVM 18+ on the system at all, skip this: Spack will build one.
Check that it identified LLVM as an external package -- LLVM should be shown with an [e] in spack spec
output rather than just a -:
spack spec -I thapi | grep llvmImportant
The external has to be declared this way rather than by spack external find, which picks variants
from executables only and never looks for the bindings. It therefore always records LLVM as
~python -- whether bindings were present or not -- and such an entry can never satisfy h2yaml:
Spack ignores it and, unless something else provides +python, builds LLVM from source. Always pass
--exclude llvm when running it.
Depending on the number of available hardware cores on your platform, you can specify the number of parallel
build processes to be used during the build of spec (THAPI in this case) using the -j option to spack
(e.g., -j16).
You can use Spack to install the required dependencies and set up the environment to build THAPI manually from source (without using Spack to build THAPI):
spack build-env thapi bashThis will spawn a new shell with the correct environment variables and dependencies set for building THAPI manually. This is useful when building THAPI for your own development work.
- THAPI Spack package (and some of its dependencies) is known to run into build failures with NVHPC SDK (available on ALCF Polaris for example). Please make sure to use GNU programming environment whenever possible.
- Users have noticed
spackprocesses getting killed (due to timeout) on login nodes on both ALCF Aurora and Polaris. Use a compute node whenever possible.
develop, which is unstable. To ensure a reliable installation,
use the latest release of Spack.
To install Spack:
git clone -c feature.manyFiles=true -b releases/latest https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh