Skip to content

About

This is a spack environment for THAPI.

Resources

Stars

6 stars

Watchers

9 watching

Forks

Repository files navigation

THAPI-spack

This repository provides a Spack environment for installing and managing THAPI.

How to Install THAPI

The easiest way to install THAPI is to use spack. This guide assumes Spack 1.0 or newer is installed and available in your PATH. If it is not, see How to Install Spack.

First, add THAPI-spack as a Spack repo (Spack clones it into its cache). Then, install THAPI:

spack repo add https://github.com/argonne-lcf/THAPI-spack.git
spack install thapi

To work from a local clone instead (e.g. to try a branch), point Spack at the repo inside it:

git clone https://github.com/argonne-lcf/THAPI-spack
spack repo add ./THAPI-spack/spack_repo/thapi

Once installed, you can load THAPI as a Spack package by doing:

spack load thapi

Tips For Speeding Up THAPI Installation

spack external find

spack external find can be used to find existing packages known to Spack on the system. This way you can avoid building them to save time when installing THAPI. Use the following command before spack install thapi to find external packages available on the system. Make sure to module load the packages (or set other environment variables like PATH) before running spack external find.

spack external find --all --exclude llvm

We always exclude llvm: the entry found by spack external find cannot satisfy h2yaml's LLVM dependency. Declare llvm with the helper instead -- see Reusing a system LLVM.

In addition to LLVM, some packages found using spack external find are known to cause build failures. If you run into such cases, use spack external find --exclude <pkg> so that Spack will build them instead. For example, to exclude bzip2, xz and curl along with llvm, you can do:

spack external find --all --exclude bzip2 --exclude xz --exclude curl --exclude llvm

Reusing a system LLVM

thapi@0.0.14: requires h2yaml, which needs llvm@18:+clang+python. +python means that the LLVM install must contain clang's Python bindings (clang/cindex.py), not just libclang.so. Building LLVM from source is by far the most time-consuming step of a THAPI install, so it is worth reusing a system LLVM install whenever one is available.

Point the helper script (scripts/gen-llvm-external.py in a clone of this repo) at any LLVM 18+ on the system (module avail llvm, /usr/lib/llvm-*, /opt/llvm*, or your site software tree) and apply what it prints. PYVER is the Python version Spack will run h2yaml with, so the overlay is laid out to match.

PYVER=$(spack spec -j h2yaml | grep -v '^==>' \
  | jq -r '[.spec.nodes[] | select(.name == "python") | .version | split(".")[:2] | join(".")]
           | unique | .[0]')

python3 scripts/gen-llvm-external.py /path/to/system/llvm --python-version $PYVER > llvm-external.yaml
spack config add -f llvm-external.yaml

The script needs only Python 3, no dependencies. If the system LLVM already has Python bindings, the script points the external straight at it. Otherwise -- a site LLVM is usually read-only, so they cannot just be dropped in next to it -- it builds an overlay in --overlay-dir (default ~/.spack): a tree of symlinks to the real install, plus the bindings, downloaded from the matching LLVM release if the system has none. The external points into that directory, so keep it.

If there is no LLVM 18+ on the system at all, skip this: Spack will build one.

Check that it identified LLVM as an external package -- LLVM should be shown with an [e] in spack spec output rather than just a -:

spack spec -I thapi | grep llvm

Important

The external has to be declared this way rather than by spack external find, which picks variants from executables only and never looks for the bindings. It therefore always records LLVM as ~python -- whether bindings were present or not -- and such an entry can never satisfy h2yaml: Spack ignores it and, unless something else provides +python, builds LLVM from source. Always pass --exclude llvm when running it.

spack install -j<core> <spec>

Depending on the number of available hardware cores on your platform, you can specify the number of parallel build processes to be used during the build of spec (THAPI in this case) using the -j option to spack (e.g., -j16).

Building THAPI Manually

You can use Spack to install the required dependencies and set up the environment to build THAPI manually from source (without using Spack to build THAPI):

spack build-env thapi bash

This will spawn a new shell with the correct environment variables and dependencies set for building THAPI manually. This is useful when building THAPI for your own development work.

Known Issues

  • THAPI Spack package (and some of its dependencies) is known to run into build failures with NVHPC SDK (available on ALCF Polaris for example). Please make sure to use GNU programming environment whenever possible.
  • Users have noticed spack processes getting killed (due to timeout) on login nodes on both ALCF Aurora and Polaris. Use a compute node whenever possible.

Miscellaneous

How to Install Spack

⚠️ Important: The default branch for Spack is develop, which is unstable. To ensure a reliable installation, use the latest release of Spack.

To install Spack:

git clone -c feature.manyFiles=true -b releases/latest https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh

About

This is a spack environment for THAPI.

Resources

Stars

6 stars

Watchers

9 watching

Forks

Releases

Packages

Used by

Contributors

Languages