271 lines
12 KiB
Markdown
271 lines
12 KiB
Markdown
# The PSI Spack Deployment
|
|
|
|
[Official Spack documentation](https://spack.readthedocs.io/en/latest/)
|
|
|
|
## How to use on Merlin systems (Merlin6/Merlin7/Ra)
|
|
|
|
Load the spack module
|
|
|
|
```shell
|
|
[stable] user@login001:~> module load spack
|
|
```
|
|
|
|
Voilà you're done you can use Spack! :)
|
|
|
|
+ Side note: the default software stack loaded for you is stable, which might not contain all the packages you're looking for, </br>
|
|
so if you want to use the unstable software stack switch to it using:
|
|
|
|
```shell
|
|
spack env activate -p unstable
|
|
```
|
|
|
|
## Find out what's already installed
|
|
|
|
```shell
|
|
[unstable] user@login001:~> spack find
|
|
==> In environment unstable
|
|
==> 1 root specs
|
|
- []
|
|
|
|
==> Included specs
|
|
-- no arch / gcc ------------------------------------------------
|
|
gromacs%gcc
|
|
|
|
-- no arch / gcc@12.3.0 -----------------------------------------
|
|
cp2k@2024.1%gcc@12.3.0 opal@master%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0
|
|
cp2k@2024.1%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0 quantum-espresso@7.3.1%gcc@12.3.0
|
|
|
|
-- no arch / oneapi@2024.1.0 ------------------------------------
|
|
gromacs@2024.1%oneapi@2024.1.0
|
|
|
|
==> Installed packages
|
|
-- linux-rhel7-x86_64 / gcc@4.8.5 -------------------------------
|
|
autoconf@2.72 curl@8.7.1 gdbm@1.23 libffi@3.4.6 mpfr@4.2.1 pkg-config@0.29.2 readline@8.2
|
|
autoconf@2.72 diffutils@3.10 gettext@0.19.8.1 libidn2@2.3.7 mpfr@4.2.1 pkg-config@0.29.2 sqlite@3.43.2
|
|
autoconf-archive@2023.02.20 diffutils@3.10 gettext@0.19.8.1 libmd@1.0.4 ncurses@6.5 py-fypp@3.1 texinfo@7.0.3
|
|
autoconf-archive@2023.02.20 expat@2.6.2 git@2.42.0 libpciaccess@0.17 ncurses@6.5 py-fypp@3.1 util-linux-uuid@2.36.2
|
|
automake@1.16.5 findutils@4.9.0 glibc@2.17 libsigsegv@2.14 nghttp2@1.57.0 py-pip@23.1.2 util-macros@1.19.3
|
|
automake@1.16.5 findutils@4.9.0 glibc@2.17 libsigsegv@2.14 ninja@1.11.1 py-setuptools@59.4.0 xz@5.4.6
|
|
berkeley-db@18.1.40 flex@2.6.3 gmake@4.4.1 libtool@2.4.7 numactl@2.0.14 py-wheel@0.41.2 zlib-ng@2.1.6
|
|
berkeley-db@18.1.40 gawk@5.3.0 gmake@4.4.1 libunistring@1.2 nvhpc@24.3 python@3.8.18 zlib-ng@2.1.6
|
|
binutils@2.42 gawk@5.3.0 gmp@6.2.1 libxcrypt@4.4.35 openssh@9.7p1 python@3.9.18 zstd@1.5.6
|
|
binutils@2.42 gcc@12.3.0 gmp@6.2.1 libxml2@2.10.3 openssl@3.3.0 python@3.11.7 zstd@1.5.6
|
|
binutils@2.42 gcc@12.3.0 krb5@1.20.1 m4@1.4.19 openssl@3.3.0 python-venv@1.0
|
|
bison@3.8.2 gcc-runtime@4.8.5 libbsd@0.12.1 m4@1.4.19 pcre2@10.43 python-venv@1.0
|
|
bzip2@1.0.8 gcc-runtime@4.8.5 libedit@3.1-20230828 mpc@1.3.1 perl@5.38.0 re2c@2.2
|
|
bzip2@1.0.8 gdbm@1.23 libevent@2.1.12 mpc@1.3.1 perl@5.38.0 readline@8.2
|
|
|
|
-- linux-rhel7-x86_64 / gcc@12.3.0 ------------------------------
|
|
abseil-cpp@20240116.2 libogg@1.3.5 py-beniget@0.3.0 py-ml-collections@0.1.0 py-tabulate@0.8.9
|
|
amrex@18.07 libpng@1.6.39 py-biopython@1.79 py-networkx@2.7.1 py-tensorboard@2.11.2
|
|
aria2@1.37.0 libssh2@1.11.0 py-bottleneck@1.3.2 py-numexpr@2.8.4 py-tensorboard-data-server@0.6.1
|
|
boost@1.85.0 libtheora@1.1.1 py-cachetools@5.2.0 py-numpy@1.17.5 py-tensorboard-plugin-wit@1.8.1
|
|
boost@1.85.0 libxc@6.2.2 py-certifi@2023.7.22 py-numpy@1.21.6 py-tensorflow@2.11.0
|
|
c-ares@1.27.0 libxsmm@1.17 py-charset-normalizer@3.3.0 py-oauthlib@3.2.2 py-termcolor@1.1.0
|
|
cmake@3.27.9 libyaml@0.2.5 py-chex@0.0.7 py-opt-einsum@3.3.0 py-tomli@2.0.1
|
|
cp2k@2024.1 llvm@16.0.6 py-contextlib2@21.6.0 py-packaging@23.1 py-toolz@0.12.0
|
|
cp2k@2024.1 lua@5.3.6 py-cython@0.29.36 py-pandas@1.3.4 py-typing-extensions@4.8.0
|
|
cuda@12.4.0 lz4@1.9.4 py-decorator@5.1.1 py-pdbfixer@1.7 py-urllib3@1.26.12
|
|
fftw@3.3.10 mesa-glu@9.0.2 py-dm-haiku@0.0.9 py-pip@23.0 py-websocket-client@1.6.3
|
|
fftw@3.3.10 mesa18@18.3.6 py-dm-tree@0.1.6 py-pip@23.1.2 py-werkzeug@2.0.2
|
|
freetype@2.10.2 metis@5.1.0 py-docker@5.0.3 py-ply@3.11 py-wheel@0.37.1
|
|
gcc-runtime@12.3.0 mithra@2.0 py-etils@0.9.0 py-poetry-core@1.8.1 py-wheel@0.41.2
|
|
gcc-runtime@12.3.0 netlib-scalapack@2.2.0 py-flatbuffers@23.5.26 py-protobuf@3.19.4 py-wrapt@1.15.0
|
|
gettext@0.19.8.1 opal@master py-flit-core@3.9.0 py-pyasn1@0.4.8 py-zipp@3.17.0
|
|
gl2ps@1.4.2 openblas@0.3.25 py-gast@0.4.0 py-pyasn1-modules@0.2.8 python@3.8.18
|
|
googletest@1.12.1 openblas@0.3.26 py-google-auth@2.27.0 py-pybind11@2.12.0 python@3.11.7
|
|
gsl@2.7.1 openblas@0.3.26 py-google-auth-oauthlib@0.4.6 py-pytest-runner@6.0.0 python-venv@1.0
|
|
h5hut@2.0.0rc6 openjdk@11.0.20.1_1 py-google-pasta@0.2.0 py-python-dateutil@2.8.2 python-venv@1.0
|
|
hdf5@1.12.3 openmm@7.5.1 py-grpcio@1.60.1 py-pythran@0.9.11 quantum-espresso@7.3.1
|
|
hdf5@1.14.3 openmpi@4.1.6 py-h5py@3.11.0 py-pytz@2023.3 re2@2023-09-01
|
|
hdf5@1.14.3 openmpi@4.1.6 py-idna@3.4 py-pyyaml@6.0 rust-bootstrap@1.78.0
|
|
hh-suite@3.3.0 osmesa@11.2.0 py-immutabledict@2.2.1 py-requests@2.31.0 slurm@23-11-0-1
|
|
hmmer@3.4 parmetis@4.0.3 py-importlib-resources@5.12.0 py-requests-oauthlib@1.3.1 swig@4.1.1
|
|
hwloc@2.9.1 pcre@8.45 py-jax@0.3.25 py-rsa@4.9 trilinos@13.4.0
|
|
jsoncpp@1.9.5 perl-data-dumper@2.173 py-jaxlib@0.3.25 py-scipy@1.7.0 unzip@6.0
|
|
kalign@3.4.0 protobuf@3.19.4 py-jmp@0.0.2 py-setuptools@57.4.0 zip@3.0
|
|
libgcrypt@1.10.3 py-absl-py@1.0.0 py-libclang@16.0.0 py-setuptools@69.2.0
|
|
libgpg-error@1.49 py-alphafold@2.3.2 py-mako@1.2.4 py-setuptools@69.2.0
|
|
libint@2.6.0 py-alphafold@2.3.2 py-markdown@3.3.4 py-setuptools-scm@8.0.4
|
|
libjpeg@9f py-astunparse@1.6.3 py-markupsafe@2.1.3 py-six@1.16.0
|
|
|
|
-- linux-rhel7-x86_64 / oneapi@2024.1.0 -------------------------
|
|
gromacs@2024.1 hwloc@2.9.1 intel-oneapi-mkl@2024.0.0 intel-oneapi-runtime@2024.1.0 intel-tbb@2021.9.0 openmpi@4.1.6 slurm@23-11-0-1
|
|
==> 257 installed packages
|
|
```
|
|
|
|
The included specs and installed packages sections shows you all the packages that are pre-installed for you. </br>
|
|
In case of a package having multiple installations, the following command helps you to find out which software you're interested about. </br>
|
|
|
|
For example let's say you are interested in py-alphafold but only in the gpu version; do:
|
|
|
|
```
|
|
[unstable] user@login001:~> spack find -vl py-alphafold
|
|
==> In environment unstable
|
|
==> 1 root specs
|
|
- dpjaszc visit
|
|
|
|
==> Included specs
|
|
-- no arch / gcc ------------------------------------------------
|
|
------- gromacs%gcc
|
|
|
|
-- no arch / gcc@12.3.0 -----------------------------------------
|
|
------- cp2k@2024.1%gcc@12.3.0 ------- opal@master%gcc@12.3.0 ------- py-alphafold@2.3.2%gcc@12.3.0
|
|
------- cp2k@2024.1%gcc@12.3.0 ------- py-alphafold@2.3.2%gcc@12.3.0 ------- quantum-espresso@7.3.1%gcc@12.3.0
|
|
|
|
-- no arch / oneapi@2024.1.0 ------------------------------------
|
|
------- gromacs@2024.1%oneapi@2024.1.0
|
|
|
|
==> Installed packages
|
|
-- linux-rhel7-x86_64 / gcc@12.3.0 ------------------------------
|
|
rrfdppp py-alphafold@2.3.2~cuda build_system=python_pip tqziprv py-alphafold@2.3.2+cuda build_system=python_pip cuda_arch=60
|
|
==> 2 installed packages
|
|
```
|
|
|
|
As you can see there are two installation of py-alphafold, one with +cuda and one without. </br>
|
|
In this particular example, there is only one variant that interests you, so you can do:
|
|
|
|
``` shell
|
|
spack load py-alphafold +cuda
|
|
```
|
|
|
|
However if multiple variants are interesting to you and you don't want to have to cite all of them you can do:
|
|
|
|
``` shell
|
|
spack load /tqziprv # refer to the installation with its hash directly.
|
|
```
|
|
|
|
## Install your own software
|
|
|
|
1. Check if your package is already implemented in Spack. <br/>
|
|
You can also use the following website: https://packages.spack.io/
|
|
|
|
```shell
|
|
spack list $pkg_name # e.g opal
|
|
```
|
|
|
|
2. Check the package variants you want to set e.g +cuda +openmp ^openmpi ...
|
|
|
|
```shell
|
|
spack info $pkg_name # e.g opal
|
|
```
|
|
|
|
3. Add your package to your personal environment
|
|
|
|
```shell
|
|
spack add $pkg_name@version +variant1 ^dep1@version1
|
|
```
|
|
|
|
+ If you want to install from local source clone your source under <br/>
|
|
/afs/psi.ch/sys/spack/user/$USER/spack-environment/$pkg_name <br/>
|
|
and tell Spack you want to use it using:
|
|
|
|
```shell
|
|
spack develop $pkg_name@version # Skip this if you don't want to develop from local source
|
|
```
|
|
4. Check the whole dependency tree of your spec and <br/>
|
|
if you don't need to set more variants
|
|
|
|
```shell
|
|
spack concretize
|
|
```
|
|
|
|
5. Install the packages in your environment
|
|
|
|
```shell
|
|
spack install # -v for verbose
|
|
```
|
|
|
|
6. Load your package
|
|
|
|
```shell
|
|
spack load $pkg_name@version +variant1 ^dep1@version1
|
|
```
|
|
|
|
7. Submit your script
|
|
|
|
```shell
|
|
sbatch batch.script
|
|
```
|
|
## Switching between environments
|
|
|
|
There are two environment available to users; stable and unstable. <br/>
|
|
The first one is loaded by default when loading the module. <br/>
|
|
If you want to switch between the two and use different software stack you can use the following command:
|
|
|
|
```shell
|
|
spack env activate -p unstable # or stable
|
|
```
|
|
|
|
## Managing your environments
|
|
|
|
When adding the following command you actually add a package to your own software stack;
|
|
|
|
```shell
|
|
spack add $pkg_name@version +variant1 ^dep1@version1
|
|
```
|
|
|
|
You can check which package will be concretized and are in your environment using;
|
|
|
|
```shell
|
|
[unstable] user@login001:~> spack find
|
|
==> In environment unstable
|
|
==> 1 root specs
|
|
- visit ~adios2+vtkm+gui
|
|
|
|
==> Included specs
|
|
-- no arch / gcc ------------------------------------------------
|
|
gromacs%gcc
|
|
|
|
-- no arch / gcc@12.3.0 -----------------------------------------
|
|
cp2k@2024.1%gcc@12.3.0 opal@master%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0
|
|
cp2k@2024.1%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0 quantum-espresso@7.3.1%gcc@12.3.0
|
|
|
|
-- no arch / oneapi@2024.1.0 ------------------------------------
|
|
gromacs@2024.1%oneapi@2024.1.0
|
|
|
|
==> Installed packages
|
|
...
|
|
==> 257 installed packages
|
|
```
|
|
|
|
The root spec is the specs that are personal to you and that will be concretized and installed. <br/>
|
|
The installed packages are either packages that were previously installed by admins or packages that you already installed but that are not root. <br/>
|
|
|
|
If you want to remove this package from your personal env, you can use the following command: <br/>
|
|
|
|
```shell
|
|
spack rm $pkg_name@version +variant1 ^dep1@version1
|
|
```
|
|
|
|
or
|
|
|
|
```shell
|
|
[stable] user@login001:~> spack config edit
|
|
spack:
|
|
...
|
|
specs:
|
|
-- - visit~adios2+gui+vtkm ^harfbuzz%gcc@12.3.0
|
|
++ []
|
|
...
|
|
```
|
|
and remove it manually from the spec list.
|
|
|
|
## How to use on other systems
|
|
|
|
### Install Spack
|
|
|
|
```shell
|
|
user@supersystem:~> cd /scratch/$USER
|
|
|
|
user@supersystem:~> git clone -c feature.manyFiles=true https://github.com/spack/spack.git
|
|
|
|
user@supersystem:~> . spack/share/spack/setup-env.sh
|
|
|
|
# Add PSI specific recipes
|
|
user@supersystem:~> git clone https://gitlab.psi.ch/lsm-hpce/alps/spack-psi.git
|
|
|
|
user@supersystem:~> spack repo add $PWD/spack-psi
|
|
|
|
user@supersystem:~> spack install $pkg_name # -v for verbose
|