# The PSI Spack Deployment
[Official Spack documentation](https://spack.readthedocs.io/en/latest/)
## How to use on Merlin systems (Merlin6/Merlin7/Ra)
Load the spack module
```shell
[stable] user@login001:~> module load spack
```
Voilà you're done you can use Spack! :)
+ Side note: the default software stack loaded for you is stable, which might not contain all the packages you're looking for,
so if you want to use the unstable software stack switch to it using:
```shell
spack env activate -p unstable
```
## Find out what's already installed
```shell
[unstable] user@login001:~> spack find
==> In environment unstable
==> 1 root specs
- []
==> Included specs
-- no arch / gcc ------------------------------------------------
gromacs%gcc
-- no arch / gcc@12.3.0 -----------------------------------------
cp2k@2024.1%gcc@12.3.0 opal@master%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0
cp2k@2024.1%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0 quantum-espresso@7.3.1%gcc@12.3.0
-- no arch / oneapi@2024.1.0 ------------------------------------
gromacs@2024.1%oneapi@2024.1.0
==> Installed packages
-- linux-rhel7-x86_64 / gcc@4.8.5 -------------------------------
autoconf@2.72 curl@8.7.1 gdbm@1.23 libffi@3.4.6 mpfr@4.2.1 pkg-config@0.29.2 readline@8.2
autoconf@2.72 diffutils@3.10 gettext@0.19.8.1 libidn2@2.3.7 mpfr@4.2.1 pkg-config@0.29.2 sqlite@3.43.2
autoconf-archive@2023.02.20 diffutils@3.10 gettext@0.19.8.1 libmd@1.0.4 ncurses@6.5 py-fypp@3.1 texinfo@7.0.3
autoconf-archive@2023.02.20 expat@2.6.2 git@2.42.0 libpciaccess@0.17 ncurses@6.5 py-fypp@3.1 util-linux-uuid@2.36.2
automake@1.16.5 findutils@4.9.0 glibc@2.17 libsigsegv@2.14 nghttp2@1.57.0 py-pip@23.1.2 util-macros@1.19.3
automake@1.16.5 findutils@4.9.0 glibc@2.17 libsigsegv@2.14 ninja@1.11.1 py-setuptools@59.4.0 xz@5.4.6
berkeley-db@18.1.40 flex@2.6.3 gmake@4.4.1 libtool@2.4.7 numactl@2.0.14 py-wheel@0.41.2 zlib-ng@2.1.6
berkeley-db@18.1.40 gawk@5.3.0 gmake@4.4.1 libunistring@1.2 nvhpc@24.3 python@3.8.18 zlib-ng@2.1.6
binutils@2.42 gawk@5.3.0 gmp@6.2.1 libxcrypt@4.4.35 openssh@9.7p1 python@3.9.18 zstd@1.5.6
binutils@2.42 gcc@12.3.0 gmp@6.2.1 libxml2@2.10.3 openssl@3.3.0 python@3.11.7 zstd@1.5.6
binutils@2.42 gcc@12.3.0 krb5@1.20.1 m4@1.4.19 openssl@3.3.0 python-venv@1.0
bison@3.8.2 gcc-runtime@4.8.5 libbsd@0.12.1 m4@1.4.19 pcre2@10.43 python-venv@1.0
bzip2@1.0.8 gcc-runtime@4.8.5 libedit@3.1-20230828 mpc@1.3.1 perl@5.38.0 re2c@2.2
bzip2@1.0.8 gdbm@1.23 libevent@2.1.12 mpc@1.3.1 perl@5.38.0 readline@8.2
-- linux-rhel7-x86_64 / gcc@12.3.0 ------------------------------
abseil-cpp@20240116.2 libogg@1.3.5 py-beniget@0.3.0 py-ml-collections@0.1.0 py-tabulate@0.8.9
amrex@18.07 libpng@1.6.39 py-biopython@1.79 py-networkx@2.7.1 py-tensorboard@2.11.2
aria2@1.37.0 libssh2@1.11.0 py-bottleneck@1.3.2 py-numexpr@2.8.4 py-tensorboard-data-server@0.6.1
boost@1.85.0 libtheora@1.1.1 py-cachetools@5.2.0 py-numpy@1.17.5 py-tensorboard-plugin-wit@1.8.1
boost@1.85.0 libxc@6.2.2 py-certifi@2023.7.22 py-numpy@1.21.6 py-tensorflow@2.11.0
c-ares@1.27.0 libxsmm@1.17 py-charset-normalizer@3.3.0 py-oauthlib@3.2.2 py-termcolor@1.1.0
cmake@3.27.9 libyaml@0.2.5 py-chex@0.0.7 py-opt-einsum@3.3.0 py-tomli@2.0.1
cp2k@2024.1 llvm@16.0.6 py-contextlib2@21.6.0 py-packaging@23.1 py-toolz@0.12.0
cp2k@2024.1 lua@5.3.6 py-cython@0.29.36 py-pandas@1.3.4 py-typing-extensions@4.8.0
cuda@12.4.0 lz4@1.9.4 py-decorator@5.1.1 py-pdbfixer@1.7 py-urllib3@1.26.12
fftw@3.3.10 mesa-glu@9.0.2 py-dm-haiku@0.0.9 py-pip@23.0 py-websocket-client@1.6.3
fftw@3.3.10 mesa18@18.3.6 py-dm-tree@0.1.6 py-pip@23.1.2 py-werkzeug@2.0.2
freetype@2.10.2 metis@5.1.0 py-docker@5.0.3 py-ply@3.11 py-wheel@0.37.1
gcc-runtime@12.3.0 mithra@2.0 py-etils@0.9.0 py-poetry-core@1.8.1 py-wheel@0.41.2
gcc-runtime@12.3.0 netlib-scalapack@2.2.0 py-flatbuffers@23.5.26 py-protobuf@3.19.4 py-wrapt@1.15.0
gettext@0.19.8.1 opal@master py-flit-core@3.9.0 py-pyasn1@0.4.8 py-zipp@3.17.0
gl2ps@1.4.2 openblas@0.3.25 py-gast@0.4.0 py-pyasn1-modules@0.2.8 python@3.8.18
googletest@1.12.1 openblas@0.3.26 py-google-auth@2.27.0 py-pybind11@2.12.0 python@3.11.7
gsl@2.7.1 openblas@0.3.26 py-google-auth-oauthlib@0.4.6 py-pytest-runner@6.0.0 python-venv@1.0
h5hut@2.0.0rc6 openjdk@11.0.20.1_1 py-google-pasta@0.2.0 py-python-dateutil@2.8.2 python-venv@1.0
hdf5@1.12.3 openmm@7.5.1 py-grpcio@1.60.1 py-pythran@0.9.11 quantum-espresso@7.3.1
hdf5@1.14.3 openmpi@4.1.6 py-h5py@3.11.0 py-pytz@2023.3 re2@2023-09-01
hdf5@1.14.3 openmpi@4.1.6 py-idna@3.4 py-pyyaml@6.0 rust-bootstrap@1.78.0
hh-suite@3.3.0 osmesa@11.2.0 py-immutabledict@2.2.1 py-requests@2.31.0 slurm@23-11-0-1
hmmer@3.4 parmetis@4.0.3 py-importlib-resources@5.12.0 py-requests-oauthlib@1.3.1 swig@4.1.1
hwloc@2.9.1 pcre@8.45 py-jax@0.3.25 py-rsa@4.9 trilinos@13.4.0
jsoncpp@1.9.5 perl-data-dumper@2.173 py-jaxlib@0.3.25 py-scipy@1.7.0 unzip@6.0
kalign@3.4.0 protobuf@3.19.4 py-jmp@0.0.2 py-setuptools@57.4.0 zip@3.0
libgcrypt@1.10.3 py-absl-py@1.0.0 py-libclang@16.0.0 py-setuptools@69.2.0
libgpg-error@1.49 py-alphafold@2.3.2 py-mako@1.2.4 py-setuptools@69.2.0
libint@2.6.0 py-alphafold@2.3.2 py-markdown@3.3.4 py-setuptools-scm@8.0.4
libjpeg@9f py-astunparse@1.6.3 py-markupsafe@2.1.3 py-six@1.16.0
-- linux-rhel7-x86_64 / oneapi@2024.1.0 -------------------------
gromacs@2024.1 hwloc@2.9.1 intel-oneapi-mkl@2024.0.0 intel-oneapi-runtime@2024.1.0 intel-tbb@2021.9.0 openmpi@4.1.6 slurm@23-11-0-1
==> 257 installed packages
```
The included specs and installed packages sections shows you all the packages that are pre-installed for you.
In case of a package having multiple installations, the following command helps you to find out which software you're interested about.
For example let's say you are interested in py-alphafold but only in the gpu version; do:
```
[unstable] user@login001:~> spack find -vl py-alphafold
==> In environment unstable
==> 1 root specs
- dpjaszc visit
==> Included specs
-- no arch / gcc ------------------------------------------------
------- gromacs%gcc
-- no arch / gcc@12.3.0 -----------------------------------------
------- cp2k@2024.1%gcc@12.3.0 ------- opal@master%gcc@12.3.0 ------- py-alphafold@2.3.2%gcc@12.3.0
------- cp2k@2024.1%gcc@12.3.0 ------- py-alphafold@2.3.2%gcc@12.3.0 ------- quantum-espresso@7.3.1%gcc@12.3.0
-- no arch / oneapi@2024.1.0 ------------------------------------
------- gromacs@2024.1%oneapi@2024.1.0
==> Installed packages
-- linux-rhel7-x86_64 / gcc@12.3.0 ------------------------------
rrfdppp py-alphafold@2.3.2~cuda build_system=python_pip tqziprv py-alphafold@2.3.2+cuda build_system=python_pip cuda_arch=60
==> 2 installed packages
```
As you can see there are two installation of py-alphafold, one with +cuda and one without.
In this particular example, there is only one variant that interests you, so you can do:
``` shell
spack load py-alphafold +cuda
```
However if multiple variants are interesting to you and you don't want to have to cite all of them you can do:
``` shell
spack load /tqziprv # refer to the installation with its hash directly.
```
## Install your own software
1. Check if your package is already implemented in Spack.
You can also use the following website: https://packages.spack.io/
```shell
spack list $pkg_name # e.g opal
```
2. Check the package variants you want to set e.g +cuda +openmp ^openmpi ...
```shell
spack info $pkg_name # e.g opal
```
3. Add your package to your personal environment
```shell
spack add $pkg_name@version +variant1 ^dep1@version1
```
+ If you want to install from local source clone your source under
/afs/psi.ch/sys/spack/user/$USER/spack-environment/$pkg_name
and tell Spack you want to use it using:
```shell
spack develop $pkg_name@version # Skip this if you don't want to develop from local source
```
4. Check the whole dependency tree of your spec and
if you don't need to set more variants
```shell
spack concretize
```
5. Install the packages in your environment
```shell
spack install # -v for verbose
```
6. Load your package
```shell
spack load $pkg_name@version +variant1 ^dep1@version1
```
7. Submit your script
```shell
sbatch batch.script
```
## Switching between environments
There are two environment available to users; stable and unstable.
The first one is loaded by default when loading the module.
If you want to switch between the two and use different software stack you can use the following command:
```shell
spack env activate -p unstable # or stable
```
## Managing your environments
When adding the following command you actually add a package to your own software stack;
```shell
spack add $pkg_name@version +variant1 ^dep1@version1
```
You can check which package will be concretized and are in your environment using;
```shell
[unstable] user@login001:~> spack find
==> In environment unstable
==> 1 root specs
- visit ~adios2+vtkm+gui
==> Included specs
-- no arch / gcc ------------------------------------------------
gromacs%gcc
-- no arch / gcc@12.3.0 -----------------------------------------
cp2k@2024.1%gcc@12.3.0 opal@master%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0
cp2k@2024.1%gcc@12.3.0 py-alphafold@2.3.2%gcc@12.3.0 quantum-espresso@7.3.1%gcc@12.3.0
-- no arch / oneapi@2024.1.0 ------------------------------------
gromacs@2024.1%oneapi@2024.1.0
==> Installed packages
...
==> 257 installed packages
```
The root spec is the specs that are personal to you and that will be concretized and installed.
The installed packages are either packages that were previously installed by admins or packages that you already installed but that are not root.
If you want to remove this package from your personal env, you can use the following command:
```shell
spack rm $pkg_name@version +variant1 ^dep1@version1
```
or
```shell
[stable] user@login001:~> spack config edit
spack:
...
specs:
-- - visit~adios2+gui+vtkm ^harfbuzz%gcc@12.3.0
++ []
...
```
and remove it manually from the spec list.
## How to use on other systems
### Install Spack
```shell
user@supersystem:~> cd /scratch/$USER
user@supersystem:~> git clone -c feature.manyFiles=true https://github.com/spack/spack.git
user@supersystem:~> . spack/share/spack/setup-env.sh
# Add PSI specific recipes
user@supersystem:~> git clone https://gitlab.psi.ch/lsm-hpce/alps/spack-psi.git
user@supersystem:~> spack repo add $PWD/spack-psi
user@supersystem:~> spack install $pkg_name # -v for verbose