Update docs
This commit is contained in:
parent
2cc87f6f44
commit
5fc6adc5ab
@ -18,6 +18,8 @@ entries:
|
||||
url: /merlin7/interactive.html
|
||||
- title: Accessing the Slurm Clusters
|
||||
url: /merlin7/slurm-access.html
|
||||
- title: Software Repositories
|
||||
url: /merlin7/software-repositories.html
|
||||
- title: How To Use Merlin7
|
||||
folderitems:
|
||||
- title: Accessing from a Linux client
|
||||
@ -38,12 +40,8 @@ entries:
|
||||
url: /merlin7/ssh-keys.html
|
||||
- title: Kerberos and AFS authentication
|
||||
url: /merlin7/kerberos.html
|
||||
- title: Software repository - Cray Modules
|
||||
url: /merlin7/cray-module-env.html
|
||||
- title: Software repository - PModules
|
||||
url: /merlin7/using-modules.html
|
||||
- title: Software repository - Spack
|
||||
url: /merlin7/spack.html
|
||||
- title: Software Repositories
|
||||
url: /merlin7/software-repositories.html
|
||||
- title: General Tools
|
||||
url: /merlin7/tools.html
|
||||
- title: Slurm General Documentation
|
||||
@ -58,6 +56,12 @@ entries:
|
||||
url: /merlin7/slurm-examples.html
|
||||
- title: Software Support
|
||||
folderitems:
|
||||
- title: PSI Modules
|
||||
url: /merlin7/pmodules.html
|
||||
- title: Spack Modules
|
||||
url: /merlin7/spack.html
|
||||
- title: Cray Modules
|
||||
url: /merlin7/cray-module-env.html
|
||||
- title: OpenMPI
|
||||
url: /merlin7/openmpi.html
|
||||
- title: ANSYS
|
||||
|
||||
@ -31,10 +31,10 @@ In addition, any job *must be submitted from a high performance storage area vis
|
||||
The **Merlin7 CPU cluster** (**`merlin7`**) is the default cluster configured in the login nodes. Any job submission will use by default this cluster, unless
|
||||
the option `--cluster` is specified with another of the existing clusters.
|
||||
|
||||
For further information about how to use this cluster, please visit: [**Merlin7 CPU Slurm Cluster documentation**](/merlin7/slurm-configuration.html).
|
||||
For further information about how to use this cluster, please visit: [**Merlin7 CPU Slurm Cluster documentation**](/merlin7/slurm-configuration.html#cpu-cluster-merlin7).
|
||||
|
||||
### Merlin7 GPU cluster access
|
||||
|
||||
The **Merlin7 GPU cluster** (**`gmerlin7`**) is visible from the login nodes. However, to submit jobs to this cluster, one needs to specify the option `--cluster=gmerlin7` when submitting a job or allocation.
|
||||
|
||||
For further information about how to use this cluster, please visit: [**Merlin7 GPU Slurm Cluster documentation**](/gmerlin7/slurm-configuration.html).
|
||||
For further information about how to use this cluster, please visit: [**Merlin7 GPU Slurm Cluster documentation**](/merlin7/slurm-configuration.html#gpu-cluster-gmerlin7).
|
||||
|
||||
@ -1,109 +0,0 @@
|
||||
---
|
||||
title: Cray Module Environment
|
||||
#tags:
|
||||
keywords: cray, module
|
||||
last_updated: 24 Mai 2023
|
||||
summary: "This document describes how to use the cray module environment on Merlin7."
|
||||
sidebar: merlin7_sidebar
|
||||
permalink: /merlin7/cray-module-env.html
|
||||
---
|
||||
|
||||
{:style="display:block; margin-left:auto; margin-right:auto"}
|
||||
|
||||
{{site.data.alerts.warning}}The Merlin7 documentation is <b>Work In Progress</b>.
|
||||
Please do not use or rely on this documentation until this becomes official.
|
||||
This applies to any page under <b><a href="https://lsm-hpce.gitpages.psi.ch/merlin7/">https://lsm-hpce.gitpages.psi.ch/merlin7/</a></b>
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
Cray Programming Environment, with Cray's compilers and MPI, are loaded by default. You can check with module list command or use the short form as shown below:
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~/slurm/ior]# module list
|
||||
Currently Loaded Modules:
|
||||
1) craype-x86-rome 2) libfabric/1.15.2.0
|
||||
3) craype-network-ofi
|
||||
4) xpmem/2.8.2-1.0_5.1__g84a27a5.shasta 5) PrgEnv-cray/8.5.0
|
||||
6) cce/17.0.0 7) cray-dsmml/0.2.2
|
||||
8) cray-libsci/23.12.5 9) cray-mpich/8.1.28
|
||||
10) craype/2.7.30 11) perftools-base/23.12.0
|
||||
12) cpe/23.12 13) cray/23.12
|
||||
```
|
||||
You will notice an unfamiliar PrgEnv-cray/8.5.0 that was loaded. This is a meta-module that Cray provides to simplify the switch of compilers and their associated dependencies and libraries,
|
||||
as a whole called Programming Environment. In the Cray Programming Environment, there are 4 key modules.
|
||||
|
||||
* `cray-libsci` is a collection of numerical routines tuned for performance on Cray systems.
|
||||
* `libfabric` is an important low-level library that allows you to take advantage of the high performance Slingshot11 network.
|
||||
* `cray-mpich` is a CUDA-aware MPI implementation.
|
||||
* `cce` is the compiler from Cray. C/C++ compilers are based on Clang/LLVM while Fortran supports Fortran 2018 standard. More info: https://user.cscs.ch/computing/compilation/cray/
|
||||
|
||||
You can switch between different programming environments. You can check the available module with the `module avail` command, as follows:
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~/slurm/ior]# module avail PrgEnv
|
||||
--------------------- /opt/cray/pe/lmod/modulefiles/core ---------------------
|
||||
|
||||
PrgEnv-aocc/8.5.0 PrgEnv-cray/8.5.0
|
||||
PrgEnv-gnu/8.5.0 PrgEnv-intel/8.5.0
|
||||
```
|
||||
{% comment %}
|
||||
If you want an in-depth information on the different programming environments, you can use the module spider command. This command allows you to explore the hierarchical structure of Lmod module environment that is in use here.
|
||||
|
||||
```bash
|
||||
$ module spider PrgEnv-cray
|
||||
---------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
PrgEnv-cray:
|
||||
---------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
Versions:
|
||||
PrgEnv-cray/8.3.0
|
||||
PrgEnv-cray/8.3.3
|
||||
|
||||
---------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
For detailed information about a specific "PrgEnv-cray" package (including how to load the modules) use the module's full name.
|
||||
Note that names that have a trailing (E) are extensions provided by other modules.
|
||||
For example:
|
||||
|
||||
$ module spider PrgEnv-cray/8.3.3
|
||||
---------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
```
|
||||
{% endcomment %}
|
||||
|
||||
## Switching compiler suites
|
||||
|
||||
Compiler suites can be exchanged with PrgEnv (Programming Environments) provided by HPE-Cray. The wrappers call the correct compiler with appropriate options to build and link applications with relevant libraries, as required by the loaded modules (only dynamic linking is supported) and therefore should replace direct calls to compiler drivers in Makefiles and build scripts.
|
||||
|
||||
To swap the the compiler suite from the default Cray to Intel, you can do the following.
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~]# module swap PrgEnv-cray/8.5.0 PrgEnv-gnu/8.5.0
|
||||
|
||||
Lmod is automatically replacing "cce/17.0.0" with "gcc-native/12.3".
|
||||
|
||||
🔥 [caubet_m@login001:~]# module list
|
||||
Currently Loaded Modules:
|
||||
1) craype-x86-rome 2) libfabric/1.15.2.0
|
||||
3) craype-network-ofi
|
||||
4) xpmem/2.8.2-1.0_5.1__g84a27a5.shasta 5) perftools-base/23.12.0
|
||||
6) cpe/23.12 7) cray/23.12
|
||||
8) gcc-native/12.3 9) craype/2.7.30
|
||||
10) cray-dsmml/0.2.2 11) cray-mpich/8.1.28
|
||||
12) cray-libsci/23.12.5 13) PrgEnv-gnu/8.5.0
|
||||
```
|
||||
|
||||
Please note that in a Cray provided `PrgEnv`, `cray-mpich` will always be used by default. This is because this MPI library has been strongly optimised for a Cray system. In the case of Intel, `cray-mpich` has been compiled with Intel compiler to improve interoperability. Please note that the same condition applies when you use the GNU and AMD (AOCC) programming environments.
|
||||
|
||||
If you would like to use a pure Intel-MPI, please refer to the the advanced guide on how to install and set up Intel such that the optimised Slingshot11 network is used.
|
||||
|
||||
You can switch versions of intel compilers by using the module swap command. If you want to know the available versions of intel compilers, you can use the `module avail` command.
|
||||
|
||||
Due to the use of wrapper, you can use the same commands when compiling with say `cce` or `gnu` compilers. In the case of `intel` compilers, you have to use the original commands, e.g. `icc`.
|
||||
|
||||
```bash
|
||||
C compiler : cc
|
||||
C++ compiler : CC
|
||||
Fortran compiler : ftn
|
||||
MPI C compiler: mpicc
|
||||
MPI C++ compiler: mpic++
|
||||
MPI Fortran compiler: mpif90
|
||||
```
|
||||
When using `gnu` compiler, you will need to specify the architecture (`-march` or `-mtune` or `--offload-arch`) you would like to optimise your code for, in this case Milan, you need to use `craype-x86-milan`.
|
||||
50
pages/merlin7/02-How-To-Use-Merlin/software-repositories.md
Normal file
50
pages/merlin7/02-How-To-Use-Merlin/software-repositories.md
Normal file
@ -0,0 +1,50 @@
|
||||
---
|
||||
title: Software repositories
|
||||
#tags:
|
||||
keywords: modules, software, stable, unstable, deprecated, spack, repository, repositories
|
||||
last_updated: 16 January 2024
|
||||
summary: "This page contains information about the different software repositories"
|
||||
sidebar: merlin7_sidebar
|
||||
permalink: /merlin7/software-repositories.html
|
||||
---
|
||||
|
||||
## Module Systems in Merlin7
|
||||
|
||||
Merlin7 provides a modular environment to ensure flexibility, compatibility, and optimized performance.
|
||||
The system supports three primary module types: PSI Environment Modules (PModules), Spack Modules, and Cray Environment Modules.
|
||||
|
||||
## PSI Environment Modules (PModules)
|
||||
|
||||
The PModules system, developed by PSI, is the officially supported module system on Merlin7. It is the preferred choice for accessing validated software across a wide range of applications.
|
||||
|
||||
Key Features:
|
||||
* **Expert Deployment:** Each package is deployed and maintained by specific experts to ensure reliability and compatibility.
|
||||
* **Broad Availability:** Commonly used software, such as OpenMPI, ANSYS, MATLAB, and other, is provided within PModules.
|
||||
* **Custom Requests:** If a package, version, or feature is missing, users can contact the support team to explore feasibility for installation.
|
||||
|
||||
{{site.data.alerts.tip}}
|
||||
For further information about <b>Pmodules</b> on Merlin7 please refer to the <b><a href="/merlin7/pmodules.html">PSI Modules</a></b> chapter.
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
## Spack Modules
|
||||
|
||||
Merlin7 also provides Spack modules, offering a modern and flexible package management system. Spack supports a wide variety of software packages and versions. For more information, refer to the **external [PSI Spack](https://gitlab.psi.ch/lsm-hpce/spack-psi) documentation**.
|
||||
|
||||
{{site.data.alerts.tip}}
|
||||
For further information about <b>Spack</b> on Merlin7 please refer to the <b><a href="/merlin7/spack.html">Spack</a></b> chapter.
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
## Cray Environment Modules
|
||||
|
||||
Merlin7 also supports Cray Environment Modules, which include compilers, MPI implementations, and libraries optimized
|
||||
for Cray systems. However, Cray modules are not recommended as the default choice due to potential backward compatibility
|
||||
issues when the Cray Programming Environment (CPE) is upgraded to a newer version.
|
||||
|
||||
Recommendations:
|
||||
* **Compiling Software:** Cray modules can be used when optimization for Cray hardware is essential.
|
||||
* **General Use:** For most applications, prefer PModules, which ensure stability, backward compatibility, and long-term support.
|
||||
|
||||
{{site.data.alerts.tip}}
|
||||
For further information about <b>CPE</b> on Merlin7 please refer to the <b><a href="/merlin7/cray-module-env.html">Cray Modules</a></b> chapter.
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
@ -1,214 +0,0 @@
|
||||
---
|
||||
title: Using PModules
|
||||
#tags:
|
||||
keywords: Pmodules, software, stable, unstable, deprecated, overlay, overlays, release stage, module, package, packages, library, libraries
|
||||
last_updated: 07 September 2022
|
||||
#summary: ""
|
||||
sidebar: merlin7_sidebar
|
||||
permalink: /merlin7/using-modules.html
|
||||
---
|
||||
|
||||
## Cray Environment Modules
|
||||
|
||||
By default, Merlin7 uses Cray Environment Modules. How to use it is explained in the [Cray Module Environment](/merlin7/cray-module-env.html) section. In addition, PSI Environment Modules, which contains a long list of software which is validated and can be used to run on Merlin7.
|
||||
|
||||
For compiling, we strongly recommend using the software provided by the [Cray Module Environment](/merlin7/cray-module-env.html), which contains compilers, MPI and libraries optimized to run on Cray, and to use the PSI Environment Modules (a.k.a. PModules) for other software (ANSYS, Matlab, etc.).
|
||||
|
||||
## PSI Environment Modules
|
||||
|
||||
On top of the operating system stack we provide different software using the PSI developed PModule system.
|
||||
|
||||
PModules is the official supported way and each package is deployed by a specific expert. Usually, in PModules
|
||||
software which is used by many people will be found.
|
||||
|
||||
If you miss any package/versions or a software with a specific missing feature, contact us. We will study if is feasible or not to install it.
|
||||
|
||||
## Module release stages
|
||||
|
||||
Three different **release stages** are available in Pmodules, ensuring proper software life cycling. These are the following: **`unstable`**, **`stable`** and **`deprecated`**
|
||||
|
||||
### Unstable release stage
|
||||
|
||||
The **`unstable`** release stage contains *unstable* releases of software. Software compilations here are usually under development or are not fully production ready.
|
||||
|
||||
This release stage is **not directly visible** by the end users, and needs to be explicitly invoked as follows:
|
||||
|
||||
```bash
|
||||
module use unstable
|
||||
```
|
||||
|
||||
Once software is validated and considered production ready, this is moved to the `stable` release stage.
|
||||
|
||||
### Stable release stage
|
||||
|
||||
The **`stable`** release stage contains *stable* releases of software, which have been deeply tested and are fully supported.
|
||||
|
||||
This is the ***default*** release stage, and is visible by default. Whenever possible, users are strongly advised to use packages from this release stage.
|
||||
|
||||
### Deprecated release stage
|
||||
|
||||
The **`deprecated`** release stage contains *deprecated* releases of software. Software in this release stage is usually deprecated or discontinued by their developers.
|
||||
Also, minor versions or redundant compilations are moved here as long as there is a valid copy in the *stable* repository.
|
||||
|
||||
This release stage is **not directly visible** by the users, and needs to be explicitly invoked as follows:
|
||||
|
||||
```bash
|
||||
module use deprecated
|
||||
```
|
||||
|
||||
However, software moved to this release stage can be directly loaded without the need of invoking it. This ensure proper life cycling of the software, and making it transparent for the end users.
|
||||
|
||||
## Module overlays
|
||||
|
||||
Recent Pmodules releases contain a feature called **Pmodules overlays**. In Merlin, overlays are used to source software from a different location.
|
||||
In that way, we can have custom private versions of software in the cluster installed on high performance storage accessed over a low latency network.
|
||||
|
||||
**Pmodules overlays** are still ***under development***, therefore consider that *some features may not work or do not work as expected*.
|
||||
|
||||
Pmodule overlays can be used from Pmodules `v1.1.5`. However, Merlin is running Pmodules `v1.0.0rc10` as the default version.
|
||||
Therefore, one needs to load first a newer version of it: this is available in the repositories and can be loaded with **`module load Pmodules/$version`** command.
|
||||
|
||||
Once running the proper Pmodules version, **overlays** are added (or invoked) with the **`module use $overlay_name`** command.
|
||||
|
||||
### overlay_merlin
|
||||
|
||||
Some Merlin software is already provided through **PModule overlays** and has been validated for using and running it in that way.
|
||||
Therefore, Melin contains an overlay called **`overlay_merlin`**. In this overlay, the software is installed in the Merlin high performance storage,
|
||||
specifically in the ``/data/software/pmodules`` directory. In general, if another copy exists in the standard repository, we strongly recommend to use
|
||||
the replica in the `overlay_merlin` overlay instead, as it provides faster access and it may also provide some customizations for the Merlin7 cluster.
|
||||
|
||||
For loading the `overlay_merlin`, please run:
|
||||
```bash
|
||||
module load Pmodules/1.1.6 # Or newer version
|
||||
module use overlay_merlin
|
||||
```
|
||||
|
||||
Then, once `overlay_merlin` is invoked, it will disable central software installations with the same version (if exist), and will be replaced
|
||||
by the local ones in Merlin. Releases from the central Pmodules repository which do not have a copy in the Merlin overlay will remain
|
||||
visible. In example, for each ANSYS release, one can identify where it is installed by searching ANSYS in PModules with the `--verbose`
|
||||
option. This will show the location of the different ANSYS releases as follows:
|
||||
* For ANSYS releases installed in the central repositories, the path starts with `/opt/psi`
|
||||
* For ANSYS releases installed in the Merlin7 repository (and/or overwritting the central ones), the path starts with `/data/software/pmodules`
|
||||
|
||||
```bash
|
||||
(base) ❄ [caubet_m@login001:/data/user/caubet_m]# module load Pmodules/1.1.6
|
||||
module load: unstable module has been loaded -- Pmodules/1.1.6
|
||||
|
||||
(base) ❄ [caubet_m@login001:/data/user/caubet_m]# module use merlin_overlay
|
||||
|
||||
(base) ❄ [caubet_m@login001:/data/user/caubet_m]# module search ANSYS --verbose
|
||||
|
||||
Module Rel.stage Group Dependencies/Modulefile
|
||||
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
||||
ANSYS/2019R3 stable Tools dependencies:
|
||||
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2019R3
|
||||
ANSYS/2020R1 stable Tools dependencies:
|
||||
modulefile: /opt/psi/Tools/modulefiles/ANSYS/2020R1
|
||||
ANSYS/2020R1-1 stable Tools dependencies:
|
||||
modulefile: /opt/psi/Tools/modulefiles/ANSYS/2020R1-1
|
||||
ANSYS/2020R2 stable Tools dependencies:
|
||||
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2020R2
|
||||
ANSYS/2021R1 stable Tools dependencies:
|
||||
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2021R1
|
||||
ANSYS/2021R2 stable Tools dependencies:
|
||||
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2021R2
|
||||
```
|
||||
|
||||
## PModules commands
|
||||
|
||||
Below is listed a summary of all available commands:
|
||||
|
||||
```bash
|
||||
module use # show all available PModule Software Groups as well as Release Stages
|
||||
module avail # to see the list of available software packages provided via pmodules
|
||||
module use unstable # to get access to a set of packages not fully tested by the community
|
||||
module load <package>/<version> # to load specific software package with a specific version
|
||||
module search <string> # to search for a specific software package and its dependencies.
|
||||
module list # to list which software is loaded in your environment
|
||||
module purge # unload all loaded packages and cleanup the environment
|
||||
```
|
||||
|
||||
### module use/unuse
|
||||
|
||||
Without any parameter, `use` **lists** all available PModule **Software Groups and Release Stages**.
|
||||
|
||||
```bash
|
||||
module use
|
||||
```
|
||||
|
||||
When followed by a parameter, `use`/`unuse` invokes/uninvokes a PModule **Software Group** or **Release Stage**.
|
||||
|
||||
```bash
|
||||
module use EM # Invokes the 'EM' software group
|
||||
module unuse EM # Uninvokes the 'EM' software group
|
||||
module use unstable # Invokes the 'unstable' Release stable
|
||||
module unuse unstable # Uninvokes the 'unstable' Release stable
|
||||
```
|
||||
|
||||
### module avail
|
||||
|
||||
This option **lists** all available PModule **Software Groups and their packages**.
|
||||
|
||||
Please run `module avail --help` for further listing options.
|
||||
|
||||
### module search
|
||||
|
||||
This is used to **search** for **software packages**. By default, if no **Release Stage** or **Software Group** is specified
|
||||
in the options of the `module search` command, it will search from the already invoked *Software Groups* and *Release Stages*.
|
||||
Direct package dependencies will be also showed.
|
||||
|
||||
```bash
|
||||
(base) [caubet_m@login001 caubet_m]$ module search openmpi/4.0.5_slurm
|
||||
|
||||
Module Release Group Requires
|
||||
---------------------------------------------------------------------------
|
||||
openmpi/4.0.5_slurm stable Compiler gcc/8.4.0
|
||||
openmpi/4.0.5_slurm stable Compiler gcc/9.2.0
|
||||
openmpi/4.0.5_slurm stable Compiler gcc/9.3.0
|
||||
openmpi/4.0.5_slurm stable Compiler intel/20.4
|
||||
|
||||
(base) [caubet_m@login001 caubet_m]$ module load intel/20.4 openmpi/4.0.5_slurm
|
||||
```
|
||||
|
||||
Please run `module search --help` for further search options.
|
||||
|
||||
### module load/unload
|
||||
|
||||
This loads/unloads specific software packages. Packages might have direct dependencies that need to be loaded first. Other dependencies
|
||||
will be automatically loaded.
|
||||
|
||||
In the example below, the ``openmpi/4.0.5_slurm`` package will be loaded, however ``gcc/9.3.0`` must be loaded as well as this is a strict dependency. Direct dependencies must be loaded in advance. Users can load multiple packages one by one or at once. This can be useful for instance when loading a package with direct dependencies.
|
||||
|
||||
```bash
|
||||
# Single line
|
||||
module load gcc/9.3.0 openmpi/4.0.5_slurm
|
||||
|
||||
# Multiple line
|
||||
module load gcc/9.3.0
|
||||
module load openmpi/4.0.5_slurm
|
||||
```
|
||||
|
||||
#### module purge
|
||||
|
||||
This command is an alternative to `module unload`, which can be used to unload **all** loaded module files.
|
||||
|
||||
```bash
|
||||
module purge
|
||||
```
|
||||
|
||||
## When to request for new PModules packages
|
||||
|
||||
### Missing software
|
||||
|
||||
If you don't find a specific software and you know from other people interesing on it, it can be installed in PModules. Please contact us
|
||||
and we will try to help with that. Deploying new software in PModules may take few days.
|
||||
|
||||
Usually installation of new software are possible as long as few users will use it. If you are insterested in to maintain this software,
|
||||
please let us know.
|
||||
|
||||
### Missing version
|
||||
|
||||
If the existing PModules versions for a specific package do not fit to your needs, is possible to ask for a new version.
|
||||
|
||||
Usually installation of newer versions will be supported, as long as few users will use it. Installation of intermediate versions can
|
||||
be supported if this is strictly justified.
|
||||
@ -12,7 +12,7 @@ This document describes generic information of how to load and run ANSYS softwar
|
||||
|
||||
## ANSYS software in Pmodules
|
||||
|
||||
The ANSYS software can be loaded through **[PModules](/merlin7/using-modules.html)**.
|
||||
The ANSYS software can be loaded through **[PModules](/merlin7/pmodules.html)**.
|
||||
|
||||
The default ANSYS versions are loaded from the central PModules repository.
|
||||
|
||||
|
||||
64
pages/merlin7/05-Software-Support/cray-module.env.md
Normal file
64
pages/merlin7/05-Software-Support/cray-module.env.md
Normal file
@ -0,0 +1,64 @@
|
||||
---
|
||||
title: Cray Programming Environment
|
||||
#tags:
|
||||
keywords: cray, module
|
||||
last_updated: 24 Mai 2023
|
||||
summary: "This document describes how to use the Cray Programming Environment on Merlin7."
|
||||
sidebar: merlin7_sidebar
|
||||
permalink: /merlin7/cray-module-env.html
|
||||
---
|
||||
|
||||
## Loading the Cray module
|
||||
|
||||
The Cray Programming Environment, with Cray's compilers and MPI, is not loaded by default.
|
||||
|
||||
To load it, one has to run the following command:
|
||||
|
||||
```bash
|
||||
module load cray
|
||||
```
|
||||
|
||||
The Cray Programming Environment will load all the necessary dependencies. In example:
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~]# module list
|
||||
Currently Loaded Modules:
|
||||
1) craype-x86-rome 2) libfabric/1.15.2.0
|
||||
3) craype-network-ofi
|
||||
4) xpmem/2.9.6-1.1_20240510205610__g087dc11fc19d 5) PrgEnv-cray/8.5.0
|
||||
6) cce/17.0.0 7) cray-libsci/23.12.5
|
||||
8) cray-mpich/8.1.28 9) craype/2.7.30
|
||||
10) perftools-base/23.12.0 11) cpe/23.12
|
||||
12) cray/23.12
|
||||
```
|
||||
|
||||
You will notice an unfamiliar `PrgEnv-cray/8.5.0` that was loaded. This is a meta-module that Cray provides to simplify the switch of compilers and their associated dependencies and libraries,
|
||||
as a whole called Programming Environment. In the Cray Programming Environment, there are 4 key modules.
|
||||
|
||||
* `cray-libsci` is a collection of numerical routines tuned for performance on Cray systems.
|
||||
* `libfabric` is an important low-level library that allows you to take advantage of the high performance Slingshot network.
|
||||
* `cray-mpich` is a CUDA-aware MPI implementation, optimized for Cray systems.
|
||||
* `cce` is the compiler from Cray. C/C++ compilers are based on Clang/LLVM while Fortran supports Fortran 2018 standard. More info: https://user.cscs.ch/computing/compilation/cray/
|
||||
|
||||
You can switch between different programming environments. You can check the available module with the `module avail` command, as follows:
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~]# module avail PrgEnv
|
||||
--------------------- /opt/cray/pe/lmod/modulefiles/core ---------------------
|
||||
|
||||
PrgEnv-cray/8.5.0 PrgEnv-gnu/8.5.0
|
||||
PrgEnv-nvhpc/8.5.0 PrgEnv-nvidia/8.5.0
|
||||
```
|
||||
## Switching compiler suites
|
||||
|
||||
Compiler suites can be exchanged with PrgEnv (Programming Environments) provided by HPE-Cray. The wrappers call the correct compiler with appropriate options to build
|
||||
and link applications with relevant libraries, as required by the loaded modules (only dynamic linking is supported) and therefore should replace direct calls to compiler
|
||||
drivers in Makefiles and build scripts.
|
||||
|
||||
To swap the the compiler suite from the default Cray to GNU compiler, one can run the following.
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~]# module swap PrgEnv-cray/8.5.0 PrgEnv-gnu/8.5.0
|
||||
|
||||
Lmod is automatically replacing "cce/17.0.0" with "gcc-native/12.3".
|
||||
```
|
||||
@ -19,7 +19,7 @@ The Merlin cluster supports OpenMPI versions across three distinct stages: stabl
|
||||
#### Stable
|
||||
|
||||
Versions in the `stable` stage are fully functional, thoroughly tested, and officially supported by the Merlin administrators.
|
||||
These versions are available via [Pmodules](/merlin7/using-modules.html) and [Spack](/merlin7/spack.html), ensuring compatibility and reliability for production use.
|
||||
These versions are available via [Pmodules](/merlin7/pmodules.html) and [Spack](/merlin7/spack.html), ensuring compatibility and reliability for production use.
|
||||
|
||||
#### Unstable
|
||||
|
||||
@ -71,7 +71,7 @@ specific pmix plugin versions available: pmix_v5,pmix_v4,pmix_v3,pmix_v2
|
||||
```
|
||||
Important Notes:
|
||||
* For OpenMPI, always use `pmix` by specifying the appropriate version (`pmix_$version`).
|
||||
When loading an OpenMPI module (via [Pmodules](/merlin7/using-modules.html) or [Spack](/merlin7/spack.html)), the corresponding PMIx version will be automatically loaded.
|
||||
When loading an OpenMPI module (via [Pmodules](/merlin7/pmodules.html) or [Spack](/merlin7/spack.html)), the corresponding PMIx version will be automatically loaded.
|
||||
* Users do not need to manually manage PMIx compatibility.
|
||||
|
||||
{{site.data.alerts.warning}}
|
||||
|
||||
153
pages/merlin7/05-Software-Support/pmodules.md
Normal file
153
pages/merlin7/05-Software-Support/pmodules.md
Normal file
@ -0,0 +1,153 @@
|
||||
---
|
||||
title: PSI Modules
|
||||
#tags:
|
||||
keywords: Pmodules, software, stable, unstable, deprecated, overlay, overlays, release stage, module, package, packages, library, libraries
|
||||
last_updated: 07 September 2022
|
||||
#summary: ""
|
||||
sidebar: merlin7_sidebar
|
||||
permalink: /merlin7/pmodules.html
|
||||
---
|
||||
|
||||
## PSI Environment Modules
|
||||
|
||||
On top of the operating system stack we provide different software using the PSI developed PModule system.
|
||||
|
||||
PModules is the official supported way and each package is deployed by a specific expert. Usually, in PModules
|
||||
software which is used by many people will be found.
|
||||
|
||||
If you miss any package/versions or a software with a specific missing feature, contact us. We will study if is feasible or not to install it.
|
||||
|
||||
### Module Release Stages
|
||||
|
||||
To ensure proper software lifecycle management, PModules uses three release stages: unstable, stable, and deprecated.
|
||||
|
||||
1. **Unstable Release Stage:**
|
||||
* Contains experimental or under-development software versions.
|
||||
* Not visible to users by default. Use explicitly:
|
||||
|
||||
```bash
|
||||
module use unstable
|
||||
```
|
||||
* Software is promoted to **stable** after validation.
|
||||
2. **Stable Release Stage:**
|
||||
* Default stage, containing fully tested and supported software versions.
|
||||
* Recommended for all production workloads.
|
||||
|
||||
3. **Deprecated Release Stage:**
|
||||
* Contains software versions that are outdated or discontinued.
|
||||
* These versions are hidden by default but can be explicitly accessed:
|
||||
|
||||
```bash
|
||||
module use deprecated
|
||||
```
|
||||
* Deprecated software can still be loaded directly without additional configuration to ensure user transparency.
|
||||
|
||||
## PModules commands
|
||||
|
||||
Below is listed a summary of common `module` commands:
|
||||
|
||||
```bash
|
||||
module use # show all available PModule Software Groups as well as Release Stages
|
||||
module avail # to see the list of available software packages provided via pmodules
|
||||
module use unstable # to get access to a set of packages not fully tested by the community
|
||||
module load <package>/<version> # to load specific software package with a specific version
|
||||
module search <string> # to search for a specific software package and its dependencies.
|
||||
module list # to list which software is loaded in your environment
|
||||
module purge # unload all loaded packages and cleanup the environment
|
||||
```
|
||||
|
||||
Please refer to the **external [PSI Modules](https://pmodules.gitpages.psi.ch/chap3.html) document** for
|
||||
detailed information about the `module` command.
|
||||
|
||||
### module use/unuse
|
||||
|
||||
Without any parameter, `use` **lists** all available PModule **Software Groups and Release Stages**.
|
||||
|
||||
```bash
|
||||
module use
|
||||
```
|
||||
|
||||
When followed by a parameter, `use`/`unuse` invokes/uninvokes a PModule **Software Group** or **Release Stage**.
|
||||
|
||||
```bash
|
||||
module use EM # Invokes the 'EM' software group
|
||||
module unuse EM # Uninvokes the 'EM' software group
|
||||
module use unstable # Invokes the 'unstable' Release stable
|
||||
module unuse unstable # Uninvokes the 'unstable' Release stable
|
||||
```
|
||||
|
||||
### module avail
|
||||
|
||||
This option **lists** all available PModule **Software Groups and their packages**.
|
||||
|
||||
Please run `module avail --help` for further listing options.
|
||||
|
||||
### module search
|
||||
|
||||
This is used to **search** for **software packages**. By default, if no **Release Stage** or **Software Group** is specified
|
||||
in the options of the `module search` command, it will search from the already invoked *Software Groups* and *Release Stages*.
|
||||
Direct package dependencies will be also showed.
|
||||
|
||||
```bash
|
||||
🔥 [caubet_m@login001:~]# module search openmpi
|
||||
|
||||
Module Rel.stage Group Overlay Requires
|
||||
--------------------------------------------------------------------------------
|
||||
openmpi/4.1.6 stable Compiler Alps gcc/12.3.0
|
||||
openmpi/4.1.6 stable Compiler Alps gcc/13.3.0
|
||||
openmpi/4.1.6 stable Compiler Alps gcc/14.2.0
|
||||
openmpi/4.1.6 stable Compiler Alps intelcc/22.2
|
||||
openmpi/5.0.5 stable Compiler Alps gcc/8.5.0
|
||||
openmpi/5.0.5 stable Compiler Alps gcc/12.3.0
|
||||
openmpi/5.0.5 stable Compiler Alps gcc/14.2.0
|
||||
openmpi/5.0.5 stable Compiler Alps intelcc/22.2
|
||||
```
|
||||
|
||||
Please run `module search --help` for further search options.
|
||||
|
||||
### module load/unload
|
||||
|
||||
This loads/unloads specific software packages. Packages might have direct dependencies that need to be loaded first. Other dependencies
|
||||
will be automatically loaded.
|
||||
|
||||
In the example below, the ``openmpi/5.0.5`` package will be loaded, however ``gcc/14.2.0`` must be loaded as well as this is a strict dependency. Direct dependencies must be loaded in advance. Users can load multiple packages one by one or at once. This can be useful for instance when loading a package with direct dependencies.
|
||||
|
||||
```bash
|
||||
# Single line
|
||||
module load gcc/14.2.0 openmpi/5.0.5
|
||||
|
||||
# Multiple line
|
||||
module load gcc/14.2.0
|
||||
module load openmpi/5.0.5
|
||||
```
|
||||
|
||||
#### module purge
|
||||
|
||||
This command is an alternative to `module unload`, which can be used to unload **all** loaded module files.
|
||||
|
||||
```bash
|
||||
module purge
|
||||
```
|
||||
|
||||
## Requesting New PModules Packages
|
||||
|
||||
The PModules system is designed to accommodate the diverse software needs of Merlin7 users. Below are guidelines for requesting new software or versions to be added to PModules.
|
||||
|
||||
### Requesting Missing Software
|
||||
|
||||
If a specific software package is not available in PModules and there is interest from multiple users:
|
||||
* **[Contact Support](/merlin7/contact.html):** Let us know about the software, and we will assess its feasibility for deployment.
|
||||
* **Deployment Timeline:** Adding new software to PModules typically takes a few days, depending on complexity and compatibility.
|
||||
* **User Involvement:** If you are interested in maintaining the software package, please inform us. Collaborative maintenance helps
|
||||
ensure timely updates and support.
|
||||
|
||||
### Requesting a Missing Version
|
||||
If the currently available versions of a package do not meet your requirements:
|
||||
* **New Versions:** Requests for newer versions are generally supported, especially if there is interest from multiple users.
|
||||
* **Intermediate Versions:** Installation of intermediate versions (e.g., versions between the current stable and deprecated versions)
|
||||
can be considered if there is a strong justification, such as specific features or compatibility requirements.
|
||||
|
||||
### General Notes
|
||||
* New packages or versions are prioritized based on their relevance and usage.
|
||||
* For any request, providing detailed information about the required software or version (e.g., name, version, features) will help
|
||||
expedite the process.
|
||||
@ -6,12 +6,6 @@ sidebar: merlin7_sidebar
|
||||
toc: false
|
||||
permalink: /merlin7/spack.html
|
||||
---
|
||||
{:style="display:block; margin-left:auto; margin-right:auto"}
|
||||
|
||||
{{site.data.alerts.warning}}The Merlin7 documentation is <b>Work In Progress</b>.
|
||||
Please do not use or rely on this documentation until this becomes official.
|
||||
This applies to any page under <b><a href="https://lsm-hpce.gitpages.psi.ch/merlin7/">https://lsm-hpce.gitpages.psi.ch/merlin7/</a></b>
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
For Merlin7 the *package manager for supercomputing* [Spack](https://spack.io/) is available. It is meant to compliment the existing PModules
|
||||
solution, giving users the opertunity to manage their own software environments.
|
||||
@ -20,5 +14,5 @@ Documentation for how to use Spack on Merlin7 is provided [here](https://gitlab.
|
||||
|
||||
## The Spack PSI packages
|
||||
|
||||
An initial collection of packages (and Spack reciepes) are located at <https://gitlab.psi.ch/lsm-hpce/spack-psi>, users can directly use these
|
||||
An initial collection of packages (and Spack reciepes) are located at **[Spack PSI](https://gitlab.psi.ch/lsm-hpce/spack-psi)**, users can directly use these
|
||||
through calls like `spack add ...`.
|
||||
Loading…
x
Reference in New Issue
Block a user