215 lines
10 KiB
Markdown
215 lines
10 KiB
Markdown
---
|
|
title: Using PModules
|
|
#tags:
|
|
keywords: Pmodules, software, stable, unstable, deprecated, overlay, overlays, release stage, module, package, packages, library, libraries
|
|
last_updated: 07 September 2022
|
|
#summary: ""
|
|
sidebar: merlin7_sidebar
|
|
permalink: /merlin7/using-modules.html
|
|
---
|
|
|
|
## Cray Environment Modules
|
|
|
|
By default, Merlin7 uses Cray Environment Modules. How to use it is explained in the [Cray Module Environment](/merlin7/cray-module-env.html) section. In addition, PSI Environment Modules, which contains a long list of software which is validated and can be used to run on Merlin7.
|
|
|
|
For compiling, we strongly recommend using the software provided by the [Cray Module Environment](/merlin7/cray-module-env.html), which contains compilers, MPI and libraries optimized to run on Cray, and to use the PSI Environment Modules (a.k.a. PModules) for other software (ANSYS, Matlab, etc.).
|
|
|
|
## PSI Environment Modules
|
|
|
|
On top of the operating system stack we provide different software using the PSI developed PModule system.
|
|
|
|
PModules is the official supported way and each package is deployed by a specific expert. Usually, in PModules
|
|
software which is used by many people will be found.
|
|
|
|
If you miss any package/versions or a software with a specific missing feature, contact us. We will study if is feasible or not to install it.
|
|
|
|
## Module release stages
|
|
|
|
Three different **release stages** are available in Pmodules, ensuring proper software life cycling. These are the following: **`unstable`**, **`stable`** and **`deprecated`**
|
|
|
|
### Unstable release stage
|
|
|
|
The **`unstable`** release stage contains *unstable* releases of software. Software compilations here are usually under development or are not fully production ready.
|
|
|
|
This release stage is **not directly visible** by the end users, and needs to be explicitly invoked as follows:
|
|
|
|
```bash
|
|
module use unstable
|
|
```
|
|
|
|
Once software is validated and considered production ready, this is moved to the `stable` release stage.
|
|
|
|
### Stable release stage
|
|
|
|
The **`stable`** release stage contains *stable* releases of software, which have been deeply tested and are fully supported.
|
|
|
|
This is the ***default*** release stage, and is visible by default. Whenever possible, users are strongly advised to use packages from this release stage.
|
|
|
|
### Deprecated release stage
|
|
|
|
The **`deprecated`** release stage contains *deprecated* releases of software. Software in this release stage is usually deprecated or discontinued by their developers.
|
|
Also, minor versions or redundant compilations are moved here as long as there is a valid copy in the *stable* repository.
|
|
|
|
This release stage is **not directly visible** by the users, and needs to be explicitly invoked as follows:
|
|
|
|
```bash
|
|
module use deprecated
|
|
```
|
|
|
|
However, software moved to this release stage can be directly loaded without the need of invoking it. This ensure proper life cycling of the software, and making it transparent for the end users.
|
|
|
|
## Module overlays
|
|
|
|
Recent Pmodules releases contain a feature called **Pmodules overlays**. In Merlin, overlays are used to source software from a different location.
|
|
In that way, we can have custom private versions of software in the cluster installed on high performance storage accessed over a low latency network.
|
|
|
|
**Pmodules overlays** are still ***under development***, therefore consider that *some features may not work or do not work as expected*.
|
|
|
|
Pmodule overlays can be used from Pmodules `v1.1.5`. However, Merlin is running Pmodules `v1.0.0rc10` as the default version.
|
|
Therefore, one needs to load first a newer version of it: this is available in the repositories and can be loaded with **`module load Pmodules/$version`** command.
|
|
|
|
Once running the proper Pmodules version, **overlays** are added (or invoked) with the **`module use $overlay_name`** command.
|
|
|
|
### overlay_merlin
|
|
|
|
Some Merlin software is already provided through **PModule overlays** and has been validated for using and running it in that way.
|
|
Therefore, Melin contains an overlay called **`overlay_merlin`**. In this overlay, the software is installed in the Merlin high performance storage,
|
|
specifically in the ``/data/software/pmodules`` directory. In general, if another copy exists in the standard repository, we strongly recommend to use
|
|
the replica in the `overlay_merlin` overlay instead, as it provides faster access and it may also provide some customizations for the Merlin7 cluster.
|
|
|
|
For loading the `overlay_merlin`, please run:
|
|
```bash
|
|
module load Pmodules/1.1.6 # Or newer version
|
|
module use overlay_merlin
|
|
```
|
|
|
|
Then, once `overlay_merlin` is invoked, it will disable central software installations with the same version (if exist), and will be replaced
|
|
by the local ones in Merlin. Releases from the central Pmodules repository which do not have a copy in the Merlin overlay will remain
|
|
visible. In example, for each ANSYS release, one can identify where it is installed by searching ANSYS in PModules with the `--verbose`
|
|
option. This will show the location of the different ANSYS releases as follows:
|
|
* For ANSYS releases installed in the central repositories, the path starts with `/opt/psi`
|
|
* For ANSYS releases installed in the Merlin7 repository (and/or overwritting the central ones), the path starts with `/data/software/pmodules`
|
|
|
|
```bash
|
|
(base) ❄ [caubet_m@login001:/data/user/caubet_m]# module load Pmodules/1.1.6
|
|
module load: unstable module has been loaded -- Pmodules/1.1.6
|
|
|
|
(base) ❄ [caubet_m@login001:/data/user/caubet_m]# module use merlin_overlay
|
|
|
|
(base) ❄ [caubet_m@login001:/data/user/caubet_m]# module search ANSYS --verbose
|
|
|
|
Module Rel.stage Group Dependencies/Modulefile
|
|
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
ANSYS/2019R3 stable Tools dependencies:
|
|
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2019R3
|
|
ANSYS/2020R1 stable Tools dependencies:
|
|
modulefile: /opt/psi/Tools/modulefiles/ANSYS/2020R1
|
|
ANSYS/2020R1-1 stable Tools dependencies:
|
|
modulefile: /opt/psi/Tools/modulefiles/ANSYS/2020R1-1
|
|
ANSYS/2020R2 stable Tools dependencies:
|
|
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2020R2
|
|
ANSYS/2021R1 stable Tools dependencies:
|
|
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2021R1
|
|
ANSYS/2021R2 stable Tools dependencies:
|
|
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2021R2
|
|
```
|
|
|
|
## PModules commands
|
|
|
|
Below is listed a summary of all available commands:
|
|
|
|
```bash
|
|
module use # show all available PModule Software Groups as well as Release Stages
|
|
module avail # to see the list of available software packages provided via pmodules
|
|
module use unstable # to get access to a set of packages not fully tested by the community
|
|
module load <package>/<version> # to load specific software package with a specific version
|
|
module search <string> # to search for a specific software package and its dependencies.
|
|
module list # to list which software is loaded in your environment
|
|
module purge # unload all loaded packages and cleanup the environment
|
|
```
|
|
|
|
### module use/unuse
|
|
|
|
Without any parameter, `use` **lists** all available PModule **Software Groups and Release Stages**.
|
|
|
|
```bash
|
|
module use
|
|
```
|
|
|
|
When followed by a parameter, `use`/`unuse` invokes/uninvokes a PModule **Software Group** or **Release Stage**.
|
|
|
|
```bash
|
|
module use EM # Invokes the 'EM' software group
|
|
module unuse EM # Uninvokes the 'EM' software group
|
|
module use unstable # Invokes the 'unstable' Release stable
|
|
module unuse unstable # Uninvokes the 'unstable' Release stable
|
|
```
|
|
|
|
### module avail
|
|
|
|
This option **lists** all available PModule **Software Groups and their packages**.
|
|
|
|
Please run `module avail --help` for further listing options.
|
|
|
|
### module search
|
|
|
|
This is used to **search** for **software packages**. By default, if no **Release Stage** or **Software Group** is specified
|
|
in the options of the `module search` command, it will search from the already invoked *Software Groups* and *Release Stages*.
|
|
Direct package dependencies will be also showed.
|
|
|
|
```bash
|
|
(base) [caubet_m@login001 caubet_m]$ module search openmpi/4.0.5_slurm
|
|
|
|
Module Release Group Requires
|
|
---------------------------------------------------------------------------
|
|
openmpi/4.0.5_slurm stable Compiler gcc/8.4.0
|
|
openmpi/4.0.5_slurm stable Compiler gcc/9.2.0
|
|
openmpi/4.0.5_slurm stable Compiler gcc/9.3.0
|
|
openmpi/4.0.5_slurm stable Compiler intel/20.4
|
|
|
|
(base) [caubet_m@login001 caubet_m]$ module load intel/20.4 openmpi/4.0.5_slurm
|
|
```
|
|
|
|
Please run `module search --help` for further search options.
|
|
|
|
### module load/unload
|
|
|
|
This loads/unloads specific software packages. Packages might have direct dependencies that need to be loaded first. Other dependencies
|
|
will be automatically loaded.
|
|
|
|
In the example below, the ``openmpi/4.0.5_slurm`` package will be loaded, however ``gcc/9.3.0`` must be loaded as well as this is a strict dependency. Direct dependencies must be loaded in advance. Users can load multiple packages one by one or at once. This can be useful for instance when loading a package with direct dependencies.
|
|
|
|
```bash
|
|
# Single line
|
|
module load gcc/9.3.0 openmpi/4.0.5_slurm
|
|
|
|
# Multiple line
|
|
module load gcc/9.3.0
|
|
module load openmpi/4.0.5_slurm
|
|
```
|
|
|
|
#### module purge
|
|
|
|
This command is an alternative to `module unload`, which can be used to unload **all** loaded module files.
|
|
|
|
```bash
|
|
module purge
|
|
```
|
|
|
|
## When to request for new PModules packages
|
|
|
|
### Missing software
|
|
|
|
If you don't find a specific software and you know from other people interesing on it, it can be installed in PModules. Please contact us
|
|
and we will try to help with that. Deploying new software in PModules may take few days.
|
|
|
|
Usually installation of new software are possible as long as few users will use it. If you are insterested in to maintain this software,
|
|
please let us know.
|
|
|
|
### Missing version
|
|
|
|
If the existing PModules versions for a specific package do not fit to your needs, is possible to ask for a new version.
|
|
|
|
Usually installation of newer versions will be supported, as long as few users will use it. Installation of intermediate versions can
|
|
be supported if this is strictly justified.
|