9.8 KiB
title, last_updated, sidebar, permalink
| title | last_updated | sidebar | permalink |
|---|---|---|---|
| Using PModules | 21 May 2021 | merlin6_sidebar | /merlin6/using-modules.html |
Environment Modules
On top of the operating system stack we provide different software using the PSI developed PModule system.
PModules is the official supported way and each package is deployed by a specific expert. Usually, in PModules software which is used by many people will be found.
If you miss any package/versions or a software with a specific missing feature, contact us. We will study if is feasible or not to install it.
Module release stages
Three different release stages are available in Pmodules, ensuring proper software life cycling. These are the following: unstable, stable and deprecated
Unstable release stage
The unstable release stage contains unstable releases of software. Software compilations here are usually under development or are not fully production ready.
This release stage is not directly visible by the end users, and needs to be explicitly invoked as follows:
module use unstable
Once software is validated and considered production ready, this is moved to the stable release stage.
Stable release stage
The stable release stage contains stable releases of software, which have been deeply tested and are fully supported.
This is the default release stage, and is visible by default. Whenever possible, users are strongly advised to use packages from this release stage.
Deprecated release stage
The deprecated release stage contains deprecated releases of software. Software in this release stage is usually deprecated or discontinued by their developers.
Also, minor versions or redundant compilations are moved here as long as there is a valid copy in the stable repository.
This release stage is not directly visible by the users, and needs to be explicitly invoked as follows:
module use deprecated
However, software moved to this release stage can be directly loaded without the need of invoking it. This ensure proper life cycling of the software, and making it transparent for the end users.
Module overlays
Recent Pmodules releases contain a feature called Pmodules overlays. In Merlin, overlays are used to source software from a different location. In that way, we can have custom private versions of software in the cluster installed on high performance storage accessed over a low latency network.
Pmodules overlays are still under development, therefore consider that some features may not work or do not work as expected.
Pmodule overlays can be used from Pmodules v1.1.5. However, Merlin is running Pmodules v1.0.0rc10 as the default version.
Therefore, one needs to load first a newer version of it: this is available in the repositories and can be loaded with module load Pmodules/$version command.
Once running the proper Pmodules version, overlays are added (or invoked) with the module use $overlay_name command.
overlay_merlin
Some Merlin software is already provided through PModule overlays and has been validated for using and running it in that way.
Therefore, Melin contains an overlay called overlay_merlin. In this overlay, the software is installed in the Merlin high performance storage,
specifically in the /data/software/pmodules directory. In general, if another copy exists in the standard repository, we strongly recommend to use
the replica in the overlay_merlin overlay instead, as it provides faster access and it may also provide some customizations for the Merlin6 cluster.
For loading the overlay_merlin, please run:
module load Pmodules/1.1.6 # Or newer version
module use overlay_merlin
Then, once overlay_merlin is invoked, it will disable central software installations with the same version (if exist), and will be replaced
by the local ones in Merlin. Releases from the central Pmodules repository which do not have a copy in the Merlin overlay will remain
visible. In example, for each ANSYS release, one can identify where it is installed by searching ANSYS in PModules with the --verbose
option. This will show the location of the different ANSYS releases as follows:
- For ANSYS releases installed in the central repositories, the path starts with
/opt/psi - For ANSYS releases installed in the Merlin6 repository (and/or overwritting the central ones), the path starts with
/data/software/pmodules
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# module load Pmodules/1.1.6
module load: unstable module has been loaded -- Pmodules/1.1.6
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# module use merlin_overlay
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# module search ANSYS --verbose
Module Rel.stage Group Dependencies/Modulefile
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ANSYS/2019R3 stable Tools dependencies:
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2019R3
ANSYS/2020R1 stable Tools dependencies:
modulefile: /opt/psi/Tools/modulefiles/ANSYS/2020R1
ANSYS/2020R1-1 stable Tools dependencies:
modulefile: /opt/psi/Tools/modulefiles/ANSYS/2020R1-1
ANSYS/2020R2 stable Tools dependencies:
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2020R2
ANSYS/2021R1 stable Tools dependencies:
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2021R1
ANSYS/2021R2 stable Tools dependencies:
modulefile: /data/software/pmodules/Tools/modulefiles/ANSYS/2021R2
PModules commands
Below is listed a summary of all available commands:
module use # show all available PModule Software Groups as well as Release Stages
module avail # to see the list of available software packages provided via pmodules
module use unstable # to get access to a set of packages not fully tested by the community
module load <package>/<version> # to load specific software package with a specific version
module search <string> # to search for a specific software package and its dependencies.
module list # to list which software is loaded in your environment
module purge # unload all loaded packages and cleanup the environment
module use/unuse
Without any parameter, use lists all available PModule Software Groups and Release Stages.
module use
When followed by a parameter, use/unuse invokes/uninvokes a PModule Software Group or Release Stage.
module use EM # Invokes the 'EM' software group
module unuse EM # Uninvokes the 'EM' software group
module use unstable # Invokes the 'unstable' Release stable
module unuse unstable # Uninvokes the 'unstable' Release stable
module avail
This option lists all available PModule Software Groups and their packages.
Please run module avail --help for further listing options.
module search
This is used to search for software packages. By default, if no Release Stage or Software Group is specified
in the options of the module search command, it will search from the already invoked Software Groups and Release Stages.
Direct package dependencies will be also showed.
(base) [caubet_m@merlin-l-001 caubet_m]$ module search openmpi/4.0.5_slurm
Module Release Group Requires
---------------------------------------------------------------------------
openmpi/4.0.5_slurm stable Compiler gcc/8.4.0
openmpi/4.0.5_slurm stable Compiler gcc/9.2.0
openmpi/4.0.5_slurm stable Compiler gcc/9.3.0
openmpi/4.0.5_slurm stable Compiler intel/20.4
(base) [caubet_m@merlin-l-001 caubet_m]$ module load intel/20.4 openmpi/4.0.5_slurm
Please run module search --help for further search options.
module load/unload
This loads/unloads specific software packages. Packages might have direct dependencies that need to be loaded first. Other dependencies will be automatically loaded.
In the example below, the openmpi/4.0.5_slurm package will be loaded, however gcc/9.3.0 must be loaded as well as this is a strict dependency. Direct dependencies must be loaded in advance. Users can load multiple packages one by one or at once. This can be useful for instance when loading a package with direct dependencies.
# Single line
module load gcc/9.3.0 openmpi/4.0.5_slurm
# Multiple line
module load gcc/9.3.0
module load openmpi/4.0.5_slurm
module purge
This command is an alternative to module unload, which can be used to unload all loaded module files.
module purge
When to request for new PModules packages
Missing software
If you don't find a specific software and you know from other people interesing on it, it can be installed in PModules. Please contact us and we will try to help with that. Deploying new software in PModules may take few days.
Usually installation of new software are possible as long as few users will use it. If you are insterested in to maintain this software, please let us know.
Missing version
If the existing PModules versions for a specific package do not fit to your needs, is possible to ask for a new version.
Usually installation of newer versions will be supported, as long as few users will use it. Installation of intermediate versions can be supported if this is strictly justified.