gitea-pages/pages/merlin7/02-How-To-Use-Merlin/cray-module-env.md

title, keywords, last_updated, summary, sidebar, permalink

title	keywords	last_updated	summary	sidebar	permalink
Cray Module Environment	cray, module	24 Mai 2023	This document describes how to use the cray module environment on Merlin7.	merlin7_sidebar	/merlin7/cray-module-env.html

{:style="display:block; margin-left:auto; margin-right:auto"}

{{site.data.alerts.warning}}The Merlin7 documentation is Work In Progress. Please do not use or rely on this documentation until this becomes official.
This applies to any page under https://lsm-hpce.gitpages.psi.ch/merlin7/ {{site.data.alerts.end}}

Cray Programming Environment, with Cray's compilers and MPI, are loaded by default. You can check with module list command or use the short form as shown below:

🔥 [caubet_m@login001:~/slurm/ior]# module list
Currently Loaded Modules:
 1) craype-x86-rome                        2) libfabric/1.15.2.0
 3) craype-network-ofi                    
 4) xpmem/2.8.2-1.0_5.1__g84a27a5.shasta   5) PrgEnv-cray/8.5.0
 6) cce/17.0.0                             7) cray-dsmml/0.2.2
 8) cray-libsci/23.12.5                    9) cray-mpich/8.1.28
10) craype/2.7.30                         11) perftools-base/23.12.0
12) cpe/23.12                             13) cray/23.12

You will notice an unfamiliar PrgEnv-cray/8.5.0 that was loaded. This is a meta-module that Cray provides to simplify the switch of compilers and their associated dependencies and libraries, as a whole called Programming Environment. In the Cray Programming Environment, there are 4 key modules.

• cray-libsci is a collection of numerical routines tuned for performance on Cray systems.
• libfabric is an important low-level library that allows you to take advantage of the high performance Slingshot11 network.
• cray-mpich is a CUDA-aware MPI implementation.
• cce is the compiler from Cray. C/C++ compilers are based on Clang/LLVM while Fortran supports Fortran 2018 standard. More info: https://user.cscs.ch/computing/compilation/cray/

You can switch between different programming environments. You can check the available module with the module avail command, as follows:

🔥 [caubet_m@login001:~/slurm/ior]# module avail PrgEnv
--------------------- /opt/cray/pe/lmod/modulefiles/core ---------------------

PrgEnv-aocc/8.5.0               PrgEnv-cray/8.5.0               
PrgEnv-gnu/8.5.0                PrgEnv-intel/8.5.0              

{% comment %} If you want an in-depth information on the different programming environments, you can use the module spider command. This command allows you to explore the hierarchical structure of Lmod module environment that is in use here.

    $ module spider PrgEnv-cray
    ---------------------------------------------------------------------------------------------------------------------------------------------------
    PrgEnv-cray:
    ---------------------------------------------------------------------------------------------------------------------------------------------------
        Versions:
            PrgEnv-cray/8.3.0
            PrgEnv-cray/8.3.3
    
    ---------------------------------------------------------------------------------------------------------------------------------------------------

  For detailed information about a specific "PrgEnv-cray" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:
 
    $ module spider PrgEnv-cray/8.3.3
    ---------------------------------------------------------------------------------------------------------------------------------------------------

{% endcomment %}

Switching compiler suites

Compiler suites can be exchanged with PrgEnv (Programming Environments) provided by HPE-Cray. The wrappers call the correct compiler with appropriate options to build and link applications with relevant libraries, as required by the loaded modules (only dynamic linking is supported) and therefore should replace direct calls to compiler drivers in Makefiles and build scripts.

To swap the the compiler suite from the default Cray to Intel, you can do the following.

🔥 [caubet_m@login001:~]# module swap PrgEnv-cray/8.5.0 PrgEnv-gnu/8.5.0

Lmod is automatically replacing "cce/17.0.0" with "gcc-native/12.3".

🔥 [caubet_m@login001:~]# module list
Currently Loaded Modules:
 1) craype-x86-rome                        2) libfabric/1.15.2.0
 3) craype-network-ofi                    
 4) xpmem/2.8.2-1.0_5.1__g84a27a5.shasta   5) perftools-base/23.12.0
 6) cpe/23.12                              7) cray/23.12
 8) gcc-native/12.3                        9) craype/2.7.30
10) cray-dsmml/0.2.2                      11) cray-mpich/8.1.28
12) cray-libsci/23.12.5                   13) PrgEnv-gnu/8.5.0

Please note that in a Cray provided PrgEnv, cray-mpich will always be used by default. This is because this MPI library has been strongly optimised for a Cray system. In the case of Intel, cray-mpich has been compiled with Intel compiler to improve interoperability. Please note that the same condition applies when you use the GNU and AMD (AOCC) programming environments.

If you would like to use a pure Intel-MPI, please refer to the the advanced guide on how to install and set up Intel such that the optimised Slingshot11 network is used.

You can switch versions of intel compilers by using the module swap command. If you want to know the available versions of intel compilers, you can use the module avail command.

Due to the use of wrapper, you can use the same commands when compiling with say cce or gnu compilers. In the case of intel compilers, you have to use the original commands, e.g. icc.

C compiler :              cc
C++ compiler :            CC
Fortran compiler :        ftn
MPI C compiler:           mpicc
MPI C++ compiler:         mpic++
MPI Fortran compiler:     mpif90

When using gnu compiler, you will need to specify the architecture (-march or -mtune or --offload-arch) you would like to optimise your code for, in this case Milan, you need to use craype-x86-milan.