5.5 KiB
title, keywords, last_updated, summary, sidebar, permalink
| title | keywords | last_updated | summary | sidebar | permalink |
|---|---|---|---|---|---|
| Cray Module Environment | cray, module | 24 Mai 2023 | This document describes how to use the cray module environment on Merlin7. | merlin7_sidebar | /merlin7/cray-module-env.html |
To switch from the PSI Module Environment to the provided Cray Programming Environment (CPE), please do the following Switch to Cray PrgEnv
$ source /etc/bash.bashrc.local.2023-04-26T164631
Cray Programming Environment, with Cray's compilers and MPI are loaded by default. You can check with module list command or use the short form as shown below. ml
$ ml #means: module list
Currently Loaded Modules:
1) craype-x86-rome 4) perftools-base/21.12.0 7) craype/2.7.13 10) cray-libsci/21.08.1.2
2) libfabric/1.15.2.0 5) xpmem/2.4.4-2.3_13.8__gff0e1d9.shasta 8) cray-dsmml/0.2.2 11) PrgEnv-cray/8.3.0
3) craype-network-ofi 6) cce/13.0.0 9) cray-mpich/8.1.12
You will notice an unfamiliar PrgEnv-cray/8.3.0 that was loaded. This is meta-module that Cray provides to simplify the switch of compilers and their associated dependencies and libraries, as a whole called Programming Environment. In the Cray Programming Environment, there are 4 key modules.
cray-libsci is a collection of numerical routines tuned for performance on Cray systems.
libfabric is an important low-level library that allows you to take advantage of the high performance Slingshot11 network.
cray-mpich is a CUDA-aware MPI implementation.
cce is the compiler from Cray. C/C++ compilers are based on Clang/LLVM while Fortran supports Fortran 2018 standard. More info: https://user.cscs.ch/computing/compilation/cray/
You can switch between different programming environments. You can check the available module with module avail command or the short form (ml av)as shown below. ml av $ ml av PrgEnv
PrgEnv-aocc/8.3.0 (D) PrgEnv-cray/8.3.3 PrgEnv-intel/8.3.0 (D) PrgEnv-nvidia/8.3.0 (D)
PrgEnv-aocc/8.3.3 PrgEnv-gnu/8.3.0 (D) PrgEnv-intel/8.3.3 PrgEnv-nvidia/8.3.3
PrgEnv-cray/8.3.0 (L,D) PrgEnv-gnu/8.3.3 PrgEnv-nvhpc/8.3.3
If you want an in-depth information on the different programming environments, you can use the module spider command. This command allows you to explore the hierarchical structure of Lmod module environment that is in use here. module spider
$ module spider PrgEnv-cray
---------------------------------------------------------------------------------------------------------------------------------------------------
PrgEnv-cray:
---------------------------------------------------------------------------------------------------------------------------------------------------
Versions:
PrgEnv-cray/8.3.0
PrgEnv-cray/8.3.3
---------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "PrgEnv-cray" package (including how to load the modules) use the module's full name. Note that names that have a trailing (E) are extensions provided by other modules. For example:
$ module spider PrgEnv-cray/8.3.3
---------------------------------------------------------------------------------------------------------------------------------------------------
Switching compiler suites
Compiler suites can be exchanged with PrgEnv (Programming Environments) provided by HPE-Cray. The wrappers call the correct compiler with appropriate options to build and link applications with relevant libraries, as required by the loaded modules (only dynamic linking is supported) and therefore should replace direct calls to compiler drivers in Makefiles and build scripts.
To swap the the compiler suite from the default Cray to Intel, you can do the following. If there are no PrgEnv loaded, then you can load the one you prefer with ml command. Swapping PrgEnv
$ module swap PrgEnv-cray PrgEnv-intel
Please note that in a Cray provided PrgEnv, cray-mpich will always be used by default. This is because this MPI library has been strongly optimised for a Cray system. In the case of Intel, cray-mpich has been compiled with Intel compiler to improve interoperability. Please note that the same condition applies when you use the GNU and AMD (AOCC) programming environments.
If you would like to use a pure Intel-MPI, please refer to the the advanced guide on how to install and set up Intel such that the optimised Slingshot11 network is used.
You can switch versions of intel compilers by using the module swap command. If you want to know the available versions of intel compilers, you can use the module avail or module spider commands.
An example to switch the version of Cray compiler while you are in the PrgEnv-cray environment module swap
$ module swap cce/13.0.0 cce/14.0.0
Due to the use of wrapper, you can use the same commands when compiling with say cce or gnu compilers. In the case of intel compilers, you have to use the original commands, e.g. icc.
C compiler : cc
C++ compiler : CC
Fortran compiler : ftn
MPI C compiler: mpicc
MPI C++ compiler: mpic++
MPI Fortran compiler: mpif90
When using gnu compiler, you will need to specify the architecture (-march or -mtune or --offload-arch) you would like to optimise your code for, in this case Milan, you need to use craype-x86-milan.