HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

ADD: small introduction to Merlin7

Browse Source
This commit is contained in:
germann_e
2023-05-24 16:01:13 +02:00
parent b5957bed56
commit 1a750b3250
8 changed files with 280 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
_data/sidebars/merlin7_sidebar.yml Normal file
View File

@ -0,0 +1,26 @@
# Follow the pattern here for the URLs -- no slash at the beginning, and include the .html. The link here is rendered exactly as is in the Markdown references.
entries:
- product: Merlin
version: 7
folders:
- title: Quick Start Guide
folderitems:
- title: Introduction
url: /merlin7/introduction.html
- title: How To Use Merlin7
folderitems:
- title: Cray systems modules
url: /merlin7/cray-module-env.html
- title: Transferring files
url: /merlin7/file-transfers.html
- title: Slurm General Documentation
folderitems:
- title: Merlin7 Infrastructure
url: /merlin7/slurm-configuration.html
- title: Slurm Batch Script Examples
url: /merlin7/slurm-examples.html
- title: Software Support
folderitems:
- title: ANSYS RSM
url: /merlin7/ansys-rsm.html

58
_data/topnav.yml
View File

@ -1,34 +1,36 @@
## Topnav single links
## if you want to list an external url, use external_url instead of url. the theme will apply a different link base.
topnav:
- title: Topnav
items:
# - title: GitHub
# external_url: https://github.com/tomjoht/documentation-theme-jekyll
- title: News
url: /news
- title: Topnav
items:
# - title: GitHub
# external_url: https://github.com/tomjoht/documentation-theme-jekyll
- title: News
url: /news
#Topnav dropdowns
topnav_dropdowns:
- title: Topnav dropdowns
folders:
- title: Quick Start
folderitems:
- title: Introduction
url: /merlin6/introduction.html
- title: Requesting Accounts
url: /merlin6/request-account.html
- title: Requesting Projects
url: /merlin6/request-project.html
- title: Accessing the Interactive Nodes
url: /merlin6/interactive.html
- title: Accessing the Slurm Clusters
url: /merlin6/slurm-access.html
- title: Merlin Slurm Clusters
folderitems:
- title: Cluster 'merlin5'
url: /merlin5/slurm-configuration.html
- title: Cluster 'merlin6'
url: /gmerlin6/slurm-configuration.html
- title: Cluster 'gmerlin6'
url: /gmerlin6/slurm-configuration.html
- title: Topnav dropdowns
folders:
- title: Quick Start
folderitems:
- title: Introduction
url: /merlin6/introduction.html
- title: Requesting Accounts
url: /merlin6/request-account.html
- title: Requesting Projects
url: /merlin6/request-project.html
- title: Accessing the Interactive Nodes
url: /merlin6/interactive.html
- title: Accessing the Slurm Clusters
url: /merlin6/slurm-access.html
- title: Clusters
folderitems:
- title: Cluster 'merlin5'
url: /merlin5/slurm-configuration.html
- title: Cluster 'merlin6'
url: /gmerlin6/slurm-configuration.html
- title: Cluster 'merlin7'
url: /merlin7/slurm-configuration.html
- title: Cluster 'gmerlin6'
url: /gmerlin6/slurm-configuration.html

18
pages/merlin7/01-Quick-Start-Guide/introduction.md Normal file
View File

@ -0,0 +1,18 @@
---
title: Introduction
#tags:
keywords: introduction, home, welcome, architecture, design
last_updated: 07 September 2022
sidebar: merlin7_sidebar
permalink: /merlin7/introduction.html
redirect_from:
- /merlin7
- /merlin7/index.html
---
## Logging in
To get onto the machine, you have to start from ela.cscs.ch or from login.psi.ch at PSI.
$ ssh psi-username@psi-dev.cscs.ch

90
pages/merlin7/02-How-To-Use-Merlin/cray-module-env.md Normal file
View File

@ -0,0 +1,90 @@
---
title: Cray Module Environment
#tags:
keywords: cray, module
last_updated: 24 Mai 2023
summary: "This document describes how to use the cray module environment on Merlin7."
sidebar: merlin7_sidebar
permalink: /merlin7/cray-module-env.html
---
To switch from the PSI Module Environment to the provided Cray Programming Environment (CPE), please do the following
Switch to Cray PrgEnv
$ source /etc/bash.bashrc.local.2023-04-26T164631
Cray Programming Environment, with Cray's compilers and MPI are loaded by default. You can check with module list command or use the short form as shown below.
ml
$ ml #means: module list
Currently Loaded Modules:
1) craype-x86-rome 4) perftools-base/21.12.0 7) craype/2.7.13 10) cray-libsci/21.08.1.2
2) libfabric/1.15.2.0 5) xpmem/2.4.4-2.3_13.8__gff0e1d9.shasta 8) cray-dsmml/0.2.2 11) PrgEnv-cray/8.3.0
3) craype-network-ofi 6) cce/13.0.0 9) cray-mpich/8.1.12
You will notice an unfamiliar PrgEnv-cray/8.3.0 that was loaded. This is meta-module that Cray provides to simplify the switch of compilers and their associated dependencies and libraries, as a whole called Programming Environment. In the Cray Programming Environment, there are 4 key modules.
cray-libsci is a collection of numerical routines tuned for performance on Cray systems.
libfabric is an important low-level library that allows you to take advantage of the high performance Slingshot11 network.
cray-mpich is a CUDA-aware MPI implementation.
cce is the compiler from Cray. C/C++ compilers are based on Clang/LLVM while Fortran supports Fortran 2018 standard. More info: https://user.cscs.ch/computing/compilation/cray/
You can switch between different programming environments. You can check the available module with module avail command or the short form (ml av)as shown below.
ml av
$ ml av PrgEnv
PrgEnv-aocc/8.3.0 (D) PrgEnv-cray/8.3.3 PrgEnv-intel/8.3.0 (D) PrgEnv-nvidia/8.3.0 (D)
PrgEnv-aocc/8.3.3 PrgEnv-gnu/8.3.0 (D) PrgEnv-intel/8.3.3 PrgEnv-nvidia/8.3.3
PrgEnv-cray/8.3.0 (L,D) PrgEnv-gnu/8.3.3 PrgEnv-nvhpc/8.3.3
If you want an in-depth information on the different programming environments, you can use the module spider command. This command allows you to explore the hierarchical structure of Lmod module environment that is in use here.
module spider
$ module spider PrgEnv-cray
---------------------------------------------------------------------------------------------------------------------------------------------------
PrgEnv-cray:
---------------------------------------------------------------------------------------------------------------------------------------------------
Versions:
PrgEnv-cray/8.3.0
PrgEnv-cray/8.3.3
---------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "PrgEnv-cray" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider PrgEnv-cray/8.3.3
---------------------------------------------------------------------------------------------------------------------------------------------------
## Switching compiler suites
Compiler suites can be exchanged with PrgEnv (Programming Environments) provided by HPE-Cray. The wrappers call the correct compiler with appropriate options to build and link applications with relevant libraries, as required by the loaded modules (only dynamic linking is supported) and therefore should replace direct calls to compiler drivers in Makefiles and build scripts.
To swap the the compiler suite from the default Cray to Intel, you can do the following. If there are no PrgEnv loaded, then you can load the one you prefer with ml command.
Swapping PrgEnv
$ module swap PrgEnv-cray PrgEnv-intel
Please note that in a Cray provided PrgEnv, cray-mpich will always be used by default. This is because this MPI library has been strongly optimised for a Cray system. In the case of Intel, cray-mpich has been compiled with Intel compiler to improve interoperability. Please note that the same condition applies when you use the GNU and AMD (AOCC) programming environments.
If you would like to use a pure Intel-MPI, please refer to the the advanced guide on how to install and set up Intel such that the optimised Slingshot11 network is used.
You can switch versions of intel compilers by using the module swap command. If you want to know the available versions of intel compilers, you can use the module avail or module spider commands.
An example to switch the version of Cray compiler while you are in the PrgEnv-cray environment
module swap
$ module swap cce/13.0.0 cce/14.0.0
Due to the use of wrapper, you can use the same commands when compiling with say cce or gnu compilers. In the case of intel compilers, you have to use the original commands, e.g. icc.
C compiler : cc
C++ compiler : CC
Fortran compiler : ftn
MPI C compiler: mpicc
MPI C++ compiler: mpic++
MPI Fortran compiler: mpif90
When using gnu compiler, you will need to specify the architecture (-march or -mtune or --offload-arch) you would like to optimise your code for, in this case Milan, you need to use craype-x86-milan.

13
pages/merlin7/02-How-To-Use-Merlin/file-transfers.md Normal file
View File

@ -0,0 +1,13 @@
---
title: Transferring files between systems
#tags:
keywords: files, transfer, scp
last_updated: 24 Mai 2023
summary: "This document describes some possibilities to transfer files from Merlin6 to Merlin7."
sidebar: merlin7_sidebar
permalink: /merlin7/file-transfers.html
---
From Merlin6 to Merlin7 you can use (ideally SSH keys should be set beforehand):
$ rsync -avAHXS ~/merlin6_localdata $USER@psi-dev.cscs.ch:/scratch/home/$USER/

25
pages/merlin7/03-Slurm-General-Documentation/slurm-examples.md Normal file
View File

@ -0,0 +1,25 @@
---
title: Slurm Examples
#tags:
keywords: slurm example, template, examples, templates, running jobs, sbatch, single core based jobs, HT, multithread, no-multithread, mpi, openmp, packed jobs, hands-on, array jobs, gpu
last_updated: 24 Mai 2023
summary: "This document shows different template examples for running jobs in the Merlin cluster."
sidebar: merlin7_sidebar
permalink: /merlin7/slurm-examples.html
---
## Single core based job examples
```bash
#!/bin/bash
#SBATCH --partition=hourly # Using 'hourly' will grant higher priority
#SBATCH --ntasks-per-core=2 # Request the max ntasks be invoked on each core
#SBATCH --hint=multithread # Use extra threads with in-core multi-threading
#SBATCH --time=00:30:00 # Define max time job will run
#SBATCH --output=myscript.out # Define your output file
#SBATCH --error=myscript.err # Define your error file
module purge
module load $MODULE_NAME # where $MODULE_NAME is a software in PModules
srun $MYEXEC # where $MYEXEC is a path to your binary file
```

29
pages/merlin7/05-Software-Support/ansys-rsm.md Normal file
View File

@ -0,0 +1,29 @@
---
title: ANSYS RSM (Remote Resolve Manager)
#tags:
keywords: software, ansys, rsm, slurm, interactive, rsm, windows
last_updated: 24 Mai 2023
summary: "This document describes how to use the ANSYS Remote Resolve Manager service in the Merlin6 cluster"
sidebar: merlin7_sidebar
permalink: /merlin7/ansys-rsm.html
---
## ANSYS RSM Configuration tool settings:
Use Merlin6 ANSYS to submit to RSM:
If you want to submit from Titan is possible too, you have to set SSH keys in Titan as described in: https://www.purdue.edu/science/scienceit/ssh-keys-windows.html
"HPC_Resource" tab configuration:
HPC Type: Slurm
Submit Host: psi-dev.cscs.ch
Slurm job arguments: --hint=nomultithread
"File Management" tab configuration:
External mechanism for the transfer (SCP, custom)
Transfer Mechanism SCP via SSH
As staging directory, use /scratch/tmp
As account, use your PSI username
SSH Keys have to be configured to make it work.

49
pages/merlin7/slurm-configuration.md Normal file
View File

@ -0,0 +1,49 @@
---
title: Slurm cluster 'merlin7'
#tags:
keywords: configuration, partitions, node definition
last_updated: 24 Mai 2023
summary: "This document describes a summary of the Merlin6 configuration."
sidebar: merlin7_sidebar
permalink: /merlin7/slurm-configuration.html
---
This documentation shows basic Slurm configuration and options needed to run jobs in the Merlin7 cluster.
### Infrastructure
#### Hardware
The current configuration for the _test_ phase is made up as:
* 9 nodes for the _PSI-Dev_ development system
* 8 nodes were meant for baremetal and k8s
* 1 login node
| Node | CPU | RAM | GRES | Notes |
| ---- | --- | --- | ---- | ----- |
| Compute node | _2x_ AMD EPYC 7713 (x86_64 Milan, 64 Cores, 3.2GHz) | 512GB DDR4 3200Mhz | _4x_ NVidia A100 (Ampere, 80GB) | |
| Login node | _2x_ AMD EPYC 7742 (x86_64 Rome, 64 Cores, 3.2GHz) | 512GB DRR4 3200Mhz | | |
#### Storage
* CephFS only for `/home` -- 1 TB
* ClusterStor L300 for `/scratch` -- 224 TB usable space
* CephRBD `/local` -- 100GB
#### Node IDs
Cray user various identifies to uniquely label each node, details on this can be found on the [Crayism page](cray-conventions.html).
The table below collates these together for the current configuration:
| Node ID | Cray XNAME | Notes |
| ---------- | ---------- | - |
| nid003204 | x1500c4s7b0n0 | login node, to which **psi-dev.cscs.ch** points |
| nid002808 | x1007c0s4b0n0 | |
| nid002809 | x1007c0s4b0n1 | |
| nid002812 | x1007c0s5b0n0 | |
| nid002813 | x1007c0s5b0n1 | |
| nid002824 | x1007c1s0b0n0 | |
| nid002825 | x1007c1s0b0n1 | |
| nid002828 | x1007c1s1b0n0 | |
| nid002829 | x1007c1s1b0n1 | |