HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

Doc changes

Browse Source
This commit is contained in:
caubet_m
2021-05-21 12:34:19 +02:00
parent 42d8f38934
commit fcfdbf1344
46 changed files with 447 additions and 528 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
pages/gmerlin6/slurm-configuration.md
View File

@ -63,13 +63,13 @@ and, if possible, they will preempt running jobs from partitions with lower *Pri
### Merlin6 GPU Accounts
Users might need to specify the Slurm account to be used. If no account is specified, the **`merlin`** **account** will be used as default:
Users need to ensure that the public **`merlin`** account is specified. No specifying account options would default to this account.
This is mostly needed by users which have multiple Slurm accounts, which may define by mistake a different account.
```bash
#SBATCH --account=merlin # Possible values: merlin, gwendolen_public, gwendolen
```
Not all accounts can be used on all partitions. This is resumed in the table below:
Not all the accounts can be used on all partitions. This is resumed in the table below:
| Slurm Account | Slurm Partitions |
|:-------------------: | :--------------: |
@ -77,13 +77,17 @@ Not all accounts can be used on all partitions. This is resumed in the table bel
| **gwendolen_public** | `gwendolen` |
| **gwendolen** | `gwendolen` |
By default, all users belong to the `merlin` and `gwendolen_public` Slurm accounts.
The `gwendolen` **account** is only available for a few set of users, other users must use `gwendolen_public` instead.
By default, all users belong to the `merlin` and `gwendolen_public` Slurm accounts. `gwendolen` is a restricted account.
For running jobs in the `gwendolen` **partition**, users must specify one of the `gwendolen_public` or `gwendolen` accounts.
The `merlin` account is not allowed in the `gwendolen` partition.
#### The 'gwendolen' accounts
### GPU specific options
For running jobs in the **`gwendolen`** partition, users must specify one of the `gwendolen_public` or `gwendolen` accounts.
The `merlin` account is not allowed to use the `gwendolen` partition.
* The **`gwendolen_public`** can be used by any Merlin user, and provides restricted resource access to **`gwendolen`**.
* The **`gwendolen`** is restricted to a set of users, and provides full access to **`gwendolen`**.
### Slurm GPU specific options
Some options are available when using GPUs. These are detailed here.
@ -115,6 +119,7 @@ for each Slurm command for further information about it (`man salloc`, `man sbat
Below are listed the most common settings:
```bash
#SBATCH --hint=[no]multithread
#SBATCH --ntasks=<ntasks>
#SBATCH --ntasks-per-gpu=<ntasks>
#SBATCH --mem-per-gpu=<size[units]>
@ -127,6 +132,17 @@ Below are listed the most common settings:
Please, notice that when defining `[<type>:]` once, then all other options must use it too!
#### Dealing with Hyper-Threading
The **`gmerlin6`** cluster contains the partition `gwendolen`, which has a node with Hyper-Threading enabled.
In that case, one should always specify whether to use Hyper-Threading or not. If not defined, Slurm will
generally use it (exceptions apply). For this machine, generally HT is recommended.
```bash
#SBATCH --hint=multithread # Use extra threads with in-core multi-threading.
#SBATCH --hint=nomultithread # Don't use extra threads with in-core multi-threading.
```
## User and job limits
The GPU cluster contains some basic user and job limits to ensure that a single user can not overabuse the resources and a fair usage of the cluster.
@ -135,8 +151,8 @@ The limits are described below.
### Per job limits
These are limits applying to a single job. In other words, there is a maximum of resources a single job can use.
Limits are defined using QoS, and this is usually set at the partition level. Limits are described in the table below with the format: `SlurmQoS(limits)`,
(list of possible `SlurmQoS` values can be listed with the command `sacctmgr show qos`):
Limits are defined using QoS, and this is usually set at the partition level. Limits are described in the table below with the format: `SlurmQoS(limits)`
(possible `SlurmQoS` values can be listed with the command `sacctmgr show qos`):
| Partition | Slurm Account | Mon-Sun 0h-24h |
|:-------------:| :----------------: | :------------------------------------------: |
@ -159,8 +175,8 @@ For full access, the `gwendolen` account is needed, and this is restricted to a
### Per user limits for GPU partitions
These limits apply exclusively to users. In other words, there is a maximum of resources a single user can use.
Limits are defined using QoS, and this is usually set at the partition level. Limits are described in the table below with the format: `SlurmQoS(limits)`,
(list of possible `SlurmQoS` values can be listed with the command `sacctmgr show qos`):
Limits are defined using QoS, and this is usually set at the partition level. Limits are described in the table below with the format: `SlurmQoS(limits)`
(possible `SlurmQoS` values can be listed with the command `sacctmgr show qos`):
| Partition | Slurm Account | Mon-Sun 0h-24h |
|:-------------:| :----------------: | :---------------------------------------------: |