HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

update

Browse Source
This commit is contained in:
caubet_m
2025-01-21 13:57:25 +01:00
parent 4727bf5591
commit 6157453523
3 changed files with 52 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
pages/merlin7/01-Quick-Start-Guide/introduction.md
View File

@ -10,14 +10,58 @@ redirect_from:
- /merlin7/index.html
---
![Work In Progress](/images/WIP/WIP1.webp){:style="display:block; margin-left:auto; margin-right:auto"}
{{site.data.alerts.warning}}The Merlin7 documentation is <b>Work In Progress</b> as the system is still evolving.
{{site.data.alerts.end}}
## About Merlin7
The Merlin7 cluster is in **preproduction** state since August 2024. We are moving the system towards production from January 2025 on, the schedule of the migration of users and communities being subject to the resolution of some remaining problems on the platform. You will be notified well in advance regarding the migration of data.
All PSI users can request access to Merlin7.
In case you identify errors or missing information, please provide feedback through [merlin-admins mailing list](mailto:merlin-admins@lists.psi.ch) mailing list or [submit a ticket using the PSI service portal](https://psi.service-now.com/psisp).
## Infrastructure
### Hardware
The Merlin7 cluster contains the following node specification:
| Node | #N | CPU | RAM | GPU | #GPUs |
| ----: | -- | --- | --- | ----: | ---: |
| Login | 2 | 2 AMD EPYC 7742 (64 Cores 2.25GHz) | 512GB | | |
| CPU | 77 | 2 AMD EPYC 7742 (64 Cores 2.25GHz) | 512GB | | |
| GPU A100 | 8 | 2 AMD EPYC 7713 (64 Cores 3.2GHz) | 512GB | A100 80GB | 4 |
| GPU GH | 5 | NVIDIA ARM Grace Neoverse v2 (144 Cores 3.1GHz) | 864GB (Unified) | GH200 120GB | 4 |
### Network
The Merlin7 cluster builds on top of HPE/Cray technologies, including a high-performance network fabric called Slingshot. This network fabric is able
to provide up to 200 Gbit/s throughput between nodes. Further information on Slignshot can be found on at [HPE](https://www.hpe.com/psnow/doc/PSN1012904596HREN) and
at <https://www.glennklockwood.com/garden/slingshot>.
Through software interfaces like [libFabric](https://ofiwg.github.io/libfabric/) (which available on Merlin7), application can leverage the network seamlessly.
### Storage
Unlike previous iteration of the Merlin HPC clusters, Merlin7 _does not_ have any local storage. Instead storage for the entire cluster is provided through
a dedicated storage appliance from HPE/Cray called [ClusterStor](https://www.hpe.com/psnow/doc/PSN1012842049INEN.pdf).
The appliance is built of several storage servers:
* 2 management nodes
* 2 MDS servers, 12 drives per server, 2.9TiB (Raid10)
* 8 OSS-D servers, 106 drives per server, 14.5 T.B HDDs (Gridraid / Raid6)
* 4 OSS-F servers, 12 drives per server 7TiB SSDs (Raid10)
With effective storage capacity of:
* 10 PB HDD
* value visible on linux: HDD 9302.4 TiB
* 162 TB SSD
* value visible on linux: SSD 151.6 TiB
* 23.6 TiB on Metadata
The storage is directly connected to the cluster (and each individual node) through the Slingshot NIC.

21
pages/merlin7/03-Slurm-General-Documentation/merlin7-configuration.md
View File

@ -8,22 +8,12 @@ sidebar: merlin7_sidebar
permalink: /merlin7/merlin7-configuration.html
---
![Work In Progress](/images/WIP/WIP1.webp){:style="display:block; margin-left:auto; margin-right:auto"}
{{site.data.alerts.warning}}The Merlin7 documentation is <b>Work In Progress</b>.
Please do not use or rely on this documentation until this becomes official.
This applies to any page under <b><a href="https://lsm-hpce.gitpages.psi.ch/merlin7/">https://lsm-hpce.gitpages.psi.ch/merlin7/</a></b>
{{site.data.alerts.end}}
This documentation shows basic Slurm configuration and options needed to run jobs in the Merlin7 cluster.
## Infrastructure
### Hardware
The current configuration for the _preproduction_ phase (and likely the production phase) is made up as:
* 92 nodes in total for Merlin7:
* 2 CPU-only login nodes
* 77 CPU-only compute nodes
* 5 GPU A100 nodes
@ -31,11 +21,12 @@ The current configuration for the _preproduction_ phase (and likely the producti
The specification of the node types is:
| Node | CPU | RAM | GRES | Notes |
| ---- | --- | --- | ---- | ----- |
| Multi-core node | _2x_ AMD EPYC 7742 (x86_64 Rome, 64 Cores, 2.25GHz) | 512GB DDR4 3200Mhz | | For both the login and CPU-only compute nodes |
| A100 node | _2x_ AMD EPYC 7713 (x86_64 Milan, 64 Cores, 3.2GHz) | 512GB DDR4 3200Mhz | _4x_ NVidia A100 (Ampere, 80GB) | |
| GH Node | _2x_ NVidia Grace Neoverse-V2 (SBSA ARM 64bit, 144 Cores, 3.1GHz) | _2x_ 480GB DDR5X (CPU + GPU) | _4x_ NVidia GH200 (Hopper, 120GB) | |
| Node | #Nodes | CPU | RAM | GRES |
| ----: | ------ | --- | --- | ---- |
| Login Nodes | 2 | _2x_ AMD EPYC 7742 (x86_64 Rome, 64 Cores, 2.25GHz) | 512GB DDR4 3200Mhz | |
| CPU Nodes | 77 | _2x_ AMD EPYC 7742 (x86_64 Rome, 64 Cores, 2.25GHz) | 512GB DDR4 3200Mhz | |
| A100 GPU Nodes | 8 | _2x_ AMD EPYC 7713 (x86_64 Milan, 64 Cores, 3.2GHz) | 512GB DDR4 3200Mhz | 4 x NV_A100 (80GB) |
| GH GPU Nodes | 5 | _2x_ NVidia Grace Neoverse-V2 (SBSA ARM 64bit, 144 Cores, 3.1GHz) | _2x_ 480GB DDR5X (CPU+GPU) | 4 x NV_GH200 (120GB) |
### Network

3
pages/merlin7/03-Slurm-General-Documentation/slurm-configuration.md
View File

@ -180,8 +180,7 @@ by **`PriorityTier`**, ensures that jobs submitted to private partitions are pri
| PartitionName | DefaultTime | MaxTime | TotalNodes | PriorityJobFactor | PriorityTier | QoS | AllowAccounts |
| -----------------: | -----------: | ----------: | --------: | ----------------: | -----------: | ----------: | -------------: |
| **asa-general** | 0-01:00:00 | 14-00:00:00 | 10 | 1 | 2 | normal | asa |
| **asa-daily** | 0-01:00:00 | 1-00:00:00 | 10 | 1000 | 2 | normal | asa |
| **asa** | 0-01:00:00 | 14-00:00:00 | 10 | 1 | 2 | normal | asa |
##### CNM / Mu3e