10 KiB
title, last_updated, sidebar, permalink
| title | last_updated | sidebar | permalink |
|---|---|---|---|
| Hardware And Software Description | 19 April 2021 | merlin6_sidebar | /gmerlin6/hardware-and-software.html |
Hardware
GPU Computing Nodes
The GPU Merlin6 cluster was initially built from recycled workstations from different groups in the BIO division. From then, little by little it was updated with new nodes from sporadic investments from the same division, and it was never possible a central big investment. Hence, due to this, the Merlin6 GPU computing cluster has a non homogeneus solution, consisting on a big variety of hardware types and components.
On 2018, for the common good, BIO decided to open the cluster to the Merlin users and make it widely accessible for the PSI scientists.
The below table summarizes the hardware setup for the Merlin6 GPU computing nodes:
| Merlin6 GPU Computing Nodes | ||||||||
|---|---|---|---|---|---|---|---|---|
| Node | Processor | Sockets | Cores | Threads | Scratch | Memory | GPUs | GPU Model |
| merlin-g-001 | Intel Core i7-5960X | 1 | 16 | 2 | 1.8TB | 128GB | 2 | GTX1080 |
| merlin-g-00[2-5] | Intel Xeon E5-2640 | 2 | 20 | 1 | 1.8TB | 128GB | 4 | GTX1080 |
| merlin-g-006 | Intel Xeon E5-2640 | 2 | 20 | 1 | 800GB | 128GB | 4 | GTX1080Ti |
| merlin-g-00[7-9] | Intel Xeon E5-2640 | 2 | 20 | 1 | 3.5TB | 128GB | 4 | GTX1080Ti |
| merlin-g-01[0-3] | Intel Xeon Silver 4210R | 2 | 20 | 1 | 1.7TB | 128GB | 4 | RTX2080Ti |
| merlin-g-014 | Intel Xeon Gold 6240R | 2 | 48 | 1 | 2.9TB | 384GB | 8 | RTX2080Ti |
| merlin-g-015 | Intel(R) Xeon Gold 5318S | 2 | 48 | 1 | 2.9TB | 384GB | 8 | RTX A5000 |
Login Nodes
The login nodes are part of the Merlin6 HPC cluster,
and are used to compile and to submit jobs to the different Merlin Slurm clusters (merlin5,merlin6,gmerlin6,etc.).
Please refer to the Merlin6 Hardware Documentation for further information.
Storage
The storage is part of the Merlin6 HPC cluster,
and is mounted in all the Slurm clusters (merlin5,merlin6,gmerlin6,etc.).
Please refer to the Merlin6 Hardware Documentation for further information.
Network
The Merlin6 cluster connectivity is based on the Infiniband FDR and EDR technologies. This allows fast access with very low latencies to the data as well as running extremely efficient MPI-based jobs. To check the network speed (56Gbps for FDR, 100Gbps for EDR) of the different machines, it can be checked by running on each node the following command:
ibstat | grep Rate
Software
In the Merlin6 GPU computing nodes, we try to keep software stack coherency with the main cluster Merlin6.
Due to this, the Merlin6 GPU nodes run:
- RedHat Enterprise Linux 7
- Slurm, we usually try to keep it up to date with the most recent versions.
- GPFS v5
- MLNX_OFED LTS v.5.2-2.2.0.0 or newer for all ConnectX-4 or superior cards.