HPCE/gitea-pages
0
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Activity

merge and move support pages

Browse Source 
this are now under the /support path, meaning that this is unified for
all clusters.
This commit is contained in:
viessm_h
2026-01-07 17:15:56 +01:00
parent 860bc998ec
commit 8acef7694c
17 changed files with 254 additions and 236 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
docs/merlin6/99-support/contact.md
View File

@@ -1,41 +0,0 @@
# Contact
## Support
Basic contact information can be also found when logging into the Merlin Login Nodes through the *Message of the Day*.
Support can be asked through:
* [PSI Service Now](https://psi.service-now.com/psisp)
* E-Mail: <merlin-admins@lists.psi.ch>
### PSI Service Now
**[PSI Service Now](https://psi.service-now.com/psisp)**: is the official tool for opening incident requests.
* PSI HelpDesk will redirect the incident to the corresponding department, or
* you can always assign it directly by checking the box `I know which service is affected` and providing the service name `Local HPC Resources (e.g. Merlin) [CF]` (just type in `Local` and you should get the valid completions).
### Contact Merlin6 Administrators
**E-Mail <merlin-admins@lists.psi.ch>**
* This is the official way to contact Merlin6 Administrators for discussions which do not fit well into the incident category.
Do not hesitate to contact us for such cases.
---
## Get updated through the Merlin User list!
Is strongly recommended that users subscribe to the Merlin Users mailing list: **<merlin-users@lists.psi.ch>**
This mailing list is the official channel used by Merlin6 administrators to inform users about downtimes,
interventions or problems. Users can be subscribed in two ways:
* *(Preferred way)* Self-registration through **[Sympa](https://psilists.ethz.ch/sympa/info/merlin-users)**
* If you need to subscribe many people (e.g. your whole group) by sending a request to the admin list **<merlin-admins@lists.psi.ch>**
and providing a list of email addresses.
---
## The Merlin Cluster Team
The PSI Merlin clusters are managed by the **[High Performance Computing and Emerging technologies Group](https://www.psi.ch/de/lsm/hpce-group)**, which
is part of the [Science IT Infrastructure, and Services department (AWI)](https://www.psi.ch/en/awi) in PSI's [Center for Scientific Computing, Theory and Data (SCD)](https://www.psi.ch/en/csd).

42
docs/merlin6/99-support/faq.md
View File

@@ -1,42 +0,0 @@
# FAQ
## How do I register for Merlin?
See [Requesting Merlin Access](../quick-start-guide/requesting-accounts.md).
## How do I get information about downtimes and updates?
See [Get updated through the Merlin User list!](contact.md#get-updated-through-the-merlin-user-list)
## How can I request access to a Merlin project directory?
Merlin projects are placed in the `/data/project` directory. Access to each project is controlled by Unix group membership.
If you require access to an existing project, please request group membership as described in [Requesting Unix Group Membership](../quick-start-guide/requesting-projects.md#requesting-unix-group-membership).
Your project leader or project colleagues will know what Unix group you should belong to. Otherwise, you can check what Unix group is allowed to access that project directory (simply run `ls -ltrhd` for the project directory).
## Can I install software myself?
Most software can be installed in user directories without any special permissions. We recommend using `/data/user/$USER/bin` for software since home directories are fairly small. For software that will be used by multiple groups/users you can also [request the admins](contact.md) install it as a [module](../how-to-use-merlin/using-modules.md).
How to install depends a bit on the software itself. There are three common installation procedures:
1. *binary distributions*. These are easy; just put them in a directory (eg `/data/user/$USER/bin`) and add that to your PATH.
2. *source compilation* using make/cmake/autoconfig/etc. Usually the compilation scripts accept a `--prefix=/data/user/$USER` directory for where to install it. Then they place files under `<prefix>/bin`, `<prefix>/lib`, etc. The exact syntax should be documented in the installation instructions.
3. *conda environment*. This is now becoming standard for python-based software, including lots of the AI tools. First follow the [initial setup instructions](../software-support/python.md#anaconda) to configure conda to use /data/user instead of your home directory. Then you can create environments like:
```bash
module load anaconda/2019.07
# if they provide environment.yml
conda env create -f environment.yml
# or to create manually
conda create --name myenv python==3.9 ...
conda activate myenv
```
## Something doesn't work
Check the list of [known problems](known-problems.md) to see if a solution is known.
If not, please [contact the admins](contact.md).

170
docs/merlin6/99-support/known-problems.md
View File

@@ -1,170 +0,0 @@
# Known Problems
## Common errors
### Illegal instruction error
It may happened that your code, compiled on one machine will not be executed on another throwing exception like **"(Illegal instruction)"**.
This is usually because the software was compiled with a set of instructions newer than the ones available in the node where the software runs,
and it mostly depends on the processor generation.
In example, `merlin-l-001` and `merlin-l-002` contain a newer generation of processors than the old GPUs nodes, or than the Merlin5 cluster.
Hence, unless one compiles the software with compatibility with set of instructions from older processors, it will not run on old nodes.
Sometimes, this is properly set by default at the compilation time, but sometimes is not.
For GCC, please refer to [GCC x86 Options](https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html) for compiling options. In case of doubts, contact us.
## Slurm
### sbatch using one core despite setting -c/--cpus-per-task
From **Slurm v22.05.6**, the behavior of `srun` has changed. Merlin has been updated to this version since *Tuesday 13.12.2022*.
`srun` will no longer read in `SLURM_CPUS_PER_TASK`, which is typically set when defining `-c/--cpus-per-task` in the `sbatch` command.
This means you will implicitly have to specify `-c\--cpus-per-task` also on your `srun` calls, or set the new `SRUN_CPUS_PER_TASK` environment variable to accomplish the same thing.
Therefore, unless this is implicitly specified, `srun` will use only one Core per task (resulting in 2 CPUs per task when multithreading is enabled)
An example for setting up `srun` with `-c\--cpus-per-task`:
```bash
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# cat mysbatch_method1
#!/bin/bash
#SBATCH -n 1
#SBATCH --cpus-per-task=8
echo 'From Slurm v22.05.8 srun does not inherit $SLURM_CPUS_PER_TASK'
srun python -c "import os; print(os.sched_getaffinity(0))"
echo 'One has to implicitly specify $SLURM_CPUS_PER_TASK'
echo 'In this example, by setting -c/--cpus-per-task in srun'
srun --cpus-per-task=$SLURM_CPUS_PER_TASK python -c "import os; print(os.sched_getaffinity(0))"
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# sbatch mysbatch_method1
Submitted batch job 8000813
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# cat slurm-8000813.out
From Slurm v22.05.8 srun does not inherit $SLURM_CPUS_PER_TASK
{1, 45}
One has to implicitly specify $SLURM_CPUS_PER_TASK
In this example, by setting -c/--cpus-per-task in srun
{1, 2, 3, 4, 45, 46, 47, 48}
```
An example to accomplish the same thing with the `SRUN_CPUS_PER_TASK` environment variable:
```bash
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# cat mysbatch_method2
#!/bin/bash
#SBATCH -n 1
#SBATCH --cpus-per-task=8
echo 'From Slurm v22.05.8 srun does not inherit $SLURM_CPUS_PER_TASK'
srun python -c "import os; print(os.sched_getaffinity(0))"
echo 'One has to implicitly specify $SLURM_CPUS_PER_TASK'
echo 'In this example, by setting an environment variable SRUN_CPUS_PER_TASK'
export SRUN_CPUS_PER_TASK=$SLURM_CPUS_PER_TASK
srun python -c "import os; print(os.sched_getaffinity(0))"
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# sbatch mysbatch_method2
Submitted batch job 8000815
(base) ❄ [caubet_m@merlin-l-001:/data/user/caubet_m]# cat slurm-8000815.out
From Slurm v22.05.8 srun does not inherit $SLURM_CPUS_PER_TASK
{1, 45}
One has to implicitly specify $SLURM_CPUS_PER_TASK
In this example, by setting an environment variable SRUN_CPUS_PER_TASK
{1, 2, 3, 4, 45, 46, 47, 48}
```
## General topics
### Default SHELL
In general, **`/bin/bash` is the recommended default user's SHELL** when working in Merlin.
Some users might notice that BASH is not the default SHELL when logging in to Merlin systems, or they might need to run a different SHELL.
This is probably because when the PSI account was requested, no SHELL description was specified or a different one was requested explicitly by the requestor.
Users can check which is the default SHELL specified in the PSI account with the following command:
```bash
getent passwd $USER | awk -F: '{print $NF}'
```
If SHELL does not correspond to the one you need to use, you should request a central change for it.
This is because Merlin accounts are central PSI accounts. Hence, **change must be requested via [PSI Service Now](contact.md#psi-service-now)**.
Alternatively, if you work on other PSI Linux systems but for Merlin you need a different SHELL type, a temporary change can be performed during login startup.
You can update one of the following files:
* `~/.login`
* `~/.profile`
* Any `rc` or `profile` file in your home directory (i.e. `.cshrc`, `.bashrc`, `.bash_profile`, etc.)
with the following lines:
```bash
# Replace MY_SHELL with the bash type you need
MY_SHELL=/bin/bash
exec $MY_SHELL -l
```
Notice that available *shells* can be found in the following file:
```bash
cat /etc/shells
```
### 3D acceleration: OpenGL vs Mesa
Some applications can run with OpenGL support. This is only possible when the node contains a GPU card.
In general, X11 with Mesa Driver is the recommended method as it will work in all cases (no need of GPUs). In example, for ParaView:
```bash
module load paraview
paraview-mesa paraview # 'paraview --mesa' for old releases
```
However, if one needs to run with OpenGL support, this is still possible by running `vglrun`. In example, for running Paraview:
```bash
module load paraview
vglrun paraview
```
Officially, the supported method for running `vglrun` is by using the [NoMachine remote desktop](../how-to-use-merlin/nomachine.md).
Running `vglrun` it's also possible using SSH with X11 Forwarding. However, it's very slow and it's only recommended when running
in Slurm (from [NoMachine](../how-to-use-merlin/nomachine.md)). Please, avoid running `vglrun` over SSH from a desktop or laptop.
## Software
### ANSYS
Sometimes, running ANSYS/Fluent requires X11 support. For that, one should run fluent as follows.
```bash
module load ANSYS
fluent -driver x11
```
### Paraview
For running Paraview, one can run it with Mesa support or OpenGL support. Please refer to [OpenGL vs Mesa](#3d-acceleration-opengl-vs-mesa) for
further information about how to run it.
### Module command not found
In some circumstances the module command may not be initialized properly. For instance, you may see the following error upon logon:
```
bash: module: command not found
```
The most common cause for this is a custom `.bashrc` file which fails to source the global `/etc/bashrc` responsible for setting up PModules in some OS versions. To fix this, add the following to `$HOME/.bashrc`:
```bash
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
```
It can also be fixed temporarily in an existing terminal by running `. /etc/bashrc` manually.

40
docs/merlin6/99-support/troubleshooting.md
View File

@@ -1,40 +0,0 @@
# Troubleshooting
For troubleshooting, please contact us through the official channels. See [Contact](contact.md)
for more information.
## Known Problems
Before contacting us for support, please check the **[Merlin6 Support: Known Problems](known-problems.md)** page to see if there is an existing
workaround for your specific problem.
## Troubleshooting Slurm Jobs
If you want to report a problem or request for help when running jobs, please **always provide**
the following information:
1. Provide your batch script or, alternatively, the path to your batch script.
2. Add **always** the following commands to your batch script
```bash
echo "User information:"; who am i
echo "Running hostname:"; hostname
echo "Current location:"; pwd
echo "User environment:"; env
echo "List of PModules:"; module list
```
3. Whenever possible, provide the Slurm JobID.
Providing this information is **extremely important** in order to ease debugging, otherwise
only with the description of the issue or just the error message is completely insufficient
in most cases.
## Troubleshooting SSH
Use the ssh command with the "-vvv" option and copy and paste (no screenshots please)
the output to your request in Service-Now. Example
```bash
ssh -Y -vvv $username@merlin-l-01.psi.ch
```

69
docs/merlin6/how-to-use-merlin/transfer-data.md
View File

@@ -6,14 +6,14 @@ Most methods allow data to be either transmitted or received, so it may make sen
initiate the transfer from either merlin or the other system, depending on the network
visibility.
- Merlin login nodes are visible from the PSI network, so direct data transfer
* Merlin login nodes are visible from the PSI network, so direct data transfer
(rsync/WinSCP) is generally preferable. This can be initiated from either endpoint.
- Merlin login nodes can access the internet using a limited set of protocols
- SSH-based protocols using port 22 (rsync-over-ssh, sftp, WinSCP, etc)
- HTTP-based protocols using ports 80 or 445 (https, WebDav, etc)
- Protocols using other ports require admin configuration and may only work with
specific hosts (ftp, rsync daemons, etc)
- Systems on the internet can access the [PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer) service
* Merlin login nodes can access the internet using a limited set of protocols:
* SSH-based protocols using port 22 (rsync-over-ssh, sftp, WinSCP, etc)
* HTTP-based protocols using ports 80 or 445 (https, WebDav, etc)
* Protocols using other ports require admin configuration and may only work with
specific hosts (ftp, rsync daemons, etc)
* Systems on the internet can access the [PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer) service
`datatransfer.psi.ch`, using ssh-based protocols and [Globus](https://www.globus.org/)
## Direct transfer via Merlin6 login nodes
@@ -27,14 +27,14 @@ for use from within the PSI network.
Rsync is the preferred method to transfer data from Linux/MacOS. It allows
transfers to be easily resumed if they get interrupted. The general syntax is:
```
```bash
rsync -avAHXS <src> <dst>
```
For example, to transfer files from your local computer to a merlin project
directory:
```
```bash
rsync -avAHXS ~/localdata user@merlin-l-01.psi.ch:/data/project/general/myproject/
```
@@ -60,25 +60,28 @@ The purpose of the software is to send a large file to someone, have that file a
## PSI Data Transfer
From August 2024, Merlin is connected to the **[PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer)** service,
`datatransfer.psi.ch`. This is a central service managed by the **[Linux team](https://linux.psi.ch/index.html)**. However, any problems or questions related to it can be directly
[reported](../99-support/contact.md) to the Merlin administrators, which will forward the request if necessary.
From August 2024, Merlin is connected to the **[PSI Data
Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer)**
service, `datatransfer.psi.ch`. This is a central service managed by the
**[Linux team](https://linux.psi.ch/index.html)**. However, any problems or
questions related to it can be directly [reported](../../support/index.md) to the
Merlin administrators, which will forward the request if necessary.
The PSI Data Transfer servers supports the following protocols:
* Data Transfer - SSH (scp / rsync)
* Data Transfer - Globus
Notice that `datatransfer.psi.ch` does not allow SSH login, only `rsync`, `scp` and [Globus](https://www.globus.org/) access is allowed.
The following filesystems are mounted:
* `/merlin/export` which points to the `/export` directory in Merlin.
* `/merlin/data/experiment/mu3e` which points to the `/data/experiment/mu3e` directories in Merlin.
* Mu3e sub-directories are mounted in RW (read-write), except for `data` (read-only mounted)
* `/merlin/data/project/general` which points to the `/data/project/general` directories in Merlin.
* Owners of Merlin projects should request explicit access to it.
* Currently, only `CSCS` is available for transferring files between PizDaint/Alps and Merlin
* `/merlin/data/project/bio` which points to the `/data/project/bio` directories in Merlin.
* `/merlin/data/user` which points to the `/data/user` directories in Merlin.
@@ -95,34 +98,42 @@ Therefore, having the Microsoft Authenticator App is required as explained [here
User data directories are mounted in RW.
!!! warning "Secure Permissions"
Please, **ensure proper secured permissions** in your `/data/user` directory. By default, when directory is created, the system applies the most restrictive permissions. However, this does not prevent users for changing permissions if they wish. At this point, users become responsible of those changes.
Please, **ensure proper secured permissions** in your `/data/user`
directory. By default, when directory is created, the system applies the
most restrictive permissions. However, this does not prevent users for
changing permissions if they wish. At this point, users become responsible
of those changes.
#### /merlin/export
Transferring big amounts of data from outside PSI to Merlin is always possible through `/export`.
!!! tip "Export Directory Access"
The `/export` directory can be used by any Merlin user. This is configured in Read/Write mode. If you need access, please, contact the Merlin administrators.
The `/export` directory can be used by any Merlin user. This is configured
in Read/Write mode. If you need access, please, contact the Merlin
administrators.
!!! warning "Export Usage Policy"
The use **export** as an extension of the quota *is forbidden*.
Auto cleanup policies in the **export** area apply for files older than 28 days.
The use **export** as an extension of the quota *is forbidden*. Auto
cleanup policies in the **export** area apply for files older than 28 days.
##### Exporting data from Merlin
For exporting data from Merlin to outside PSI by using `/export`, one has to:
* From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to
`/export`. Ensure to properly secure your directories and files with proper permissions.
* Once data is copied, from **`datatransfer.psi.ch`**, copy the data from `/merlin/export` to outside PSI
* From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to
`/export`. Ensure to properly secure your directories and files with proper permissions.
* Once data is copied, from **`datatransfer.psi.ch`**, copy the data from `/merlin/export` to outside PSI
##### Importing data to Merlin
For importing data from outside PSI to Merlin by using `/export`, one has to:
* From **`datatransfer.psi.ch`**, copy the data from outside PSI to `/merlin/export`.
* From **`datatransfer.psi.ch`**, copy the data from outside PSI to `/merlin/export`.
Ensure to properly secure your directories and files with proper permissions.
* Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`).
* Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`).
#### Request access to your project directory
@@ -148,10 +159,10 @@ Merlin6 is fully accessible from within the PSI network. To connect from outside
Merlin provides a command for mounting remote file systems, called `merlin_rmount`. This
provides a helpful wrapper over the Gnome storage utilities, and provides support for a wide range of remote file formats, including
- SMB/CIFS (Windows shared folders)
- WebDav
- AFP
- FTP, SFTP
- [others](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_the_desktop_environment_in_rhel_8/managing-storage-volumes-in-gnome_using-the-desktop-environment-in-rhel-8#gvfs-back-ends_managing-storage-volumes-in-gnome)
* SMB/CIFS (Windows shared folders)
* WebDav
* AFP
* FTP, SFTP
* [others](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_the_desktop_environment_in_rhel_8/managing-storage-volumes-in-gnome_using-the-desktop-environment-in-rhel-8#gvfs-back-ends_managing-storage-volumes-in-gnome)
[More instruction on using `merlin_rmount`](../software-support/merlin-rmount.md)

2
docs/merlin6/introduction.md
View File

@@ -1,7 +1,7 @@
# Introduction
!!! note
Merlin6 is no longer accessible for new users! Please have a look at [Merlin 7](../../merlin7/01-Quick-Start-Guide/introduction.md) instead.
Merlin6 is no longer accessible for new users! Please have a look at [Merlin 7](../merlin7/01-Quick-Start-Guide/introduction.md) instead.
## Slurm clusters