merge and move support pages
this are now under the /support path, meaning that this is unified for all clusters.
This commit is contained in:
@@ -6,14 +6,14 @@ Most methods allow data to be either transmitted or received, so it may make sen
|
||||
initiate the transfer from either merlin or the other system, depending on the network
|
||||
visibility.
|
||||
|
||||
- Merlin login nodes are visible from the PSI network, so direct data transfer
|
||||
* Merlin login nodes are visible from the PSI network, so direct data transfer
|
||||
(rsync/WinSCP) is generally preferable. This can be initiated from either endpoint.
|
||||
- Merlin login nodes can access the internet using a limited set of protocols
|
||||
- SSH-based protocols using port 22 (rsync-over-ssh, sftp, WinSCP, etc)
|
||||
- HTTP-based protocols using ports 80 or 445 (https, WebDav, etc)
|
||||
- Protocols using other ports require admin configuration and may only work with
|
||||
specific hosts (ftp, rsync daemons, etc)
|
||||
- Systems on the internet can access the [PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer) service
|
||||
* Merlin login nodes can access the internet using a limited set of protocols:
|
||||
* SSH-based protocols using port 22 (rsync-over-ssh, sftp, WinSCP, etc)
|
||||
* HTTP-based protocols using ports 80 or 445 (https, WebDav, etc)
|
||||
* Protocols using other ports require admin configuration and may only work with
|
||||
specific hosts (ftp, rsync daemons, etc)
|
||||
* Systems on the internet can access the [PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer) service
|
||||
`datatransfer.psi.ch`, using ssh-based protocols and [Globus](https://www.globus.org/)
|
||||
|
||||
## Direct transfer via Merlin6 login nodes
|
||||
@@ -27,14 +27,14 @@ for use from within the PSI network.
|
||||
Rsync is the preferred method to transfer data from Linux/MacOS. It allows
|
||||
transfers to be easily resumed if they get interrupted. The general syntax is:
|
||||
|
||||
```
|
||||
```bash
|
||||
rsync -avAHXS <src> <dst>
|
||||
```
|
||||
|
||||
For example, to transfer files from your local computer to a merlin project
|
||||
directory:
|
||||
|
||||
```
|
||||
```bash
|
||||
rsync -avAHXS ~/localdata user@merlin-l-01.psi.ch:/data/project/general/myproject/
|
||||
```
|
||||
|
||||
@@ -60,25 +60,28 @@ The purpose of the software is to send a large file to someone, have that file a
|
||||
|
||||
## PSI Data Transfer
|
||||
|
||||
From August 2024, Merlin is connected to the **[PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer)** service,
|
||||
`datatransfer.psi.ch`. This is a central service managed by the **[Linux team](https://linux.psi.ch/index.html)**. However, any problems or questions related to it can be directly
|
||||
[reported](../99-support/contact.md) to the Merlin administrators, which will forward the request if necessary.
|
||||
From August 2024, Merlin is connected to the **[PSI Data
|
||||
Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer)**
|
||||
service, `datatransfer.psi.ch`. This is a central service managed by the
|
||||
**[Linux team](https://linux.psi.ch/index.html)**. However, any problems or
|
||||
questions related to it can be directly [reported](../../support/index.md) to the
|
||||
Merlin administrators, which will forward the request if necessary.
|
||||
|
||||
The PSI Data Transfer servers supports the following protocols:
|
||||
|
||||
* Data Transfer - SSH (scp / rsync)
|
||||
* Data Transfer - Globus
|
||||
|
||||
Notice that `datatransfer.psi.ch` does not allow SSH login, only `rsync`, `scp` and [Globus](https://www.globus.org/) access is allowed.
|
||||
|
||||
The following filesystems are mounted:
|
||||
|
||||
* `/merlin/export` which points to the `/export` directory in Merlin.
|
||||
* `/merlin/data/experiment/mu3e` which points to the `/data/experiment/mu3e` directories in Merlin.
|
||||
* Mu3e sub-directories are mounted in RW (read-write), except for `data` (read-only mounted)
|
||||
|
||||
* `/merlin/data/project/general` which points to the `/data/project/general` directories in Merlin.
|
||||
* Owners of Merlin projects should request explicit access to it.
|
||||
* Currently, only `CSCS` is available for transferring files between PizDaint/Alps and Merlin
|
||||
|
||||
* `/merlin/data/project/bio` which points to the `/data/project/bio` directories in Merlin.
|
||||
* `/merlin/data/user` which points to the `/data/user` directories in Merlin.
|
||||
|
||||
@@ -95,34 +98,42 @@ Therefore, having the Microsoft Authenticator App is required as explained [here
|
||||
User data directories are mounted in RW.
|
||||
|
||||
!!! warning "Secure Permissions"
|
||||
Please, **ensure proper secured permissions** in your `/data/user` directory. By default, when directory is created, the system applies the most restrictive permissions. However, this does not prevent users for changing permissions if they wish. At this point, users become responsible of those changes.
|
||||
Please, **ensure proper secured permissions** in your `/data/user`
|
||||
directory. By default, when directory is created, the system applies the
|
||||
most restrictive permissions. However, this does not prevent users for
|
||||
changing permissions if they wish. At this point, users become responsible
|
||||
of those changes.
|
||||
|
||||
#### /merlin/export
|
||||
|
||||
Transferring big amounts of data from outside PSI to Merlin is always possible through `/export`.
|
||||
|
||||
!!! tip "Export Directory Access"
|
||||
The `/export` directory can be used by any Merlin user. This is configured in Read/Write mode. If you need access, please, contact the Merlin administrators.
|
||||
The `/export` directory can be used by any Merlin user. This is configured
|
||||
in Read/Write mode. If you need access, please, contact the Merlin
|
||||
administrators.
|
||||
|
||||
!!! warning "Export Usage Policy"
|
||||
The use **export** as an extension of the quota *is forbidden*.
|
||||
|
||||
Auto cleanup policies in the **export** area apply for files older than 28 days.
|
||||
The use **export** as an extension of the quota *is forbidden*. Auto
|
||||
cleanup policies in the **export** area apply for files older than 28 days.
|
||||
|
||||
##### Exporting data from Merlin
|
||||
|
||||
For exporting data from Merlin to outside PSI by using `/export`, one has to:
|
||||
* From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to
|
||||
`/export`. Ensure to properly secure your directories and files with proper permissions.
|
||||
* Once data is copied, from **`datatransfer.psi.ch`**, copy the data from `/merlin/export` to outside PSI
|
||||
|
||||
* From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to
|
||||
`/export`. Ensure to properly secure your directories and files with proper permissions.
|
||||
* Once data is copied, from **`datatransfer.psi.ch`**, copy the data from `/merlin/export` to outside PSI
|
||||
|
||||
##### Importing data to Merlin
|
||||
|
||||
For importing data from outside PSI to Merlin by using `/export`, one has to:
|
||||
* From **`datatransfer.psi.ch`**, copy the data from outside PSI to `/merlin/export`.
|
||||
|
||||
* From **`datatransfer.psi.ch`**, copy the data from outside PSI to `/merlin/export`.
|
||||
|
||||
Ensure to properly secure your directories and files with proper permissions.
|
||||
* Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`).
|
||||
|
||||
* Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`).
|
||||
|
||||
#### Request access to your project directory
|
||||
|
||||
@@ -148,10 +159,10 @@ Merlin6 is fully accessible from within the PSI network. To connect from outside
|
||||
|
||||
Merlin provides a command for mounting remote file systems, called `merlin_rmount`. This
|
||||
provides a helpful wrapper over the Gnome storage utilities, and provides support for a wide range of remote file formats, including
|
||||
- SMB/CIFS (Windows shared folders)
|
||||
- WebDav
|
||||
- AFP
|
||||
- FTP, SFTP
|
||||
- [others](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_the_desktop_environment_in_rhel_8/managing-storage-volumes-in-gnome_using-the-desktop-environment-in-rhel-8#gvfs-back-ends_managing-storage-volumes-in-gnome)
|
||||
* SMB/CIFS (Windows shared folders)
|
||||
* WebDav
|
||||
* AFP
|
||||
* FTP, SFTP
|
||||
* [others](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/using_the_desktop_environment_in_rhel_8/managing-storage-volumes-in-gnome_using-the-desktop-environment-in-rhel-8#gvfs-back-ends_managing-storage-volumes-in-gnome)
|
||||
|
||||
[More instruction on using `merlin_rmount`](../software-support/merlin-rmount.md)
|
||||
|
||||
Reference in New Issue
Block a user