diff --git a/pages/merlin6/02-How-To-Use-Merlin/transfer-data.md b/pages/merlin6/02-How-To-Use-Merlin/transfer-data.md index d7e054c..cc2765f 100644 --- a/pages/merlin6/02-How-To-Use-Merlin/transfer-data.md +++ b/pages/merlin6/02-How-To-Use-Merlin/transfer-data.md @@ -21,8 +21,8 @@ visibility. - HTTP-based protocols using ports 80 or 445 (https, WebDav, etc) - Protocols using other ports require admin configuration and may only work with specific hosts (ftp, rsync daemons, etc) -- Systems on the internet can access the Remote Access Merlin servers - (ra-merlin\*.psi.ch) using ssh-based protocols +- Systems on the internet can access the [PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer) service + `datatransfer.psi.ch`, using ssh-based protocols and [Globus](https://www.globus.org/) ## Direct transfer via Merlin6 login nodes @@ -68,32 +68,38 @@ The purpose of the software is to send a large file to someone, have that file a **[SWITCHfilesender](https://filesender.switch.ch/filesender2/?s=upload)** is fully integrated with PSI, therefore, PSI employees can log in by using their PSI account (through Authentication and Authorization Infrastructure / AAI, by selecting PSI as the institution to be used for log in). -## Remote Access Servers +## PSI Data Transfer -Two servers are enabled for data transfers originating from outside PSI. -This is a central service managed by a different team, which is managing the different Remote Access -services at PSI for the different facilities (including the one for Merlin). However, any problems -or questions related to it can be directly [reported](/merlin6/contact.html) to the Merlin adminstrators, -which will forward the request if necessary. +From August 2024, Merlin is connected to the **[PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer)** service, +`datatransfer.psi.ch`. This is a central service managed by the Linux team. However, any problems or questions related to it can be directly +[reported](/merlin6/contact.html) to the Merlin adminstrators, which will forward the request if necessary. -These Remote Access Merlin servers are the following: -* **`ra-merlin-01.psi.ch`** -* **`ra-merlin-02.psi.ch`** +The PSI Data Transfer servers supports the following protocols: +* Data Transfer - SSH (scp / rsync) +* Data Transfer - Globus -Both servers have mounted the following Merlin filesystems: -* `/data/project` directories mounted in RW on demand. Project responsibles must request it. -* `/data/user` mounted in RW (read-write) -* `/data/experiment/mu3e` directories mounted in RW (read-write), except `data` (read-only mounted) -* `/export` directory in RW (read-write). `/export` is also visible from login nodes. +Notice that `datatransfer.psi.ch` does not allow SSH login, only `rsync`, `scp` and [Globus](https://www.globus.org/) access is allowed. -Access to the Remote Access server uses ***Multi factor authentication*** (MFA). +The following filesystems are mounted: +* `/merlin/bio` which points to the `/data/project/bio` directories in Merlin. +* `/merlin/user` which points to the `/data/user` directories in Merlin. +* `/merlin/export` which points to the `/export` directory in Merlin. +* `/merlin/cscs` which points to the `/data/project/general/cscs` directories in Merlin. +* `/merlin/mu3e` which points to the `/data/experiment/mu3e` directories in Merlin. + * Mu3e sub-directories are mounted in RW (read-write), except for `data` (read-only mounted) + +Access to the PSI Data Transfer uses ***Multi factor authentication*** (MFA). Therefore, having the Microsoft Authenticator App is required as explained [here](https://www.psi.ch/en/computing/change-to-mfa). +{{site.data.alerts.tip}}Please follow the +Official PSI Data Transfer documentation for further instructions. +{{site.data.alerts.end}} + ### Directories -#### /data/user +#### /merlin/user -User data directories are mounted in RW on both 'ra-merlin-01' and 'ra-merlin-02'. +User data directories are mounted in RW. {{site.data.alerts.warning}}Please, ensure proper secured permissions in your '/data/user' directory. By default, when directory is created, the system applies the most restrictive @@ -101,7 +107,7 @@ permissions. However, this does not prevent users for changing permissions if th point, users become responsible of those changes. {{site.data.alerts.end}} -#### /export +#### /merlin/export Transferring big amounts of data from outside PSI to Merlin is always possible through `/export`. @@ -118,41 +124,23 @@ This is configured in Read/Write mode. If you need access, please, contact the M For exporting data from Merlin to outside PSI by using `/export`, one has to: * From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to `/export`. Ensure to properly secure your directories and files with proper permissions. - * Once data is copied, from **ra-merlin-01.psi.ch** or **ra-merlin-02.psi.ch**, copy the data from `/export` to outside PSI. + * Once data is copied, from **`datatransfer.psi.ch`**, copy the data from `/merlin/export` to outside PSI ##### Importing data to Merlin For importing data from outside PSI to Merlin by using `/export`, one has to: - * From **ra-merlin-01.psi.ch** or **ra-merlin-02.psi.ch**, copy the data from outside PSI to `/export`. + * From **`datatransfer.psi.ch`**, copy the data from outside PSI to `/merlin/export`. Ensure to properly secure your directories and files with proper permissions. * Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`). -#### /data/project +#### Request access to your project directory -Optionally, instead of using `/export`, experiments with a Merlin project can request Read/Write or Read/Only access to their project directory. +Optionally, instead of using `/export`, Merlin project owners can request Read/Write or Read/Only access to their project directory. {{site.data.alerts.tip}}Merlin projects can request direct access. -This can be configured in Read/Write or Read/Only modes. If your project needs access, please, -contact the Merlin administrators. +This can be configured in Read/Write or Read/Only modes. If your project needs access, please, contact the Merlin administrators. {{site.data.alerts.end}} -### Accepted protocols - -Accepted protocols for Remote Access Merlin servers are the following: -* **sftp**: **``sftp``** command or similar X11/Windows/MacOS based programs. -* **ssh**: **`scp`** command (as well as **WinSCP** and similar programs) or **`rsync`** command -* **~~Globus Online~~**: ***not available yet.*** - -### Remote Access Servers Policies - -SSH is one of the allowed protocols. -* Please, **absolutely never** use this servers as a login node. -* Please avoid copying files to the *home* directories. -* Please **never use SSH Keys** for accessing these servers. Accessing through SSH keys will be denied in the upcomig months. - -Only ``/data/user`, `/data/project` and `/export` directories should be used on these nodes, -and exclusively for transferring data from/to PSI to/from outside PSI. - ## Connecting to Merlin6 from outside PSI Merlin6 is fully accessible from within the PSI network. To connect from outside you can use: