Update Data Transfer to use datatransfer.psi.ch
This commit is contained in:
parent
a0e38e612f
commit
b63ecb4141
@ -21,8 +21,8 @@ visibility.
|
||||
- HTTP-based protocols using ports 80 or 445 (https, WebDav, etc)
|
||||
- Protocols using other ports require admin configuration and may only work with
|
||||
specific hosts (ftp, rsync daemons, etc)
|
||||
- Systems on the internet can access the Remote Access Merlin servers
|
||||
(ra-merlin\*.psi.ch) using ssh-based protocols
|
||||
- Systems on the internet can access the [PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer) service
|
||||
`datatransfer.psi.ch`, using ssh-based protocols and [Globus](https://www.globus.org/)
|
||||
|
||||
|
||||
## Direct transfer via Merlin6 login nodes
|
||||
@ -68,32 +68,38 @@ The purpose of the software is to send a large file to someone, have that file a
|
||||
|
||||
**[SWITCHfilesender](https://filesender.switch.ch/filesender2/?s=upload)** is fully integrated with PSI, therefore, PSI employees can log in by using their PSI account (through Authentication and Authorization Infrastructure / AAI, by selecting PSI as the institution to be used for log in).
|
||||
|
||||
## Remote Access Servers
|
||||
## PSI Data Transfer
|
||||
|
||||
Two servers are enabled for data transfers originating from outside PSI.
|
||||
This is a central service managed by a different team, which is managing the different Remote Access
|
||||
services at PSI for the different facilities (including the one for Merlin). However, any problems
|
||||
or questions related to it can be directly [reported](/merlin6/contact.html) to the Merlin adminstrators,
|
||||
which will forward the request if necessary.
|
||||
From August 2024, Merlin is connected to the **[PSI Data Transfer](https://www.psi.ch/en/photon-science-data-services/data-transfer)** service,
|
||||
`datatransfer.psi.ch`. This is a central service managed by the Linux team. However, any problems or questions related to it can be directly
|
||||
[reported](/merlin6/contact.html) to the Merlin adminstrators, which will forward the request if necessary.
|
||||
|
||||
These Remote Access Merlin servers are the following:
|
||||
* **`ra-merlin-01.psi.ch`**
|
||||
* **`ra-merlin-02.psi.ch`**
|
||||
The PSI Data Transfer servers supports the following protocols:
|
||||
* Data Transfer - SSH (scp / rsync)
|
||||
* Data Transfer - Globus
|
||||
|
||||
Both servers have mounted the following Merlin filesystems:
|
||||
* `/data/project` directories mounted in RW on demand. Project responsibles must request it.
|
||||
* `/data/user` mounted in RW (read-write)
|
||||
* `/data/experiment/mu3e` directories mounted in RW (read-write), except `data` (read-only mounted)
|
||||
* `/export` directory in RW (read-write). `/export` is also visible from login nodes.
|
||||
Notice that `datatransfer.psi.ch` does not allow SSH login, only `rsync`, `scp` and [Globus](https://www.globus.org/) access is allowed.
|
||||
|
||||
Access to the Remote Access server uses ***Multi factor authentication*** (MFA).
|
||||
The following filesystems are mounted:
|
||||
* `/merlin/bio` which points to the `/data/project/bio` directories in Merlin.
|
||||
* `/merlin/user` which points to the `/data/user` directories in Merlin.
|
||||
* `/merlin/export` which points to the `/export` directory in Merlin.
|
||||
* `/merlin/cscs` which points to the `/data/project/general/cscs` directories in Merlin.
|
||||
* `/merlin/mu3e` which points to the `/data/experiment/mu3e` directories in Merlin.
|
||||
* Mu3e sub-directories are mounted in RW (read-write), except for `data` (read-only mounted)
|
||||
|
||||
Access to the PSI Data Transfer uses ***Multi factor authentication*** (MFA).
|
||||
Therefore, having the Microsoft Authenticator App is required as explained [here](https://www.psi.ch/en/computing/change-to-mfa).
|
||||
|
||||
{{site.data.alerts.tip}}Please follow the
|
||||
<b><a href="https://www.psi.ch/en/photon-science-data-services/data-transfer">Official PSI Data Transfer</a></b> documentation for further instructions.
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
### Directories
|
||||
|
||||
#### /data/user
|
||||
#### /merlin/user
|
||||
|
||||
User data directories are mounted in RW on both 'ra-merlin-01' and 'ra-merlin-02'.
|
||||
User data directories are mounted in RW.
|
||||
|
||||
{{site.data.alerts.warning}}Please, <b>ensure proper secured permissions</b> in your '/data/user'
|
||||
directory. By default, when directory is created, the system applies the most restrictive
|
||||
@ -101,7 +107,7 @@ permissions. However, this does not prevent users for changing permissions if th
|
||||
point, users become responsible of those changes.
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
#### /export
|
||||
#### /merlin/export
|
||||
|
||||
Transferring big amounts of data from outside PSI to Merlin is always possible through `/export`.
|
||||
|
||||
@ -118,41 +124,23 @@ This is configured in Read/Write mode. If you need access, please, contact the M
|
||||
For exporting data from Merlin to outside PSI by using `/export`, one has to:
|
||||
* From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to
|
||||
`/export`. Ensure to properly secure your directories and files with proper permissions.
|
||||
* Once data is copied, from **ra-merlin-01.psi.ch** or **ra-merlin-02.psi.ch**, copy the data from `/export` to outside PSI.
|
||||
* Once data is copied, from **`datatransfer.psi.ch`**, copy the data from `/merlin/export` to outside PSI
|
||||
|
||||
##### Importing data to Merlin
|
||||
|
||||
For importing data from outside PSI to Merlin by using `/export`, one has to:
|
||||
* From **ra-merlin-01.psi.ch** or **ra-merlin-02.psi.ch**, copy the data from outside PSI to `/export`.
|
||||
* From **`datatransfer.psi.ch`**, copy the data from outside PSI to `/merlin/export`.
|
||||
Ensure to properly secure your directories and files with proper permissions.
|
||||
* Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`).
|
||||
|
||||
#### /data/project
|
||||
#### Request access to your project directory
|
||||
|
||||
Optionally, instead of using `/export`, experiments with a Merlin project can request Read/Write or Read/Only access to their project directory.
|
||||
Optionally, instead of using `/export`, Merlin project owners can request Read/Write or Read/Only access to their project directory.
|
||||
|
||||
{{site.data.alerts.tip}}<b>Merlin projects can request direct access.</b>
|
||||
This can be configured in Read/Write or Read/Only modes. If your project needs access, please,
|
||||
contact the Merlin administrators.
|
||||
This can be configured in Read/Write or Read/Only modes. If your project needs access, please, contact the Merlin administrators.
|
||||
{{site.data.alerts.end}}
|
||||
|
||||
### Accepted protocols
|
||||
|
||||
Accepted protocols for Remote Access Merlin servers are the following:
|
||||
* **sftp**: **``sftp``** command or similar X11/Windows/MacOS based programs.
|
||||
* **ssh**: **`scp`** command (as well as **WinSCP** and similar programs) or **`rsync`** command
|
||||
* **~~Globus Online~~**: ***not available yet.***
|
||||
|
||||
### Remote Access Servers Policies
|
||||
|
||||
SSH is one of the allowed protocols.
|
||||
* Please, **absolutely never** use this servers as a login node.
|
||||
* Please avoid copying files to the *home* directories.
|
||||
* Please **never use SSH Keys** for accessing these servers. Accessing through SSH keys will be denied in the upcomig months.
|
||||
|
||||
Only ``/data/user`, `/data/project` and `/export` directories should be used on these nodes,
|
||||
and exclusively for transferring data from/to PSI to/from outside PSI.
|
||||
|
||||
## Connecting to Merlin6 from outside PSI
|
||||
|
||||
Merlin6 is fully accessible from within the PSI network. To connect from outside you can use:
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user