Transfer data update

This commit is contained in:
2020-07-27 16:33:04 +02:00
parent cdbbb30504
commit 19d295df0c

View File

@ -41,12 +41,24 @@ local computer and merlin.
Two servers are enabled for exporting data from Merlin to outside PSI.
These Remote Access Merlin servers are the following:
* **'ra-merlin-01.psi.ch'**
* **'ra-merlin-02.psi.ch'**
* **'ra-merlin-01.psi.ch'**: normal password authentication with PSI password
* `/data/user` mounted in RO (read-only)
* `/export` directory in RW (read-write). `/export` is also visible from login nodes.
* **'ra-merlin-02.psi.ch'**: ***Two factor authentication*** (2FA), required **RSA SecurID** token (same as VPN)
* `/data/project` directories mounted in RW on demand. Project responsibles must request it.
* `/data/user` mounted in RO (read-only)
* `/export` directory in RW (read-write). `/export` is also visible from login nodes.
Due to security reasons, currently only ``/data/user`` directories are exported in Read-Only mode.
This policy will change in the future but in the meantime we allow users to export their files
in an easy way.
In the future, **'ra-merlin-01.psi.ch'** will be also configured with 2FA and will mount the same
as **'ra-merlin-02.psi.ch'**. In the meantime, we keep **'ra-merlin-01.psi.ch'** with standard authentication
until we can ensure that most of the Merlin users have a RSA SecurID token or until PSI security policy makes
its use mandatory.
### Directories
#### /data/user
User data directories are mounted in RO on both servers. Mounting it in RW is under evaluation.
{{site.data.alerts.warning}}Please, <b>ensure proper secured permissions</b> in your '/data/user'
directory. By default, when directory is created, the system applies the most restrictive
@ -54,9 +66,36 @@ permissions. However, this does not prevent users for changing permissions if th
point, users become responsible of those changes.
{{site.data.alerts.end}}
Transferring big amounts of data from outside PSI to Merlin is not possible yet, unless
this is done from PSI (i.e. transferring files from CSCS to Merlin is possible by using
`scp` / `rsync` commands from the Merlin6 login nodes).
#### /export
Transferring big amounts of data from outside PSI to Merlin is always possible through `/export`.
{{site.data.alerts.tip}}<b>The '/export' directory can be used by any Merlin user.</b>
This is configured in Read/Write mode. If you need access, please, contact the Merlin administrators.
{{site.data.alerts.end}}
##### Exporting data from Merlin
For exporting data from Merlin to outside PSI by using `/export`, one has to:
* From a Merlin login node, copy your data from any directory (i.e. `/data/project`, `/data/user`, `/scratch`) to
`/export`. Ensure to properly secure your directories and files with proper permissions.
* Once data is copied, from **ra-merlin-01.psi.ch** or **ra-merlin-02.psi.ch**, copy the data from `/export` to outside PSI.
##### Importing data to Merlin
For importing data from outside PSI to Merlin by using `/export`, one has to:
* From **ra-merlin-01.psi.ch** or **ra-merlin-02.psi.ch**, copy the data from outside PSI to `/export`.
Ensure to properly secure your directories and files with proper permissions.
* Once data is copied, from a Merlin login node, copy your data from `/export` to any directory (i.e. `/data/project`, `/data/user`, `/scratch`).
#### /data/project
Optionally, instead of using `/export`, experiments with a Merlin project can request Read/Write or Read/Only access to their project directory.
{{site.data.alerts.tip}}<b>Merlin projects can request direct access on 'ra-merlin-02.psi.ch'</b>
This can be configured in Read/Write or Read/Only modes. If your project needs access, please,
contact the Merlin administrators.
{{site.data.alerts.end}}
### Accepted protocols
@ -65,13 +104,15 @@ Accepted protocols for Remote Access Merlin servers are the following:
* **ssh**: **`scp`** command (as well as **WinSCP** and similar programs) or **`rsync`** command
* **~~Globus Online~~**: ***not available yet.***
### About Remote Access Servers
### Remote Access Servers Policies
SSH is one of the allowed protocols.
* Please, **absolutely never** use this servers as a login node.
* Please avoid copying files to the *home* directories.
* Please **never use SSH Keys** for accessing these servers. Accessing through SSH keys will be denied in the upcomig months.
Only ``/data/user`` should be used on these nodes, and exclusively for transferring data to outside PSI.
Only ``/data/user`, `/data/project` and `/export` directories should be used on these nodes,
and exclusively for transferring data from/to PSI to/from outside PSI.
## Connecting to Merlin6 from outside PSI