HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

transfer-data.md storage.md updated with exports

Browse Source
This commit is contained in:
caubet_m
2020-08-06 12:53:37 +02:00
parent 0d874eaaa0
commit 9537dcd2e4
2 changed files with 23 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
pages/merlin6/02 accessing-merlin6/storage.md
View File

@ -34,6 +34,8 @@ Merlin6 offers the following directory classes for users:
* For BIO experiments, a dedicated ``/data/project/bio/$projectname`` exists. * For BIO experiments, a dedicated ``/data/project/bio/$projectname`` exists.
* ``/scratch``: Local *scratch* disk (only visible by the node running a job). * ``/scratch``: Local *scratch* disk (only visible by the node running a job).
* ``/shared-scratch``: Shared *scratch* disk (visible from all nodes). * ``/shared-scratch``: Shared *scratch* disk (visible from all nodes).
* ``/export``: Export directory for data transfer, visible from `ra-merlin-01.psi.ch`, `ra-merlin-02.psi.ch` and Merlin login nodes.
* Refer to **[Transferring Data](/merlin6/transfer-data.html)** for more information about the export area and data transfer service.
Properties of the directory classes: Properties of the directory classes:
@ -45,9 +47,11 @@ Properties of the directory classes:
| /data/project/general/$projectname | GRP [1TB:1.074TB] | GRP [1M:1.1M] | Subject to project requirements. | Subject to project requirements. | no | Project responsible for backup | | /data/project/general/$projectname | GRP [1TB:1.074TB] | GRP [1M:1.1M] | Subject to project requirements. | Subject to project requirements. | no | Project responsible for backup |
| /scratch | *Undef* | *Undef* | N/A | N/A | no | N/A | | /scratch | *Undef* | *Undef* | N/A | N/A | no | N/A |
| /shared-scratch | USR [512GB:2TB] | USR [2M:2.5M] | Up to x2 when strongly justified. | Changeable when justified. | no | N/A | | /shared-scratch | USR [512GB:2TB] | USR [2M:2.5M] | Up to x2 when strongly justified. | Changeable when justified. | no | N/A |
| /export | USR [10MB:20TB] | USR [512K:5M] | Soft can be temporary increased. | Changeable when justified. | no | N/A |
* **Important note:** the use of **scratch** areas as an extension of the quota **is forbidden**. **scratch** areas **must not contain** final data. {{site.data.alerts.warning}}The use of <b>scratch</b> and <b>export</b> areas as an extension of the quota <i>is forbidden</i>. <b>scratch</b> and <b>export</b> areas <i>must not contain</i> final data.
Auto cleanup policies in the **scratch** areas are applied. <br><b><i>Auto cleanup policies</i></b> in the <b>scratch</b> and <b>export</b> areas are applied.
{{site.data.alerts.end}}
### User home directory ### User home directory
@ -166,6 +170,18 @@ The properties of the available scratch storage spaces are given in the followin
* Temporary files *must be deleted at the end of the job by the user*. * Temporary files *must be deleted at the end of the job by the user*.
* Remaining files will be deleted by the system if detected. * Remaining files will be deleted by the system if detected.
* Files not accessed within 28 days will be automatically cleaned up by the system. * Files not accessed within 28 days will be automatically cleaned up by the system.
* If for some reason the scratch areas get full admins have the rights to cleanup the oldest data. * If for some reason the scratch areas get full, admins have the rights to cleanup the oldest data.
### Export directory
Export directory is exclusively intended for transferring data from outside PSI to Merlin and viceversa. Is a temporary directoy with an auto-cleanup policy.
Please read **[Transferring Data](/merlin6/transfer-data.html)** for more information about it.
#### Export directory policy
* Temporary files *must be deleted at the end of the job by the user*.
* Remaining files will be deleted by the system if detected.
* Files not accessed within 28 days will be automatically cleaned up by the system.
* If for some reason the export area gets full, admins have the rights to cleanup the oldest data
--- ---

4
pages/merlin6/02 accessing-merlin6/transfer-data.md
View File

@ -75,6 +75,10 @@ Transferring big amounts of data from outside PSI to Merlin is always possible t
This is configured in Read/Write mode. If you need access, please, contact the Merlin administrators. This is configured in Read/Write mode. If you need access, please, contact the Merlin administrators.
{{site.data.alerts.end}} {{site.data.alerts.end}}
{{site.data.alerts.warning}}The use <b>export</b> as an extension of the quota <i>is forbidden</i>.
<br><b><i>Auto cleanup policies</i></b> in the <b>export</b> area apply for files older than 28 days.
{{site.data.alerts.end}}
##### Exporting data from Merlin ##### Exporting data from Merlin
For exporting data from Merlin to outside PSI by using `/export`, one has to: For exporting data from Merlin to outside PSI by using `/export`, one has to: