From 1e9c8a95d0af33d5b223388f2361b1e7c672379a Mon Sep 17 00:00:00 2001 From: caubet_m Date: Wed, 6 May 2020 13:41:28 +0200 Subject: [PATCH] Scratch policy update --- pages/merlin6/02 accessing-merlin6/storage.md | 21 ++++++++++++------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/pages/merlin6/02 accessing-merlin6/storage.md b/pages/merlin6/02 accessing-merlin6/storage.md index 5de9c34..388e6ff 100644 --- a/pages/merlin6/02 accessing-merlin6/storage.md +++ b/pages/merlin6/02 accessing-merlin6/storage.md @@ -39,12 +39,15 @@ Properties of the directory classes: | Directory | Block Quota [Soft:Hard] | Block Quota [Soft:Hard] | Quota Change Policy: Block | Quota Change Policy: Files | Backup | Backup Policy | | ---------------------------------- | ----------------------- | ----------------------- |:--------------------------------- |:-------------------------------- | ------ | :----------------------------- | -| /psi/home/$username | USR [10GB:11GB] | *Undef* | Up to x2 when strictly justified. | N/A | yes | Daily snapshots for 1 week | +| /psi/home/$username | USR [10GB:11GB] | *Undef* | Up to x2 when strongly justified. | N/A | yes | Daily snapshots for 1 week | | /data/user/$username | USR [1TB:1.074TB] | USR [1M:1.1M] | Inmutable. Need a project. | Changeable when justified. | no | Users responsible for backup | | /data/project/bio/$projectname | GRP [1TB:1.074TB] | GRP [1M:1.1M] | Subject to project requirements. | Subject to project requirements. | no | Project responsible for backup | | /data/project/general/$projectname | GRP [1TB:1.074TB] | GRP [1M:1.1M] | Subject to project requirements. | Subject to project requirements. | no | Project responsible for backup | | /scratch | *Undef* | *Undef* | N/A | N/A | no | N/A | -| /shared-scratch | *Undef* | *Undef* | N/A | N/A | no | N/A | +| /shared-scratch | USR [512GB:2TB] | USR [2M:2.5M] | Up to x2 when strongly justified. | Changeable when justified. | no | N/A | + +* **Important note:** the use of **scratch** areas as an extension of the quota **is forbidden**. **scratch** areas **must not contain** final data. +Auto cleanup policies in the **scratch** areas are applied. ### User home directory @@ -149,12 +152,12 @@ and all tasks need to do I/O on the same temporary files. The properties of the available scratch storage spaces are given in the following table -| Cluster | Service | Scratch | Scratch Mountpoint | Shared Scratch | Shared Scratch Mountpoint | Comments | -| ------- | -------------- | ------------ | ------------------ | -------------- | ------------------------- | ------------------------------------- | -| merlin5 | computing node | 50GB / SAS | ``/scratch`` | ``N/A`` | ``N/A`` | ``merlin-c-[01-64]`` | -| merlin6 | login node | 100GB / SAS | ``/scratch`` | ``N/A`` | ``N/A`` | ``merlin-l-0[1,2]`` | -| merlin6 | computing node | 1.3TB / NVMe | ``/scratch`` | 50TB / GPFS | ``/shared-scratch`` | ``merlin-c-[001-022,101-122,201-222`` | -| merlin6 | login node | 2.0TB / NVMe | ``/scratch`` | ``N/A`` | ``N/A`` | ``merlin-l-00[1,2]`` | +| Cluster | Service | Scratch | Scratch Mountpoint | Shared Scratch | Shared Scratch Mountpoint | Comments | +| ------- | -------------- | ------------ | ------------------ | -------------- | ------------------------- | -------------------------------------- | +| merlin5 | computing node | 50GB / SAS | ``/scratch`` | ``N/A`` | ``N/A`` | ``merlin-c-[01-64]`` | +| merlin6 | login node | 100GB / SAS | ``/scratch`` | 50TB / GPFS | ``/shared-scratch`` | ``merlin-l-0[1,2]`` | +| merlin6 | computing node | 1.3TB / NVMe | ``/scratch`` | 50TB / GPFS | ``/shared-scratch`` | ``merlin-c-[001-024,101-124,201-224]`` | +| merlin6 | login node | 2.0TB / NVMe | ``/scratch`` | 50TB / GPFS | ``/shared-scratch`` | ``merlin-l-00[1,2]`` | #### Scratch directories policy @@ -162,5 +165,7 @@ The properties of the available scratch storage spaces are given in the followin * By default, *always* use **local** first and only use **shared** if your specific use case requires it. * Temporary files *must be deleted at the end of the job by the user*. * Remaining files will be deleted by the system if detected. + * Files not accessed within 28 days will be automatically cleaned up by the system. + * If for some reason the scratch areas get full admins have the rights to cleanup the oldest data. ---