HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity
Files
master
gitea-pages/pages/merlin6/98-announcements/downtimes.md
caubet_m fcfdbf1344 Doc changes
2021-05-21 12:34:19 +02:00

3.3 KiB
Raw Permalink Blame History

title, last_updated, sidebar, permalink
title last_updated sidebar permalink
Downtimes 28 June 2019 merlin6_sidebar /merlin6/downtimes.html

On the first Monday of each month the Merlin6 cluster might be subject to interruption due to maintenance. Users will be informed with at least one week in advance when a downtime is scheduled for the next month.

Downtimes will be informed to users through the merlin-users@lists.psi.ch mail list. Also, a detailed description for the nexts scheduled interventions will be available in Next Scheduled Downtimes).

Scheduled Downtime Draining Policy

Scheduled downtimes mostly affecting the storage and Slurm configurantions may require draining the nodes. When this is required, users will be informed accordingly. Two different types of draining are possible:

  • soft drain: new jobs may be queued on the partition, but queued jobs may not be allocated nodes and run from the partition. Jobs already running on the partition continue to run. This will be the default drain method.
  • hard drain: no new jobs may be queued on the partition (job submission requests will be denied with an error message), but jobs already queued on the partition may be allocated to nodes and run.

Unless explicitly specified, the default draining policy for each partition will be the following:

  • The daily and general partitions will be soft drained 12h before the downtime.
  • The hourly partition will be soft drained 1 hour before the downtime.
  • The gpu and gpu-short partitions will be soft drained 1 hour before the downtime.

Finally, remaining running jobs will be killed by default when the downtime starts. In some specific rare cases jobs will be just paused and resumed back when the downtime finished.

Draining Policy Summary

The following table contains a summary of the draining policies during a Schedule Downtime:

Partition Drain Policy Default Drain Type Default Job Policy
general 12h before the SD soft drain Kill running jobs when SD starts
daily 12h before the SD soft drain Kill running jobs when SD starts
hourly 1h before the SD soft drain Kill running jobs when SD starts
gpu 1h before the SD soft drain Kill running jobs when SD starts
gpu-short 1h before the SD soft drain Kill running jobs when SD starts
gfa-asa 1h before the SD soft drain Kill running jobs when SD starts

Next Scheduled Downtimes

The table below shows a description for the next Scheduled Downtime:

From To Service Description
05.09.2020 8am 05.09.2020 6pm
  • Note: An e-mail will be sent when the services are fully available.