Minibutler is looking for new SwissFEL data and submitting analysis jobs to SLURM. This is the little prototype brother of apocalypse, which will replace it later with merged functionality.

Usage

After installation (first clone this repository into the pgroup's /work, then pip install -e . --user in the project directory into the suitable python environment for analysis).

In src we have to adapt calc_script.sh, calc_job.py and calculators.py to the current pgroup, e.g. include special paths, SLURM reservations, etc.

We can then start minibutler with a given directory to watch for new runs:

minibutler --watch-path /sf/cristallina/data/p19150/raw

As soon as the files of a new run appear on the filesystem the scheduled_jobs from the calculator.py module are submitted for execution.

Reprocessing

We can trigger the reprocessing with the current set of functions via a named pipe. For example to reprocess run 112 write into the pipe the run number, e.g. echo 112 > reprocess_pipe in the same directory as the script.