Minibutler is looking for new SwissFEL data and submitting analysis jobs to SLURM. This is the little prototype brother of apocalypse, which will replace it later with merged functionality.

Usage

After installation (first clone this repository into the pgroup's /work, then pip install -e . --user in the project directory into the suitable python environment for analysis).

The pgroup is only defined in src/minibutler/config.py, all paths are derived from it there. By default it is guessed from the location of the repository (via cristallina.utils.heuristic_extract_pgroup), so there is nothing to change for a new pgroup. Other adaptations, e.g. SLURM reservations or the calculations themselves, go into calc_script.sh and calculators.py.

We can then start minibutler, which watches /sf/cristallina/data/<pgroup>/raw for new runs:

minibutler

or with an explicitly given pgroup and/or directory to watch:

minibutler --pgroup p19150 --watch-path /sf/cristallina/data/p19150/raw

As soon as the files of a new run appear on the filesystem the scheduled_jobs from the calculator.py module are submitted for execution.

Reprocessing

We can trigger the reprocessing with the current set of functions via a named pipe. For example to reprocess run 112 write into the pipe the run number, e.g. echo 112 > reprocess_pipe in the same directory as the script.