BronkasandClaude Opus 5.5
734dd7dbe1
The pgroup is now either given via --pgroup or guessed from the repository location with cristallina.utils.heuristic_extract_pgroup. All paths are derived from it in config.py and the pgroup is passed on to the SLURM jobs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.4 KiB
minibutler
Minibutler is looking for new SwissFEL data and submitting analysis jobs to SLURM. This is the little prototype brother of apocalypse, which will replace it later with merged functionality.
Usage
After installation (first clone this repository into the pgroup's /work, then pip install -e . --user in the project directory into the suitable python environment for analysis).
The pgroup is only defined in src/minibutler/config.py, all paths are derived from it there. By default it is guessed from the location of the repository (via cristallina.utils.heuristic_extract_pgroup), so there is nothing to change for a new pgroup. Other adaptations, e.g. SLURM reservations or the calculations themselves, go into calc_script.sh and calculators.py.
We can then start minibutler, which watches /sf/cristallina/data/<pgroup>/raw for new runs:
minibutler
or with an explicitly given pgroup and/or directory to watch:
minibutler --pgroup p19150 --watch-path /sf/cristallina/data/p19150/raw
As soon as the files of a new run appear on the filesystem the scheduled_jobs from the calculator.py module are submitted for execution.
Reprocessing
We can trigger the reprocessing with the current set of functions via a named pipe. For example to reprocess run 112 write into the pipe the run number, e.g. echo 112 > reprocess_pipe in the same directory as the script.