Jungfraujoch/docs/RUGNUX.md

# rugnux

`rugnux` is the **offline** crystallographic data-analysis tool of Jungfraujoch — the
data-processing half of the system (see [Naming](NAMING.md) for where the name comes from).
It takes an existing HDF5 dataset, runs the full analysis pipeline — spot finding, indexing,
geometry refinement, Bragg integration and (optionally) scaling and merging — and writes the
results to a `_process.h5` file, plus reflection files (`.mtz`/`.cif`/`.hkl`) when merging is
requested.

It runs the *same* analysis code as the online and interactive tools, just driven from the
command line over a file rather than a live detector stream.

```
rugnux {<options>} <input.h5>
```

Run it with no arguments to print the usage.

> **Note.** `rugnux` is under very active development. This page describes the tool and
> its options at a high level; the authoritative, always-current list of options is the program's
> own usage message — run `rugnux` with no arguments.

```{contents} On this page
:local:
:depth: 2
```

## Quick start

Four commands cover most of what people ask of `rugnux`. Each takes the **master** file of the
dataset — one written by Jungfraujoch, a DECTRIS EIGER master, or an NXmx master written by another
facility's toolchain; a PILATUS miniCBF sweep works too (see [What rugnux reads](RUGNUX_FORMATS.md))
— and names its output files from `-o`:

```
# 1. everything from the data - index, integrate, scale and merge with the defaults
rugnux -o myrun dataset_master.h5

# 2. with a reference dataset of the same crystal form: it fixes the space group and the cell,
#    resolves the indexing ambiguity, and hands over its R-free set
rugnux -o myrun -z reference.mtz dataset_master.h5

# 3. with a known structure: R-work / R-free and sigma_A-weighted 2mFo-DFc / mFo-DFc maps
rugnux -o myrun --model model.pdb dataset_master.h5

# 4. with the space group and the cell pinned (-S takes either spelling: P43212 or 96)
rugnux -o myrun -S P43212 -C 79,79,38,90,90,90 dataset_master.h5
```

Parallelism needs no asking for: a run already uses the machine's threads. `-N` is there to *limit*
that, or to lift the per-image loop's default ceiling of 16 workers per GPU.

Nothing more is needed to pick the workflow: a dataset carrying a **goniometer axis** is processed
as a rotation sweep, one without as **independent stills**, and scaling and merging run by default
in both. A rotation run that merges — the default — leaves seven files next to each other:

```
myrun.mtz            merged intensities + French-Wilson amplitudes, for CCP4 / phenix
myrun.cif            the same, as mmCIF - the self-describing format, and what to deposit
myrun.hkl            the same, as SHELX HKLF 4 - feed this to SHELXC / SHELXD / ANODE
myrun_unmerged.mtz   every observation before scaling, for pointless / aimless / careless -
                     the largest file of the run (--no-export-unmerged skips it)
myrun_P1.mtz         the same observations merged in P1, so a wrong space-group call can be
                     re-merged or re-refined without reprocessing (--no-p1-crosscheck skips it)
myrun_report.txt     what the run determined: cell, space group, statistics, warnings
myrun_image.dat      one row per image, for plotting how the crystal behaved over the sweep
```

The two MTZ extras are most of the bytes a run writes — worth knowing when sizing a scratch
directory for a campaign, and both have off switches.

Read `myrun_report.txt` first: it says which space group was chosen and on what evidence, how far
the data go, and anything that needs attention.

A few things worth knowing before reaching for more flags:

- **The written reflections stop where CC½ falls through 0.30.** Every reflection file is
  resolution-trimmed automatically (`--resolution-cutoff cc-logistic`, one shell past the crossing);
  `--resolution-cutoff off` keeps the full measured range, `--scaling-high-resolution` fixes the
  limit by hand. A rugnux file reaching less far than another program's on the same data is usually
  this default at work, not lost data.
- **Rotation data are best left de novo.** Pinning the cell and space group (recipe 4) is the
  normal thing to do for **serial stills**, where the `ffbidx` indexer needs a cell; on a rotation
  sweep it tends to *degrade* low-symmetry cases, so prefer recipe 1 and let the run determine both
  (see [Rotation data](RUGNUX_TUTORIAL.md#rotation-data)). `-S` takes a Hermann-Mauguin symbol (`P43212`) or a
  space-group number (`96`), whichever is to hand.
- **Anomalous data are there without `-A`.** A rotation merge always keeps the Bijvoet split: a
  default run's `.mtz` carries `I(+)`/`I(-)` beside `IMEAN`, and its `.hkl` the ±hkl mates —
  `FRIEDELS_LAW= TRUE` in the report says how the *statistics* were counted, not that the signal was
  averaged away. What `-A` changes is the counting basis and the error model: each hand becomes a
  merged observation of its own, so multiplicity, completeness and ⟨I/σ⟩ are counted anomalously and
  the sigmas are refitted on the Bijvoet-separated merge. Reach for it for anomalous statistics; the
  signal itself is in the file either way. (Stills merges carry no split by default — there `-A` is
  what creates one.)
- **A model names the enantiomorph.** Where the data accept the model — it is tested against a
  null of the same model in random orientations, and `MODEL_FIT=` in the report says the verdict —
  `--model` settles which of P4<sub>1</sub>2<sub>1</sub>2 and P4<sub>3</sub>2<sub>1</sub>2 the merged
  reflections are *labelled* with — a choice no merged intensity can make. It is a label and nothing
  more: the two groups have the same rotation operations, so no reflection moves, and in particular
  I(+) and I(-) are left exactly as measured. Whether the model agrees with the data about the hand
  is then a real question, and the anomalous difference map answers it — a run says so when the
  density at the model's atoms comes out inverted.
- **`-z` and `--model` overlap but are not the same.** A reference MTZ steers the processing from
  the start; a model scores the merge and settles the frame it is written in — where the data
  accept it; a model they reject changes nothing. Either resolves an
  [indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity), which on serial data decides whether the merged
  intensities are usable at all.
- **`--scaling-high-resolution <d>`**, where the resolution is already known, sharpens both the
  space-group search and the error model.
- Everything else is in [Running rugnux](RUGNUX_TUTORIAL.md#running-rugnux) and the full
  [Command-line options](RUGNUX_ADVANCED.md#command-line-options).


## The rest of the manual

One page per job, so the answer needed is near the top of a short page:

- [What a rugnux run does](RUGNUX_OVERVIEW.md) — the pipeline from images to merged reflections,
  in order. Read this one first.
- [Installing rugnux](RUGNUX_INSTALL.md) — packages, the release archive, GPU drivers, building
  from source, hardware.
- [What rugnux reads](RUGNUX_FORMATS.md) — will it open your data: NXmx / EIGER masters, PILATUS
  miniCBF, one sweep per input.
- [Running rugnux](RUGNUX_TUTORIAL.md) — a first run in detail, rotation and serial data, and every
  file a run writes.
- [rugnux with other programs](RUGNUX_INTEGRATION.md) — the reflection-file conventions, the
  unmerged export, and worked command lines for phenix, REFMAC, POINTLESS / AIMLESS, careless and
  Phaser.
- [The results report](RUGNUX_REPORT.md) — the `KEY= value` interface, sweep quality and the
  anisotropy section.
- [Advanced usage](RUGNUX_ADVANCED.md) — reference data and the indexing ambiguity, model
  validation, re-merging, and the full command-line option tables.
- [Detector calibration](RUGNUX_CALIBRATION.md) — the geometry from a calibrant's powder rings
  (`--mode calibration`).
- [CPU/GPU data analysis](CPU_DATA_ANALYSIS.md) — the algorithms behind all of it.

## Where it fits among the three analysis tools

| Tool | Mode | Driven by | Output |
| --- | --- | --- | --- |
| [`jfjoch_broker`](JFJOCH_BROKER.md) | Online, real-time streaming analysis on FPGA + GPU | HTTP/REST + ZeroMQ | Live results and statistics, images streamed to [`jfjoch_writer`](JFJOCH_WRITER.md) |
| [`jfjoch_viewer`](JFJOCH_VIEWER.md) | Interactive, on-screen exploration | Qt desktop application | On screen; a processing job can write the same files as `rugnux` |
| **`rugnux`** | **Offline batch processing of a stored dataset** | **Command-line interface** | **`_process.h5`, and `.mtz`/`.cif`/`.hkl` when merging** |

Use `rugnux` to re-analyse data after acquisition, to experiment with processing
parameters, or to produce merged intensities for downstream structure solution.