Jungfraujoch/docs/RUGNUX.md

# rugnux

`rugnux` is the **offline** crystallographic data-analysis tool of Jungfraujoch — the
data-processing half of the system (see [Naming](NAMING.md) for where the name comes from).
It takes an existing HDF5 dataset, runs the full analysis pipeline — spot finding, indexing,
geometry refinement, Bragg integration and (optionally) scaling and merging — and writes the
results to a `_process.h5` file, plus reflection files (`.mtz`/`.cif`/`.hkl`) when merging is
requested.

It runs the *same* analysis code as the online and interactive tools, just driven from the
command line over a file rather than a live detector stream.

> **Note.** `rugnux` is under very active development. This page describes the tool and
> its options at a high level; the authoritative, always-current list of options is the program's
> own usage message — run `rugnux` with no arguments.

## Where it fits among the three analysis tools

| Tool | Mode | Driven by | Output |
| --- | --- | --- | --- |
| [`jfjoch_broker`](JFJOCH_BROKER.md) | Online, real-time streaming analysis on FPGA + GPU | HTTP/REST + ZeroMQ | Live results and statistics, images streamed to [`jfjoch_writer`](JFJOCH_WRITER.md) |
| [`jfjoch_viewer`](JFJOCH_VIEWER.md) | Interactive, on-screen exploration | Qt desktop application | Displayed on screen (results not saved to disk) |
| **`rugnux`** | **Offline batch processing of a stored dataset** | **Command-line interface** | **`_process.h5`, and `.mtz`/`.cif`/`.hkl` when merging** |

Use `rugnux` to re-analyse data after acquisition, to experiment with processing
parameters, or to produce merged intensities for downstream structure solution.

## Hardware

As with the rest of Jungfraujoch, **serious performance requires an NVIDIA GPU**. The CUDA build
provides the GPU fast-feedback indexer (`ffbidx`) and the GPU FFT indexer (`fft`); without CUDA
only the CPU `fftw` indexer is available. Spot finding, integration and scaling run on the CPU and
scale with the thread count (`-N`).

## Input and output

**Input** is a single Jungfraujoch HDF5 master file (NXmx-based). If the dataset already contains
stored spot lists, two-pass rotation indexing can reuse them instead of re-running spot finding on
the first pass.

**Output** (controlled by `-o, --output-prefix`, default `output`):

- `<prefix>_process.h5` — NXmx-compliant HDF5 with derived metadata (spots, indexing,
  integration, azimuthal integration, per-image statistics). See
  [HDF5 / NeXus data format](HDF5.md) for the layout. Written by default only when **not** merging
  (i.e. under `--no-merge`); add `--write-process-h5` to also write it when merging.
- Merging is **on by default** (`--no-merge` disables it). The merged reflections are written as
  `<prefix>.cif` (mmCIF — the default), or `<prefix>.mtz` / `<prefix>.hkl` depending on
  `--scaling-output`. Both the mmCIF and the MTZ carry the **refined unit cell** (from rotation
  indexing) and the **space group determined from systematic absences** (constrained to the indexed
  lattice symmetry). No-reference scaling additionally emits per-iteration `<prefix>_iterN_scale.dat`.

Merged statistics (⟨I/σ⟩, CC1/2, completeness, …), the error model and timing are printed to the
console.

## Re-scaling and re-merging (`rugnux --scale`)

The `--scale` mode re-scales and merges the *already-integrated* reflections stored in a
`_process.h5` file, without re-running spot finding or integration. Use it to re-merge quickly with a
different space group, resolution limit, anomalous setting or reference MTZ. It reuses the same
`-o/-N/-s/-e/-S/-A/-B/-z/--scaling-*` options as the full run, and (unlike the full pipeline) does
not run a space-group search, so pass `-S` for the correct symmetry.

## Quick start

### Rotation data

Index, integrate, scale and merge a rotation sweep, fully de novo:

```
rugnux rotation_master.h5 \
    -o lyso_rot -N 32 \
    --scaling-high-resolution 1.4
```

Because the dataset carries a rotation goniometer axis, it is processed as **rotation data by
default**: two-pass rotation indexing (index the sweep once, then process every frame against that
lattice) with the **`rot3d`** partiality model (rotation partials combined into 3D fulls). Scaling
and merging run **by default** (for both rotation and stills; `--no-merge` turns them off); the unit
cell is taken from the rotation indexer and the space group is determined from systematic absences,
and both are written
into the merged `.cif`.

Run **fully de novo** (no `-C`/`-S`) for the best result — supplying a cell or space group up front
tends to *degrade* low-symmetry cases. `--scaling-high-resolution` (set it to your expected
resolution) sharpens both the space-group search and the error model. To tune the first pass use
`--two-pass-rotation=100` (or `-R100` — the first-pass image count); to force the sweep to be
treated as independent stills use `--force-still`.

### Still / serial data

A dataset with **no goniometer axis** (e.g. a serial grid scan) is processed as **independent
stills automatically** — no flag needed. Known-cell indexing with the GPU fast-feedback indexer,
then merge against a reference structure:

```
rugnux serial_master.h5 \
    -o lyso_serial -N 32 \
    -X ffbidx -C 79,79,38,90,90,90 -S 96 \
    --spot-sigma 4 \
    -z reference.mtz \
    --scaling-high-resolution 1.8
```

`ffbidx` requires a known cell (`-C`) and is the indexer of choice for sparse serial stills. For
weak serial data, tightening spot finding with `--spot-sigma 4` typically raises the indexing rate
substantially. If a dataset *does* carry a goniometer axis but you want per-frame stills processing
anyway, add `--force-still`.

## Command-line options

General:

| Option | Description |
| --- | --- |
| `-o, --output-prefix <txt>` | Output file prefix (default: `output`) |
| `-N, --threads <num>` | Number of worker threads (default: 1) |
| `-s, --start-image <num>` | First image to process (default: 0) |
| `-e, --end-image <num>` | Last image to process (default: all) |
| `-t, --stride <num>` | Process every *n*-th image (default: 1) |
| `-v, --verbose` | Verbose output |

Modes (default: full analysis — spot finding, indexing, integration and merging):

| Option | Description |
| --- | --- |
| `--azint-only` | Only run azimuthal integration (no spot finding/indexing); writes `<prefix>_process.h5` |
| `--scale` | Only re-scale/merge the already-integrated reflections in the input `_process.h5` (no re-integration) |

Spot finding:

| Option | Description |
| --- | --- |
| `--spot-sigma <num>` | Noise sigma level for spot finding (default: 3.0) |
| `--spot-threshold <num>` | Photon-count threshold for spot finding (default: 10) |
| `--spot-high-resolution <num>` | High-resolution limit for spot finding, Å (default: 1.5) |
| `--max-spots <num>` | Maximum spot count (default: 250) |
| `--detect-ice-rings[=on\|off]` | Flag ice-ring spots and exclude ice-ring reflections from scaling/merging; overrides the dataset setting (default: use the dataset value) |

Azimuthal integration (the radial profile behind the per-image ice-ring score):

| Option | Description |
| --- | --- |
| `-q, --azim-q-spacing <num>` | Q bin spacing, 1/Å (default: 0.01; finer resolves the narrow ice rings) |
| `--azim-min-q <num>` | Minimum Q, 1/Å |
| `--azim-max-q <num>` | Maximum Q, 1/Å |
| `--azim-phi-bins <num>` | Number of azimuthal (phi) bins (default: 1) |

Indexing:

A dataset with a **rotation goniometer axis** is processed as rotation data (two-pass rotation
indexing) by default; a dataset without one is processed as independent stills. `--force-still`
overrides the former; the `-R` / `--single-pass-rotation` / `--force-rotation-lattice` flags request
rotation explicitly and pick the pass or lattice.

| Option | Description |
| --- | --- |
| `--force-still` | Treat a rotation (goniometer) dataset as independent stills instead of rotation |
| `-X, --indexing-algorithm <txt>` | `FFBIDX` \| `FFT` \| `FFTW` \| `Auto` \| `None` |
| `-C, --unit-cell <cell>` | Reference unit cell `"a,b,c,alpha,beta,gamma"` (required by `ffbidx`) |
| `-S, --space-group <num>` | Space group number (used for indexing and scaling) |
| `-r, --refine <txt>` | Geometry refinement: `none` \| `orientation` \| `beam_and_lattice` (default) |
| `-R, --two-pass-rotation[=num]` | Two-pass offline rotation indexing (default for goniometer data; optional first-pass image count, default 100) |
| `--single-pass-rotation[=num]` | Online-like single-pass rotation indexing (optional min angular range, deg) |
| `--redo-rotation-spots` | Redo spot finding for the two-pass rotation first pass |
| `--force-rotation-lattice <vec>` | Force rotation lattice (9 floats, Å), skipping the first pass |

Indexer choice in brief: `ffbidx` (GPU) refines toward a **known cell** and is best for sparse
serial stills; `fft` (GPU) / `fftw` (CPU) index **de novo** and suit strong rotation data. See the
[CPU/GPU data-analysis reference](CPU_DATA_ANALYSIS.md) for the algorithms.

Scaling and merging:

| Option | Description |
| --- | --- |
| `--no-merge` | Skip scaling and merging (on by default); write only the per-image `_process.h5` |
| `-A, --anomalous` | Anomalous mode (keep Friedel pairs separate) |
| `-B, --refine-bfactor` | Refine a per-image B-factor |
| `--scale-fulls` / `--no-scale-fulls` | rot3d: refit a per-frame scale on the combined fulls (XDS order, Unity model); on by default for rotation data, off for stills |
| `--smooth-g[=deg]` | rot3d: smooth the per-frame scale *G* over a degree range before the 3D combine (XDS DELPHI-like; default 5° for rotation, 0 = off) |
| `--capture-uncertainty <num>` | rot3d: systematic sigma on under-captured fulls, ~num·(1−captured_fraction)·I (default: 1.0 for rotation, 0 otherwise) |
| `--scaling-high-resolution <num>` | High-resolution limit for scaling, Å (default: no limit) |
| `--min-partiality <num>` | Minimum partiality to accept a reflection (default: 0.02) |
| `--reject-outliers <num>` | Per-observation outlier rejection, N σ from the per-reflection median (default: 6 for `rot3d`, off otherwise) |
| `--reject-delta-cchalf <num>` | Drop images with ΔCC1/2 below mean − N·stddev (default: off) |
| `--min-image-cc <num>` | Per-image CC limit, percent (default: no limit) |
| `--scaling-iterations <num>` | Scaling iterations with no reference data (default: 3) |
| `--scaling-output <txt>` | Reflection output format: `cif` (mmCIF, default) \| `mtz` \| `txt` |
| `-z, --reference-mtz <file>` | Reference MTZ (enables reference-driven scaling) |
| `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |

Integration:

| Option | Description |
| --- | --- |
| `--integrator <txt>` | Spot integrator: `gaussian` (profile-fit, default) \| `empirical` \| `boxsum` (classical fallback) |
| `--integration-radius <r>` | Signal-box radius `r1`, or `r1,r2,r3` (px). One value ⇒ `r2=r1+2`, `r3=r1+4` |
| `--bandwidth <num>` | Relative X-ray bandwidth FWHM (e.g. `0.01` for a 1% DMM); default from file or 0 (monochromatic) |