Jungfraujoch/docs/RUGNUX.md

rugnux

rugnux is the offline crystallographic data-analysis tool of Jungfraujoch — the data-processing half of the system (see Naming for where the name comes from). It takes an existing HDF5 dataset, runs the full analysis pipeline — spot finding, indexing, geometry refinement, Bragg integration and (optionally) scaling and merging — and writes the results to a _process.h5 file, plus reflection files (.mtz/.cif/.hkl) when merging is requested.

It runs the same analysis code as the online and interactive tools, just driven from the command line over a file rather than a live detector stream.

Note. rugnux is under very active development. This page describes the tool and its options at a high level; the authoritative, always-current list of options is the program's own usage message — run rugnux with no arguments.

Where it fits among the three analysis tools

Tool	Mode	Driven by	Output
jfjoch_broker	Online, real-time streaming analysis on FPGA + GPU	HTTP/REST + ZeroMQ	Live results and statistics, images streamed to jfjoch_writer
jfjoch_viewer	Interactive, on-screen exploration	Qt desktop application	Displayed on screen (results not saved to disk)
rugnux	Offline batch processing of a stored dataset	Command-line interface	_process.h5, and .mtz/.cif/.hkl when merging

Use rugnux to re-analyse data after acquisition, to experiment with processing parameters, or to produce merged intensities for downstream structure solution.

Hardware

As with the rest of Jungfraujoch, serious performance requires an NVIDIA GPU. The CUDA build provides the GPU fast-feedback indexer (ffbidx) and the GPU FFT indexer (fft); without CUDA only the CPU fftw indexer is available. Spot finding, integration and scaling run on the CPU and scale with the thread count (-N).

Input and output

Input is a single Jungfraujoch HDF5 master file (NXmx-based). If the dataset already contains stored spot lists, two-pass rotation indexing can reuse them instead of re-running spot finding on the first pass.

Output (controlled by -o, --output-prefix, default output):

• <prefix>_process.h5 — NXmx-compliant HDF5 with derived metadata (spots, indexing, integration, azimuthal integration, per-image statistics). See HDF5 / NeXus data format for the layout. Written by default only when not merging (i.e. under --no-merge); add --write-process-h5 to also write it when merging.
• Merging is on by default (--no-merge disables it). The merged reflections are written as <prefix>.cif (mmCIF — the default), or <prefix>.mtz / <prefix>.hkl depending on --scaling-output. Both the mmCIF and the MTZ carry the refined unit cell (from rotation indexing) and the space group determined from systematic absences (constrained to the indexed lattice symmetry). No-reference scaling additionally emits per-iteration <prefix>_iterN_scale.dat.

Merged statistics (⟨I/σ⟩, CC1/2, completeness, …), the error model and timing are printed to the console. By default the written resolution is trimmed automatically where CC1/2 falls off (--resolution-cutoff cc-logistic, CC1/2 target 0.30); set --scaling-high-resolution to fix the limit by hand, or --resolution-cutoff off to keep the full range.

Re-scaling and re-merging (rugnux --scale)

The --scale mode re-scales and merges the already-integrated reflections stored in a _process.h5 file, without re-running spot finding or integration. Use it to re-merge quickly with a different space group, resolution limit, anomalous setting or reference MTZ. It reuses the same -o/-N/-s/-e/-S/-A/-B/-z/--scaling-* options as the full run, and (unlike the full pipeline) does not run a space-group search, so pass -S for the correct symmetry.

Quick start

Rotation data

Index, integrate, scale and merge a rotation sweep, fully de novo:

rugnux rotation_master.h5 \
    -o lyso_rot -N 32 \
    --scaling-high-resolution 1.4

Because the dataset carries a rotation goniometer axis, it is processed as rotation data by default: two-pass rotation indexing (index the sweep once, then process every frame against that lattice) with the rot3d partiality model (rotation partials combined into 3D fulls). Scaling and merging run by default (for both rotation and stills; --no-merge turns them off); the unit cell is taken from the rotation indexer and the space group is determined from systematic absences, and both are written into the merged .cif.

Run fully de novo (no -C/-S) for the best result — supplying a cell or space group up front tends to degrade low-symmetry cases. --scaling-high-resolution (set it to your expected resolution) sharpens both the space-group search and the error model. To tune the first pass use --two-pass-rotation=100 (or -R100 — the first-pass image count); to force the sweep to be treated as independent stills use --force-still.

After the per-frame scale-fulls step, rotation scaling applies two correction surfaces, on by default (--no-scaling-corrections disables both):

• Decay — a global Debye–Waller relative-B over the run, for the radiation damage that weakens later frames more at high resolution (a resolution×time systematic the resolution-flat per-frame scale cannot remove). It only engages when the total relative-B exceeds a physical floor (2 Ų).
• Absorption — a smooth multiplicative factor over the diffracted-beam direction in the goniometer frame (path length through the crystal). Negligible at hard X-rays / thin crystals; it matters at low photon energy. Its benefit shows up most on model-based metrics: a smooth absorption error largely cancels among symmetry mates (little effect on the error model / ISa) but still biases the intensities, so it measurably lowers R_free.

Both are cross-validated — fitted on even-numbered frames and kept only if they improve the held-out odd-frame symmetry-equivalent agreement by a clear margin (and vice versa) — so where the systematic is absent they are a no-op rather than a source of added noise; that is why they are safe to leave on. They run on the GPU when one is present, at negligible cost.

Still / serial data

A dataset with no goniometer axis (e.g. a serial grid scan) is processed as independent stills automatically — no flag needed. Known-cell indexing with the GPU fast-feedback indexer, then merge against a reference structure:

rugnux serial_master.h5 \
    -o lyso_serial -N 32 \
    -X ffbidx -C 79,79,38,90,90,90 -S 96 \
    --spot-sigma 4 \
    -z reference.mtz \
    --scaling-high-resolution 1.8

ffbidx requires a known cell (-C) and is the indexer of choice for sparse serial stills. For weak serial data, tightening spot finding with --spot-sigma 4 typically raises the indexing rate substantially. If a dataset does carry a goniometer axis but you want per-frame stills processing anyway, add --force-still.

Command-line options

General:

Option	Description
-o, --output-prefix <txt>	Output file prefix (default: output)
-N, --threads <num>	Number of worker threads (default: all hardware threads)
-s, --start-image <num>	First image to process (default: 0)
-e, --end-image <num>	Last image to process (default: all)
-t, --stride <num>	Process every n-th image (default: 1)
-v, --verbose	Verbose output

Modes (default: full analysis — spot finding, indexing, integration and merging):

Option	Description
--azint-only	Only run azimuthal integration (no spot finding/indexing); writes <prefix>_process.h5
--scale	Only re-scale/merge the already-integrated reflections in the input _process.h5 (no re-integration)

Spot finding:

Option	Description
--spot-sigma <num>	Noise sigma level for spot finding (default: 3.0)
--spot-threshold <num>	Photon-count threshold for spot finding (default: 10)
--spot-high-resolution <num>	High-resolution limit for spot finding, Å (default: 1.5)
--max-spots <num>	Maximum spot count (default: 250)
--detect-ice-rings[=on|off]	Flag ice-ring spots (de-prioritised in indexing) and exclude ice-ring reflections from scaling/merging; overrides the dataset/master-file setting (default: use the dataset value)

Azimuthal integration (the radial profile behind the per-image ice-ring score):

Option	Description
-q, --azim-q-spacing <num>	Q bin spacing, 1/Å (default: 0.01; finer resolves the narrow ice rings)
--azim-min-q <num>	Minimum Q, 1/Å
--azim-max-q <num>	Maximum Q, 1/Å
--azim-phi-bins <num>	Number of azimuthal (phi) bins (default: 1)
--polarization-correction <on|off>	Enable/disable the azimuthal polarization correction
--solid-angle-correction <on|off>	Enable/disable the azimuthal solid-angle correction

Indexing:

A dataset with a rotation goniometer axis is processed as rotation data (two-pass rotation indexing) by default; a dataset without one is processed as independent stills. --force-still overrides the former; the -R / --single-pass-rotation / --force-rotation-lattice flags request rotation explicitly and pick the pass or lattice.

Option	Description
--force-still	Treat a rotation (goniometer) dataset as independent stills instead of rotation
-X, --indexing-algorithm <txt>	FFBIDX | FFT | FFTW | Auto | None
-C, --unit-cell <cell>	Reference unit cell "a,b,c,alpha,beta,gamma" (required by ffbidx)
-S, --space-group <num>	Space group number (used for indexing and scaling)
-r, --refine <txt>	Geometry refinement: none | orientation | beam_and_lattice (default)
-R, --two-pass-rotation[=num]	Two-pass offline rotation indexing (default for goniometer data; optional first-pass image count, default 100)
--single-pass-rotation[=num]	Online-like single-pass rotation indexing (optional min angular range, deg)
--redo-rotation-spots	Redo spot finding for the two-pass rotation first pass
--force-rotation-lattice <vec>	Force rotation lattice (9 floats, Å), skipping the first pass

Indexer choice in brief: ffbidx (GPU) refines toward a known cell and is best for sparse serial stills; fft (GPU) / fftw (CPU) index de novo and suit strong rotation data. See the CPU/GPU data-analysis reference for the algorithms.

Scaling and merging:

Option	Description
--no-merge	Skip scaling and merging (on by default); write only the per-image _process.h5
-A, --anomalous	Anomalous mode (keep Friedel pairs separate)
-B, --refine-bfactor	Refine a per-image B-factor (stills only)
--scale-fulls / --no-scale-fulls	rot3d: refit a per-frame scale on the combined fulls (XDS order, Unity model); on by default for rotation data, off for stills
--smooth-g[=deg]	rot3d: smooth the per-frame scale G over a degree range before the 3D combine (XDS DELPHI-like; default 5° for rotation, 0 = off)
--no-scaling-corrections	rot3d: disable the default-on decay + absorption correction surfaces fitted on the fulls after scale-fulls (see below)
--capture-uncertainty <num>	rot3d: systematic sigma on under-captured fulls, ~num·(1−captured_fraction)·I (default: 1.0 for rotation, 0 otherwise)
--min-captured-fraction <num>	rot3d: drop a combined full whose rocking curve was captured below this fraction — edge-of-sweep truncated fulls (default: 0.7 for rotation, 0 otherwise; 0 = off)
--scaling-high-resolution <num>	High-resolution limit for scaling, Å — manual override (default: no limit; disables the automatic cutoff below)
--resolution-cutoff <txt>	Automatic high-resolution cutoff for the written reflections and reported shells: cc-logistic | off (default: cc-logistic; ignored when --scaling-high-resolution is set)
--resolution-cc-target <num>	CC1/2 target defining the cc-logistic fall-off (default: 0.30)
--resolution-shells <num>	Number of resolution shells in the reported statistics table (default: 10)
--min-partiality <num>	Minimum partiality to accept a reflection (default: 0.02)
--reject-outliers <num>	Per-observation outlier rejection, N σ from the per-reflection median (default: 6 for rot3d, off otherwise)
--reject-delta-cchalf <num>	Drop images with ΔCC1/2 below mean − N·stddev (default: off)
--min-image-cc <num>	Per-image CC limit, percent (default: no limit)
--mosaicity <num>	Diagnostic: fix the scaling mosaicity (°) instead of using the per-image seed
--scaling-iterations <num>	Scaling iterations with no reference data (default: 3)
--scaling-output <txt>	Reflection output format: cif (mmCIF, default) | mtz | txt
-z, --reference-mtz <file>	Reference MTZ (enables reference-driven scaling)
--reference-column <label>	Reference MTZ column to use (default: auto — F-model, else IMEAN/I/…)
--write-process-h5	Also write the (large) _process.h5 when merging (default: only .mtz/.cif)

Integration:

Option	Description
--integrator <txt>	Spot integrator: gaussian (profile-fit, default) | empirical | boxsum (classical fallback)
--integration-radius <r>	Signal-box radius r1, or r1,r2,r3 (px). One value ⇒ r2=r1+2, r3=r1+4
--bandwidth <num>	Relative X-ray bandwidth FWHM (e.g. 0.01 for a 1% DMM); default from file or 0 (monochromatic)

Geometry overrides (defaults are taken from the input file; override them to reprocess with a corrected geometry):

Option	Description
--beam-x <num>	Beam centre X (pixel)
--beam-y <num>	Beam centre Y (pixel)
--detector-distance <num>	Detector distance (mm)
--wavelength <num>	Wavelength (Å)
--rot1 <num>	PONI detector rotation 1 (rad)
--rot2 <num>	PONI detector rotation 2 (rad)
--polarization <num>	Polarization factor