Jungfraujoch/docs/RUGNUX_OVERVIEW.md

# What Rugnux does

The map of a run, in the order it happens — one paragraph per stage, each linking into the
[data-analysis reference](CPU_DATA_ANALYSIS.md) where the algorithm lives. The walk-through is a
rotation run with the defaults; stills differences are at the end.

![The rotation pipeline: pass 1 integrates half the sweep at the file's geometry, pass 2 the whole sweep at the refined one, and the better of the two goes on to the space group and the written merge](images/rugnux_pipeline.svg)

**Open the dataset.** The geometry, wavelength and goniometer come from the file
([What Rugnux reads](RUGNUX_FORMATS.md)). A goniometer axis that turns makes it a rotation run; none, or one
that stands still from frame to frame, makes it serial stills — nothing is asked of the user.

**Pre-scan.** A projection of frames (60 by default) spread over the first part of the sweep, the part the geometry pre-pass integrates, finds the beam-stop shadow and masks
it ([§1.5](CPU_DATA_ANALYSIS_IMAGE.md)), masks pixels its frames show to be defective
([§1.6](CPU_DATA_ANALYSIS_IMAGE.md)), measures the beam centre from the isotropy of the
scattered background and compares it with the file's ([§1.4](CPU_DATA_ANALYSIS_IMAGE.md)), and
reads how wide this crystal's spots are, which sets the integration radius
([§9.5](CPU_DATA_ANALYSIS_INTEGRATION.md)), and how much longer they are along their radius than
across it, which is the beam's bandwidth where the file does not state one
([§9.6](CPU_DATA_ANALYSIS_INTEGRATION.md)), and how much wider still they grow away from the beam,
which sets the integration footprint ([§9.1](CPU_DATA_ANALYSIS_INTEGRATION.md)). In a GPU build the
pre-scan runs on the card.

**Spots.** Every image is decoded — on the GPU straight from the compressed chunk
([§0](CPU_DATA_ANALYSIS_IMAGE.md)) — and one fused pass computes the azimuthal profile and finds
the spots against each image's own per-resolution-ring noise
([§2–§3](CPU_DATA_ANALYSIS_IMAGE.md)). The ice-ring score is read off the same profile.

**Indexing.** The spots of a sample of frames are rotated back to a common crystal frame and the
FFT search looks for periodicity over thousands of directions; candidate cells are
Niggli-reduced, classified by Bravais lattice, refined both constrained and triclinic, and decided
on how many validation frames each actually indexes
([§4–§7](CPU_DATA_ANALYSIS_INDEXING.md)). A de-novo run also tries a second hypothesis with the
shortest accepted axis lowered from 10 to 5 Å, for small-molecule cells. The first pass is indexed a
second time at the beam centre the pre-scan measured; the file's centre is kept unless it indexes
nothing, gives an axis harmonic of the measured centre's lattice, or loses to it when both first passes
are merged ([§1.4](CPU_DATA_ANALYSIS_IMAGE.md)). A failed pass triggers the discrete rescues — the
rotation-axis sign, the first-pass depth ladder, the long-axis rescue — before anything is given up on.

**First integration pass.** At the geometry in the file, every frame of the first half of the sweep, at least 120° of it
(`--prepass-fraction`) is predicted
([§8](CPU_DATA_ANALYSIS_INTEGRATION.md)) and profile-fit integrated
([§9](CPU_DATA_ANALYSIS_INTEGRATION.md)); partials are combined into fulls, scaled and merged
([§10](CPU_DATA_ANALYSIS_INTEGRATION.md)).

**Geometry post-refinement.** From those reflections the detector distance, beam centre and the
cell scale / rotation axis are refined over all those frames at once, each step committed only if it
improves a held-out residual ([§7.5](CPU_DATA_ANALYSIS_INDEXING.md)).

**Second pass.** The sweep is re-indexed de novo and re-integrated at the refined geometry; this
pass is the canonical output, and a guard compares the two passes and keeps the better one
(reported as `PASS=` / `PASS_DECISION=` in [the report](RUGNUX_REPORT.md)).

**Space group.** On the P1 merge of the final pass, the point group is scored operator by operator
on resolution-normalised intensities and the screw axes, glide planes and centring are read from the
systematic absences, with the centre of symmetry from the intensity distribution where the absences
leave it open ([§13.1](CPU_DATA_ANALYSIS_DECISIONS.md)); twinning and translational
pseudo-symmetry are checked beside it ([§13.2](CPU_DATA_ANALYSIS_DECISIONS.md)). `CANNOT_DETERMINE` and an enantiomorphic pair are real
answers here, not evasions.

**Scale and merge.** In the determined group: per-frame scales, the cross-validated correction
surfaces (decay, absorption, modulation), the error model and ISa, outlier rejection, the
CC1/2-based resolution cut, the anisotropy description, French–Wilson amplitudes and the R-free
flags ([§10](CPU_DATA_ANALYSIS_INTEGRATION.md),
[§13.3–§13.5](CPU_DATA_ANALYSIS_DECISIONS.md)).

**Model: validation and molecular replacement.** Only with a model (`--model` or `--mr`, PDB or
mmCIF). With `--model` the model is placed as a rigid body and checked against a null; where that does
not place it, or the model is a prediction, it is placed by molecular replacement instead, and `--mr`
goes to molecular replacement straight away. The search places as many copies as the cell holds, in
every space group the merge cannot tell apart and in the twin hypothesis' subgroup; the placed model
then gives R-work/R-free and σA-weighted maps. The model never changes how the data are processed
([Model and reference](RUGNUX_MODEL_AND_REFERENCE.md),
[§14](CPU_DATA_ANALYSIS_DECISIONS.md)). It runs on every GPU of the machine, and on a Mac its
model density and rotation function on the Apple GPU ([GPU data analysis](GPU_DATA_ANALYSIS.md#molecular-replacement)).

**Write.** The merged `.mtz` / `.cif` / `.hkl`, the unmerged MTZ and the
results report land next to each other ([Output files](RUGNUX_TUTORIAL.md#output-files)); with a
model, the maps and the placed model are written too, and where molecular replacement chooses another
space group or the twin hypothesis' subgroup, that merge is written beside the answer as
`<prefix>_mr.mtz`/`.cif` with the model placed there, `<prefix>_mr_model.pdb`.

**Stills instead.** Serial data skip the two-pass machinery: each image is indexed independently
(with the known-cell `ffbidx` indexer where a cell is given), partiality comes from a per-crystal
orientation-tilt post-refinement rather than a rocking curve, and a merohedral indexing ambiguity
has to be broken per image, at integration time, against a reference or a model
([Advanced ▸ the indexing ambiguity](RUGNUX_ADVANCED.md#the-indexing-ambiguity)). The input may be
several files given together - Jungfraujoch masters, or the files an XFEL pipeline writes, read without
a geometry file ([Serial XFEL inputs](RUGNUX_FORMATS.md#serial-xfel-inputs)). Before the images are
processed the beam centre is decided on a sample of frames, the file's against the measured ones, and a
given unit cell also decides the detector distance there; the cell is a starting point the frames
refine, and only the frames of its crystal form are merged. With a cell, the geometry refinement places
each detector module as well ([§1.4](CPU_DATA_ANALYSIS_IMAGE.md#14-the-beam-centre),
[§7.7](CPU_DATA_ANALYSIS_INDEXING.md#77-stills-a-given-cell-and-the-detector-modules)). The report adds
R_split and a CC1/2 from frame half-sets, as CrystFEL computes them ([Report](RUGNUX_REPORT.md)).

![The stills pipeline: the beam centre (and, with a cell, the distance and the module placement) decided on the frames first, every image indexed on its own, and the crystals scaled and merged together](images/rugnux_pipeline_stills.svg)