Files
Jungfraujoch/docs/RUGNUX_OVERVIEW.md
leonarski_fandClaude Opus 5.5 9b4acd6ab4 RESULT-CHANGING: FFT indexing finds rows shorter than its floor; the short-axis pass is retired
The per-direction peak search starts at the 10 A floor, so a lattice row shorter than that was
reported at a multiple of itself past it and the cell built from such rows was an exact integer
supercell. A second first-pass hypothesis at a 5 A floor (the short-axis pass) won the true cell
back on frame counts, and could not reach an axis below 5 A at all.

On rotation data (a scanning axis) every shortlisted row reported within three times the floor is
now asked, by a direct sum over the accumulated spots, at each whole fraction of its length down
to 3 A; the shortest fraction that keeps half the row's amplitude replaces it. The cells of those
rows are reduced (widened: a small cell's in-plane combinations fill the standard window) and
refined beside the standard cells, and ranked with them, so where a cell and its supercell index
alike the smaller one is taken. Protein shortlists have no rows that short, so nothing changes for
them; stills and grid scans keep the search they had. The short-axis pass, its speculative pool and
its live warm-up are removed; the refined-length bound of the cell fit goes from 5 to 2.5 A.

The 40 % overlap test of the candidate refinement no longer deletes a cell for its overlap with an
integer multiple or sub-cell of it - one lattice contains the other, so that overlap is total by
construction. Such candidates are kept after every cell the test accepts, which keep their order,
so the first lattice and the extra crystals are what they were. The rotation indexer carries the
refined integer supercells of the cell it chooses, and the twin-composite test asks them as well as
the standing cell: once the search offers a twin domain's own cell, the coincidence lattice it used
to stand on is one of those.

Measured (battery subsets vs rc181-b2): a small-molecule crystal with an axis under 5 A, indexed as an
index-5 supercell now gives its true cell and group; the 13 sets the short-axis pass rescued keep
their verdicts; the sparse reticular twin keeps its domain and overlap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdL4qBGYRiSXSGXfCLu
2026-11-03 12:08:44 +01:00

8.9 KiB

What Rugnux does

The map of a run, in the order it happens — one paragraph per stage, each linking into the data-analysis reference where the algorithm lives. The walk-through is a rotation run with the defaults; stills differences are at the end.

The rotation pipeline: pass 1 integrates half the sweep at the file's geometry, pass 2 the whole sweep at the refined one, and the better of the two goes on to the space group and the written merge; with a model, the placement and the refinement follow, deciding nothing

Open the dataset. The geometry, wavelength and goniometer come from the file (What Rugnux reads). A goniometer axis that turns makes it a rotation run; none, or one that stands still from frame to frame, makes it serial stills — nothing is asked of the user.

Pre-scan. A projection of frames (60 by default) spread over the first part of the sweep, the part the geometry pre-pass integrates, finds the beam-stop shadow and masks it (§1.5), masks pixels its frames show to be defective (§1.6), measures the beam centre from the isotropy of the scattered background and compares it with the file's (§1.4), and reads how wide this crystal's spots are, which sets the integration radius (§9.5), and how much longer they are along their radius than across it, which is the beam's bandwidth where the file does not state one (§9.6), and how much wider still they grow away from the beam, which sets the integration footprint (§9.1). In a GPU build the pre-scan runs on the card.

Spots. Every image is decoded — on the GPU straight from the compressed chunk (§0) — and one fused pass computes the azimuthal profile and finds the spots against each image's own per-resolution-ring noise (§2–§3). The ice-ring score is read off the same profile.

Indexing. The spots of a sample of frames are rotated back to a common crystal frame and the FFT search looks for periodicity over thousands of directions; candidate cells are Niggli-reduced, classified by Bravais lattice, refined both constrained and triclinic, and decided on how many validation frames each actually indexes (§4–§7). A lattice row shorter than the search's 10 Å floor, as small-molecule cells have, is recognised under the multiple of it the search reports, down to 3 Å. The first pass is indexed a second time at the beam centre the pre-scan measured; the file's centre is kept unless it indexes nothing, gives an axis harmonic of the measured centre's lattice, or loses to it when both first passes are merged (§1.4). A failed pass triggers the discrete rescues — the rotation-axis sign, the first-pass depth ladder, the long-axis hypothesis — before anything is given up on.

First integration pass. At the geometry in the file, every frame of the first half of the sweep, at least 120° of it (--prepass-fraction) is predicted (§8) and profile-fit integrated (§9); partials are combined into fulls, scaled and merged (§10).

Geometry post-refinement. From those reflections the detector distance, beam centre and the cell scale / rotation axis are refined over all those frames at once, each step committed only if it improves a held-out residual (§7.5).

Second pass. The sweep is re-indexed de novo and re-integrated at the refined geometry; this pass is the canonical output, and a guard compares the two passes and keeps the better one (reported as PASS= / PASS_DECISION= in the report).

Space group. On the P1 merge of the final pass, the point group is scored operator by operator on resolution-normalised intensities and the screw axes, glide planes and centring are read from the systematic absences, with the centre of symmetry from the intensity distribution where the absences leave it open (§13.1); twinning and translational pseudo-symmetry are checked beside it (§13.2). CANNOT_DETERMINE and an enantiomorphic pair are real answers here, not evasions.

Scale and merge. In the determined group: per-frame scales, the cross-validated correction surfaces (decay, absorption, modulation), the error model and ISa, outlier rejection, the CC1/2-based resolution cut, the anisotropy description, French–Wilson amplitudes and the R-free flags (§10, §13.3–§13.5).

Model: validation and molecular replacement. Only with a model (--model or --mr, PDB or mmCIF). With --model the model is placed as a rigid body and checked against a null; where that does not place it, or the model is a prediction, it is placed by molecular replacement instead, and --mr goes to molecular replacement straight away. The search places as many copies as the cell holds, in every space group the merge cannot tell apart and in the twin hypothesis' subgroup; the placed model then gives R-work/R-free and σA-weighted maps. The model never changes how the data are processed (Model and reference, §14). It runs on every GPU of the machine, and on a Mac its model density and rotation function on the Apple GPU (GPU data analysis).

Model: refinement. A placed model is then refined against the merge it was placed on (--no-refine skips it): its chains as rigid bodies and B per residue where that lowers R-free, then every atom's position and B by a restrained maximum-likelihood refinement whose weights are chosen by R-free in short trials run side by side; the refined model gives its own maps and the blobs of unexplained density. Like the validation it decides nothing - the merge, R_WORK/R_FREE and the placed model's files stay as they were, and the refined results are written beside them under REFINE_ (Model and reference, §15). On a GPU it takes seconds, with the weight trials spread over every card (GPU data analysis).

Write. The merged .mtz / .cif / .hkl, the unmerged MTZ and the results report land next to each other (Output files); with a model, the maps and the placed model are written too, and beside them the refined model, its maps and its blobs; where molecular replacement chooses another space group or the twin hypothesis' subgroup, that merge is written beside the answer as <prefix>_mr.mtz/.cif with the model placed there, <prefix>_mr_model.pdb.

Stills instead. Serial data skip the two-pass machinery: each image is indexed independently (with the known-cell ffbidx indexer where a cell is given), partiality comes from a per-crystal orientation-tilt post-refinement rather than a rocking curve, and where the lattice admits a merohedral indexing ambiguity the crystals are put into one indexing before the first merge: from the data alone, each crystal into the indexing in which it agrees best with all the others, where the space group is given (-S); or per image against a reference or a model, which then decides the indexing the merge is written in. Either way a crystal whose own refined cell already tells the indexings apart keeps its indexing (Advanced ▸ the indexing ambiguity). The input may be several files given together - Jungfraujoch masters, or the files an XFEL pipeline writes, read without a geometry file (Serial XFEL inputs). Before the images are processed the beam centre is decided on a sample of frames, the file's against the measured ones, and a given unit cell also decides the detector distance there; the cell is a starting point the frames refine, and only the frames of its crystal form are merged. With a cell, the geometry refinement places each detector module as well (§1.4, §7.7). The report adds R_split and a CC1/2 from frame half-sets, as CrystFEL computes them (Report).

The stills pipeline: the beam centre (and, with a cell, the distance and the module placement) decided on the frames first, every image indexed on its own, the crystals put into one indexing where the lattice admits two, then scaled and merged together; with a model, the placement and the refinement follow, deciding nothing