leonarski_fandFilip Leonarski
19b135186d
* Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
5.9 KiB
What Rugnux does
The map of a run, in the order it happens — one paragraph per stage, each linking into the data-analysis reference where the algorithm lives. The walk-through is a rotation run with the defaults; stills differences are at the end.
Open the dataset. The geometry, wavelength and goniometer come from the file (What Rugnux reads). A goniometer axis that turns makes it a rotation run; none, or one that stands still from frame to frame, makes it serial stills — nothing is asked of the user.
Pre-scan. A projection of frames (60 by default) spread over the first part of the sweep, the part the geometry pre-pass integrates, finds the beam-stop shadow and masks it (§1.5), masks pixels its frames show to be defective (§1.6), measures the beam centre from the isotropy of the scattered background and compares it with the file's (§1.4), and reads how wide this crystal's spots are, which sets the integration radius (§9.5), and how much longer they are along their radius than across it, which is the beam's bandwidth where the file does not state one (§9.6), and how much wider still they grow away from the beam, which sets the integration footprint (§9.1). In a GPU build the pre-scan runs on the card.
Spots. Every image is decoded — on the GPU straight from the compressed chunk (§0) — and one fused pass computes the azimuthal profile and finds the spots against each image's own per-resolution-ring noise (§2–§3). The ice-ring score is read off the same profile.
Indexing. The spots of a sample of frames are rotated back to a common crystal frame and the FFT search looks for periodicity over thousands of directions; candidate cells are Niggli-reduced, classified by Bravais lattice, refined both constrained and triclinic, and decided on how many validation frames each actually indexes (§4–§7). A de-novo run also tries a second hypothesis with the shortest accepted axis lowered from 10 to 5 Å, for small-molecule cells. The first pass is indexed a second time at the beam centre the pre-scan measured; the file's centre is kept unless it indexes nothing, gives an axis harmonic of the measured centre's lattice, or loses to it when both first passes are merged (§1.4). A failed pass triggers the discrete rescues — the rotation-axis sign, the first-pass depth ladder, the long-axis rescue — before anything is given up on.
First integration pass. At the geometry in the file, every frame of the first half of the sweep, at least 120° of it
(--prepass-fraction) is predicted
(§8) and profile-fit integrated
(§9); partials are combined into fulls, scaled and merged
(§10).
Geometry post-refinement. From those reflections the detector distance, beam centre and the cell scale / rotation axis are refined over all those frames at once, each step committed only if it improves a held-out residual (§7.5).
Second pass. The sweep is re-indexed de novo and re-integrated at the refined geometry; this
pass is the canonical output, and a guard compares the two passes and keeps the better one
(reported as PASS= / PASS_DECISION= in the report).
Space group. On the P1 merge of the final pass, the point group is scored operator by operator
on resolution-normalised intensities and the screw axes, glide planes and centring are read from the
systematic absences, with the centre of symmetry from the intensity distribution where the absences
leave it open (§13.1); twinning and translational
pseudo-symmetry are checked beside it (§13.2). CANNOT_DETERMINE and an enantiomorphic pair are real
answers here, not evasions.
Scale and merge. In the determined group: per-frame scales, the cross-validated correction surfaces (decay, absorption, modulation), the error model and ISa, outlier rejection, the CC1/2-based resolution cut, the anisotropy description, French–Wilson amplitudes and the R-free flags (§10, §13.3–§13.5).
Write. The merged .mtz / .cif / .hkl, the unmerged MTZ and the
results report land next to each other (Output files); with
--model, validation runs first - the model placed as a rigid body, or by molecular replacement
where that does not place it or the model is a prediction - and the maps and the placed model are
written too, with <prefix>_mr.mtz where the model chooses another group or the twin hypothesis'
subgroup (§14).
Stills instead. Serial data skip the two-pass machinery: each image is indexed independently
(with the known-cell ffbidx indexer where a cell is given), partiality comes from a per-crystal
orientation-tilt post-refinement rather than a rocking curve, and a merohedral indexing ambiguity
has to be broken per image, at integration time, against a reference or a model
(Advanced ▸ the indexing ambiguity).
