Merge branch 'review-fixes-rc175' into rc175: review fixes (quality guard serial on one card, DeviceBytes as measured, CBF plain stores, GPU test guards), docs brought up to the code, GPU_DATA_ANALYSIS, acknowledgements, CHANGELOG

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

compression/CBFByteOffset.cpp

@@ -6,11 +6,6 @@
 #include <cstring>
 #include <string>
 
-#if defined(__SSE2__) || defined(_M_X64) || defined(_M_AMD64)
-#  include <emmintrin.h>
-#  define MINICBF_STREAM_STORE 1
-#endif
-
 #include "../common/JFJochException.h"
 
 namespace {
@@ -19,26 +14,6 @@ int16_t ReadI16(const uint8_t *p) { int16_t v; std::memcpy(&v, p, 2); return v;
 int32_t ReadI32(const uint8_t *p) { int32_t v; std::memcpy(&v, p, 4); return v; }
 int64_t ReadI64(const uint8_t *p) { int64_t v; std::memcpy(&v, p, 8); return v; }
 
-// Nothing on this side reads the decoded image back: it goes to the GPU by DMA, or a CPU engine
-// walks it once and drops it. An ordinary store therefore pays twice over - it reads every cache
-// line before overwriting it, and it evicts 72 MB of live cache to make room - so the pixels go
-// straight past the cache. Measured on an 18 Mpx frame: 27 ms of ordinary stores against 11 ms of
-// these. The bytes written are the same either way; the fence is what makes them visible to the
-// copy that follows.
-void StorePixel(int32_t *out, int32_t value) {
-#ifdef MINICBF_STREAM_STORE
-    _mm_stream_si32(out, value);
-#else
-    *out = value;
-#endif
-}
-
-void StoreFence() {
-#ifdef MINICBF_STREAM_STORE
-    _mm_sfence();
-#endif
-}
-
 }  // namespace
 
 // The x-CBF_BYTE_OFFSET scheme: a running value, each pixel stored as a delta in the smallest
@@ -75,12 +50,10 @@ void DecodeCBFByteOffset(const uint8_t *data, size_t size, int32_t *out, size_t
         }
 
         value += delta;
-        StorePixel(out + written, static_cast<int32_t>(value));
+        out[written] = static_cast<int32_t>(value);
         written++;
     }
 
-    StoreFence();
-
     if (written != n_pixels)
         throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
                               "miniCBF byte-offset stream ended after " + std::to_string(written)

docs/ACKNOWLEDGEMENT.md

@@ -21,11 +21,13 @@ image-header metadata, and the attention paid to diffraction anisotropy, each de
 pipeline that takes the decisions an experienced crystallographer would take while processing a
 dataset, and reports each decision together with the evidence it rests on, so that a bare
 invocation gives the best answer available. autoPROC is the inspiration and the target for that
-behaviour, and two of rugnux's working principles come from it. The first is that the metadata in
-the image headers - beam centre, detector geometry, rotation axis - may be wrong and must not be
-trusted blindly, a point argued in particular by Clemens Vonrhein. The second, which follows from
-it, is that the beam centre should be determined from the images themselves; rugnux does this in
-its own way, described in §1.4 of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md). The
+behaviour, and two of rugnux's working principles come from it. The first, and the one that has
+guided rugnux most, we owe to Clemens Vonrhein: a processing pipeline should not take the
+experimental metadata it is given - the beam centre, the detector geometry, the goniometer and its
+rotation axis - on trust, but verify it against the images and, where it is wrong, correct it. We
+are grateful to him for arguing this so clearly and for so long. The second follows from it: the
+beam centre should be determined from the images themselves; rugnux does this in its own way,
+described in §1.4 of the [data-analysis reference](CPU_DATA_ANALYSIS_IMAGE.md). The
 reasons rugnux gives for a weak stretch of a sweep also use autoPROC's wording ("loss of centring
 during crystal rotation"). C. Vonrhein, C. Flensburg, P. Keller, A. Sharff, O. Smart, W. Paciorek,
 T. Womack and G. Bricogne, "Data processing and analysis with the autoPROC toolbox" (2011),
@@ -75,6 +77,13 @@ B. Couturier, V. V. Gligorov and L. Lacassagne, "SparseCCL: Connected Components
 Analysis for sparse images" (2019), DASIP 2019, 65-70
 [doi:10.1109/DASIP48288.2019.9049184](https://doi.org/10.1109/DASIP48288.2019.9049184).
 
+**XDS and DISTL** — a spot is bounded by its shape (how much of its bounding box it fills) rather than
+by its size alone, as XDS (cited under [Integration and rotation geometry](#integration-and-rotation-geometry)) and DISTL bound theirs; the statistic
+is rugnux's own, the choice of shape over size is theirs. Z. Zhang, N. K. Sauter, H. van den Bedem,
+G. Snell and A. M. Deacon, "Automated diffraction image analysis and spot searching for high-throughput
+crystal screening" (2006), J. Appl. Cryst. 39, 112-119
+[doi:10.1107/S0021889805040677](https://doi.org/10.1107/S0021889805040677).
+
 ### Indexing
 
 **Fourier-transform indexing** — indexing a diffraction pattern, with no prior knowledge of the
@@ -84,7 +93,7 @@ one-dimensional projection form of that idea described in the next paragraph. G.
 in *Proceedings of the EEC Cooperative Workshop on Position-Sensitive Detector Software
 (Phase III)*, LURE, Paris, p. 28 (a workshop proceedings without a DOI).
 
-**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/)** — the Rossmann FFT autoindexing algorithm and
+**[MOSFLM](https://www.mrc-lmb.cam.ac.uk/mosflm/mosflm/)** — the Rossmann FFT autoindexing algorithm and
 post-refinement practice, including which parameters are safe to refine per image and which must be
 refined over a wedge. The autoindexing algorithm itself — projecting the reciprocal-space points
 onto many directions and Fourier-transforming the 1D projection histograms — is I. Steller,
@@ -216,7 +225,7 @@ steps are restricted when one is detected - see the translational-pseudo-symmetr
 the intensity a twinned crystal records is (1−α) I(h) + α I(Th), and the disagreement
 H = |I(h) − I(Th)|/(I(h) + I(Th)) of the pairs a law relates is uniform on [0, 1−2α], so the median
 gives the fraction. T. O. Yeates, "Detecting and overcoming crystal twinning" (1997), Methods Enzymol.
-276, 344-358 [doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879(97)76068-3).
+276, 344-358 [doi:10.1016/S0076-6879(97)76068-3](https://doi.org/10.1016/S0076-6879%2897%2976068-3).
 
 **A first-pass cell that is the coincidence lattice of two twin domains** (twinning by reticular
 merohedry of index 2 or 3, e.g. obverse/reverse rhombohedral domains or a cubic crystal twinned about
@@ -438,8 +447,8 @@ Problems Whose Variables Separate" (1973), SIAM J. Numer. Anal. 10, 413-432
 for solving separable nonlinear least squares problems" (1975), BIT 15, 49-57
 [doi:10.1007/BF01932995](https://doi.org/10.1007/BF01932995).
 
-**Predicted models** — `--mr` (experimental) places an AlphaFold prediction (or any model) by molecular
-replacement. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction
+**Predicted models** — `--model` (and `--mr`) place an AlphaFold prediction (or any model the rigid
+body does not place) by molecular replacement. J. Jumper, R. Evans, A. Pritzel et al., "Highly accurate protein structure prediction
 with AlphaFold" (2021), Nature 596, 583-589
 [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2); the models are those of
 the AlphaFold Protein Structure Database: M. Varadi, S. Anyango, M. Deshpande et al., "AlphaFold
@@ -465,7 +474,7 @@ crystallographic software" (2007), J. Appl. Cryst. 40, 658-674
 [doi:10.1107/S0021889807021206](https://doi.org/10.1107/S0021889807021206). Whether one copy fits
 the cell is read off the Matthews coefficient: B. W. Matthews, "Solvent content of protein
 crystals" (1968), J. Mol. Biol. 33, 491-497
-[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836(68)90205-2). How many copies
+[doi:10.1016/0022-2836(68)90205-2](https://doi.org/10.1016/0022-2836%2868%2990205-2). How many copies
 to search for is weighed by a distribution of the Matthews coefficient over protein crystals made from
 the mode and mean of Kantardjieff and Rupp's: K. A. Kantardjieff and B. Rupp, "Matthews coefficient
 probabilities: Improved estimates for unit cell contents of proteins, DNA, and protein-nucleic acid
@@ -486,7 +495,10 @@ log-likelihood gain - the sum of sigma_A^4 / 2 over the reflections - is too sma
 out is not searched for: A. J. McCoy, R. D. Oeffner, A. G. Wrobel, J. R. M. Ojala, K. Tryggvason,
 B. Lohkamp and R. J. Read, "Ab initio solution of macromolecular crystal structures without direct
 methods" (2017), Proc. Natl. Acad. Sci. USA 114, 3637-3641
-[doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). The logarithm of the Bessel
+[doi:10.1073/pnas.1701640114](https://doi.org/10.1073/pnas.1701640114). Translational NCS is looked
+for in the native Patterson before the search, an off-origin peak read as Read, Adams and McCoy read
+it (see
+[Translational pseudo-symmetry](#space-group-twinning-and-pseudo-symmetry) above). The logarithm of the Bessel
 function I0 in it is evaluated with the polynomial approximations of M. Abramowitz and I. A. Stegun,
 "Handbook of Mathematical Functions" (1964), National Bureau of Standards, 9.8.1-9.8.2. A predicted
 model is cut into rigid domains by its predicted aligned error, as Oeffner et al. (above) do before

docs/CHANGELOG.md

@@ -7,9 +7,11 @@
 * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection.
 * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets.
 * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand.
-* Rugnux: Experimental `--mr` places a predicted or experimental model by molecular replacement to settle space group, hand and twinning, writing a different answer as `<prefix>_mr.mtz`.
+* Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`.
 * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed.
-* Fixes: The TCP image pusher no longer blocks the broker's receiver, the master file appears only after its data files, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges.
+* jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection.
+* jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images.
+* Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges.
 
 ### 1.0.0-rc.174
 

docs/CPU_DATA_ANALYSIS.md

@@ -17,7 +17,7 @@ This document describes the crystallographic algorithms implemented in Jungfrauj
 11. space-group determination from the merged intensities (Laue group, screw axes, glide planes, centering, the centre of symmetry), the twinning check and the translational pseudo-symmetry check,
 12. auxiliary statistics (Wilson plot, ⟨I/σ(I)⟩, CC1/2, CCref),
 13. amplitude estimation (French–Wilson) and R-free test-set flagging,
-14. optional model-based validation: rigid-body placement of a supplied model, R-free against it, sigma_A-weighted 2mFo−DFc / mFo−DFc electron-density maps, and an anomalous difference map with the strongest anomalous sites named.
+14. optional model-based validation: rigid-body placement of a supplied model (molecular replacement where the rigid body does not place it, or for a predicted model), R-free against it, sigma_A-weighted 2mFo−DFc / mFo−DFc electron-density maps, and an anomalous difference map with the strongest anomalous sites named.
 
 
 The reference is split into four parts, in pipeline order; the section numbers run continuously
@@ -31,8 +31,8 @@ across them and are the ones the rest of the documentation cites.
   reflection prediction, profile-fitted integration, scaling, merging, mosaicity and the auxiliary
   statistics.
 - [Space group and validation (§13–§14)](CPU_DATA_ANALYSIS_DECISIONS.md) — the space-group search,
-  twinning and translational pseudo-symmetry, the resolution cutoff, diffraction anisotropy, and
-  model-based validation.
+  twinning and translational pseudo-symmetry, the resolution cutoff, diffraction anisotropy,
+  model-based validation and molecular replacement.
 
 ## References
 
@@ -82,6 +82,11 @@ The methods draw on, and in places reimplement, solutions from:
 - K. Diederichs & P. A. Karplus, *Nat. Struct. Biol.* **4** (1997), 269-275, and P. A. Karplus & K. Diederichs, *Science* **336** (2012), 1030-1033 (R_meas / R_pim, CC1/2 and CC\*).
 - M. S. Weiss, "Global indicators of X-ray data quality", *J. Appl. Cryst.* **34** (2001), 130-135 (R_pim).
 - IUCr Commission on Crystallographic Nomenclature, "Statistical descriptors in crystallography", *Acta Cryst.* **A45** (1989), 63-75, and *Acta Cryst.* **A51** (1995), 565-569 (uncertainty conventions).
+- J. Navaza, "AMoRe: an automated package for molecular replacement", *Acta Cryst.* **A50** (1994), 157-163 (the rotation function on an interpolated molecular transform, §14.10).
+- R. A. Crowther & D. M. Blow, "A method of positioning a known molecule in an unknown crystal structure", *Acta Cryst.* **23** (1967), 544-548 (the full-symmetry translation function, §14.10).
+- B. W. Matthews, "Solvent content of protein crystals", *J. Mol. Biol.* **33** (1968), 491-497, and K. A. Kantardjieff & B. Rupp, "Matthews coefficient probabilities", *Protein Sci.* **12** (2003), 1865-1871 (how many copies the cell holds, §14.10).
+- R. J. Read, "Pushing the boundaries of molecular replacement with maximum likelihood", *Acta Cryst.* **D57** (2001), 1373-1382; A. J. McCoy, R. W. Grosse-Kunstleve, L. C. Storoni & R. J. Read, "Likelihood-enhanced fast translation functions", *Acta Cryst.* **D61** (2005), 458-464; A. J. McCoy et al., "Phaser crystallographic software", *J. Appl. Cryst.* **40** (2007), 658-674; A. J. McCoy et al., *Proc. Natl. Acad. Sci. USA* **114** (2017), 3637-3641 (the likelihood target, the solution criterion and the expected gain of molecular replacement, §14.10).
+- R. D. Oeffner, T. Croll, C. Millán et al., "Putting AlphaFold models to work with phenix.process_predicted_model and ISOLDE", *Acta Cryst.* **D78** (2022), 1303-1314, and J. Jumper et al., "Highly accurate protein structure prediction with AlphaFold", *Nature* **596** (2021), 583-589 (preparing a predicted model for molecular replacement, §14.10).
 
 (list is not exhaustive; the full citations, with DOIs, are in [ACKNOWLEDGEMENT.md](ACKNOWLEDGEMENT.md))
 

docs/CPU_DATA_ANALYSIS_DECISIONS.md

@@ -124,7 +124,7 @@ Everything lands in `<prefix>_report.txt` section 4 (`ANISOTROPY_*` keys), in th
 
 ## 14. Model-based validation: R-free against a model and electron-density maps
 
-Offline (`rugnux --model model.pdb`) the merged data can be scored against a supplied atomic model and electron-density maps computed — enough to confirm that a model fits the data and to inspect the density, not a substitute for refinement. **The structure itself is not refined**; the model is re-fractionalized into the data unit cell and then **placed as one rigid body** (§14.8), and the observed amplitudes are the French–Wilson $|F|$ from §10.8, so the R-free and the maps use exactly the same amplitudes as the written reflection file. The model, structure-factor, bulk-solvent and FFT machinery is provided by GEMMI.
+Offline (`rugnux --model model.pdb`) the merged data can be scored against a supplied atomic model and electron-density maps computed — enough to confirm that a model fits the data and to inspect the density, not a substitute for refinement. **The structure itself is not refined**; the model is re-fractionalized into the data unit cell and then **placed as one rigid body** (§14.8) - or, where the rigid body does not place it and for a predicted model, by molecular replacement (§14.10) - and the observed amplitudes are the French–Wilson $|F|$ from §10.8, so the R-free and the maps use exactly the same amplitudes as the written reflection file. The model, structure-factor, bulk-solvent and FFT machinery is provided by GEMMI.
 
 ### 14.1 Model structure factors
 
@@ -167,8 +167,8 @@ $m$ and $D$ are estimated on each dataset separately, so two datasets of one cry
 A model supplied with `--model` is a **hypothesis about the crystal**, not an instruction. It is
 always scaled, always placed (§14.8) and always scored, and its R-factors and maps are always
 reported — but before it is allowed to change anything about the reflections that are written, the
-data have to accept it. There are only two such things (§14.6), and where the model claims neither the
-question is never put: see the end of this section.
+data have to accept it. There are only two such things (§14.6); the same verdict also says whether the
+rigid body placed the model at all (below).
 
 **No fixed threshold on R can decide that**, and this was measured three ways. The classical acentric
 random-structure value is $R = 0.586$ at unit scale, but the scale here is fitted to minimise the very
@@ -235,10 +235,8 @@ merges in 2 s. What remains is the cost of the *slowest* replicate: how many pla
 orientation needs varies by nearly half between them, so the concurrency saturates around 4×, and
 more threads than replicates buy nothing. The orientations are drawn up front, in order, from the
 fixed seed, so replicate $i$ gets the same orientation whatever order the threads run in — a σ that
-depended on the scheduling would not be a measurement. Where the null is skipped, `--model` costs
-what it did before the null existed (measured: indistinguishable, ~4 s against ~4 s, on a loaded
-machine where a single run scatters by a second). Where it runs, that is the price of the answer
-being a measurement rather than a threshold.
+depended on the scheduling would not be a measurement. That is the price of the answer being a
+measurement rather than a threshold.
 
 ### 14.6 Aligning the data to the model: enantiomorph and indexing ambiguity
 
@@ -273,7 +271,7 @@ The refinement walks a coarse-to-fine ladder, 6 Å → 4.5 Å → 3.5 Å, each z
 
 The refinement sees **only the working reflections**. The step is then committed only if it lowers **R-free**, computed at full resolution on the free set it never saw; otherwise the model is put back exactly where it was read and the maps are the ones it would have given. The whole addition costs about two seconds. Note that the origin is a gauge in some space groups — free in all three directions in $P1$, and along the unique axis in a polar group — so those components of the translation are undetermined; nothing is done about that beyond the Levenberg–Marquardt damping and the R-free gate, which between them make an undetermined direction harmless rather than unstable.
 
-Where a CUDA GPU is present, the refinement's target is evaluated on it: the same density, the same composition of $F_\mathrm{calc}$ from one copy of the model, the same bulk-solvent mask with its islands removed, the same scale fit and the same Jacobian, computed on the device, where a fit takes a fraction of a second rather than several seconds. The device is chosen once per validation - the real fit and every replicate of the null of §14.5 on the same one - and the CPU is used where there is no GPU, where the card has too little free memory for the fit's buffers, or after a CUDA failure, in which case the whole validation is run again on the CPU. The two agree to rounding and not bit for bit: the device measures distances in single precision and transforms with cuFFT instead of FFTW, and GEMMI's scale fit, whose stopping rule leaves it unconverged at about $10^{-4}$ of $|F|$, can respond to that rounding with a step of its own. A rigid-body step accepted or rejected on an R-free difference below about $5\cdot10^{-4}$ can therefore go either way between the two; every kernel is deterministic, so either one gives the same answer on every run.
+Where a CUDA GPU is present, the refinement's target is evaluated on it: the same density, the same composition of $F_\mathrm{calc}$ from one copy of the model, the same bulk-solvent mask with its islands removed, the same scale fit and the same Jacobian, computed on the device, where a fit takes a fraction of a second rather than several seconds. The device is chosen once per validation, before anything is computed, and never revisited - the real fit and every replicate of the null of §14.5 on the same one. The structure factors go to the GPU where their buffers fit a quarter of the card's total memory (judged on the total, so the same input on the same machine always takes the same path), the rigid-body engines take up to another quarter, and the scale fits of §14.2 run on the GPU too, in engines of their own that take at most an eighth of the card; a part that does not fit, or a cell too small for the GPU's gridding, runs on the CPU, as everything does where there is no GPU. A CUDA failure ends the validation, not the run: the validation reports why it did not finish, decides nothing, and the reflection files are those of a run without a model. The two agree to rounding and not bit for bit: the device measures distances in single precision and transforms with cuFFT instead of FFTW, and GEMMI's scale fit, whose stopping rule leaves it unconverged at about $10^{-4}$ of $|F|$, can respond to that rounding with a step of its own. A rigid-body step accepted or rejected on an R-free difference below about $5\cdot10^{-4}$ can therefore go either way between the two; every kernel is deterministic, so either one gives the same answer on every run.
 
 The rotation and translation actually taken are reported (`RIGID_BODY_ROTATION_DEG`, `RIGID_BODY_SHIFT_A`), together with the R-free before it (`R_FREE_BEFORE_RIGID_BODY`), so what the placement bought is visible.
 
@@ -283,5 +281,46 @@ The coordinates **as placed** are written as `<prefix>_model.cif`, and as `<pref
 
 Where the run wrote a twin hypothesis (§13.2), the model is scored against that merge too: as merged, and **as twinned** by its law — $|F_\mathrm{twin}(h)|^2 = (1-\alpha)|F_\mathrm{model}(h)|^2 + \alpha|F_\mathrm{model}(Th)|^2$ from the placed, scaled model, one overall scale refitted at each $\alpha$, $\alpha$ scanned from 0 to 0.5 in steps of 0.01 on the working set and R-free read at the chosen one. The placement, the anisotropic $B$ and the bulk solvent are those of the untwinned fit (§14.2, §14.8), so this is the same few-parameter comparison as §14.3, not a twin refinement. Against the answer's merge the model is scored as in §14.3, the subgroup model's structure factors simply read at each of the higher group's reflections.
 
-The maps and the placed model are then made from whichever merge the model fits better on R-free — the lower of the subgroup's plain and twinned R-free against the answer's — and the report says which (`MAPS_FROM=`) and with what R-free for each. The map coefficients are those of §14.4 against the chosen merge's amplitudes, not detwinned. **This chooses the maps and nothing else**: the space group, the answer's reflection files and every number in the report outside the twin keys are what the run decided without the model, because a model can be wrong and the reflection files must not depend on one (§14.5). The structure factors come from the same GPU engine as §14.1 where there is a GPU.
+The maps and the placed model are then made from whichever merge the model fits better on R-free — the lower of the subgroup's plain and twinned R-free against the answer's — and the report says which (`MAPS_FROM=`) and with what R-free for each. The map coefficients are those of §14.4 against the chosen merge's amplitudes, not detwinned. **This chooses the maps, and never the answer**: the space group, the answer's reflection files and every number in the report outside the twin keys are what the run decided without the model, because a model can be wrong and the reflection files must not depend on one (§14.5). Where the model also fits the subgroup's merge against its own null (§14.5), it has chosen the subgroup, and that merge is written beside the answer as `<prefix>_mr.mtz`/`.cif`, with the model as placed there. The structure factors come from the same GPU engine as §14.1 where there is a GPU.
 
+### 14.10 Molecular replacement: placing a model the rigid body cannot
+
+The rigid body of §14.8 only moves a model that already sits near its place. A model it does not
+place - the null of §14.5 is not beaten in the answer's merge nor in the twin hypothesis' one - is
+searched for from scratch, and so is a **predicted** model, which has no placement of its own, and any
+model given with `--mr`. The kind of model is read from the file: a prediction (ModelCIF's
+`_ma_qa_metric`, or no cell and one B value per residue between 0 and 100) is trimmed to the residues
+of pLDDT 70 and above, its B-factors made from the pLDDT and its coordinate error from them (following
+Oeffner et al., 2022), and cut into rigid domains where the AlphaFold DB's predicted-aligned-error file
+sits beside it; anything else is an experimental structure, used as its polymer alone.
+
+The search is the classical one, scored by likelihood. A rotation function over the point group, its
+best peaks refined off the sampling grid (following Navaza, 1994); for each orientation a full-symmetry
+translation function computed as one Fourier synthesis (following Crowther & Blow, 1967), in **every
+space group the merge cannot tell apart** and in the twin hypothesis' subgroup; and the best
+placements of each group put through the rigid body, the bulk solvent and R-free. Further copies are
+added one at a time, up to the copy number the cell can hold by its Matthews coefficient (Matthews,
+1968; the probabilities of Kantardjieff & Rupp, 2003), each with the copies already placed as a known
+partial structure and ranked by the log-likelihood gain of the Rice/Woolfson distributions of
+$E_\mathrm{obs}$ given $E_\mathrm{calc}$, with $\sigma_A$ falling off with the coordinate error
+(following Read, 2001, and McCoy et al., 2005 and 2007). A model whose expected gain (McCoy et al.,
+2017) is too small to be found is not searched for. A copy is kept where it stands out from the
+translation function's own peaks, packs without clashing, and lowers R-free; in $P1$, where the first
+copy's position is a gauge, it has to stand out in the rotation function instead.
+
+Translational NCS changes what the search can tell apart (Read, Adams & McCoy, 2013): with a
+pseudo-translation, one copy beside a translated twin of itself fits the data as well in a group whose
+screw axes the translation turns into the chosen group's, so such a choice is left undecided unless the
+copies, carried into the other group and refined there, fit clearly worse; and where the vector is half
+a lattice vector, the placement is compared with every move of itself the vector allows, on the weak
+reflections it modulates.
+
+The choice between groups and merges is a hypothesis comparison like §14.5: the candidate the choice
+falls on has to beat a null of the same model placed from random orientations, and lead the other
+groups by a margin in R-free. What the choice may do is limited exactly as for the rigid body, and
+less: it may relabel the **enantiomorph** of the answer, past the anomalous check of §14.7, and never
+reindexes; another group with the answer's rotations, or the twin hypothesis' subgroup, is written
+beside the answer as `<prefix>_mr.mtz`/`.cif` with the model as placed there, and the answer's own
+files stay what the run decided. The rotation and translation functions, the rigid body and the
+structure factors run on the GPU where there is one. The keys, thresholds and table are described in
+[the results report](RUGNUX_REPORT.md#molecular-replacement).

docs/CPU_DATA_ANALYSIS_IMAGE.md

@@ -8,9 +8,9 @@ Part of the [CPU/GPU data-analysis reference](CPU_DATA_ANALYSIS.md); the section
 ```
 
 
-## 0. Getting the image onto the GPU: device-side bitshuffle+LZ4 decoding
+## 0. Getting the image onto the GPU: device-side decoding
 
-Images arrive bitshuffle+LZ4 compressed (HDF5 filter 32008), and everything from §1 onwards runs on
+HDF5 images mostly arrive bitshuffle+LZ4 compressed (HDF5 filter 32008), and everything from §1 onwards runs on
 the GPU when one is present. Instead of decompressing on the host and uploading the image, the
 compressed chunk is uploaded — a few MB rather than tens of MB — and decoded on the device. The
 approach follows Jon Wright (ESRF); the kernels are Jungfraujoch's own.
@@ -45,9 +45,21 @@ image to device memory and the second un-transposing and preprocessing out of it
 The block offsets inside the container can only be discovered by reading the block lengths in
 order, so that scan stays on the host.
 
-Only `BSHUF_LZ4` is decoded on the device. For the zstd variants (`BSHUF_ZSTD`, `BSHUF_ZSTD_RLE`,
-`BSHUF_ZSTD_RLE_HUFF`), and for uncompressed or float images, `BSLZ4DecoderGPU::Supports()` returns
-false and the pipeline decompresses on the host and uploads as before.
+Two stored formats of other detectors also reach the card as the file stores them. A miniCBF frame's
+`x-CBF_BYTE_OFFSET` stream is decoded on the device into the same `int32` image: the stream is cut
+into short parts; for every byte offset a part can be entered at, one thread works out where the
+decode leaves the part, how many pixels it emitted and their running sum; a scan composing those maps
+gives every part its true entry and running value; and a second walk writes the pixels. A stream that
+ends before the image is refused, as on the host. A marCCD frame's 16-bit pixels cross the bus at two
+bytes each and are widened by the preprocessing kernel itself, which tests the error marker and the
+saturation cap on the pixel's value, so a marCCD's 65535 - its saturation marker, not an error - is
+treated exactly as in the `int32` image the host would have made. Both are staged through the
+engine's own page-locked buffer, as the BSLZ4 chunks are; `.cbf.gz` is still inflated on the host.
+
+For the zstd variants (`BSHUF_ZSTD`, `BSHUF_ZSTD_RLE`, `BSHUF_ZSTD_RLE_HUFF`), and for other
+uncompressed or float images, the pipeline decompresses on the host and uploads as before. The host
+decoders are the CPU build's path, not a fallback: a frame the device refuses is not retried on the
+host.
 
 The container arrives off the network or off disk and is not trusted. Everything checkable on the
 host — declared sizes, the block scan, a block size that is not a multiple of 8 elements, a block

docs/CPU_DATA_ANALYSIS_INDEXING.md

@@ -230,6 +230,16 @@ also runs (rotation only, and not when a cell was given — `-C` already lowers
 and its answer is adopted only when it is an integer sub-cell of the standing one and ties or beats
 it on the validation frames. Everywhere else the standard pass's answer stands.
 
+**A near tie is decided early and checked again late.** The first pass decides on validation frames
+from the start of the sweep, so it can act at once - also while a collection is still being written.
+Where one of its decisions was a near tie that later frames could tip (the lattice, the indexing
+path, the beam centre, the spot-finding settings), the pass is run again once every frame is in, on a
+copy of the run taken as it began and deciding on the rest of the sweep too, and only where that
+re-check ends elsewhere is the whole run made again with every near tie decided late; a run that found
+no lattice after carrying a near tie is made again likewise. The answer is therefore the one deciding
+late throughout gives, at the cost of the re-checks (measured: a median of 0.9 s on the sets that had
+one).
+
 **A first pass that ends with no usable lattice tries a leaner, shallower one.** A crystal sitting in
 a crystalline powder floods the pass with spots that are not its own — measured, 2112 spots a frame
 against the 80 a clean crystal on the same beamline gives — and the FFT then takes its cell out of the

docs/GPU_DATA_ANALYSIS.md

@@ -0,0 +1,182 @@
+# GPU data analysis
+
+Jungfraujoch runs most of its crystallographic analysis on NVIDIA GPUs where one is present: the
+per-image loop of `jfjoch_broker`, `jfjoch_viewer` and `rugnux`, and most of `rugnux`'s
+dataset-level steps - the pre-scan, scaling and merging, model validation and molecular replacement.
+This page lists the GPU engines (all of them in `.cu` files under `image_analysis/`), with what each
+computes. The methods themselves are described once, in the
+[CPU/GPU data analysis](CPU_DATA_ANALYSIS.md) reference and its siblings, which each entry links to.
+
+**Every engine has a CPU counterpart** that computes the same thing, and the CPU-only build
+(`JFJOCH_USE_CUDA=OFF`) runs the whole pipeline on it; the one exception is the `ffbidx` indexer,
+which needs a GPU (the CPU build indexes with `fftw`). How closely the two agree is stated engine by
+engine below and in the source: where the arithmetic is integer, or the order of every sum is fixed to
+the host's, the result is the same to the bit; where the GPU uses float distances, cuFFT for FFTW, or a
+different (fixed) summation order, the two agree to rounding. Every engine is deterministic on its
+own - no floating-point atomics decide a result - so the same input gives the same output on every
+run of a GPU build.
+
+## Getting the image onto the card
+
+- **Bitshuffle+LZ4 decoder** (`BSLZ4DecoderGPU.cu`) - decompresses the stored chunk on the device, so
+  only the compressed bytes cross PCIe, and un-transposes it straight into the preprocessing step.
+  The same pixels as the host decoder. The approach is Jon Wright's (ESRF).
+  [§0](CPU_DATA_ANALYSIS_IMAGE.md#0-getting-the-image-onto-the-gpu-device-side-decoding)
+- **CBF byte-offset decoder** (`CBFDecoderGPU.cu`) - decodes a miniCBF `x-CBF_BYTE_OFFSET` stream on
+  the device into the same int32 pixels as the host decoder, refusing a stream that ends before the
+  image does. [§0](CPU_DATA_ANALYSIS_IMAGE.md#0-getting-the-image-onto-the-gpu-device-side-decoding)
+- **marCCD frames** (`ImagePreprocessorGPU.cu`) - the stored 16-bit pixels are uploaded as they are
+  and widened to int32 on the card, inside preprocessing.
+  [§0](CPU_DATA_ANALYSIS_IMAGE.md#0-getting-the-image-onto-the-gpu-device-side-decoding)
+- **Preprocessing** (`ImagePreprocessorGPU.cu`, `ImagePreprocessorBufferGPU.cu`) - applies the pixel
+  mask and flags error and saturated pixels, counting each, and keeps the preprocessed image on the
+  device for every engine below; on request it gathers the pixels of the strong spots, or small
+  squares around given positions, back to the host. Integer, so identical to the host.
+  [§1.6](CPU_DATA_ANALYSIS_IMAGE.md#16-defective-pixels)
+
+## Per image: profiles and spots
+
+- **Azimuthal integration** (`AzIntEngineGPU.cu`) - bins every pixel into its resolution (and
+  optionally azimuth) bin with the geometric corrections applied and sums them, as a radial profile.
+  Fixed-point sums, so the profile does not depend on the order the threads add in.
+  [§2](CPU_DATA_ANALYSIS_IMAGE.md#2-azimuthal-integration-radial-profiles)
+- **ROI integration** (`ROIIntegrationGPU.cu`) - sums the pixels of user-defined detector regions of
+  interest (box, circle, azimuthal), each pixel feeding any subset of them, on the EIGER/CPU path where
+  the FPGA does not. Integer sums. [FPGA: ROI integration](FPGA_DATA_ANALYSIS.md#region-of-interest-roi-integration)
+- **Local-statistics spot finder** (`ImageSpotFinderGPU.cu`) - flags strong pixels by their
+  signal-to-noise against a local box. [§3.1](CPU_DATA_ANALYSIS_IMAGE.md#31-strong-pixel-detection-by-local-statistics)
+- **Adaptive spot finder** (`AdaptiveSpotFinderGPU.cu`) - reduces the image into resolution rings
+  once, and from that one pass gives both the azimuthal profile and each ring's robust background,
+  which sets the self-calibrating detection threshold; the result is intersected with the local test
+  above. [§3.2](CPU_DATA_ANALYSIS_IMAGE.md#32-adaptive-self-calibrating-detection)
+- **Spot extractor** (`SpotExtractorGPU.cu`) - joins the flagged pixels into connected components and
+  sums each into a spot descriptor on the device, so only the spot list comes back. The same
+  components, in the same order, as the host labelling (the sparse formulation of the ACTS/traccc
+  project). [§3.4](CPU_DATA_ANALYSIS_IMAGE.md#34-connected-component-labeling-ccl)
+
+## Pre-scan accumulators
+
+These read a subset of the frames before processing starts, on every card at once, and are added
+together across cards; their sums are integers, so the split of the frames between the cards changes
+nothing.
+
+- **Beam-stop projection and mask** (`ShadowAccumulatorGPU.cu`, `ShadowMaskGPU.cu`) - accumulates the
+  per-pixel maximum, sum and count of the raw counts straight from the compressed frames, then draws
+  the beam-stop shadow mask from them, step for step as the host does.
+  [§1.5](CPU_DATA_ANALYSIS_IMAGE.md#15-finding-the-beam-stop)
+- **Hot pixels** (`HotPixelsGPU.cu`) - per frame, the ring-sector and ring order statistics, the levels
+  made from them and the per-pixel counts of lit frames, from which persistently hot pixels are masked.
+  Exact order statistics and integer sums: the host's result to the bit.
+  [§1.6](CPU_DATA_ANALYSIS_IMAGE.md#16-defective-pixels)
+- **Beam centre from the background** (`BeamCenterBackgroundGPU.cu`) - bins the usable pixels about
+  a trial centre and sums each cell, the two passes of the background-isotropy fit; the fit itself
+  stays on the host. [§1.4](CPU_DATA_ANALYSIS_IMAGE.md#14-the-beam-centre)
+- **Beam centre by FFT** (`BeamCenterFFTGPU.cu`) - the self-convolutions that score every candidate
+  centre at once, their combination into one score surface, and the shortlist of its peaks, with only
+  the shortlist brought back. Falls back to FFTW on the host where the card has no room.
+  [§1.4](CPU_DATA_ANALYSIS_IMAGE.md#14-the-beam-centre)
+
+## Indexing
+
+- **FFT indexer** (`FFTIndexerGPU.cu`) - projects the spots' reciprocal-space vectors onto a dense set
+  of directions, histograms each projection and transforms it with cuFFT, and picks each direction's
+  most prominent lattice peak; the candidate vectors then go to the host's lattice reduction.
+  [§5.1](CPU_DATA_ANALYSIS_INDEXING.md#51-directional-projections-and-histograms),
+  [§5.2](CPU_DATA_ANALYSIS_INDEXING.md#52-fft-peak-picking-and-candidate-vectors)
+- **Fast-feedback indexer** (`ffbidx`) - third-party, from the fast-feedback-indexer project (PSI); a
+  known-cell indexer for serial stills, used as fetched.
+  [§4](CPU_DATA_ANALYSIS_INDEXING.md#4-indexing-overview)
+
+## Prediction and integration
+
+- **Still prediction** (`BraggPredictionGPU.cu`) - the reflections of a lattice within the
+  excitation-error cutoff, with their detector positions and per-reflection corrections.
+  [§8.2](CPU_DATA_ANALYSIS_INTEGRATION.md#82-still-prediction-excitation-error-cutoff)
+- **Rotation prediction** (`BraggPredictionRotGPU.cu`) - the reflections crossing the Ewald sphere
+  within an image's rotation range, with their rotation angle and partiality; they are ordered, and
+  under a cap the best-recorded ones selected, on the device, in exactly the order the host gives.
+  [§8.3](CPU_DATA_ANALYSIS_INTEGRATION.md#83-rotation-prediction-laue-equation--partiality-model)
+- **Bragg integration** (`BraggIntegrationEngineGPU.cu`) - for every predicted reflection: the
+  three-ring regions and neighbour ownership, box summation, the per-shell profile learned from the
+  strong spots and rebuilt per reflection, the radial background-curvature correction, and the profile
+  fit. Matches the CPU engine to floating-point precision.
+  [§9.2](CPU_DATA_ANALYSIS_INTEGRATION.md#92-box-summation-seed-and-fallback),
+  [§9.3](CPU_DATA_ANALYSIS_INTEGRATION.md#93-profile-fitted-extraction-default)
+
+## Scaling and merging (rotation)
+
+One engine (`RotationScaleMergeGPU.cu`) keeps every observation resident on the card for the whole
+scaling run; each per-group sum is a segmented reduction in a fixed order, so it is deterministic.
+
+- **Ingest** - the integrated reflections uploaded frame by frame and taken apart on the card; the
+  frame flux applied; the sort into raw-hkl runs and the resolution cut; the recomputed rotation angles
+  (delta-phi), the rocking events relaid, and the partiality recomputed from the smoothed mosaicity.
+  The same sort and cut as the host, to the bit.
+  [§10.2](CPU_DATA_ANALYSIS_INTEGRATION.md#102-partiality-models),
+  [§10.6](CPU_DATA_ANALYSIS_INTEGRATION.md#106-rotation-datasets-combining-partials-into-fulls-3d-integration)
+- **Per-frame scaling** (`ReduceGroupMeans`, `FitPerFrameG`, `UpdateCorr`, `SmoothCorr`, the zeta and
+  frame filters, `PerFrameCC`) - the reference intensity of every symmetry group, the per-frame scale
+  fitted against it, its smoothing over the sweep, and the per-frame correlation behind
+  `--min-image-cc`. [§10.1](CPU_DATA_ANALYSIS_INTEGRATION.md#101-observation-model),
+  [§10.3](CPU_DATA_ANALYSIS_INTEGRATION.md#103-smoothing-of-per-frame-scales)
+- **Combine** - each reflection's partials summed into a full, with its capture fraction and
+  variance; then the fulls' own per-frame scale and its smoothing.
+  [§10.6](CPU_DATA_ANALYSIS_INTEGRATION.md#106-rotation-datasets-combining-partials-into-fulls-3d-integration)
+- **Correction surfaces** - the decay, absorption and modulation surfaces' terms, their reference and
+  their least-squares sums over the fulls, on held-out subsets.
+  [§10.6](CPU_DATA_ANALYSIS_INTEGRATION.md#106-rotation-datasets-combining-partials-into-fulls-3d-integration)
+- **Merge** - the inverse-variance merge (I(+)/I(-) and the mean), the half-sets for CC1/2, R_meas,
+  the per-group scatter, the anomalous (Bijvoet) excess and the error-model samples.
+  [§10.4](CPU_DATA_ANALYSIS_INTEGRATION.md#104-merging-estimator),
+  [§10.5](CPU_DATA_ANALYSIS_INTEGRATION.md#105-merging-statistics)
+- **Error model** (`ErrorModelGPU.cu`) - the binned fit of the error model's a and b, the bin medians
+  identical to the host's; a and b can differ from the host fit in their last bits (summation order).
+  [§10.5](CPU_DATA_ANALYSIS_INTEGRATION.md#105-merging-statistics)
+- **Wilson outliers** (`WilsonOutliersGPU.cu`) - the per-observation Wilson-statistics rejection,
+  the host's result to the bit. [§13.3](CPU_DATA_ANALYSIS_DECISIONS.md#133-outlier-rejection)
+- **Pair sort** (`PairSortGPU.cu`) - sorts the (deviation, weight) pairs of the outlier test's
+  weighted median; the same sequence as `std::sort`.
+  [§13.3](CPU_DATA_ANALYSIS_DECISIONS.md#133-outlier-rejection)
+
+## Model validation
+
+The model's structure factors and the rigid body run on the card where there is one; the GPU and CPU
+agree to rounding (float distances, cuFFT for FFTW, the crystal's symmetry composed in reciprocal
+space instead of by symmetrizing the grid).
+
+- **Model density** (`ModelDensityGPU.cu`) - the density of one copy of the model's atoms on a grid,
+  as a gather over bricks of the grid, bit-identical from run to run.
+  [§14.1](CPU_DATA_ANALYSIS_DECISIONS.md#141-model-structure-factors)
+- **Bulk-solvent mask** (`ModelMaskGPU.cu`) - the solvent mask from every symmetry image of every
+  atom, with island removal and the shrink, as gemmi computes it.
+  [§14.2](CPU_DATA_ANALYSIS_DECISIONS.md#142-bulk-solvent-and-scaling)
+- **Structure factors and maps** (`ModelStructureFactorsGPU.cu`) - F_calc and F_mask from the two grids
+  by cuFFT, composed over the group's operators and read off at the data's reflections; and the
+  inverse transform of map coefficients into maps.
+  [§14.1](CPU_DATA_ANALYSIS_DECISIONS.md#141-model-structure-factors),
+  [§14.4](CPU_DATA_ANALYSIS_DECISIONS.md#144-electron-density-maps)
+- **Scale fit** (`ModelScaleGPU.cu`) - the sums of the overall-scale, anisotropic-B and bulk-solvent
+  fit over the reflections, in double and a fixed order, for many fits at once (the solvent grid, the
+  null replicates); the Levenberg-Marquardt control stays on the host. Used where the pool fits an
+  eighth of the card, the CPU fit otherwise.
+  [§14.2](CPU_DATA_ANALYSIS_DECISIONS.md#142-bulk-solvent-and-scaling)
+- **Rigid body** (`RigidBodyGPU.cu`) - the rigid-body target for a placement: the model moved, its
+  F_calc and mask, the scale, the residuals and their Jacobian; used for the placement itself and for
+  every replicate of the null. [§14.8](CPU_DATA_ANALYSIS_DECISIONS.md#148-rigid-body-placement-of-the-model),
+  [§14.5](CPU_DATA_ANALYSIS_DECISIONS.md#145-does-the-model-fit-the-null-it-is-scored-against)
+
+## Molecular replacement
+
+- **Molecular transform** (`MolecularReplacementGPU.cu`, `MapToFPhiGPU`) - the Fourier transform of
+  the search model's density in a large box, by cuFFT, where it fits a quarter of the card.
+- **Rotation function** (`MolecularReplacementGPU.cu`) - for every trial orientation, the model's
+  intensities summed over the copies, normalised by resolution bin and correlated with the data's
+  normalised intensities.
+- **Translation function** (`MolecularReplacementGPU.cu`) - for each orientation, the function over
+  every position in the cell, beside the copies already placed where there are any, and its mean,
+  spread and highest point.
+
+The placements are then refined, scored and checked against their null with the rigid body and the
+structure factors above; the likelihood scoring runs on the host. The search is described in
+[§14.10](CPU_DATA_ANALYSIS_DECISIONS.md#1410-molecular-replacement-placing-a-model-the-rigid-body-cannot), its keys and decision rules in
+[Molecular replacement](RUGNUX_REPORT.md#molecular-replacement).

docs/RUGNUX.md

@@ -2,20 +2,29 @@
 
 `rugnux` is the **offline** crystallographic data-analysis tool of Jungfraujoch — the
 data-processing half of the system (see [Naming](NAMING.md) for where the name comes from).
-It takes an existing HDF5 dataset, runs the full analysis pipeline — spot finding, indexing,
-geometry refinement, Bragg integration and (optionally) scaling and merging — and writes the
-results to a `_process.h5` file, plus reflection files (`.mtz`/`.cif`/`.hkl`) when merging is
-requested.
+It takes a stored dataset, runs the full analysis pipeline — spot finding, indexing, geometry
+refinement, Bragg integration, scaling and merging — and writes the merged reflections
+(`.mtz`/`.cif`/`.hkl`), the unmerged MTZ and the results report; the per-image `_process.h5` is
+written only on request (`--write-process-h5`), or as the only output with `--no-merge`.
 
-It runs the *same* analysis code as the online and interactive tools, just driven from the
-command line over a file rather than a live detector stream.
+It runs the *same* analysis code as the online and interactive tools, driven from the command line
+over files.
 
 ```
-rugnux {<options>} <input.h5>
+rugnux {<options>} <input.h5> [<broker URL>]
 ```
 
 Run it with no arguments to print the usage.
 
+Given the URL of a `jfjoch_broker` (`http://host:port/`) and the path the master file *will* have,
+`rugnux` follows the broker's current collection while it is being written: the data files are read
+as the broker reports them closed, processing starts on the first images, and the run ends with the
+images the collection delivered. A choice made early on a near tie is checked again on every frame
+once all are in, and the run is made again where that check ends elsewhere, so the answer is the one
+the whole sweep gives. This is for rotation data and `--mode mx`; the collection's token is read from
+`JUNGFRAUJOCH_HTTP_TOKEN` and sent only to the broker. With the master file already on disk, the URL
+is ignored.
+
 > **Note.** `rugnux` is under very active development. This page describes the tool and
 > its options at a high level; the authoritative, always-current list of options is the program's
 > own usage message — run `rugnux` with no arguments.
@@ -41,6 +50,7 @@ rugnux -o myrun dataset_master.h5
 rugnux -o myrun -z reference.mtz dataset_master.h5
 
 # 3. with a known structure: R-work / R-free and sigma_A-weighted 2mFo-DFc / mFo-DFc maps
+#    (a predicted model, or one the rigid body cannot place, goes to molecular replacement)
 rugnux -o myrun --model model.pdb dataset_master.h5
 
 # 4. with the space group and the cell pinned (-S takes either spelling: P43212 or 96)

docs/RUGNUX_ADVANCED.md

@@ -164,6 +164,35 @@ the data cell and then placed as **one rigid body**, so a deposited model from a
 quite isomorphous still sits where the density is. The placement runs on the GPU where there is one
 and on the CPU otherwise, with nothing to set; the two agree to rounding, not bit for bit.
 
+The rigid body only moves a model that already sits near its place, so its placement is **checked**:
+the model counts as placed where its fit beats the same model refitted from random orientations (the
+null behind `MODEL_FIT`, below), in the answer's merge or in the twin hypothesis' one. An experimental
+model the rigid body does not place — a deposited structure of another crystal form, a model of a
+subgroup the data were merged above, a file with no crystal cell — is then searched for by
+**molecular replacement**: as many copies as the cell holds, in every space group the merge cannot tell
+apart and in the twin hypothesis' subgroup (the search and its keys are in
+[Molecular replacement](RUGNUX_REPORT.md#molecular-replacement)). A **predicted** model — AlphaFold or
+ModelCIF, recognised from the file's content (pLDDT in the B column) — has no placement of its own and
+goes to molecular replacement straight away, trimmed to the residues of pLDDT 70 and above and, where
+the AlphaFold DB's predicted-aligned-error JSON sits beside it under its own name, cut into rigid
+domains. `--mr <file>` is `--model` with the rigid body skipped: molecular replacement straight away,
+whatever the model. The report says which way the model was placed, `MODEL_PLACEMENT=` `RIGID_BODY`,
+`MR` or `NOT_PLACED`; a model placed nowhere is a result, not a warning, and the reflection files are
+then what a run without a model writes. Where molecular replacement placed it, the R-factors, the maps
+and the placed model below are those of its placement. Molecular replacement is part of the full
+`--mode mx` run, rotation and serial stills alike; `--mode scale` (below) places the model by the
+rigid body alone.
+
+A placed model changes two things in `<prefix>.mtz`/`.cif`/`.hkl` and nothing else: the
+**enantiomorph** label and, placed by the rigid body, the
+[indexing ambiguity](#the-indexing-ambiguity) (molecular replacement relabels the hand, past the same
+anomalous check, and never reindexes). Where the model chooses **another space group** with the
+answer's rotations, or the **twin hypothesis' subgroup**, the answer's files stay the run's own and that
+merge is written beside them as `<prefix>_mr.mtz`/`.cif` — from the merge it was chosen on, with the
+twin law and fraction where it is the subgroup — together with the model as placed there,
+`<prefix>_mr_model.pdb`/`.cif`; the report names them (`MODEL_CHOICE_FILE_PREFIX=`,
+`MODEL_CHOICE_SPACE_GROUP=`).
+
 Because the model moves, the input file no longer describes these maps, so the model **as placed** is
 written as `<prefix>_model.cif` — the input's chains, residues, ligands, waters, B-factors,
 occupancies and anisotropic *U*s, at the coordinates the maps were computed from, in the same unit
@@ -180,8 +209,9 @@ in a subgroup with a twin law, written when the adopted space group may have abs
 [the twin hypothesis](RUGNUX_REPORT.md#the-twin-hypothesis-twin_subgroup_)) — the model is scored
 against that merge as well, plainly and as twinned by the law, and the maps and `<prefix>_model.cif`
 come from whichever of the two merges the model fits better on R-free. The report says which
-(`MAPS_FROM=`) and gives the R-free of both. The model only chooses the maps: `<prefix>.mtz` and the
-space group are the run's own either way. Both are written whenever the
+(`MAPS_FROM=`) and gives the R-free of both. `<prefix>.mtz` and the space group are the run's own
+either way; where the model also fits that merge against its null, it has chosen the subgroup, and the
+merge is written once more as `<prefix>_mr.mtz`/`.cif` (above). Both are written whenever the
 maps are, including for a model the data rejected: the rejection is a result, and it is exactly the
 case where someone wants to look at the model in the density.
 
@@ -240,10 +270,11 @@ in P1 unless `-S` says otherwise.
 
 A reference MTZ (`-z`) is accepted here on **stills** data, where it fixes the space group and cell,
 reports CC<sub>ref</sub> and hands over its R-free flags; on rotation data the rotation scaler
-declines it and the run stops with a message saying so. `--model` works here exactly as in a full
-run: on rotation data, where the [indexing ambiguity](#the-indexing-ambiguity) is one choice for
-the whole dataset, it can settle the enantiomorph label and the indexing — under the same fit test
-as always — and the re-merged reflections are written in what it settled. What no re-merge can
+declines it and the run stops with a message saying so. `--model` works here as in a full run, except
+that the model is placed by the rigid body only: a model it does not place, or a prediction, is not
+searched for by molecular replacement, and `--mr` is refused. On rotation data, where the
+[indexing ambiguity](#the-indexing-ambiguity) is one choice for the whole dataset, it can settle the
+enantiomorph label and the indexing — under the same fit test as always — and the re-merged reflections are written in what it settled. What no re-merge can
 repair is a **stills** `_process.h5` integrated without a reference: there the ambiguity was
 decided per image, at integration time, and the distinction is gone from the stored reflections.
 
@@ -294,7 +325,7 @@ General:
 | Option | Description |
 | --- | --- |
 | `-o, --output-prefix <txt>` | Output file prefix (default: `output`) |
-| `-N, --threads <num>` | Number of worker threads (default, and for any value ≤ 0: all hardware threads). Some stages take fewer, because past a point more workers make them slower: the per-image loop of `--mode mx` uses at most 16 per GPU unless `-N` was given a positive value, and first-pass spot finding and the beam-stop pre-scan have ceilings of their own that `-N` does not lift. Scaling, merging and the space-group search use the full count |
+| `-N, --threads <num>` | Number of worker threads (default, and for any value ≤ 0: every CPU the run may use - its affinity and cgroup quota). Some stages take fewer, because past a point more workers make them slower: the per-image loop of `--mode mx` uses at most 16 per GPU unless `-N` was given a positive value, and first-pass spot finding and the beam-stop pre-scan have ceilings of their own that `-N` does not lift. Scaling, merging and the space-group search use the full count |
 | `-s, --start-image <num>` | First image to process (default: 0) |
 | `-e, --end-image <num>` | Last image to process (default: all) |
 | `-t, --stride <num>` | Process every *n*-th image (default: 1) |
@@ -424,7 +455,8 @@ Scaling and merging:
 | `--scaling-iterations <num>` | Cap on the per-frame scaling iterations; the loop stops earlier once the scales settle, and a merge that hits the cap is flagged `SCALING_NOT_CONVERGED` (default: 100) |
 | `-z, --reference <file>` | Reference data of the same crystal form — an MTZ or a PDB structure-factor mmCIF (`-sf.cif`), gzipped or not, recognised by content; `--reference-mtz` is the same option: fixes the space group, its setting and the cell, resolves the [indexing ambiguity](#the-indexing-ambiguity), hands over the R-free set and reports CC<sub>ref</sub>. Not a scale anchor |
 | `--reference-column <label>` | Reference column to use, by MTZ label — an mmCIF under the labels `gemmi cif2mtz` gives it (default: auto — F-model, else IMEAN/I/…, else FP/FOBS/F) |
-| `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](#validating-against-a-model-rugnux---model)). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](#the-indexing-ambiguity) where no `-z` did. For serial stills given `-C` / `-S`, the model's structure factors become the per-image reference |
+| `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](#validating-against-a-model-rugnux---model)). An experimental model is placed as a rigid body, and by molecular replacement where the rigid body does not place it; a prediction (AlphaFold or ModelCIF) goes to molecular replacement straight away (`MODEL_PLACEMENT=`). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](#the-indexing-ambiguity) where no `-z` did; another space group or the twin hypothesis' subgroup it chooses is written beside the answer as `<prefix>_mr.mtz`/`.cif`. For serial stills given `-C` / `-S`, the model's structure factors become the per-image reference |
+| `--mr <file>` | As `--model`, but placed by molecular replacement straight away, without trying the rigid body first. `--mode mx` only; not with `--model` |
 | `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |
 | `--developer` | Write the full `<prefix>_report.txt`: the pipeline-internal keys and the long explanations the default report leaves out. Nothing is computed differently — the same report, rendered in full (see [The results report](RUGNUX_REPORT.md#the-developer-report)) |
 | `--finalist-ledger` | Report the full-resolution evidence for each space group the search considered, not only the one it adopted, and the twin-immune zone evidence of the adopted group's 2-folds (report-only; the decision is unchanged). Both are read on a merge of the observations in P1 that only this option makes, so it costs one extra merge; rotation only, and not with a fixed centred `-S` |

docs/RUGNUX_INSTALL.md

@@ -115,10 +115,15 @@ build has no CUDA there and needs nothing installed beyond Xcode and CMake.
 As with the rest of Jungfraujoch, **serious performance requires an NVIDIA GPU**. The CUDA build
 provides the GPU fast-feedback indexer (`ffbidx`) and the GPU FFT indexer (`fft`); without CUDA
 only the CPU `fftw` indexer is available. With a GPU present most of the per-image pipeline runs on
-the device — bitshuffle+LZ4 decompression, image preprocessing, azimuthal integration, spot finding,
-prediction and Bragg integration — as do the pre-scan (beam-stop mask, background beam-centre fit,
-spot widths, defective pixels), rotation scaling and merging with its correction surfaces, and the
-`--model` rigid-body placement, with CPU implementations as the fallback where there is no GPU. The choice is automatic;
+the device — bitshuffle+LZ4 decompression, miniCBF byte-offset decoding and marCCD widening, image
+preprocessing, azimuthal integration, spot finding, prediction and Bragg integration — as do the
+pre-scan (beam-stop mask, background beam-centre fit, spot widths, defective pixels; one accumulator
+per card), rotation scaling and merging from the ingest of the reflections onwards (sorting and the
+resolution cut included) with its correction surfaces, and `--model`: the structure factors, scale
+fits and rigid-body placement of the validation and the rotation and translation functions of
+molecular replacement. The CPU implementations are the path where there is no GPU, and the two agree
+to rounding rather than bit for bit (parts of the scaling and the model fits round differently
+on the card), so a GPU and a CPU run can differ in the last digits of the merged data. The choice is automatic;
 a CUDA build runs on the CPU when the GPU is hidden from it (`CUDA_VISIBLE_DEVICES= rugnux ...`).
 The thread count (`-N`) governs the CPU side of all of it.
 

docs/RUGNUX_INTEGRATION.md

@@ -274,8 +274,10 @@ intensities could not decide: which of several space groups the data are in.
 
 ## Molecular replacement with Phaser
 
-rugnux does not do molecular replacement, so Phaser is the next program for anyone who has a search
-model. Both CCP4 and phenix ship it — `phaser` and `phenix.phaser`, the same 2.8.3 build in the
+rugnux places a model by molecular replacement only to decide what the merged data cannot decide by
+themselves - the space group, the hand, twinning (`--model`, `--mr`; see
+[Validating against a model](RUGNUX_ADVANCED.md#validating-against-a-model-rugnux---model)) - so Phaser
+is still the next program for anyone solving a structure with a search model. Both CCP4 and phenix ship it — `phaser` and `phenix.phaser`, the same 2.8.3 build in the
 versions this was checked against — and either takes the merged `myrun.mtz` as it is written.
 
 **No `LABIN`, no label choices.** Phaser reads the cell, the space group and the resolution range

docs/RUGNUX_OVERVIEW.md

@@ -64,8 +64,10 @@ flags ([§10](CPU_DATA_ANALYSIS_INTEGRATION.md),
 
 **Write.** The merged `.mtz` / `.cif` / `.hkl`, the unmerged MTZ and the
 results report land next to each other ([Output files](RUGNUX_TUTORIAL.md#output-files)); with
-`--model`, validation runs first and the maps and the placed model are written too
-([§14](CPU_DATA_ANALYSIS_DECISIONS.md)).
+`--model`, validation runs first - the model placed as a rigid body, or by molecular replacement
+where that does not place it or the model is a prediction - and the maps and the placed model are
+written too, with `<prefix>_mr.mtz` where the model chooses another group or the twin hypothesis'
+subgroup ([§14](CPU_DATA_ANALYSIS_DECISIONS.md)).
 
 **Stills instead.** Serial data skip the two-pass machinery: each image is indexed independently
 (with the known-cell `ffbidx` indexer where a cell is given), partiality comes from a per-crystal

docs/RUGNUX_REPORT.md

@@ -977,7 +977,8 @@ nothing else.
 
 A `--model` is placed by molecular replacement where it has no placement of its own - a prediction - or
 where the rigid body did not place it (`MODEL_PLACEMENT=` above), and `--mr <file>` takes a model there
-straight away, without trying the rigid body first. The search's keys (`MR_*`) and its table are in
+straight away, without trying the rigid body first. It runs in a full `--mode mx` run, rotation and
+serial stills alike; `--mode scale` places a model by the rigid body only. The search's keys (`MR_*`) and its table are in
 `5. MODEL VALIDATION`. Where it places the model, the R-factors, the maps and CC(model, data) of that
 section are of its placement, validated against the merge it was chosen on with nothing left to decide
 (the maps `<prefix>_mr_*` where that merge is `<prefix>_mr`); `MODEL_FIT` stays the rigid body's. Where

docs/RUGNUX_TUTORIAL.md

@@ -290,6 +290,11 @@ Absorption is corrected only by the empirical surface a rotation run fits and ke
   free set as `<prefix>.mtz`, for refinement with that law. With `--model`, the maps come from
   whichever of the two merges the model fits better. See
   [The twin hypothesis](RUGNUX_REPORT.md#the-twin-hypothesis-twin_subgroup_).
+- `<prefix>_mr.mtz` / `.cif`, with `<prefix>_mr_model.pdb` / `.cif` — only with `--model` (or `--mr`),
+  where the model, placed as a rigid body or by molecular replacement, chooses another space group
+  with the answer's rotations or the twin hypothesis' subgroup: that merge, labelled with the group
+  the model chose, beside the run's own answer, which is left as it was. See
+  [Validating against a model](RUGNUX_ADVANCED.md#validating-against-a-model-rugnux---model).
 - `<prefix>_report.txt` — the **results report**: what the run determined, in a form both a person and
   a beamline script can read. Always written, next to the files above. See
   [The results report](RUGNUX_REPORT.md#the-results-report).

docs/index.rst

@@ -33,6 +33,7 @@ Jungfraujoch is distributed under the GPLv3 license.
    CPU_DATA_ANALYSIS_INDEXING
    CPU_DATA_ANALYSIS_INTEGRATION
    CPU_DATA_ANALYSIS_DECISIONS
+   GPU_DATA_ANALYSIS
 
 .. toctree::
    :maxdepth: 1

image_analysis/scale_merge/RotationScaleMerge.cpp

@@ -701,11 +701,15 @@ size_t RotationScaleMerge::DeviceBytes() const {
     const size_t bytes = BYTES_PER_OBS * static_cast<size_t>(n_partials_obs) + BYTES_PER_EVENT * events
                          + BYTES_PER_HKL * rawrun_start.size();
     // The resident ingest's own peak comes before any of that, over every reflection read and every run
-    // before the resolution cut: the observation arrays, the sort key and the integrated delta_phi of
-    // each (66 bytes) and 24 bytes of the sort's or the cut's working arrays beside them, plus sixty-four
-    // bytes per run at the cut (IngestSort).
-    constexpr size_t INGEST_BYTES_PER_REFLECTION = 90, INGEST_BYTES_PER_RUN = 64;
-    const size_t ingest = INGEST_BYTES_PER_REFLECTION * ingest_reflections + INGEST_BYTES_PER_RUN * ingest_runs;
+    // before the resolution cut, measured with cudaMemGetInfo. Beside the observation arrays, the sort key
+    // and the integrated delta_phi of each (66 bytes): the second radix sort holds the frame order, the
+    // permutation, both keys and cub's alternate key and value buffers, 102.6 bytes per reflection in
+    // all; the cut (IngestSort) 98 per reflection and 56 per run. The upload before them holds the 66
+    // bytes and the two 64 MB staging buffers.
+    constexpr size_t INGEST_BYTES_PER_REFLECTION = 103, INGEST_BYTES_PER_RUN = 58;
+    constexpr size_t INGEST_ARRAY_BYTES = 66, INGEST_STAGING_BYTES = size_t(160) << 20;
+    const size_t ingest = std::max(INGEST_BYTES_PER_REFLECTION * ingest_reflections + INGEST_BYTES_PER_RUN * ingest_runs,
+                                   INGEST_ARRAY_BYTES * ingest_reflections + INGEST_STAGING_BYTES);
     return std::max(bytes + bytes / 5, ingest);
 }
 

image_analysis/structure_refinement/ModelValidation.cpp

@@ -1381,7 +1381,7 @@ ModelValidationResult Validate(const std::vector<MergedReflection> &merged,
     }
 #ifdef JFJOCH_USE_CUDA
     } catch (const RigidBodyGPUFailure &) {
-        throw;   // not the null's failure but the device's: the whole validation runs again on the CPU
+        throw;   // not the null's failure but the device's: it ends the whole validation (ValidateAgainstModel)
 #endif
     } catch (const std::exception &e) {
         // The null could not be built, so no decision is licensed and none is taken - which is the

image_analysis/structure_refinement/MolecularReplacement.cpp

@@ -1603,6 +1603,7 @@ LikelihoodPeak LikelihoodTranslationZ(const LikelihoodSetup &setup, const std::v
 
 // --- translational NCS ---
 
+// Following Read, Adams & McCoy (2013) Acta Cryst. D69, 176-183
 TranslationalNcs DetectTranslationalNcs(const std::vector<MergedReflection> &merged, const gemmi::UnitCell &cell,
                                         const gemmi::SpaceGroup &sg) {
     constexpr double D_MIN = 5.0, D_MAX = 10.0, MIN_DISTANCE_A = 15.0, MIN_HEIGHT = 0.2;

rugnux/Rugnux.h

@@ -791,11 +791,6 @@ class Rugnux {
     // fed to the second pass's Bragg prediction. Empty outside the rotation two-pass.
     std::vector<float> prepass_mosaicity_;
 
-    // The two-pass geometry pre-pass's scaling engine: its card memory (RotationScaleMerge::DeviceBytes)
-    // and the reflections it read. A refined pass reckons the engines it is about to build from them,
-    // per reflection - the quality guard's merges the pre-pass frames again. Zero outside the two-pass.
-    size_t prepass_engine_bytes_ = 0, prepass_engine_reflections_ = 0;
-
     // Two-pass geometry pre-pass: the detector geometry the first pass ends on - beam_x, beam_y in px,
     // distance in mm, then rot1 and rot2 in rad - applied by Run() before the second pass. Empty when
     // the pre-pass committed no change (the second pass then reproduces the first).

rugnux/RugnuxScaleMerge.cpp

@@ -887,7 +887,7 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
             && !experiment_.GetRefineRotationWedgeInScaling() && !rot_ss.GetRotationWedgeForScaling().has_value();
     // With two GPUs or more the guard's merge has an engine of its own on the second card, and the
     // main engine does not wait for it: its ingest and first merge run beside the guard's merge. With
-    // one, the guard's engine goes beside the main one on the card where the two fit (see below). The
+    // one, the guard's merge runs first and the main engine waits for the card (see below). The
     // guard reads the outcomes only while it ingests - the first merge writes per-frame fields back
     // into them (mosaicity among them, which the ingest reads) - so that merge holds its write-back
     // until the guard has ingested, and nothing else waits. Neither merge reads anything the other
@@ -917,13 +917,15 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
         return ScaleMergeResult{std::move(r.merged), std::move(r.statistics), r.cc_half_before_corrections};
     };
     std::future<ScaleMergeResult> guard_ahead;
-    if (merge_first_part && guard_on_second_gpu && !(postrefine_probe_ && !geometry_prepass && postrefine_probe_only_))
+    if (merge_first_part && !(postrefine_probe_ && !geometry_prepass && postrefine_probe_only_))
         guard_ahead = std::async(std::launch::async, [&] {
-            pin_gpu(1);   // the card and its NUMA node, as an image-loop worker has them
+            if (guard_on_second_gpu)
+                pin_gpu(1);   // the card and its NUMA node, as an image-loop worker has them
             return merge_guard();
         });
     // The scaling engine's ingest reads the outcomes and nothing else either, so its host half runs
-    // beside the post-refinement too.
+    // beside the post-refinement too; with one GPU the device half waits for the guard to hand the
+    // card back.
     Logger engine_log = Logger::Buffered();
     std::future<void> engine_ahead;
     if (is_rotation && postrefine_probe_ && !geometry_prepass && !postrefine_probe_only_
@@ -1082,50 +1084,14 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
                         config_.nthreads, engine_log, config_.observation_dump_path);
             rsm->IngestHost();
         }
-        const int32_t cards = get_gpu_count();
-        constexpr size_t CARD_BYTES_RESERVED = size_t(3) << 30;
-        // With one card the guard's engine goes beside this one where the two fit the card's total,
-        // and otherwise has the card to itself first. Both are reckoned, per reflection, at what the
-        // pre-pass's engine took for the same frames' reflections: the guard merges those frames again,
-        // and this engine merges more of the same crystal, which adds observations in proportion and
-        // rocking events and distinct hkl at most in proportion. Judged on the total, as the
-        // all-observation arm below is, so that where the guard runs does not depend on the moment.
-        size_t guard_bytes = 0;
-        if (merge_first_part && !guard_ahead.valid()) {
-            const auto &outcomes = indexer->GetIntegrationOutcome();
-            double n_first = 0.0, n_all = 0.0;
-            for (size_t o = 0; o < outcomes.size(); ++o) {
-                n_all += static_cast<double>(outcomes[o].reflections.size());
-                if (o < static_cast<size_t>(prepass_images))
-                    n_first += static_cast<double>(outcomes[o].reflections.size());
-            }
-            const double per_reflection = prepass_engine_reflections_ > 0
-                    ? static_cast<double>(prepass_engine_bytes_) / static_cast<double>(prepass_engine_reflections_) : 0.0;
-            const auto guard_estimate = static_cast<size_t>(per_reflection * n_first);
-            const auto engine_estimate = static_cast<size_t>(per_reflection * n_all);
-            const size_t card_bytes = cards > 0 ? get_gpu_total_memory(0) : 0;
-            const bool beside = cards == 1 && per_reflection > 0.0
-                                && engine_estimate + guard_estimate + CARD_BYTES_RESERVED <= card_bytes;
-            if (cards == 1)
-                logger.Info("Two-pass: the quality guard's merge runs {} this pass's engine on the card ({:.1f} GB "
-                            "{} {:.1f} GB on a {:.1f} GB card)", beside ? "beside" : "before",
-                            guard_estimate / 1e9, beside ? "beside" : "and", engine_estimate / 1e9, card_bytes / 1e9);
-            if (beside) {
-                guard_bytes = guard_estimate;
-                guard_ahead = std::async(std::launch::async, [&] { return merge_guard(); });
-            } else {
-                guard_merge = merge_guard();
-                guard_log.ReplayInto(logger);
-            }
+        // With one card the guard's merge has the card to itself first.
+        if (merge_first_part && !guard_on_second_gpu) {
+            guard_merge = guard_ahead.valid() ? guard_ahead.get() : merge_guard();
+            guard_log.ReplayInto(logger);
         }
         rsm->IngestDevice();
         release_gather();
-        if (geometry_prepass) {
-            prepass_engine_bytes_ = rsm->DeviceBytes();
-            prepass_engine_reflections_ = 0;
-            for (const auto &io : indexer->GetIntegrationOutcome())
-                prepass_engine_reflections_ += io.reflections.size();
-        }
+        const int32_t cards = get_gpu_count();
         engine_log.ReplayInto(logger);
         rsm->SetLogger(logger);
         // On a card of its own from three cards up (card 0 has this engine, card 1 the quality guard's
@@ -1140,8 +1106,9 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
         if (!geometry_prepass && !experiment_.GetGemmiSpaceGroup().has_value() && rsm->GetSearchMinZeta() > 0.0
             && cards > 0) {
             const int32_t device = std::min(cards - 1, 2);
+            constexpr size_t CARD_BYTES_RESERVED = size_t(3) << 30;
             if (cards >= 3
-                || 2 * rsm->DeviceBytes() + guard_bytes + CARD_BYTES_RESERVED <= get_gpu_total_memory(device)) {
+                || 2 * rsm->DeviceBytes() + CARD_BYTES_RESERVED <= get_gpu_total_memory(device)) {
                 rsm_all = rsm->CloneIngest(device, all_arm_log);
                 rsm_all->SetSearchMinZeta(0.0);
                 rsm_all->SetExportScaledFulls(false);   // as scale_and_merge sets them for a search merge

tests/CalcBraggPredictionTest.cpp

@@ -525,6 +525,7 @@ TEST_CASE("BraggPredictionRot_ColumnRangeKeepsEveryReflection", "[portable]") {
 }
 
 #ifdef JFJOCH_USE_CUDA
+#include "../common/CUDAWrapper.h"
 #include "../image_analysis/bragg_prediction/BraggPredictionGPU.h"
 #include "../image_analysis/bragg_prediction/BraggPredictionRotGPU.h"
 #include "../image_analysis/bragg_prediction/BraggPredictionRot.h"
@@ -532,6 +533,8 @@ TEST_CASE("BraggPredictionRot_ColumnRangeKeepsEveryReflection", "[portable]") {
 #include <algorithm>
 
 TEST_CASE("BraggPredictionGPU") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     DiffractionExperiment experiment(DetJF4M());
     experiment.DetectorDistance_mm(100.0).BeamX_pxl(1500.0).BeamY_pxl(1000.0)
     .IncidentEnergy_keV(13.0);
@@ -568,6 +571,8 @@ TEST_CASE("BraggPredictionGPU") {
 }
 
 TEST_CASE("BraggPredictionGPU_Rot1_Rot2") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     DiffractionExperiment experiment(DetJF4M());
     experiment.DetectorDistance_mm(100.0).BeamX_pxl(1500.0).BeamY_pxl(1000.0)
     .PoniRot1_rad(2.0/180.0 * PI).PoniRot2_rad(3.0/180.0 * PI)
@@ -605,6 +610,8 @@ TEST_CASE("BraggPredictionGPU_Rot1_Rot2") {
 }
 
 TEST_CASE("BraggPredictionGPU_systematic_absences") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     DiffractionExperiment experiment(DetJF4M());
     experiment.DetectorDistance_mm(120.0).BeamX_pxl(1500.0).BeamY_pxl(1000.0)
         .IncidentEnergy_keV(12.0);
@@ -664,6 +671,8 @@ TEST_CASE("BraggPredictionGPU_systematic_absences") {
 }
 
 TEST_CASE("BraggPredictionGPU_backscattering") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     DiffractionExperiment experiment(DetJF9M());
     experiment.DetectorDistance_mm(120.0).BeamX_pxl(1500.0).BeamY_pxl(1000.0)
         .IncidentEnergy_keV(3.0/WVL_1A_IN_KEV);
@@ -691,6 +700,8 @@ TEST_CASE("BraggPredictionGPU_backscattering") {
 }
 
 TEST_CASE("BraggPrediction_CPU_GPU_consistency_tilted") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     // Verify CPU and GPU implementations produce identical results with tilted detector
     DiffractionExperiment experiment(DetJF4M());
     experiment.DetectorDistance_mm(100.0).BeamX_pxl(1500.0).BeamY_pxl(1000.0)
@@ -761,6 +772,8 @@ TEST_CASE("BraggPrediction_CPU_GPU_consistency_tilted") {
 }
 
 TEST_CASE("BraggPredictionRot_CPU_GPU_consistency_tilted") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     // The rotation counterpart of the test above. Same purpose: every per-reflection quantity the
     // two implementations both produce has to agree - the Lorentz-polarization factor and the sensor
     // efficiency each in their own field, so neither can hide behind the other in a product.
@@ -818,6 +831,8 @@ TEST_CASE("BraggPredictionRot_CPU_GPU_consistency_tilted") {
     CHECK(max_flight > 1.0f);
 }
 TEST_CASE("BraggPrediction_CPU_GPU_consistency_large_cell") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     // A frame predicting more than kPredictionCapacity: both paths grow their buffer, so both predict the
     // same reflections. The CPU used to stop at the capacity and return the first 20000 of the walk.
     const auto experiment = LargeCellExperiment();
@@ -863,6 +878,8 @@ TEST_CASE("BraggPrediction_CPU_GPU_consistency_large_cell") {
 }
 
 TEST_CASE("BraggPredictionRotGPU_DeviceOrderMatchesHost") {
+    if (get_gpu_count() == 0)
+        SKIP("No CUDA GPU present");
     // The GPU predictor orders and selects its reflections on the device. What it returns has to be what
     // OrderOutput and TruncateToOutput make of the same predictions on the host: the full list in hkl,
     // delta_phi order, and under a cap the best-recorded ones in TruncateToOutput's order.

tools/battery/README.md

@@ -244,7 +244,7 @@ R=build/rugnux/rugnux
 
 $B run --rugnux $R                                  # full battery: open + inhouse
 $B run --rugnux $R --tier smoke                     # 21 sets covering readers, symmetry, known hard cases
-$B run --rugnux $R --tier ci                        # 26 sets every recent release passed (CI, below), plus the wrong-model null
+$B run --rugnux $R --tier ci                        # 28 sets every recent release passed (CI, below), plus the wrong-model null
 $B nullmodel --rugnux $R                            # the wrong-model null on its own (CI's second step)
 $B run --rugnux $R --only 5reo,lyso_ref             # named sets
 $B run --rugnux $R --arm private                    # the private arm, on its own