v1.0.0-rc.160 (#70)

This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell.
* rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged.
* rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set.
* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme.
* rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free.
* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
* Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md.
* Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>

docs/CHANGELOG.md

@@ -1,14 +1,56 @@
 # Changelog
 ## 1.0.0
+### 1.0.0-rc.160
+This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
+
+* rugnux: Rotation **geometry post-refinement** is now on by default (`--rotation-no-postrefine` to disable; also a viewer checkbox). A first pass integrates at the header geometry, then the shared detector distance + beam centre and the crystal cell/goniometer-axis are post-refined over all frames (cross-validated, committed only for a small < 1 % move, with the gauge-weak beam centre restrained toward the header); a second pass re-indexes de novo and re-integrates at the refined geometry. The refined pass is the canonical `<prefix>_*` output; the header-geometry pass is kept as `<prefix>_01_*`.
+* rugnux: Optional per-batch **relative-B** correction for rotation (`--relative-b[=deg]`, default 10°-of-rotation batches when bare, off otherwise) - a cross-validated, curvature-smoothed resolution×dose correction beyond the single global decay slope.
+* rugnux: Always-on **radiation-damage report** for rotation - the per-image scale correlation-to-merge and mosaicity versus dose, plus the relative B-factor change over the run (first→last) as a scalar and a per-batch relative-B curve, printed to the log and written to the merged mmCIF. Report-only; it never alters the merge.
+* rugnux: De-novo space-group search ranks candidate lattice **centerings by net absences** (systematically-absent minus violating), not the gross absent count, fixing an over-centering of a genuinely C-centred lattice to F.
+* rugnux: Record the **producing software** (name and version) and the refined **detector distance and beam centre** in the merged mmCIF (and the software in the MTZ history).
+* Bragg integration: Carry the box-sum observed centroid through the profile-fit path, so the observed spot centroid is emitted in every integrator mode.
+* rugnux: Report **ISa** as the counting-subtracted strong-reflection asymptote, not `1/b` of the whole-range fit; it also sets the merged-sigma floor. CC1/2, R-meas and per-obs sigmas unchanged.
+* Frontend: Azimuthal-integration Q fields (Q spacing / Low Q / High Q) accept 5 decimals (was 3), matching the 1e-5 `q_spacing` minimum; number-field precision is now configurable.
+* rugnux: Add a dataset-wide **Wilson B-factor** estimate to the merged output (mmCIF, stats table, log); the per-image viewer Wilson B emits NaN for implausible fits.
+* rugnux: De-novo space-group search vetoes a merohedral-twin over-promotion whose systematic error-model `b` balloons past a calibrated bound (keeps R3 as R3, not R32).
+* rugnux: De-novo space-group search decides lattice **centering** from the strength (mean I/sigma) of the systematically-absent class, not a per-reflection violation count.
+* rugnux: Recover lattice **centering** on weak / low-energy data via a floor-independent test (rate of significant absent vs present reflections), fixing a missed I-centring at 5/13 keV.
+* rugnux: Report anomalous signal-to-noise **SigAno** = <|I(+)-I(-)|>/<sigma> per shell and overall (mmCIF PDBx items + stats-table column); anomalous merges only.
+* rugnux: De-novo space-group search recovers a genuine high-symmetry group on weak data with a broken sigma model by confirming on the systematic-`b` test alone (restores an F432 case).
+* rugnux: Print the adopted **space group and unit cell** as a one-line summary at the end of the run (de-novo or user-fixed `-S`).
+* rugnux: Score the radiation-damage **decay** cross-validation on a sigma-independent (R-meas-like) metric, so a spurious slope can't pass by reshaping sigmas.
+* rugnux: Fix de-novo rotation indexing committing a spurious axis-multiple supercell (collapsing to P1) via a cross-scheme smaller-cell tie-break on near-integer volume ratios.
+* rugnux: Widen refined-cell angle bounds to [30, 150] deg (rotation candidate and per-frame stills refinement); check refined angles against the reference cell.
+* rugnux: `-S`/`--space-group` now accepts a Hermann-Mauguin symbol (e.g. `P43212`) as well as a space-group number.
+* rugnux: Warn when the chosen cell/space group carries an indexing (merohedral) ambiguity needing a reference to resolve.
+* Indexing: Requesting the FFTW (CPU) indexer on a GPU node now fails with a clear, actionable message (rotation always uses the GPU FFT indexer there).
+* rugnux: Stills `--refine-geometry[=N|off]` - first-pass bundle-adjust of beam/distance/cell then re-index (default ON with a reference cell); accepts reference `F`/`FP` columns.
+* rugnux: Per-image geometry refinement `-r flex` tries all three algorithms per image and keeps the best (old name `multi` kept as an alias).
+* rugnux: Experimental stills partiality `--still-partiality` (Gaussian excitation-error) and `--partiality-uncertainty <num>` down-weighting the least-complete partials.
+* rugnux: Default the stills Bragg-integration box to r=6 (integration radii 6, 8, 12).
+* rugnux: Self-referenced stills scale in a single pass (fixing a weak-data collapse); a reference MTZ (`-z`) fixes SG/cell/ambiguity but never anchors the scale (stills and rotation).
+* rugnux: Cap normalised intensity (E^2) on second-lattice overlaps in the de-novo space-group search, so strong overlaps don't skew the symmetry decision.
+* rugnux: Fix `--scale` on a self-contained `_process.h5` (stored reflections and error model reload correctly).
+* rugnux: Add `--spot-low-resolution <num>` (default 50 A) and `--min-pix-per-spot <num>` (default 2) to tune spot finding on weak serial data.
+* jfjoch_viewer: Expose stills processing settings in the settings dock, rename geometry-refinement `multi` to `flex`, and refit the initial image on resize.
+* jfjoch_writer: Remove the CBF and TIFF image writers - only NXmx HDF5 is written (all three layouts remain).
+* Reader: Treat a negative `total_flux` in a stored dataset as unknown/absent rather than a valid flux.
+* Packaging: Build the self-contained Linux viewer against a static libdbus with glib disabled; add parallel image-build and in-container viewer-verification scripts.
+* rugnux: Write anomalous data as a standard CCP4 anomalous MTZ (one row per reflection: `IMEAN`, `I(+)`/`I(-)`, `F`/`F(+)`/`F(-)`), readable by aimless/mtz2sca/ANODE.
+* rugnux: Always write merged reflections as both `<prefix>.mtz` and `<prefix>.cif`; the `--scaling-output` selector and text `.hkl` output are removed.
+* rugnux: Add a detector-plane **modulation** (flat-field) correction surface to rotation scaling (cross-validated, on by default; `--no-scaling-corrections` disables all), dropping R-meas.
+* rugnux: Add optional **stills detector-plane modulation** (`--stills-modulation`, default off) - the same cross-validated surface for the on-the-fly stills path.
+* Bragg integration: Local background is now a **symmetric trimmed mean** of the ring (`--background-trim <f>`, default 0.10; monochromatic), improving <I/sigma> and resolution-edge CC1/2.
+
 ### 1.0.0-rc.159
 This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
 
 * rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell.
 * rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged.
 * rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set.
-* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme.
+* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal (which has no twin laws).
 * rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free.
-* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
+* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes a cubic case (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
 * Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md.
 * Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).
 

docs/CPU_DATA_ANALYSIS.md

@@ -352,6 +352,16 @@ with $R(\phi)$ constructed from the axis-angle representation of the goniometer
 
 Refinement is performed in stages with decreasing acceptance tolerance for including reflections (three stages, indexing tolerance $0.3\to0.2\to0.1$), which stabilizes convergence when starting from imperfect indexing and approximate geometry.
 
+### 7.5 Rotation geometry post-refinement (two-pass)
+
+The refinement above (§7.2) runs per image against that image's spots. For rotation data an additional **post-refinement** (on by default; `--rotation-no-postrefine` disables it) improves the detector distance, beam centre and crystal cell/axis using **all** frames at once, then re-integrates:
+
+1. **Pass 1** integrates, scales and merges at the header geometry.
+2. Against the merged fulls, the shared detector distance and beam centre and the crystal cell / goniometer-axis are refined jointly over all frames (Ceres, robust loss). The fit is **cross-validated** on a deterministic frame split — refined on one half, gated on the held-out half — and the geometry is **committed only for a small move** (a distance/beam shift under ~1 %); a larger move is treated as a fit instability and rejected. The beam centre is gauge-weak in a rotation series (it trades off against the crystal orientation), so it is restrained toward the header value.
+3. **Pass 2** re-indexes de novo and re-integrates at the committed geometry, reusing pass-1's space group for the merge only.
+
+The refined pass is written as the canonical `<prefix>_*` output; the pass-1 (header-geometry) result is kept alongside as `<prefix>_01_*` for comparison.
+
 ---
 
 ## 8. Reflection prediction
@@ -446,7 +456,9 @@ $
 $
 with a Poisson-like uncertainty $\sigma(\hat{I})=\max\!\big(1,\ r_\sigma\hat{I},\ \sqrt{S}\big)$, i.e. $\sqrt{S}$ floored both at 1 and at a small fraction $r_\sigma$ of the intensity. A reflection is accepted as “observed” only if all signal pixels were valid and $n_B$ exceeds a minimum. This box sum is the classical estimator; it is used directly with `--integrator boxsum`, and otherwise seeds the profile fit below.
 
-For the **profile-fit path on broadband (still) data**, the background mean is additionally computed with a single high-outlier reject (drop ring pixels above $\hat{b}+3\sqrt{\hat{b}}$, then recompute): a bandwidth-streaked high-resolution spot or a close neighbour can leak into the ring and bias the mean high, over-subtracting and driving weak high-resolution intensities negative. A clean Poisson background is essentially unchanged by the cut. The reject is **not** applied to plain box summation (`--integrator boxsum`) or to monochromatic/rotation data.
+**Trimmed-mean background (monochromatic, default on).** On monochromatic data — rotation *and* still (the discriminator is the beam, not the acquisition mode) — the ring mean $\hat{b}$ is by default replaced with a **symmetric trimmed mean**: the ring pixels are sorted, the lowest and highest fraction $f$ are dropped, and the central $(1-2f)$ are averaged ($f=0.10$ by default, `--background-trim`; $f=0$ restores the plain mean). Because $\hat{I}=S-n_S\hat{b}$ is a small difference of large numbers for weak reflections, a per-pixel background bias $\delta\hat{b}$ becomes a *fractional* intensity bias $\approx n_S\,\delta\hat{b}/\hat{I}$ that grows as $\hat{I}$ shrinks — worst at the resolution edge. The plain mean reads high there because neighbour-spot wings that survive the signal-disk mask, tails and zingers are one-sided (positive) contaminants; dropping the extreme ring pixels removes that bias while a clean Poisson ring is essentially unchanged. In practice this lowers the resolution-edge $R_\text{meas}$ several-fold, raises $\langle I/\sigma\rangle$, and rescues the high-resolution CC$_{1/2}$ on data where the plain mean had collapsed it (at a small CC$_{1/2}$ cost in already-clean shells, from the slightly higher variance of the trimmed estimate). It is applied to the shared background used by both the box sum and the profile fit.
+
+For the **profile-fit path on broadband (non-zero bandwidth: pink-beam / DMM) data**, the trimmed mean is *not* used; instead the background mean is computed with a single high-outlier reject (drop ring pixels above $\hat{b}+3\sqrt{\hat{b}}$, then recompute): a bandwidth-streaked high-resolution spot or a close neighbour can leak into the ring and bias the mean high, over-subtracting and driving weak high-resolution intensities negative. A clean Poisson background is essentially unchanged by the cut. Neither robustification is applied to plain box summation (`--integrator boxsum`).
 
 ### 9.3 Profile-fitted extraction (default)
 
@@ -512,7 +524,7 @@ The partiality applied is fixed by the data type and scaling stage, not chosen f
 
 2. **Unity** ($P_{ij}=1$): used for the scale-on-fulls refit (§10.6), where each observation is already a complete reflection.
 
-3. **Fixed**: use the per-reflection partiality carried from prediction. Still/serial images are predicted with $P=1$, so their scaling is effectively unity/fixed — there is no excitation-error still-partiality model.
+3. **Fixed**: use the per-reflection partiality carried from prediction. Still/serial images are predicted with $P=1$ by default, so their scaling is effectively unity/fixed. An optional excitation-error still-partiality model (`--still-partiality`) instead weights each stills reflection by a Gaussian $\exp(-\Delta_\mathrm{Ewald}^2/2\sigma^2)$ in its distance from the Ewald sphere, with a companion merge term (`--partiality-uncertainty`) that adds an intensity- and $(1-P)$-proportional sigma to down-weight the least-complete reflections.
 
 Reflections below a minimum partiality can be rejected from merging to avoid unstable corrections.
 
@@ -543,9 +555,12 @@ Per-shell and overall merging statistics are computed on corrected intensities,
 - mean $I/\sigma(I)$,
 - $R_\mathrm{meas}$ (the redundancy-independent Diederichs–Karplus form) from within‑HKL deviations,
 - $\mathrm{CC}_{1/2}$ (half-set correlation) and, when a reference dataset is supplied, $\mathrm{CC}_\mathrm{ref}$,
-- completeness against the enumerated reflections for the cell and symmetry.
+- completeness against the enumerated reflections for the cell and symmetry,
+- the anomalous signal-to-noise $\mathrm{SigAno}$ (below).
 
-The error model is refined as $\sigma_\mathrm{corr}^2 = a\,\sigma^2 + (b\,\langle I\rangle)^2$ with a systematic floor $\sigma\ge b|I|$; the asymptotic signal-to-noise $\mathrm{ISa}=1/b$ is reported and written to the output files.
+The error model is refined as $\sigma_\mathrm{corr}^2 = a\,\sigma^2 + (b\,\langle I\rangle)^2$, with $a$ set by the scatter of weak (counting-limited) reflections and $b$ the intensity-proportional systematic scatter of the strong ones. **ISa** is the asymptotic ($I\to\infty$) signal-to-noise — by definition the reproducibility limit of the strongest reflections (Diederichs, *Acta Cryst.* **D66** (2010) 733) — and is read directly from the strong symmetry equivalents as the counting-subtracted fractional scatter of well-measured reflection groups (a robust median over strong groups; the $I/\sigma$ threshold is relaxed on weak or radiation-damaged data that has few strong reflections), rather than as $1/b$ of the whole-range fit, whose $b$ is raised slightly by an intermediate-intensity excess and so understates the limit. The reported ISa and the merged-intensity systematic floor $\sigma \ge b_\mathrm{ISa}\,|I|$ both use this asymptotic value, so a high-multiplicity merged $I/\sigma$ approaches ISa; the per-observation $\sigma_\mathrm{corr}$ (the merge weights) uses the whole-range $a,b$ and is unchanged.
+
+**Anomalous signal-to-noise (SigAno).** The strength of the anomalous signal is reported per shell and overall as $\mathrm{SigAno}=\langle|\Delta I|\rangle / \langle\sigma(\Delta I)\rangle$, where $\Delta I = I(+)-I(-)$ over acentric reflections measured in both Bijvoet hands and $\sigma(\Delta I)=\sqrt{\sigma_+^2+\sigma_-^2}$. It is computed from the **full-multiplicity** inverse-variance $I(+)/I(-)$ split (the same one written to the output), i.e. from all observations rather than a half-set. For pure noise $\mathrm{SigAno}$ approaches the half-normal value $\sqrt{2/\pi}\approx0.8$, and it rises above $1$ once a real anomalous difference is present. A half-set anomalous correlation ("$\mathrm{CC}_\mathrm{anom}$") is deliberately **not** used: its two half estimates $\Delta I_0,\Delta I_1$ are complementary partitions of one observation pool ($\Delta I_0+\Delta I_1=2\,\Delta I_\mathrm{full}$), so subtracting the two Bijvoet hands cancels the large common intensity that keeps $\mathrm{CC}_{1/2}$ non-negative and leaves only the small anomalous signal against the per-half split noise; once the anomalous signal-to-noise per half drops below $1$ that correlation is driven towards $-1$ rather than $0$, misrepresenting a weak-but-real signal, whereas $\mathrm{SigAno}$ has no such floor. It is emitted only when an anomalous split was made, using the standard PDBx items `_reflns.pdbx_absDiff_over_sigma_anomalous` (overall) and `_reflns_shell.pdbx_absDiff_over_sigma_anomalous` (per shell), and appears as the `SigAno` column of the printed merge-statistics table.
 
 ### 10.6 Rotation datasets: combining partials into fulls (3D integration)
 
@@ -559,12 +574,15 @@ The combine groups each reflection's partials into rocking events (contiguous ru
 
 The fulls are then re-scaled in the XDS sense — a per-image scale refit directly on the complete reflections under the unity partiality model — and merged (§10.4). Because every merged observation is now a counting-statistics-limited full rather than a partiality-divided slice, the error model reaches a far higher asymptotic $I/\sigma$.
 
-After scale-fulls, two **optional correction surfaces** can be fitted on the combined fulls (rotation only, both **off by default**), each an alternating multiplicative refinement of the per-full scale against the merged reference:
+After scale-fulls, three **correction surfaces** are fitted on the combined fulls (rotation path, **on by default**; disable all with `--no-scaling-corrections`), each an alternating multiplicative refinement of the per-full scale against the merged reference:
 
-- **Decay** (`-B`). Radiation damage weakens later frames more at higher resolution — a resolution×time (Debye–Waller) systematic the resolution-flat per-image scale cannot capture. A single global relative-$B$ rate is fitted, $\ln(I_\mathrm{ref}/I_\mathrm{obs}) = 2\,(\mathrm{d}B/\mathrm{d}n)\,(n-\bar n)\,s^2$ (frame $n$, $s^2 = 1/4d^2$), and folded into the scale. It engages only when the total relative-$B$ over the run exceeds a physical floor (2 Ų); below that the decay is negligible and "correcting" it only spreads symmetry equivalents (which sit at the same $s^2$ but different frames).
-- **Absorption** (`--absorption`). A smooth multiplicative factor over the diffracted-beam direction expressed in the goniometer (crystal) frame: each full's predicted detector position gives the lab diffracted direction, de-rotated by the spindle so a fixed crystal-frame direction is sampled at many rotation angles and its grid cell is well-determined. Negligible at hard X-rays / thin crystals; it matters at low photon energy. Its gain is largest on model-based metrics — a smooth absorption error largely *cancels* among symmetry mates (small effect on the error model / ISa) but still biases the intensities from their true values (a measurable $R_\mathrm{free}$ improvement).
+- **Decay.** Radiation damage weakens later frames more at higher resolution — a resolution×time (Debye–Waller) systematic the resolution-flat per-image scale cannot capture. A single global relative-$B$ rate is fitted, $\ln(I_\mathrm{ref}/I_\mathrm{obs}) = 2\,(\mathrm{d}B/\mathrm{d}n)\,(n-\bar n)\,s^2$ (frame $n$, $s^2 = 1/4d^2$), and folded into the scale. It engages only when the total relative-$B$ over the run exceeds a physical floor (2 Ų); below that the decay is negligible and "correcting" it only spreads symmetry equivalents (same $s^2$, different frames). An optional **per-batch relative-$B$** (`--relative-b[=deg]`, off unless requested; 10°-of-rotation batches by default) extends the single global rate to a smooth $B(n)$ curve — the same $s^2$-weighted decay fit solved independently over short frame batches, curvature-penalized so it cannot over-fit and cross-validated like the surfaces below — for crystals whose decay is non-linear in dose.
+- **Absorption.** A smooth multiplicative factor over the diffracted-beam direction expressed in the goniometer (crystal) frame: each full's predicted detector position gives the lab diffracted direction, de-rotated by the spindle so a fixed crystal-frame direction is sampled at many rotation angles and its grid cell is well-determined. Negligible at hard X-rays / thin crystals; it matters at low photon energy.
+- **Modulation** (detector-plane flat-field). A smooth multiplicative factor over where each reflection lands on the detector (predicted $x,y$): symmetry-equivalents land at different positions as the crystal rotates, over-determining the surface. It absorbs detector-response and geometric systematics that inflate $R_\mathrm{meas}$. The same 16×16 detector-frame surface is available for the stills path (`--stills-modulation`, off by default), where serial data repeatedly hammers the same detector regions.
 
-Both surfaces are **cross-validated**: fitted on even-numbered frames and kept only if they improve the held-out odd-frame symmetry-equivalent agreement by a clear margin (and vice versa). A surface fitted to noise where its systematic is absent therefore does not generalize and is discarded — an opt-in correction never adds scatter.
+Each surface is **cross-validated**: fitted on even-numbered frames and kept only if it improves the held-out odd-frame agreement by a clear margin (and vice versa), scored by a **σ-independent, $R_\mathrm{meas}$-like** fractional agreement $\sum|I_s-I_\mathrm{ref}|/\sum|I_\mathrm{ref}|$ rather than a studentized $\chi^2$ — so a surface cannot "pass" by reshaping the sigmas instead of tightening the intensities. A surface fitted to noise where its systematic is absent does not generalize and is discarded — a correction never adds scatter.
+
+**Radiation-damage report (rotation, report-only).** Independently of whether any decay correction is applied, rugnux measures and reports the relative Debye–Waller $B$ across the sweep: the per-image scale's correlation to the merge and the per-image mosaicity versus frame (dose), together with a per-batch relative-$B$ curve whose first→last change is a single headline number (measured before any decay correction, against the least-damaged early wedge). It is written to the log and to the merged mmCIF as a data-quality-vs-dose diagnostic and **never** alters the merged intensities — distinct from the decay correction above, which does fold into the scale.
 
 ### 10.7 R-free test-set flags
 
@@ -604,7 +622,7 @@ A reference dataset (`--reference-mtz`) supplies known intensities for the same
 
 **Fix the space group and cell.** Unless overridden on the command line (`-S` for the space group, `-C` for the cell), the reference's space group is adopted and its cell is used as the soft reference cell — indexing may still drift the cell within tolerance, so a small mismatch between reference and data is absorbed rather than rejected. This applies to both stills and rotation data.
 
-**Resolve the indexing (merohedral) ambiguity.** When the lattice symmetry is higher than the crystal's Laue symmetry (e.g. $P3$, $P4$, $P6$, $C2$), more than one indexing of the same lattice is geometrically valid, and the two solutions produce *different* merged intensities that a self-consistent scale cannot tell apart — only an external reference can. The candidate reindexings are the identity together with the twin-law cosets of the metric symmetry (from the unit-cell metric and the Laue group); each is scored by the intensity correlation $\mathrm{CC}_\mathrm{ref}$ of the reindexed merge against the reference, and the data are re-merged in the best-correlating indexing. The reindex is **metric-preserving** — only the $hkl$ labels change, the cell is unchanged — and it is a no-op for a holohedral crystal, which has no twin laws (e.g. lysozyme, where the lattice and Laue symmetry coincide). For rotation data this is done once, after the space group is determined; the reference is *not* used to scale the rotation merge, which stays self-consistent (its $\mathrm{ISa}$ comes from the data alone). For stills the reference is the per-image scale target of the on-the-fly scaling (§10.2).
+**Resolve the indexing (merohedral) ambiguity.** When the lattice symmetry is higher than the crystal's Laue symmetry (e.g. $P3$, $P4$, $P6$, $C2$), more than one indexing of the same lattice is geometrically valid, and the two solutions produce *different* merged intensities that a self-consistent scale cannot tell apart — only an external reference can. The candidate reindexings are the identity together with the twin-law cosets of the metric symmetry (from the unit-cell metric and the Laue group); each is scored by the intensity correlation $\mathrm{CC}_\mathrm{ref}$ of the reindexed merge against the reference, and the data are re-merged in the best-correlating indexing. The reindex is **metric-preserving** — only the $hkl$ labels change, the cell is unchanged — and it is a no-op for a holohedral crystal, which has no twin laws (the lattice and Laue symmetry coincide). For rotation data this is done once, after the space group is determined; the reference is *not* used to scale the rotation merge, which stays self-consistent (its $\mathrm{ISa}$ comes from the data alone). For stills the reference is the per-image scale target of the on-the-fly scaling (§10.2).
 
 ---
 
@@ -646,6 +664,8 @@ $
 $
 A linear regression of $\log\langle I\rangle$ vs $1/d^2$ provides an estimate of $B$, subject to basic quality checks (e.g. $R^2$ threshold).
 
+A **dataset-wide** Wilson $B$ is also estimated over the merged reflections — restricted to the meaningful resolution range (skipping the low-resolution non-linear region below ~4 Å and shells past the signal limit $\langle I/\sigma\rangle < 1$, so it is insensitive to how far the merged data extend) — and written to the merged mmCIF as `_reflns.B_iso_Wilson_estimate`, the analogue of XDS's Wilson-line $B$. It is diagnostic only and is not fed back into scaling. The **per-image** estimate (used for the live radiation-damage plot) is accepted only when the fit is well-correlated and physically plausible ($0 < B < 200$ Ų); on a bad frame (an indexing glitch, too few reflections) the Wilson line runs wildly steep, so an implausible $B$ is reported as NaN rather than a spurious hundreds-of-Ų value.
+
 ---
 
 ## 13. Practical notes and limitations
@@ -654,7 +674,7 @@ A linear regression of $\log\langle I\rangle$ vs $1/d^2$ provides an estimate of
 - **Space-group symmetry** beyond centering absences is not necessarily enforced during prediction/integration unless the space group is supplied and used downstream.
 - **Resolution masking and ice rings** are controllable; including ice-ring spots in indexing can improve robustness for some samples but may bias refinement in others.
 - **Rotation vs still modes** differ substantially in prediction and scaling: partiality is angle-driven in rotation data, while stills are predicted (within an excitation-error window) and scaled with unit partiality.
-- **Space-group determination.** When no space group is supplied, a POINTLESS-like search scores Laue-group symmetry (CC of $I(h)$ vs $I(Rh)$ plus merge self-consistency) and detects screw/centering absences from the $P1$-merged intensities. The self-consistency test is calibrated so a merohedral twin — whose twin law forces non-equivalent reflections together and inflates the merged $\chi^2$ — stays in its true lower symmetry rather than being over-promoted to the holohedral group.
+- **Space-group determination.** When no space group is supplied, a POINTLESS-like search scores Laue-group symmetry (CC of $I(h)$ vs $I(Rh)$ plus merge self-consistency) and detects screw/centering absences from the $P1$-merged intensities. The self-consistency test is calibrated so a merohedral twin — whose twin law forces non-equivalent reflections together and inflates the merged $\chi^2$ — stays in its true lower symmetry rather than being over-promoted to the holohedral group. Because a partial twin's within-orbit $\chi^2$ can nonetheless look self-consistent, a chi²-passing promotion is additionally **vetoed** when merging its extra operator balloons the error-model $b$ (the intensity-proportional systematic) relative to the confirmed subgroup: a genuine symmetry step gains multiplicity without inflating $b$, whereas a twin forces non-equivalent reflections together and $b$ balloons. **Centering** is accepted when the systematically-absent class is weak relative to the present one by *either* of two floor-independent tests — its mean signed $I/\sigma$ well below the present mean, *or* its rate of individually-significant reflections well below the present class's own significant rate. The second test matters on weak / low-energy data, where a positive intensity floor (background/profile leakage) lifts the absent class's mean $I/\sigma$ to $\sim1.5$–$2.3$ instead of $\sim0$ and, when the present class is itself weak, inflates the plain mean ratio past its bound and hides a real centering (an $I$-centred cubic recorded at 5 keV was otherwise kept primitive); a false centering fails both tests because its absent class is as strong as the present one. When several centerings pass, they are ranked by their **net** systematic absences (absent minus violating), not the gross absent count, so a super-centering (e.g. $F$ over a true $C$) whose extra, only-half-populated absent class merely dilutes the strength ratio does not out-rank the correct lower centering.
 - **Twinning check.** A Padilla–Yeates $L$-test ($\langle|L|\rangle$, $\langle L^2\rangle$) and the second moment $\langle I^2\rangle/\langle I\rangle^2$ (taken per resolution shell with noise-only shells skipped and Wilson outliers rejected, so a single strong reflection in a collapsed-mean shell cannot skew it) are written to the merged mmCIF as a twinning diagnostic. Twinning is only flagged in Laue classes where a merohedral twin law can exist; the holohedral high-symmetry classes ($4/mmm$, $6/mmm$, $m\bar{3}m$, and $\bar{3}m$ on a rhombohedral lattice) are exempt, so a low $\langle|L|\rangle$ there is reported as a statistical artefact rather than twinning.
 - **Outlier rejection.** Merging applies an optional per-observation median-based $N\sigma$ cut (default 6σ for `rot3d`) and an optional per-crystal $\Delta\mathrm{CC}_{1/2}$ image rejection (`--reject-delta-cchalf`, CrystFEL-style, off by default). The same $N\sigma$ cut is fed back into the error model: after an initial $a,b$ fit the parameters are re-fit once on the reflections that survive rejection (dropping any whose squared deviation exceeds $N\sigma^2\,[a\,\sigma^2 + (b\,\langle I\rangle)^2]$), so the calibrated errors describe the reflections that actually enter the merge rather than the pre-rejection pool.
 - **Automatic resolution cutoff.** By default the reported/written high-resolution limit is trimmed where $\mathrm{CC}_{1/2}$ falls off (logistic, target 0.30); `--scaling-high-resolution` overrides it and `--resolution-cutoff off` disables it.
@@ -699,4 +719,4 @@ Two maps are formed with the model phases $\varphi_\mathrm{model}$: a $2F_o-F_c$
 The model fixes a definite hand and indexing, but the merged data need not share them, so before comparison the observed reflections are brought into the model's frame.
 
 - **Enantiomorph / screw.** When the data space group is the enantiomorph of the model's (e.g. data $P4_12_12$, model $P4_32_12$; or $P3_1/P3_2$), the two are **indistinguishable from merged intensities** — $|F_\mathrm{calc}|$ is invariant under the change of hand, so R-free cannot choose between them and probing would be meaningless. The hand is therefore taken from the model: the observed reflections are reindexed by the change-of-hand operator into the model's enantiomorph. Only the map phases (the density's hand) depend on this choice.
-- **Indexing (merohedral) ambiguity.** When the crystal has a merohedral ambiguity (§10.9), the observed intensities *do* differ between indexings, and the right one is chosen against the best available reference. **If a reference MTZ was supplied, the data were already reindexed to agree with it** (§10.9 — by the reference-intensity correlation, at the merge stage for rotation data or per image in stills scaling), and model validation keeps that authoritative choice. **Only with a model and no reference** does validation resolve the ambiguity itself, as a fallback: the scaled model is fit to each reindexing of the data (identity plus the twin-law cosets) and the one giving the **lowest R-free** is kept. This matters for a multi-dataset campaign — a single shared reference fixes one indexing convention for every dataset, whereas an independent per-dataset lowest-R-free choice could send borderline datasets to different conventions. A no-op either way for a holohedral crystal (no twin laws), e.g. lysozyme.
+- **Indexing (merohedral) ambiguity.** When the crystal has a merohedral ambiguity (§10.9), the observed intensities *do* differ between indexings, and the right one is chosen against the best available reference. **If a reference MTZ was supplied, the data were already reindexed to agree with it** (§10.9 — by the reference-intensity correlation, at the merge stage for rotation data or per image in stills scaling), and model validation keeps that authoritative choice. **Only with a model and no reference** does validation resolve the ambiguity itself, as a fallback: the scaled model is fit to each reindexing of the data (identity plus the twin-law cosets) and the one giving the **lowest R-free** is kept. This matters for a multi-dataset campaign — a single shared reference fixes one indexing convention for every dataset, whereas an independent per-dataset lowest-R-free choice could send borderline datasets to different conventions. A no-op either way for a holohedral crystal (no twin laws).

docs/HDF5.md

@@ -8,7 +8,7 @@ extra entries do not exist in NXmx and are documented here so that the layout is
 reusable.
 
 This page documents the **file layout and the data fields**. The operational behaviour of the
-writer (running, republishing, file finalisation, CBF/TIFF output) is described in
+writer (running, republishing, file finalisation) is described in
 [jfjoch_writer](JFJOCH_WRITER.md). The wire format that feeds the writer is described in
 [CBOR messages](CBOR.md); fields below frequently correspond one-to-one to CBOR message fields, and
 that document is a useful companion for their meaning.

docs/IMAGE_STREAM.md

@@ -188,7 +188,7 @@ For every image stream socket, downstream code must send the following message t
 ```json
 {
   "run_number":135,
-  "run_name": "lysozyme_1",
+  "run_name": "sample_1",
   "socket_number": 1,
   "processed_images":250,
   "ok": true
@@ -201,7 +201,7 @@ If not, it is possible to include error message:
 ```json
 {
   "run_number":135,
-  "run_name": "lysozyme_1",
+  "run_name": "sample_1",
   "socket_number": 1,
   "processed_images": 0,
   "ok": false,

docs/JFJOCH_WRITER.md

@@ -108,8 +108,8 @@ For example `header_appendix` of `{"param1": "test1", "param2": ["test1", "test2
   "filename": "dataset_name_data_000001.h5",
   "nimages": 1000,
   "file_number": 0,
-  "sample_name": "lysozyme",
-  "run_name": "lyso_cryo",
+  "sample_name": "my_sample",
+  "run_name": "my_run",
   "run_number": 25,
   "experiment_group": "p00001",
   "beam_x_pxl": 1200,
@@ -153,11 +153,10 @@ If data collection was configured with a `header_appendix` containing a key `hdf
 JSON object of numbers and strings, those entries are written to `/entry/user`.
 
 ## Other formats (CBF and TIFF)
-In addition to HDF5 format, Jungfraujoch allows to save images in the Crystallographic Binary File (CBF) format. 
-CBF files are written according to miniCBF format, with only basic header, and always with 32-bit signed integer format. 
-Dynamic range is reduced to max 2^24, negative numbers are zeroed, and masked, and/or bad pixels are set to -1. 
-
-Also writing to TIFF files is possible, though no metadata are saved in this case.
+Earlier versions could also write Crystallographic Binary File (CBF, miniCBF) and TIFF images. These
+writers have been removed: Jungfraujoch now writes only NXmx HDF5. The `CBF` and `TIFF` values are
+retained in the file-format enum for wire back-compatibility, but a request to write either format
+is rejected.
 
 ## No file option(s)
 There are two options to disable writing of files by the writer:

docs/RUGNUX.md

@@ -44,11 +44,21 @@ the first pass.
   integration, azimuthal integration, per-image statistics). See
   [HDF5 / NeXus data format](HDF5.md) for the layout. Written by default only when **not** merging
   (i.e. under `--no-merge`); add `--write-process-h5` to also write it when merging.
-- Merging is **on by default** (`--no-merge` disables it). The merged reflections are written as
-  `<prefix>.cif` (mmCIF — the default), or `<prefix>.mtz` / `<prefix>.hkl` depending on
-  `--scaling-output`. Both the mmCIF and the MTZ carry the **refined unit cell** (from rotation
-  indexing) and the **space group determined from systematic absences** (constrained to the indexed
-  lattice symmetry). No-reference scaling additionally emits per-iteration `<prefix>_iterN_scale.dat`.
+- Merging is **on by default** (`--no-merge` disables it). The merged reflections are written in
+  **three** formats — each has its uses downstream:
+  - `<prefix>.mtz` — CCP4 MTZ (`IMEAN`/`I(+)`/`I(-)`, French–Wilson `F`, `FreeR_flag`) for the CCP4 /
+    phenix reflection tools.
+  - `<prefix>.cif` — mmCIF, for deposition and as the self-describing native format (also carries the
+    merging statistics, ISa, twinning and radiation-damage indicators).
+  - `<prefix>.hkl` — SHELX **HKLF 4** text (`h k l I σ(I)`, fixed `3I4,2F8.2`), the direct input for
+    **SHELXC / ANODE / SHELXD**. Bijvoet mates are written separately (`I(+)` at `+hkl`, `I(-)` at
+    `-hkl`) so the anomalous signal is preserved; intensities are put on a common scale so the largest
+    value fits the fixed-width field (the absolute scale is irrelevant to SHELXC/ANODE), and the file
+    ends with the `0 0 0` terminator record.
+
+  All three carry the **refined unit cell** (from rotation indexing) and the **space group determined
+  from systematic absences** (constrained to the indexed lattice symmetry). No-reference scaling
+  additionally emits per-iteration `<prefix>_iterN_scale.dat`.
 
 Merged statistics (⟨I/σ⟩, CC1/2, completeness, …), the error model and timing are printed to the
 console. By default the written resolution is trimmed automatically where CC1/2 falls off
@@ -71,7 +81,7 @@ Index, integrate, scale and merge a rotation sweep, fully de novo:
 
 ```
 rugnux rotation_master.h5 \
-    -o lyso_rot -N 32 \
+    -o rotation_run -N 32 \
     --scaling-high-resolution 1.4
 ```
 
@@ -89,22 +99,46 @@ resolution) sharpens both the space-group search and the error model. To tune th
 `--two-pass-rotation=100` (or `-R100` — the first-pass image count); to force the sweep to be
 treated as independent stills use `--force-still`.
 
-After the per-frame scale-fulls step, rotation scaling applies two **correction surfaces**, **on by
-default** (`--no-scaling-corrections` disables both):
+By default a rotation run also **post-refines the geometry** in a second pass: the first pass
+integrates and merges at the header geometry, then the detector distance + beam centre and the crystal
+cell / rotation-axis are refined against the merged fulls (cross-validated, and committed only for a
+small < 1 % move, with the gauge-weak beam centre restrained toward the header), and the second pass
+re-indexes de novo and re-integrates at the refined geometry. The refined pass is the canonical
+`<prefix>_*` output; the header-geometry pass is kept alongside as `<prefix>_01_*` for comparison.
+Disable it with `--rotation-no-postrefine`.
+
+After the per-frame scale-fulls step, rotation scaling applies three **correction surfaces**, **on by
+default** (`--no-scaling-corrections` disables all):
 
 - **Decay** — a global Debye–Waller relative-*B* over the run, for the radiation damage that weakens
   later frames more at high resolution (a resolution×time systematic the resolution-flat per-frame
-  scale cannot remove). It only engages when the total relative-*B* exceeds a physical floor (2 Ų).
+  scale cannot remove). It only engages when the total relative-*B* exceeds a physical floor (2 Ų). An
+  optional `--relative-b[=deg]` extends this single global rate to a smooth per-batch relative-*B* curve
+  (default 10°-of-rotation batches when bare, off otherwise), cross-validated like the surfaces here, for
+  crystals whose decay is non-linear in dose.
 - **Absorption** — a smooth multiplicative factor over the diffracted-beam direction in the goniometer
   frame (path length through the crystal). Negligible at hard X-rays / thin crystals; it matters at
   low photon energy. Its benefit shows up most on model-based metrics: a smooth absorption error
   largely cancels among symmetry mates (little effect on the error model / ISa) but still biases the
   intensities, so it measurably lowers *R*<sub>free</sub>.
+- **Modulation** — a smooth multiplicative factor over the position where a reflection lands on the
+  detector (a flat-field: detector-response and geometric systematics that vary across the detector
+  plane). Symmetry-equivalents of one reflection land at different detector positions as the crystal
+  rotates, which over-determines the surface. Because it lives in the detector frame (not the
+  rotation) the same correction concept applies to stills. This is the largest of the three on
+  JUNGFRAU data — it lowers *R*<sub>meas</sub> by several to tens of percent on datasets that carry a
+  detector systematic, while holding or improving CC<sub>1/2</sub> and the anomalous signal.
 
-Both are **cross-validated** — fitted on even-numbered frames and kept only if they improve the
-held-out odd-frame symmetry-equivalent agreement by a clear margin (and vice versa) — so where the
-systematic is absent they are a no-op rather than a source of added noise; that is why they are safe
-to leave on. They run on the GPU when one is present, at negligible cost.
+All three are **cross-validated** — fitted on even-numbered frames and kept only if they improve the
+held-out odd-frame symmetry-equivalent agreement by a clear margin (and vice versa). The agreement is
+scored as a σ-independent, *R*<sub>meas</sub>-like fractional deviation, so a surface can never pass
+cross-validation by merely reshaping the sigmas; where the systematic is absent the surface is a no-op
+rather than a source of added noise, which is why they are safe to leave on.
+
+Independently of any correction, a rotation run prints a **radiation-damage report** — the per-image
+scale correlation-to-merge and mosaicity versus dose, and the relative *B*-factor change over the run
+(first→last) together with a per-batch relative-*B* curve, also written to the merged mmCIF. It is a
+data-quality-vs-dose diagnostic and never alters the merged intensities.
 
 ### Still / serial data
 
@@ -114,7 +148,7 @@ then merge against a reference structure:
 
 ```
 rugnux serial_master.h5 \
-    -o lyso_serial -N 32 \
+    -o serial_run -N 32 \
     -X ffbidx -C 79,79,38,90,90,90 -S 96 \
     --spot-sigma 4 \
     -z reference.mtz \
@@ -153,6 +187,8 @@ Spot finding:
 | `--spot-sigma <num>` | Noise sigma level for spot finding (default: 3.0) |
 | `--spot-threshold <num>` | Photon-count threshold for spot finding (default: 10) |
 | `--spot-high-resolution <num>` | High-resolution limit for spot finding, Å (default: 1.5) |
+| `--spot-low-resolution <num>` | Low-resolution limit for spot finding, Å (default: 50; lower it, e.g. 24, to exclude the direct-beam halo on weak serial data) |
+| `--min-pix-per-spot <num>` | Minimum connected strong pixels per spot (default: 2; serial data can index better with 1 and a higher `--spot-threshold`) |
 | `--max-spots <num>` | Maximum spot count (default: 250) |
 | `--detect-ice-rings[=on\|off]` | Flag ice-ring spots (de-prioritised in indexing) and exclude ice-ring reflections from scaling/merging; overrides the dataset/master-file setting (default: use the dataset value) |
 
@@ -179,12 +215,14 @@ rotation explicitly and pick the pass or lattice.
 | `--force-still` | Treat a rotation (goniometer) dataset as independent stills instead of rotation |
 | `-X, --indexing-algorithm <txt>` | `FFBIDX` \| `FFT` \| `FFTW` \| `Auto` \| `None` |
 | `-C, --unit-cell <cell>` | Reference unit cell `"a,b,c,alpha,beta,gamma"` (required by `ffbidx`) |
-| `-S, --space-group <num>` | Space group number (used for indexing and scaling) |
-| `-r, --refine <txt>` | Geometry refinement: `none` \| `orientation` \| `beam_and_lattice` (default) |
+| `-S, --space-group <num\|symbol>` | Space group number (`92`) or Hermann-Mauguin symbol (`P43212`) — for indexing and scaling |
+| `-r, --refine <txt>` | Geometry refinement: `none` \| `orientation` \| `beam_and_lattice` (default) \| `flex` (try all three per image, keep whichever indexes the most spots; alias `multi`) |
 | `-R, --two-pass-rotation[=num]` | Two-pass offline rotation indexing (default for goniometer data; optional first-pass image count, default 100) |
 | `--single-pass-rotation[=num]` | Online-like single-pass rotation indexing (optional min angular range, deg) |
 | `--redo-rotation-spots` | Redo spot finding for the two-pass rotation first pass |
 | `--force-rotation-lattice <vec>` | Force rotation lattice (9 floats, Å), skipping the first pass |
+| `--rotation-no-postrefine` | Rotation: disable the default-on two-pass geometry post-refine (see the rotation section) |
+| `--refine-geometry[=N\|off]` | Stills: extra first pass that bundle-adjusts the shared beam/distance/cell from N strongly-indexed frames (default 200) then re-indexes; default ON for stills with a reference cell (`-C` / `-z`), `=off` disables |
 
 Indexer choice in brief: `ffbidx` (GPU) refines toward a **known cell** and is best for sparse
 serial stills; `fft` (GPU) / `fftw` (CPU) index **de novo** and suit strong rotation data. See the
@@ -199,7 +237,11 @@ Scaling and merging:
 | `-B, --refine-bfactor` | Refine a per-image B-factor (stills only) |
 | `--scale-fulls` / `--no-scale-fulls` | rot3d: refit a per-frame scale on the combined fulls (XDS order, Unity model); on by default for rotation data, off for stills |
 | `--smooth-g[=deg]` | rot3d: smooth the per-frame scale *G* over a degree range before the 3D combine (XDS DELPHI-like; default 5° for rotation, 0 = off) |
-| `--no-scaling-corrections` | rot3d: disable the default-on decay + absorption correction surfaces fitted on the fulls after scale-fulls (see below) |
+| `--no-scaling-corrections` | rot3d: disable the default-on decay + absorption + modulation correction surfaces fitted on the fulls after scale-fulls (see below) |
+| `--relative-b[=deg]` | rot3d: fit a per-batch relative-*B* beyond the single decay slope over deg-degree batches, cross-validated (default 10° when bare; off otherwise) |
+| `--still-partiality` | Experimental (stills): weight reflections by a Gaussian excitation-error partiality instead of treating each as a full |
+| `--partiality-uncertainty <num>` | Stills: extra merge sigma ~num·(1−partiality)·⟨I⟩ on partials (use with `--still-partiality`; default 0, ~2.5 recommended) |
+| `--stills-modulation` | Experimental (stills): fit a detector-plane modulation (flat-field) surface, cross-validated (default off) |
 | `--capture-uncertainty <num>` | rot3d: systematic sigma on under-captured fulls, ~num·(1−captured_fraction)·I (default: 1.0 for rotation, 0 otherwise) |
 | `--min-captured-fraction <num>` | rot3d: drop a combined full whose rocking curve was captured below this fraction — edge-of-sweep truncated fulls (default: 0.7 for rotation, 0 otherwise; 0 = off) |
 | `--scaling-high-resolution <num>` | High-resolution limit for scaling, Å — manual override (default: no limit; disables the automatic cutoff below) |
@@ -212,7 +254,6 @@ Scaling and merging:
 | `--min-image-cc <num>` | Per-image CC limit, percent (default: no limit) |
 | `--mosaicity <num>` | Diagnostic: fix the scaling mosaicity (°) instead of using the per-image seed |
 | `--scaling-iterations <num>` | Scaling iterations with no reference data (default: 3) |
-| `--scaling-output <txt>` | Reflection output format: `cif` (mmCIF, default) \| `mtz` \| `txt` |
 | `-z, --reference-mtz <file>` | Reference MTZ (enables reference-driven scaling) |
 | `--reference-column <label>` | Reference MTZ column to use (default: auto — F-model, else IMEAN/I/…) |
 | `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |
@@ -223,6 +264,7 @@ Integration:
 | --- | --- |
 | `--integrator <txt>` | Spot integrator: `gaussian` (profile-fit, default) \| `empirical` \| `boxsum` (classical fallback) |
 | `--integration-radius <r>` | Signal-box radius `r1`, or `r1,r2,r3` (px). One value ⇒ `r2=r1+2`, `r3=r1+4` |
+| `--background-trim <f>` | Monochromatic (rotation + still): symmetric trimmed-mean fraction for the background ring, 0≤f<0.5 (default 0.10; 0 = plain mean) — removes the high-side bias that over-subtracts weak high-angle spots |
 | `--bandwidth <num>` | Relative X-ray bandwidth FWHM (e.g. `0.01` for a 1% DMM); default from file or 0 (monochromatic) |
 
 Geometry overrides (defaults are taken from the input file; override them to reprocess with a corrected geometry):

docs/conf.py

@@ -9,7 +9,7 @@
 project = 'Jungfraujoch'
 copyright = '2024, Paul Scherrer Institute'
 author = 'Filip Leonarski'
-release = '1.0.0-rc.159'
+release = '1.0.0-rc.160'
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

docs/python_client/README.md

@@ -22,8 +22,8 @@ communicate through network calls or other mechanisms.
 
 This Python package is automatically generated by the [OpenAPI Generator](https://openapi-generator.tech) project:
 
-- API version: 1.0.0-rc.159
-- Package version: 1.0.0-rc.159
+- API version: 1.0.0-rc.160
+- Package version: 1.0.0-rc.160
 - Generator version: 7.20.0
 - Build package: org.openapitools.codegen.languages.PythonClientCodegen
 

docs/python_client/docs/FileWriterFormat.md

@@ -1,6 +1,6 @@
 # FileWriterFormat
 
-NoFileWritten - no files are written at all NXmxOnlyData - only data files are written, no master file NXmxLegacy - legacy format with soft links to data files in the master file; necessary for DECTRIS Albula 4.0 and DECTRIS Neggia   NXmxVDS - newer format with virtual dataset linking data files in the master file, also includes better metadata handling NXmxIntegrated - single HDF5 per dataset CBF - CBF format (limited metadata) TIFF - TIFF format (no metadata) 
+NoFileWritten - no files are written at all NXmxOnlyData - only data files are written, no master file NXmxLegacy - legacy format with soft links to data files in the master file; necessary for DECTRIS Albula 4.0 and DECTRIS Neggia NXmxVDS - newer format with virtual dataset linking data files in the master file, also includes better metadata handling NXmxIntegrated - single HDF5 per dataset CBF - DEPRECATED, no longer supported; kept for back compatibility only. Requests using this value are rejected. Only HDF5 formats are written. TIFF - DEPRECATED, no longer supported; kept for back compatibility only. Requests using this value are rejected. Only HDF5 formats are written. 
 
 ## Enum
 

docs/python_client/docs/GeomRefinementAlgorithm.md

@@ -1,6 +1,6 @@
 # GeomRefinementAlgorithm
 
-Selection of an post-indexing least-square diffraction geometry refinement algorithm used by Jungfraujoch. BeamCenter - This option is refining both beam center and lattice (restricted to a chosen/detected Bravais lattice). OrientationOnly - This option is refining only orientation of the lattice. 
+Selection of an post-indexing least-square diffraction geometry refinement algorithm used by Jungfraujoch. BeamCenter - This option is refining both beam center and lattice (restricted to a chosen/detected Bravais lattice). OrientationOnly - This option is refining only orientation of the lattice. Flex - Tries all per-image refinements and keeps whichever indexes the most spots, letting the pipeline decide (the rugnux flex mode). 
 
 ## Enum
 
@@ -8,6 +8,8 @@ Selection of an post-indexing least-square diffraction geometry refinement algor
 
 * `ORIENTATIONONLY` (value: `'OrientationOnly'`)
 
+* `FLEX` (value: `'Flex'`)
+
 * `NONE` (value: `'None'`)
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)