diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 1af6797c..99720192 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -5,6 +5,7 @@ This is an UNSTABLE release. It includes many experimental features, as well as * The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one, so a module built while a kernel update is being applied loads after the reboot. * The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents, where the `vm_flags` kernel interface was backported into the 5.14 kernel. +* rugnux: `--mode scale` reports the Wilson B-factor estimate instead of `WILSON_B= nan`. * rugnux: `--export-unmerged` writes the integrated observations as `_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike; each rotation reflection's partials are summed into one full, and `--export-unmerged-partials` writes one row per image instead. Lattice-centring absences are not written; screw and glide absences are. ### 1.0.0-rc.163 diff --git a/docs/CPU_DATA_ANALYSIS.md b/docs/CPU_DATA_ANALYSIS.md index 7f74ada2..675d74b0 100644 --- a/docs/CPU_DATA_ANALYSIS.md +++ b/docs/CPU_DATA_ANALYSIS.md @@ -664,7 +664,9 @@ The width is the bandwidth streak alone, and deliberately **not** the profile's Growing the ring also grows the neighbour exclusion, so on a crowded pattern fewer background pixels survive; a reflection left with too few is rejected outright. On the data this was measured on the loss is under 0.1% of reflections, but it is not structurally zero. -Only the ring moves. The signal disk $r_1$ stays circular, deliberately: it sets $n_S$ and it sets $\mathrm{var}(\hat b)$, and with `--integrator boxsum` it drives the all-or-nothing “every signal pixel valid” acceptance gate (§9.2), so growing it would reject any box sum carrying a single bad pixel anywhere along a long streak. What a circular $r_1$ loses is flux, and that loss is a function of resolution alone, which the per-shell scale absorbs. +Only the ring moves. The signal disk $r_1$ stays circular, deliberately: it sets $n_S$, it sets $\mathrm{var}(\hat b)$, it is the domain the profile *width* is learned over (§9.3), and with `--integrator boxsum` it drives the all-or-nothing “every signal pixel valid” acceptance gate (§9.2), so growing it would reject any box sum carrying a single bad pixel anywhere along a long streak. In the default `gaussian` mode $r_1$ does not set the intensity at all — the fit grid, $\lceil r_2\rceil$, does. + +What a circular $r_1$ loses is flux, and that loss is **not** a function of resolution alone: measured per reflection, it carries a directional component worth several Ų with a definite principal axis, on top of the isotropic part. Nor is there anything in the merge to absorb it. There is **no per-shell scale**, and there cannot usefully be one: every scale in §10 is fitted against a reference built from a reflection's own symmetry equivalents, and equivalents share $s^2$ exactly, so any function of $s^2$ lies in the exact null space of the whole scaling model — a per-shell parameter would have zero residual to fit against. (XDS and DIALS have the same null space, for the same reason.) The isotropic part of the loss is instead degenerate with the overall Wilson $B$ and is silently reported as part of it, so **the reported `WILSON_B` / `_reflns.B_iso_Wilson_estimate` carries an $r_1$-dependent contribution**: measured across a constant-ring-area radius sweep it falls monotonically as the disk grows, by 0.5 Ų on sharp strong data and by up to ~10 Ų on weak wide-spot data. What this costs the *data* is much less than what it costs the flux, because most of the loss is matched by a proportional $\sigma$: it moves no CC$_{1/2}$ and no $R_\text{meas}$, and — to within a few hundredths of an ångström — no resolution cut. ### 9.2 Box summation (seed and fallback) @@ -923,7 +925,7 @@ $ $ A linear regression of $\log\langle I\rangle$ vs $1/d^2$ provides an estimate of $B$, subject to basic quality checks (e.g. $R^2$ threshold). -A **dataset-wide** Wilson $B$ is also estimated over the merged reflections — restricted to the meaningful resolution range (skipping the low-resolution non-linear region below ~4 Å and shells past the signal limit $\langle I/\sigma\rangle < 1$, so it is insensitive to how far the merged data extend) — and written to the merged mmCIF as `_reflns.B_iso_Wilson_estimate`, the analogue of XDS's Wilson-line $B$. It is diagnostic only and is not fed back into scaling. The **per-image** estimate (used for the live radiation-damage plot) is accepted only when the fit is well-correlated and physically plausible ($0 < B < 200$ Ų); on a bad frame (an indexing glitch, too few reflections) the Wilson line runs wildly steep, so an implausible $B$ is reported as NaN rather than a spurious hundreds-of-Ų value. +A **dataset-wide** Wilson $B$ is also estimated over the merged reflections — restricted to the meaningful resolution range (skipping the low-resolution non-linear region below ~4 Å and shells past the signal limit $\langle I/\sigma\rangle < 1$, so it is insensitive to how far the merged data extend) — and written to the merged mmCIF as `_reflns.B_iso_Wilson_estimate` (and reported as `WILSON_B=`), the analogue of XDS's Wilson-line $B$. It is diagnostic only and is not fed back into scaling. It is also where the flux the fixed integration disk clips lands: that loss is degenerate with an overall $B$, so the reported number carries an $r_1$-dependent contribution and is not a property of the crystal alone (§9.1). The **per-image** estimate (used for the live radiation-damage plot) is accepted only when the fit is well-correlated and physically plausible ($0 < B < 200$ Ų); on a bad frame (an indexing glitch, too few reflections) the Wilson line runs wildly steep, so an implausible $B$ is reported as NaN rather than a spurious hundreds-of-Ų value. --- diff --git a/image_analysis/bragg_integration/BraggStencil.h b/image_analysis/bragg_integration/BraggStencil.h index f87f859d..d85866f2 100644 --- a/image_analysis/bragg_integration/BraggStencil.h +++ b/image_analysis/bragg_integration/BraggStencil.h @@ -14,10 +14,21 @@ // reflection's own tails and measures them as background. // // So the RING is an ellipse, elongated along the beam->reflection direction by k_sigma times that -// streak, and left alone tangentially. The SIGNAL disk stays a circle: r1 governs n_inner, var_bkg -// and the all-or-nothing `n_inner_valid == n_inner` acceptance gate, so growing it rejects every -// reflection with one bad pixel anywhere along a long streak. The flux it loses is a function of -// resolution alone and the per-shell scale absorbs it. +// streak, and left alone tangentially. The SIGNAL disk stays a circle: r1 governs n_inner, var_bkg, +// the box-sum acceptance gate (all-or-nothing `n_inner_valid == n_inner`, so growing it rejects every +// reflection with one bad pixel anywhere along a long streak) and the domain the profile WIDTH is +// learned over. In the default gaussian mode it does not set the intensity - the fit grid, ceil(r2), +// does. +// +// What the fixed circular disk loses is flux, and the loss is NOT a function of resolution alone: +// measured, it carries a directional component worth several A^2 with a definite principal axis. Nor +// is there anything to absorb it. There is no per-shell scale in the merge, and there cannot be a +// useful one: every scale is fitted against a reference built from a reflection's own symmetry +// equivalents, and equivalents share s^2 exactly, so any function of s^2 lies in the exact null space +// of the whole scaling model. The isotropic part is instead degenerate with the Wilson B and is +// silently reported as part of it - measured across a radius sweep, 0.5 A^2 on sharp strong data and +// up to ~10 A^2 on weak wide-spot data. Most of the loss is matched by a proportional sigma, so it +// moves no CC1/2 and no R_meas. // // The width is the BANDWIDTH STREAK ALONE, and deliberately not the profile's full radial variance // (which also carries a sensor parallax term and a weak-spot capture term). Those two were measured diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index 7c685358..ef677a21 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -24,6 +24,7 @@ #include "../common/PixelMask.h" #include "../common/print_license.h" #include "../common/JFJochMath.h" +#include "../image_analysis/bragg_integration/CalcISigma.h" #include "../image_analysis/geom_refinement/Calibrants.h" #include "../image_analysis/indexing/IndexerThreadPool.h" #include "../image_analysis/LoadFCalcFromMtz.h" @@ -1523,6 +1524,18 @@ static int RunRugnux(int argc, char **argv) { matched, merged_reflections.size()); } + // Dataset-wide Wilson B-factor estimate (like XDS's WILSON LINE B), as the full pipeline reports + // it (Rugnux.cpp). Diagnostic only - it is not fed back into scaling; it lands in the printed + // statistics, the mmCIF and the log. Without it --mode scale reports WILSON_B= nan. + { + const GlobalWilsonB wilson = CalcGlobalWilsonB(merged_reflections); + merged_statistics.wilson_b = wilson.b; + merged_statistics.wilson_b_correlation = wilson.correlation; + if (std::isfinite(wilson.b) && wilson.b > 0.0) + logger.Info("Wilson B-factor estimate: {:.2f} A^2 (correlation {:.3f}, {} shells)", + wilson.b, wilson.correlation, wilson.n_shells); + } + std::cout << merged_statistics; // Space-group determination lives in the full rugnux pipeline; --scale only consumes a space