Bragg integration: clip the background ring high side instead of trimming it
The r2..r3 background ring was averaged with a 10% SYMMETRIC trimmed mean. A symmetric trim is not a consistent estimator of the mean of a right-skewed (Poisson) sample: on a clean Poisson ring it sits ~0.1 ct/px BELOW the true mean at every level, and with ~50 signal pixels in the r1 disk that under-subtraction adds ~5 counts to every partial on every frame. Measured two independent ways on four rotation datasets - stored background_mean against a plain ring mean over the same pixels on reflection-free frames, and directly on apertures that provably hold no reflection. Empty-aperture pedestal, counts: plain mean -0.03..-0.20, 10% symmetric trim +5.05..+6.34, 4 sigma clip +0.02..+0.54. Replace it with a high-side-only sigma clip at mean + n*sqrt(mean), n = 4 for monochromatic data. It rejects the same one-sided contamination the trim was there for - better, in fact: a 40 px neighbour core at +100 ct shifts the trim by +10.1 ct/px, because a symmetric trim collapses once contamination exceeds ~10% of the ring, versus +0.009 ct/px at 4 sigma. False rejection on a clean ring is 0.04-0.39%. Broadband data keep their tuned 3 sigma clip unchanged. The trim stays reachable with --background-trim for back compatibility; setting either estimator clears the other, so they can never stack. --integrator boxsum does not take the clip (matching what the shipped clip already did), so it now uses the plain ring mean unless --background-trim is given. The intensities get measurably more accurate: per-shell agreement with an independent processing of the same images improves on 14 of 16 crystals (weighted -0.0347, outermost shell 12/4), the outermost-shell R_meas NUMERATOR - absolute scatter, not a denominator effect - falls 13.5% median on 16/5, and CC1/2 in the outer shell improves on 14/7. EXPECT <I/sigma> TO FALL AND EDGE R_meas TO RISE. Both are inflated by information-free counts, so both get worse when the bias is removed; neither is evidence against this change. That fingerprint is exactly how the trimmed mean was accepted in the first place. Known cost: over the 37-crystal rotation battery the de-novo space-group count goes 34 OK / 3 DIFF to 33 / 4. The single regression is a two-lattice crystal whose merge fails the absolute-sanity gate under either background (R_meas 63.5%, CC1/2 72.2%) and which carries an unresolved indexing ambiguity on the very operator being scored, so its operator CC is diluted by construction. No other crystal changes space group, and twin protection is not weakened - the H-ratio veto that refuses genuinely twinned crystals gets MORE decisive (1.63 -> 1.84, 2.83 -> 3.99). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -55,12 +55,13 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
|
||||
return Finalize(predicted, npredicted, results, image_number);
|
||||
|
||||
const int W = static_cast<int>(xpixel), H = static_cast<int>(ypixel);
|
||||
const bool do_clip = apply_bkg_clip && mode != IntegratorMode::BoxSum;
|
||||
const bool do_clip = bkg_clip_nsigma > 0.0f && mode != IntegratorMode::BoxSum;
|
||||
|
||||
// Symmetric trimmed-mean background fraction (BraggIntegrationSettings, rugnux --background-trim):
|
||||
// replaces the r2..r3 ring MEAN with an f-trimmed mean, robust to the high-side contamination
|
||||
// (neighbour wings, tails) that biases the plain mean up and makes it over-subtract weak high-angle
|
||||
// reflections. Already forced to 0 for broadband/stills by the base ctor (they keep the clip below).
|
||||
// reflections. The base ctor has already forced it to 0 whenever the clip below is in force, which
|
||||
// is the default - the two are alternatives.
|
||||
const double bkg_trim_frac = bkg_trim;
|
||||
|
||||
auto grid_idx = [this](int dx, int dy) { return (dy + R) * G + (dx + R); };
|
||||
@@ -138,9 +139,9 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
|
||||
out.bkg = s / static_cast<double>(hi - lo);
|
||||
}
|
||||
} else if (do_clip) {
|
||||
// One high-outlier sigma-clip pass on the background ring (stills-only): reject pixels
|
||||
// above mean + 3*sqrt(mean) to strip a bandwidth-streaked neighbour that biases the mean.
|
||||
const double thr = out.bkg + 3.0 * std::sqrt(std::max(out.bkg, 1.0));
|
||||
// One high-outlier sigma-clip pass on the background ring: reject pixels above
|
||||
// mean + n*sqrt(mean) to strip a neighbour core or a zinger that biases the mean.
|
||||
const double thr = out.bkg + bkg_clip_nsigma * std::sqrt(std::max(out.bkg, 1.0));
|
||||
double s = 0.0;
|
||||
int n = 0;
|
||||
for (int y = y0; y <= y1; ++y)
|
||||
|
||||
Reference in New Issue
Block a user