Radial background: one mechanism for stills and rotation, decided per run on empty apertures (D5)

Wherever the correction is set for the whole run its curve is the frame's peak-excluded ring
background (the adaptive spot finder's, whole frame) and the ring's clip follows it - as stills have
done since integ-pal - now rotation too. One empty-aperture measurement (BackgroundAperture) decides
it for both workflows:

- The apertures' corrected difference is now taken as the integrator applies the correction: the
  clip follows the curve (each ring pixel's threshold raised by the curve's rise from the centre, in
  whole counts) as well as the kernel correction. Before, the tally clipped flat, so it measured a
  correction the engine never applies.
- Rotation measures the stencils the spot width can give it (r1 = 4, 5, 6 with the ring-area rule,
  RotationStencil) and decides on the one it gets, and separately on the pre-adaptive stencil the
  geometry pre-pass and the starvation guard return to.
- Stills: on wherever the corrected ring predicts the empty disk better (unchanged rule).
  Rotation: the frame's curve on every image only where it takes >= 10% of the apertures' mean
  square error away (ROTATION_RADIAL_MIN_GAIN); otherwise the existing per-image ice gate with the
  reflections' curve is left exactly as it was. Why a margin on rotation only: a still has no other
  correction, a sweep has one. Measured over 18 rotation sets (forced on every image): photon-counting
  sweeps gain 0.1-5% on the apertures (z 3-4) and lose ISa/anomalous signal when corrected
  (insulin, lysozyme, cytochrome c; 9yzk R-free +0.009 at 1.2%), as the earlier stills-rule trial
  found; the home-source CCD and smooth-ice runs gain 13-31%.

Result (CPU A/B vs 87684012f, 18 rotation sets): 12 .hkl byte-identical (every photon-counting
control without ice), 5uth R-free 0.210 -> 0.201 / ISa 10.4 -> 10.8, the ice set neutral; stills:
lcls242_s turns the correction on (the clip-following tally), R-free 0.2399 -> 0.2406.
No integration kernel changed; the tally is host code. Tests: BackgroundAperture_CorrectedClipFollowsTheCurve,
RadialHelps margins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit dd3f3f2da39f5338e7dda1874bbdd535229bce98)

docs/CPU_DATA_ANALYSIS_INTEGRATION.md

@@ -145,13 +145,20 @@ The clip cuts only the high tail, which matters: the **symmetric** trimmed mean
 
 Both estimators are computed in the shared background pass, but only the trim reaches plain box summation: the high-side clip is skipped for `--integrator boxsum`, which therefore uses the plain ring mean unless `--background-trim` is given.
 
-**Radial background correction (opt-in).** A ring mean estimates the background *under* the signal disk correctly only if the background is flat there. The signal disk and the ring are concentric, so for a background that is **linear** in position $\langle B\rangle_\mathrm{ring}=\langle B\rangle_\mathrm{disk}$ identically — a plane or gradient fit buys exactly nothing. The leading error is the **curvature** of the radial background, which is negligible on a smooth background but reaches tens of counts on a single reflection sitting on a sharp powder ring. That error is a kernel over radial offset,
+**Radial background correction.** A ring mean estimates the background *under* the signal disk correctly only if the background is flat there. The signal disk and the ring are concentric, so for a background that is **linear** in position $\langle B\rangle_\mathrm{ring}=\langle B\rangle_\mathrm{disk}$ identically — a plane or gradient fit buys exactly nothing. The leading error is the **curvature** of the radial background, which is negligible on a smooth background but reaches tens of counts on a single reflection sitting on a sharp powder ring. That error is a kernel over radial offset,
 $
 \delta \hat b \;=\; \textstyle\sum_k \kappa_k\, \bar B(r_0+k),
 $
-with $\kappa$ the annulus-minus-disk histogram of the stencil over radial offset, averaged over azimuth, and $\bar B(r)$ the image's own radial background curve. With the fixed circular stencil ($k=0$, §9.1) that single kernel serves every reflection. An elongated ring does not: its radial-offset histogram depends on how far that particular reflection's ring was grown, so $\kappa$ becomes a small table of kernels, indexed by the growth rounded to whole pixels. The azimuthal average survives the change unaltered, because the stencil is rebuilt in the reflection's own frame at each azimuth and so stays radially aligned: what is averaged over is the sub-pixel phase of the detector grid against the radius, which is what genuinely differs between reflections. Applying it costs one short dot product per reflection and no extra pixel reads; correcting the background *scalar* means the box sum, the profile fit and the variance all pick it up. The curve is accumulated from the same annulus pixels the background pass already reads (a pixel's radius is the reflection's radius plus the pixel's projection on the beam→reflection direction, so no per-pixel square root is needed) and specifically from the **clipped** pixels, or it would carry neighbour tails and zingers — which is why the correction is inert under `--integrator boxsum`, that path having no clip pass.
+with $\kappa$ the annulus-minus-disk histogram of the stencil over radial offset, averaged over azimuth, and $\bar B(r)$ the image's own radial background curve. With the fixed circular stencil ($k=0$, §9.1) that single kernel serves every reflection. An elongated ring does not: its radial-offset histogram depends on how far that particular reflection's ring was grown, so $\kappa$ becomes a small table of kernels, indexed by the growth rounded to whole pixels. The azimuthal average survives the change unaltered, because the stencil is rebuilt in the reflection's own frame at each azimuth and so stays radially aligned: what is averaged over is the sub-pixel phase of the detector grid against the radius, which is what genuinely differs between reflections. Applying it costs one short dot product per reflection and no extra pixel reads; correcting the background *scalar* means the box sum, the profile fit and the non-signal variance all pick it up. The variance carried to the merge, $\mathrm{var}_\mathrm{bkg} = g\,(n_S\hat b + n_S^2\,\mathrm{var}(\hat b))$, moves with the corrected $\hat b$ in its first term — the Poisson share of the background *under the disk* — while $\mathrm{var}(\hat b)$, the noise of the ring estimate, and $\sigma$, taken from the counts in the disk, do not depend on a smooth curve and stay.
 
-The model is a function of **radius alone**, so it is applied only where that is true of the background. `--background-radial` takes `on`, `off` or `auto`. In Rugnux it is **`auto` by default** (the broker keeps it off); under `auto` each image's peak-excluded ice score (§3.3) is taken after spot detection and before integration, and the correction is applied to that image when the score reaches the same `--ice-min-score` gate. Smooth powder ice *is* a radial feature and is corrected; ice made of discrete crystallite spots — which the profile channel is blind to and the spot channel catches — leaves no smooth ring to model, and correcting it makes matters worse. Measured against a fixed atomic model, comparing ice bands with resolution-matched decoy bands carrying no ice: on a crystal with pure smooth ice the correction removes **43 % of the bands' excess amplitude**, and the improvement is **7× larger inside the bands than outside**, which is its stated mechanism; on a crystal whose ice is textured the same correction *increased* the excess amplitude by half; on a clean crystal it is inert to four decimal places. Auto engages only where a peak-excluded score exists (adaptive spot finding, §3.2) — a plain azimuthal profile carries the Bragg peaks and cannot support an absolute threshold, so without one auto leaves the correction off.
+The curve $\bar B(r)$ is the adaptive spot finder's peak-excluded background per resolution ring of the **whole frame** (§3.2), one value per pixel of radius: every pixel of the frame, against the few dozen to few thousand reflections whose own rings would otherwise sample it. Where that curve is in use the ring's **high-side clip follows it**: a ring pixel at radial offset $u$ is clipped against the threshold plus the curve's rise $\bar B(r_0+u)-\bar B(r_0)$, in whole counts. On a background falling steeply away from the beam a flat threshold cuts the beam-side half of the ring as outliers and the background reads low — up to 150× too strong a low-resolution reflection on a binned CCD near the beam stop. Without a frame curve (fixed-threshold spot finding) the curve is accumulated from the clipped annulus pixels of the predicted reflections themselves, and the clip stays flat.
+
+**When it is applied.** `--background-radial` takes `on`, `off` or `auto`; in Rugnux it is **`auto` by default** (the broker keeps it off), and auto is decided **per run, in the pre-scan, on empty apertures** — the same measurement that, on stills, also chooses how far the ring reaches (the $r_3$, up to the one the run starts from, whose ring predicts the empty disk best). On a grid of apertures away from every found spot, the ring's prediction of the empty disk is taken twice: plain, and with the correction applied exactly as the integrator applies it (the clip following the curve, the kernel's correction). With nothing in the aperture each difference is the error the ring makes in a reflection's background, noise plus bias. Rotation measures every stencil the spot width can give it (and the one it starts from, which the geometry pre-pass and the starvation guard return to) and reads the one it gets.
+
+- **Stills** have no other correction, so it is applied to every image of a run where it lowers the apertures' mean square error at all.
+- **Rotation** has one: under auto, each image whose peak-excluded ice score (§3.3) reaches `--ice-min-score` is corrected with the curve of its reflections' rings. The frame's curve on every image replaces that only where it takes **at least 10 %** of the apertures' mean square error away. On photon-counting sweeps the apertures gain 0.1–5 % (statistically significant, $z\approx3$–4), and correcting every image there was measured to cost ISa and anomalous signal; where it pays — the rings of home-source CCD frames, a smooth ice ring — they gain 13–31 %. A run below the margin keeps the per-image ice correction, bit for bit.
+
+The model is a function of **radius alone**, so it is applied only where that is true of the background. The per-image ice gate rests on this: smooth powder ice *is* a radial feature and is corrected; ice made of discrete crystallite spots — which the profile channel is blind to and the spot channel catches — leaves no smooth ring to model, and correcting it makes matters worse. Measured against a fixed atomic model, comparing ice bands with resolution-matched decoy bands carrying no ice: on a crystal with pure smooth ice the correction removes **43 % of the bands' excess amplitude**, and the improvement is **7× larger inside the bands than outside**, which is its stated mechanism; on a crystal whose ice is textured the same correction *increased* the excess amplitude by half; on a clean crystal it is inert to four decimal places. A strongly polarised beam or panels of different response make the background non-radial too; the apertures see that as a ring the correction predicts *worse*, and a run where they do keeps it off. Auto engages only where a peak-excluded background exists (adaptive spot finding, §3.2) — a plain azimuthal profile carries the Bragg peaks and cannot support an absolute threshold, so without one auto leaves the correction off.
 
 ### 9.3 Profile-fitted extraction (default)
 

image_analysis/MXAnalysisWithoutFPGA.cpp

@@ -237,9 +237,11 @@ void MXAnalysisWithoutFPGA::Analyze(DataMessage &output,
         // Where no such background exists (no adaptive finder), auto leaves the correction off: the
         // plain profile carries the Bragg peaks and cannot support an absolute threshold.
         //
-        // On stills the correction also takes its CURVE from that background, and auto applies it to
-        // every image that has one (rugnux's pre-scan turns it off for a run whose empty apertures say
-        // the radial model does not hold there). A serial frame has a few dozen predicted reflections, so their own
+        // On stills - and wherever the correction is set for the whole run rather than left to auto, as
+        // rugnux's pre-scan sets it on a rotation run whose empty apertures ask for it - the correction
+        // takes its CURVE from that background, and on stills auto applies it to every image that has one
+        // (the pre-scan turns it off for a run whose empty apertures say the radial model does not hold
+        // there). A serial frame has a few dozen predicted reflections, so their own
         // ring pixels - where the curve comes from otherwise - sample the radial background sparsely,
         // while the finder's rings hold every pixel of the frame. And the correction is the measured
         // curvature itself: where the background has none it is zero, so there is nothing for a gate
@@ -254,7 +256,7 @@ void MXAnalysisWithoutFPGA::Analyze(DataMessage &output,
             const std::vector<float> &ring_bkg = spot_finding_settings.adaptive_threshold
                                                  ? adaptiveSpotFinder->GetRingBackground() : no_rings;
             const bool stills = !experiment.IsRotationIndexing();
-            if (stills)
+            if (stills || !bragg_engine->IsBackgroundRadialAuto())
                 bragg_engine->RadialBackgroundFromRings(ring_bkg, integration.GetQBinCount(),
                                                         integration.Settings().GetLowQ_recipA(),
                                                         integration.Settings().GetQSpacing_recipA());

image_analysis/bragg_integration/BackgroundAperture.cpp

@@ -91,9 +91,17 @@ void background_aperture::MeasureFrame(ImagePreprocessorBuffer &image, int width
         }
         disk_v /= static_cast<double>(disk.size());
         disk_b /= static_cast<double>(disk.size());
+        // The curve at the aperture's centre, which the corrected clip measures the ring's rise from.
+        double centre_b = 0.0;
+        if (!rings.empty()) {
+            const auto [px, py] = modules.ImageToPlane(panel, cx[a], cy[a]);
+            centre_b = rings.At(2.0 * PI / geometry.PxlToRes(static_cast<float>(px), static_cast<float>(py)));
+        }
 
-        // Every candidate ring as the integrator reads it: the mean, then one high-side clip pass. The
-        // radial correction is taken over the ring's geometry, as the integrator's kernel is.
+        // Every candidate ring as the integrator reads it: the mean, then one high-side clip pass. With the
+        // radial correction, as the integrator applies it: the clip follows the curve (each pixel's
+        // threshold raised by the curve's rise from the centre to it, in whole counts -
+        // RadialClipOffset), and the correction is taken over the ring's geometry, as the kernel is.
         std::vector<double> d_plain(r3.size()), d_radial(r3.size());
         bool measured = true;
         for (size_t k = 0; k < r3.size() && measured; ++k) {
@@ -112,17 +120,24 @@ void background_aperture::MeasureFrame(ImagePreprocessorBuffer &image, int width
             }
             const double mean = sum / n;
             const double thr = mean + clip_nsigma * std::sqrt(gain * std::max(mean, static_cast<double>(gain)));
-            double kept = 0.0;
-            int n_kept = 0;
-            for (const auto &p : ring)
-                if (p.d2 < r3_sq && (clip_nsigma <= 0.0f || p.v <= thr)) {
+            double kept = 0.0, kept_radial = 0.0;
+            int n_kept = 0, n_kept_radial = 0;
+            for (const auto &p : ring) {
+                if (p.d2 >= r3_sq)
+                    continue;
+                if (clip_nsigma <= 0.0f || p.v <= thr) {
                     kept += p.v;
                     ++n_kept;
                 }
-            measured = n_kept > 0;
+                if (clip_nsigma <= 0.0f || p.v <= thr + static_cast<double>(std::lround(p.b - centre_b))) {
+                    kept_radial += p.v;
+                    ++n_kept_radial;
+                }
+            }
+            measured = n_kept > 0 && n_kept_radial > 0;
             if (measured) {
                 d_plain[k] = disk_v - kept / n_kept;
-                d_radial[k] = d_plain[k] - (disk_b - sum_b / n);
+                d_radial[k] = disk_v - kept_radial / n_kept_radial - (disk_b - sum_b / n);
             }
         }
         if (!measured)
@@ -179,3 +194,9 @@ int background_aperture::Best(const std::vector<double> &mean_square) {
         return -1;
     return static_cast<int>(std::min_element(mean_square.begin(), mean_square.end()) - mean_square.begin());
 }
+
+bool background_aperture::RadialHelps(const Tally &tally, size_t k, double min_gain) {
+    if (tally.apertures == 0 || tally.apertures_radial != tally.apertures || k >= tally.sum_d2.size())
+        return false;
+    return tally.sum_d2_radial[k] < (1.0 - min_gain) * tally.sum_d2[k];
+}

image_analysis/bragg_integration/BackgroundAperture.h

@@ -32,12 +32,14 @@
 // mean + nsigma * sqrt(gain * mean)).
 //
 // The same apertures also say whether the radial background-curvature correction helps: each difference
-// is taken a second time with the correction applied - the frame's own ring background (RingBackground)
-// averaged over the ring's pixels minus over the disk's, which is what the integrator's kernel takes off a
-// reflection. Where the background really is a function of the radius the corrected ring predicts the
-// disk better; where it is not - a strongly polarised beam, panels of different response - it predicts it
-// worse, and the run measures which. The ring is chosen on the plain differences first, so its reach does
-// not rest on the radial model; the correction is then kept where it improves that ring.
+// is taken a second time with the correction applied as the integrator applies it - the ring's clip
+// following the frame's own ring background (RingBackground), and that background averaged over the
+// ring's pixels minus over the disk's, which is what the integrator's kernel takes off a reflection.
+// Where the background really is a function of the radius the corrected ring predicts the disk better;
+// where it is not - a strongly polarised beam, panels of different response - it predicts it worse, and
+// the run measures which. On stills the ring is chosen on the plain differences first, so its reach does
+// not rest on the radial model; the correction is then kept where it improves that ring. Rotation measures
+// every stencil the spot width can give it, and reads the one it gets.
 namespace background_aperture {
 
 // The squared disk-minus-ring differences, summed per candidate r3 over every aperture measured: plain,
@@ -56,6 +58,12 @@ struct Tally {
     [[nodiscard]] double RadialZ(size_t k) const;
 };
 
+// A signal disk r1, the inner edge of its ring r2, and the candidate outer edges r3.
+struct Stencil {
+    float r1 = 0.0f, r2 = 0.0f;
+    std::vector<float> r3;
+};
+
 // The candidate outer radii: from r2 + 2 up to max_r3, one pixel apart.
 [[nodiscard]] std::vector<float> CandidateR3(float r2, float max_r3);
 
@@ -69,6 +77,17 @@ void MeasureFrame(ImagePreprocessorBuffer &image, int width, int height,
                   const RingBackground &rings, const DiffractionGeometry &geometry,
                   const DetectorModules &modules, Tally &tally);
 
+// Whether the radial correction makes candidate k's ring predict the disk better - its mean square
+// difference lower by at least the fraction min_gain - on apertures that were all measured both ways.
+[[nodiscard]] bool RadialHelps(const Tally &tally, size_t k, double min_gain = 0.0);
+
+// The part of the background error the correction must take away on a rotation run. A still has no other
+// correction, so any gain is taken there; a rotation run already corrects the frames whose ice score shows
+// a smooth ring, and the frame's curve on every frame replaces that only for a clear gain. Measured: on
+// photon-counting sweeps the apertures gain 0.1-5% (z 3-4) and every image corrected costs ISa and
+// anomalous signal; where it pays - home-source CCD rings, a smooth ice ring - they gain 13-31%.
+constexpr double ROTATION_RADIAL_MIN_GAIN = 0.10;
+
 // The candidate with the smallest mean square difference, or -1 where nothing was measured.
 [[nodiscard]] int Best(const std::vector<double> &mean_square);
 

rugnux/Rugnux.h

@@ -1335,7 +1335,7 @@ class Rugnux {
             std::vector<FootprintSpot> footprint;
             std::vector<float> spot_q;
             size_t width_images = 0;
-            background_aperture::Tally apertures;
+            std::vector<background_aperture::Tally> apertures;   // one per candidate stencil
         };
         struct Defects {
             size_t pixels = 0;

rugnux/RugnuxPreScan.cpp

@@ -88,6 +88,13 @@
 using namespace rugnux_internal;
 
 namespace {
+    // The rotation stencil the spot width gives a signal radius r1: r2 follows it, and r3 keeps the
+    // r2..r3 background ring at the area it has at the shipped 4/6/13.
+    background_aperture::Stencil RotationStencil(float r1) {
+        const float r2 = r1 + 2.0f;
+        return {r1, r2, {std::sqrt(r2 * r2 + 133.0f)}};
+    }
+
     // A detector-plane displacement resolved ACROSS the spindle - the only component the tolerance
     // above is about. Along the spindle the error translates the derotated cloud rigidly and the FFT
     // amplitude cannot see it at all; across it the peaks smear and the amplitude falls. (The lattice
@@ -364,21 +371,29 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
         return w;
     };
 
-    // Stills: how far the background ring should reach, measured on empty apertures of the same frames
-    // (BackgroundAperture.h) at the signal disk and gap the run integrates with. The candidates run up
-    // to the ring the run starts from, so where the noise wins that ring is what is kept. The ring is
+    // Empty apertures of the same frames (BackgroundAperture.h): whether the radial background correction
+    // makes the ring predict the background under the disk better, and on stills also how far the ring
+    // should reach. Stills measure at the signal disk and gap the run integrates with, the candidate rings
+    // running up to the one the run starts from, so where the noise wins that ring is what is kept.
+    // Rotation does not know its stencil yet - the spot width decides it from these same frames - so each
+    // stencil it can end up with is measured, and the one it does is read (RotationStencil). The ring is
     // clipped at the gain the run integrates at.
     const bool stills = !experiment_.IsRotationIndexing();
     const BraggIntegrationSettings aperture_bis = prescan_x.GetBraggIntegrationSettings();
-    const std::vector<float> aperture_r3 = stills
-            ? background_aperture::CandidateR3(aperture_bis.GetR2(), aperture_bis.GetR3()) : std::vector<float>();
+    std::vector<background_aperture::Stencil> aperture_stencils;
+    if (stills)
+        aperture_stencils.push_back({aperture_bis.GetR1(), aperture_bis.GetR2(),
+                                     background_aperture::CandidateR3(aperture_bis.GetR2(), aperture_bis.GetR3())});
+    else
+        for (const float r1 : {4.0f, 5.0f, 6.0f})   // the radii spot_width::R1ForWidth chooses from
+            aperture_stencils.push_back(RotationStencil(r1));
     const float aperture_gain = aperture_bis.GetPixelGain();
 
     // The spots of one image, and what is taken off them.
     const auto find_spots = [&](PreScanWorker &w, const CompressedImage &image, int image_idx,
                                 bool for_width, std::vector<spot_width::FluxCurve> &curves,
                                 std::vector<float> &spot_q, std::vector<FootprintSpot> &footprint,
-                                background_aperture::Tally &apertures) {
+                                std::vector<background_aperture::Tally> &apertures) {
         try {
             // As in the image loops: a frame the device cannot decode goes to the host decoder.
             ImageStatistics stats;
@@ -422,7 +437,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
             MeasureSpotsOnTiles(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
                                 static_cast<int>(prescan_x.GetYPixelsNumConv()), prescan_x.GetDiffractionGeometry(),
                                 prescan_x.GetModules(), spots, curves, footprint, aperture_gain);
-        if (for_width && !aperture_r3.empty()) {
+        if (for_width) {
             // The frame's ring background, as the integration of this frame will read it - from the
             // adaptive finder, which has it whole-frame and peak-excluded.
             const std::vector<float> no_rings;
@@ -431,11 +446,14 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
                                        prescan_mapping->GetQBinCount(),
                                        prescan_mapping->Settings().GetLowQ_recipA(),
                                        prescan_mapping->Settings().GetQSpacing_recipA());
-            background_aperture::MeasureFrame(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
-                                              static_cast<int>(prescan_x.GetYPixelsNumConv()), spots,
-                                              aperture_bis.GetR1(), aperture_bis.GetR2(), aperture_r3,
-                                              aperture_bis.GetBackgroundClipNSigma(), aperture_gain, rings,
-                                              prescan_x.GetDiffractionGeometry(), prescan_x.GetModules(), apertures);
+            apertures.resize(aperture_stencils.size());
+            for (size_t k = 0; k < aperture_stencils.size(); ++k)
+                background_aperture::MeasureFrame(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
+                                                  static_cast<int>(prescan_x.GetYPixelsNumConv()), spots,
+                                                  aperture_stencils[k].r1, aperture_stencils[k].r2,
+                                                  aperture_stencils[k].r3, aperture_bis.GetBackgroundClipNSigma(),
+                                                  aperture_gain, rings, prescan_x.GetDiffractionGeometry(),
+                                                  prescan_x.GetModules(), apertures[k]);
         }
     };
 
@@ -446,8 +464,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
     std::vector<spot_width::FluxCurve> shape_curves;
     // Every spot's measured widths, for the spot footprint (SpotFootprint.h).
     std::vector<FootprintSpot> footprint_spots;
-    // The empty apertures of the frames the width is measured on (stills).
-    background_aperture::Tally aperture_tally;
+    // The empty apertures of the frames the width is measured on, one tally per aperture_stencils entry.
+    std::vector<background_aperture::Tally> aperture_tally(aperture_stencils.size());
     // Images the width ended up being measured on, for the log: the tiers below stop as soon as the
     // answer has settled, so this is a property of the crystal and worth reporting.
     size_t width_images = 0;
@@ -565,7 +583,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
         std::vector<std::vector<spot_width::FluxCurve>> curves_of(ordinals.size());
         std::vector<std::vector<float>> spot_q_of(ordinals.size());
         std::vector<std::vector<FootprintSpot>> footprint_of(ordinals.size());
-        std::vector<background_aperture::Tally> apertures_of(ordinals.size());
+        std::vector<std::vector<background_aperture::Tally>> apertures_of(ordinals.size());
         // Frames read for the bandwidth alone, past the tier the width settled on. Their spots stay
         // out of the powder measurement, which is taken on the frames it always was.
         std::vector<char> shape_only(ordinals.size(), 0);
@@ -683,7 +701,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
         // In sample order, over the frames the width was taken on.
         for (size_t i = 0; i < ordinals.size(); i++)
             if (position[i] < width_images)
-                aperture_tally.Add(apertures_of[i]);
+                for (size_t k = 0; k < apertures_of[i].size(); ++k)
+                    aperture_tally[k].Add(apertures_of[i][k]);
         // In sample order, so the table does not depend on how the workers interleaved.
         for (const auto &f : footprint_of)
             footprint_spots.insert(footprint_spots.end(), f.begin(), f.end());
@@ -733,13 +752,15 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
             // The ring is chosen on the plain differences, so its reach does not rest on the radial
             // model; the correction is then kept, where it was left to auto, if it makes that ring predict
             // the disk better. Read only where every frame gave a ring background.
-            const std::vector<double> plain = aperture_tally.MeanSquare(false);
-            const std::vector<double> radial = aperture_tally.apertures_radial == aperture_tally.apertures
-                                               ? aperture_tally.MeanSquare(true) : std::vector<double>();
+            const background_aperture::Tally &tally = aperture_tally.front();
+            const std::vector<float> &aperture_r3 = aperture_stencils.front().r3;
+            const std::vector<double> plain = tally.MeanSquare(false);
+            const std::vector<double> radial = tally.apertures_radial == tally.apertures
+                                               ? tally.MeanSquare(true) : std::vector<double>();
             const int best = background_aperture::Best(plain);
             const bool use_radial = radial_setting.has_value()
                                     ? *radial_setting
-                                    : best >= 0 && !radial.empty() && radial[best] < plain[best];
+                                    : best >= 0 && background_aperture::RadialHelps(tally, best);
             if (best < 0) {
                 logger.Info("Background ring: no empty aperture measured on {} images, keeping r3={:.1f}",
                             width_images, bis.GetR3());
@@ -757,10 +778,10 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
                 logger.Info("Background ring: on {} empty apertures from {} images the r{:.0f}..r3 ring predicts "
                             "the r{:.0f} disk with a mean square error of (r3:value){}, and with the radial "
                             "curvature correction{} => integration radii r1={:.1f} r2={:.1f} r3={:.1f}, radial "
-                            "correction {} (z = {:.1f} at that ring)", aperture_tally.apertures, width_images, bis.GetR2(), bis.GetR1(),
+                            "correction {} (z = {:.1f} at that ring)", tally.apertures, width_images, bis.GetR2(), bis.GetR1(),
                             table(plain), radial.empty() ? std::string(" (no ring background)") : table(radial),
                             bis.GetR1(), bis.GetR2(), bis.GetR3(), use_radial ? "on" : "off",
-                            aperture_tally.RadialZ(best));
+                            tally.RadialZ(best));
                 Note(fmt::format("background ring {:.0f}..{:.0f} px, radial correction {}, measured on empty "
                                  "apertures", bis.GetR2(), bis.GetR3(), use_radial ? "on" : "off"));
             }
@@ -773,9 +794,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
                             "integration radius at r1={:.1f}", width_curves.size(), width_images,
                             experiment_.GetBraggIntegrationSettings().GetR1());
             } else {
-                const float r1 = spot_width::R1ForWidth(*r80);
-                const float r2 = r1 + 2.0f;
-                const float r3 = std::sqrt(r2 * r2 + 133.0f);
+                const background_aperture::Stencil st = RotationStencil(spot_width::R1ForWidth(*r80));
+                const float r1 = st.r1, r2 = st.r2, r3 = st.r3.front();
                 BraggIntegrationSettings bis = experiment_.GetBraggIntegrationSettings();
                 // Remembered where the APERTURE moved, so the starvation guard in RunAllPasses has
                 // something to give back and cannot announce a fallback that changes nothing.
@@ -799,6 +819,50 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
                                  *r80, r1, r2, r3));
             }
 
+            // The radial background correction, where it was left to auto, decided on the empty apertures of
+            // each stencil the run may integrate with: the one the width gave it, and the one it started
+            // from, which the geometry pre-pass and the starvation guard go back to. Rotation's own default
+            // is the per-image correction of the frames the ice score marks (MXAnalysisWithoutFPGA), so the
+            // frame's curve replaces it on every frame only where the apertures show it taking a real part
+            // of the background error away (background_aperture::ROTATION_RADIAL_MIN_GAIN).
+            const auto decide_radial = [&](BraggIntegrationSettings &bis, const char *which) {
+                if (bis.GetBackgroundRadialCorrection().has_value())
+                    return;
+                size_t k = 0;
+                while (k < aperture_stencils.size()
+                       && !(aperture_stencils[k].r1 == bis.GetR1() && aperture_stencils[k].r2 == bis.GetR2()
+                            && aperture_stencils[k].r3.front() == bis.GetR3()))
+                    ++k;
+                if (k == aperture_stencils.size() || aperture_tally[k].apertures == 0) {
+                    logger.Info("Background ring ({}): r1={:.1f} r2={:.1f} r3={:.2f} not measured on empty "
+                                "apertures; radial correction per image", which, bis.GetR1(), bis.GetR2(),
+                                bis.GetR3());
+                    return;
+                }
+                const background_aperture::Tally &tally = aperture_tally[k];
+                const std::vector<double> plain = tally.MeanSquare(false), radial = tally.MeanSquare(true);
+                const bool on = background_aperture::RadialHelps(tally, 0,
+                                                                 background_aperture::ROTATION_RADIAL_MIN_GAIN);
+                if (on)
+                    bis.BackgroundRadialCorrection(true);
+                logger.Info("Background ring ({}): on {} empty apertures from {} images the r{:.0f}..r{:.0f} ring "
+                            "predicts the r{:.0f} disk with a mean square error of {:.4g}, and {} with the radial "
+                            "curvature correction from the frame's curve (z = {:.1f}) => radial correction {}",
+                            which, tally.apertures, width_images, bis.GetR2(), bis.GetR3(), bis.GetR1(), plain[0],
+                            radial.empty() ? std::string("none (no ring background)")
+                                           : fmt::format("{:.4g}", radial[0]),
+                            tally.RadialZ(0), on ? "on every image" : "per image, where the ice score sees a ring");
+            };
+            {
+                BraggIntegrationSettings bis = experiment_.GetBraggIntegrationSettings();
+                decide_radial(bis, "canonical");
+                experiment_.ImportBraggIntegrationSettings(bis);
+                if (bragg_before_adaptive_)
+                    decide_radial(*bragg_before_adaptive_, "pre-adaptive");
+                if (bis.GetBackgroundRadialCorrection() == std::optional<bool>(true))
+                    Note("radial background correction on, measured on empty apertures");
+            }
+
             // How far the spots reach away from the beam, where the radius chosen above at 5 A no
             // longer holds them (SpotFootprint.h). It changes the integration only where a spot
             // outgrows the r1 disk.

rugnux/rugnux_cli.cpp

@@ -255,7 +255,7 @@ void print_usage() {
     std::cout << "   --integration-high-resolution <num> High resolution limit for prediction/integration. If omitted (or 0), integration extends as far as the detector reaches" << std::endl;
     std::cout << "   --max-hkl <n>                   Predict reflections with |h|,|k|,|l| <= n. Default: derived per crystal from the refined cell (ceil(longest axis / d_min) + 1), which is the exact bound - set it only to override that" << std::endl;
     std::cout << "   --background-clip <n>           High-side clip of the background ring at mean + n*sqrt(mean) (default 4; 0 = off). This is the default background estimator - it rejects neighbour cores and zingers without the symmetric trim's Poisson skew bias. Ignored by --integrator boxsum" << std::endl;
-    std::cout << "   --background-radial[=on|off|auto] Correct the background ring for the CURVATURE of the radial background (default auto). The signal disk and the background ring are concentric, so a background linear in position cancels between them and only curvature survives - which on a smooth ice ring reaches +26 counts on a single reflection. =auto applies it per image where that image's ice score shows a smooth powder ring, which is where a radius-only background model holds; on ice made of discrete crystallite spots there is no such ring and the correction makes the bias worse. On stills the curve is taken from the spot finder's peak-excluded background of the whole frame, and =auto applies it to every image of a run where it makes the background ring predict the background of empty apertures better (measured in the pre-scan): a smooth solvent or carrier hump curves the background as much as ice does. Costs one short dot product per reflection and no extra pixel reads" << std::endl;
+    std::cout << "   --background-radial[=on|off|auto] Correct the background ring for the CURVATURE of the radial background (default auto). The signal disk and the background ring are concentric, so a background linear in position cancels between them and only curvature survives - which on a smooth ice ring reaches +26 counts on a single reflection. Where it is on, the curve is the spot finder's peak-excluded background of the whole frame, and the ring's outlier clip follows it, so a background falling steeply from the beam is not cut as outliers. =auto decides it per run in the pre-scan, on empty apertures of the pre-scan frames: on stills it is applied to every image where it makes the background ring predict the background of empty apertures better at all (a smooth solvent or carrier hump curves the background as much as ice does); on rotation data it is applied to every image where it takes at least 10% of that error away, and otherwise only to the images whose ice score shows a smooth powder ring, with the curve from the reflections' own rings (ice made of discrete crystallite spots leaves no such ring, and the correction makes the bias worse). Costs one short dot product per reflection and no extra pixel reads" << std::endl;
     std::cout << "   --background-trim <f>           Use the old symmetric trimmed mean for the background ring instead of the clip (0<=f<0.5; 0.10 was the former default). Switches --background-clip off. A symmetric trim is biased low on Poisson data and adds ~5 counts to every partial, so this is for back compatibility only; 0 = plain ring mean" << std::endl;
     std::cout << "   --overlap <txt>                 What to do where two predicted reflections share signal pixels: off|reject|exclude (default exclude). A pixel inside two signal disks belongs to the nearer centre; before this, nothing kept a neighbour's flux out of a reflection's own disk, so on a dense pattern a crowded reflection read high. exclude drops the shared PIXELS from the profile fit, which renormalises itself, so the reflection is kept; reject instead drops the whole reflection when less than --overlap-minpk of its expected profile is cleanly its own (what XDS calls MINPK). --integrator boxsum has no profile to renormalise with, so exclude does nothing there and only reject acts" << std::endl;
     std::cout << "   --overlap-minpk <f>             Least fraction of a reflection's expected profile that must be usable for the reflection to be kept (default 0.75, XDS MINPK). Governs both ways part of a profile is lost: the fraction that must be READABLE - not masked, untrusted, in a detector gap or overloaded - in every profile mode, and, under --overlap reject, the fraction that must be cleanly the reflection's own. --integrator boxsum has no profile to renormalise with, so there a signal disk with any unreadable pixel is still discarded outright and the reject fraction is by disk AREA, which cuts harder" << std::endl;
@@ -3439,7 +3439,7 @@ static int RunRugnux(int argc, char **argv, bool no_gpu) {
         logger.Info("Background ring: radial curvature correction {}",
                     radial.has_value() ? (*radial ? "on" : "off")
                                        : rotation_indexing
-                                             ? "auto (per image, where the ice score sees a smooth ring)"
+                                             ? "auto (decided in the pre-scan on empty apertures; otherwise per image, where the ice score sees a smooth ring)"
                                              : "auto (decided in the pre-scan on empty apertures; the curve from the whole frame)");
     }
 

tests/BackgroundApertureTest.cpp

@@ -113,6 +113,39 @@ TEST_CASE("BackgroundAperture_RadialCorrectionHelpsOnARadialBackground", "[Integ
     // The pre-scan's question: at the ring the plain differences choose, does the correction help.
     const int best = background_aperture::Best(plain);
     CHECK(radial[best] < 0.8 * plain[best]);
+    CHECK(background_aperture::RadialHelps(tally, best));
+    CHECK(background_aperture::RadialHelps(tally, best, background_aperture::ROTATION_RADIAL_MIN_GAIN));
+    CHECK(!background_aperture::RadialHelps(tally, best, 0.99));
     // With the curvature taken out, the noise wins again and a wide ring is best.
     CHECK(r3[background_aperture::Best(radial)] >= 12.0f);
 }
+
+// A background falling steeply away from the beam: the flat clip cuts the beam-side half of every ring
+// and the ring reads low. The integrator's clip follows the curve when the correction is on, and so does
+// the apertures' corrected measurement - which then predicts the disk far better.
+TEST_CASE("BackgroundAperture_CorrectedClipFollowsTheCurve", "[Integration]") {
+    const DiffractionGeometry g = Geometry();
+    const auto background = [](double r) { return 50000.0 - 100.0 * r; };
+    std::mt19937 rng(5);
+    ImagePreprocessorBuffer img(static_cast<size_t>(W) * H);
+    for (int y = 0; y < H; y++)
+        for (int x = 0; x < W; x++) {
+            std::poisson_distribution<int> p(background(std::hypot(x - 300.0, y - 300.0)));
+            img[static_cast<size_t>(y) * W + x] = p(rng);
+        }
+    const float low_q = 0.05f, dq = 0.005f;
+    std::vector<float> profile(1000);
+    for (size_t i = 0; i < profile.size(); i++) {
+        const double q = low_q + (i + 0.5) * dq;
+        const double r = 60.0 / 0.075 * std::tan(2.0 * std::asin(q * 1.0 / (4.0 * PI)));
+        profile[i] = static_cast<float>(background(r));
+    }
+    const RingBackground rings(profile, static_cast<int32_t>(profile.size()), low_q, dq);
+    background_aperture::Tally tally;
+    background_aperture::MeasureFrame(img, W, H, {}, 6.0f, 8.0f, {14.0f}, 4.0f, 1.0f, rings, g, DetectorModules(),
+                                      tally);
+    REQUIRE(tally.apertures > 100);
+    const auto plain = tally.MeanSquare(false), radial = tally.MeanSquare(true);
+    CHECK(radial[0] < 0.1 * plain[0]);
+    CHECK(background_aperture::RadialHelps(tally, 0));
+}