Bragg integration: keep the background ring clear only of neighbours with flux on the frame

The r2 regions set aside from a reflection's r2..r3 background ring were
those of every prediction in the +-4 sigma rocking window. On a finely
sliced dense pattern most of them are the tails of reflections recorded on
the frames either side, which fill every ring while the frame shows nothing
there. 9ac2ca677 kept the reflections those rings starved by taking the ring
whole, neighbour pixels included, and relying on the high-side clip.

A prediction now masks the ring only where it puts at least 5% of its flux
on the frame (partiality >= 0.05). A ring still starved by those neighbours
has real flux in it, and the reflection is dropped, as before 9ac2ca677:
taking it whole let the neighbours' wings into the background. The CPU mask
holds two levels (tail, flux); the GPU writes them in two launches so the
flux mark wins, with no atomics.

The count the widened-radius guard reads keeps measuring the pattern's
density over every prediction, tails included, so the guard decides on
the quantity its 1.13% bound was read off. Read on the gated mask it let a
cubic set keep r1=6, and R_meas went 36.9 -> 43.2%. Masked but
unreadable ring pixels are no longer counted as the neighbours' doing.

Fixed radius, same code base, --model, the 0.05 deg / 7200-frame set:
                      pre-9ac2ca677  9ac2ca677  this
  rings starved            88.9%       88.9%    21.5%
  partials ingested         7.4M        67M      53M
  completeness             17.9%       79.7%    85.1%
  ISa                      12.8        14.3     14.7
  R_meas                    6.8%        8.4%     7.6%
  R_free                   0.139       0.176    0.171
  radial misfit            0.11        0.185    0.066
  R vs model, common hkl to 0.69 A (61k):
                           0.133       0.130    0.129
  (the R_free rise over pre-9ac2ca677 is composition: 63k -> 319k
  reflections to 0.69 A, the added ones weaker.) Gate at 0.2 instead:
  0.1% starved but radial misfit 0.35, R_free 0.180.

A crystal whose header-geometry pass indexes a 7x supercell:
9ac2ca677's whole rings raised that pass's I/sigma >= 2 count past the
refined pass's by more than 10%. RefinedPassIsWorse then sent the run
back to the supercell. Now: the true cell and space group as before
9ac2ca677, same ISa, 104M partials in the header pass against 219M.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-09-28 17:13:16 +02:00
1 parent de352fdaf3
commit 5803dee2ce
10 files changed
+194 -103

No files matched your search

@@ -88,14 +88,18 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
auto grid_idx = [this](int dx, int dy) { return (dy + R) * G + (dx + R); };
// --- Reflection mask: mark the r2 signal region of every predicted reflection so a neighbour's
// signal is excluded from this reflection's r2..r3 background ring. The region is the INNER
// stencil ellipse, taken in each neighbour's OWN frame - an elongated neighbour whose streak
// is still marked as a disk would leak its tails into this reflection's ring. ---
// --- Reflection mask: mark the r2 signal region of every predicted reflection, so a neighbour's
// signal is excluded from this reflection's r2..r3 background ring. MASK_FLUX where the
// neighbour puts flux on this frame (NEIGHBOUR_MASK_MIN_PARTIALITY) - only those are excluded
// - and MASK_TAIL where only the tail of a rocking curve does, which is counted for the density
// guard but read as background. The region is the INNER stencil ellipse, taken in each
// neighbour's OWN frame - an elongated neighbour whose streak is still marked as a disk would
// leak its tails into this reflection's ring. ---
refl_mask.Clear();
owner.Clear();
for (size_t i = 0; i < npredicted; ++i) {
const auto &r = predicted[i];
const uint8_t level = r.partiality >= NEIGHBOUR_MASK_MIN_PARTIALITY ? MASK_FLUX : MASK_TAIL;
const BraggStencil st = MakeBraggStencil(r.predicted_x, r.predicted_y, stencil);
const int x0 = std::max(0, static_cast<int>(std::floor(r.predicted_x - st.ex_in - 1.0f)));
const int x1 = std::min(W - 1, static_cast<int>(std::ceil(r.predicted_x + st.ex_in + 1.0f)));
@@ -104,7 +108,10 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
for (int y = y0; y <= y1; ++y)
for (int x = x0; x <= x1; ++x) {
const auto d = BraggStencilDistances(st, x - r.predicted_x, y - r.predicted_y);
if (d.inner < r2_sq) refl_mask.At(x, y) = 1;
if (d.inner < r2_sq) {
uint8_t &m = refl_mask.At(x, y);
m = std::max(m, level);
}
}
}
@@ -155,7 +162,6 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
std::vector<Rough> rough(npredicted);
double inv_d2_min = std::numeric_limits<double>::max(), inv_d2_max = 0.0;
std::vector<int32_t> bkg_vals; // reused per reflection for the trimmed-mean background (idea 1)
std::vector<int32_t> bkg_vals_neighbour; // the ring pixels a neighbour's region holds, likewise
// Radial background curve, accumulated from the annulus pixels this pass already reads. A pixel's
// radius is the reflection's radius plus the pixel's projection on the beam->reflection direction,
@@ -207,17 +213,13 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
int n_disk = 0, n_own = 0; // pixels in the signal disk, and how many are this reflection's
double bkg_sum = 0.0;
int n_bkg = 0;
// Ring pixels a NEIGHBOUR's signal region occupies. refl_mask marks d.inner < r2_sq and the
// ring is d.inner >= r2_sq, so the two are complementary and a masked ring pixel is always
// someone else's - never this reflection's own core. That makes this an exact count of what
// the pattern's density took, separable from what the detector took.
int n_bkg_neighbour = 0;
// The readable pixels among those, kept apart so a ring the neighbours starved can fall back
// on them (below).
double bkg_sum_neighbour = 0.0;
int n_bkg_neighbour_valid = 0;
// Readable ring pixels, and those of them no predicted neighbour's signal region covers.
// refl_mask marks d.inner < r2_sq and the ring is d.inner >= r2_sq, so the two are
// complementary and a masked ring pixel is always someone else's - never this reflection's
// own core. That makes the difference an exact count of what the pattern's density took,
// separable from what the detector took.
int n_ring = 0, n_clean = 0;
bkg_vals.clear();
bkg_vals_neighbour.clear();
for (int y = y0; y <= y1; ++y)
for (int x = x0; x <= x1; ++x) {
const auto d = BraggStencilDistances(st, x - r.predicted_x, y - r.predicted_y);
@@ -243,16 +245,11 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
y_sum += y;
++n_inner_valid;
} else if (d.inner >= r2_sq && d.outer < r3_sq) {
if (refl_mask.Get(x, y)) {
++n_bkg_neighbour;
if (valid(px)) {
bkg_sum_neighbour += static_cast<double>(px);
++n_bkg_neighbour_valid;
if (bkg_trim_frac > 0.0) bkg_vals_neighbour.push_back(px);
}
continue;
}
if (!valid(px)) continue;
++n_ring;
const uint8_t m = refl_mask.Get(x, y);
if (m == 0) ++n_clean;
if (m == MASK_FLUX) continue;
bkg_sum += static_cast<double>(px);
if (bkg_trim_frac > 0.0) bkg_vals.push_back(px);
++n_bkg;
@@ -273,24 +270,11 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
// than to a measurement. Counted here because it is the only direct evidence of a radius that has
// outgrown the pattern it is integrating (BraggIntegrationCounts).
const bool keep_partial = full || mode != IntegratorMode::BoxSum;
if (keep_partial && n_bkg <= 5) {
++counts.bkg_starved;
// Would the ring have been enough without the neighbours? Then it is the pattern, not the
// detector, that took it.
if (n_bkg + n_bkg_neighbour > 5) ++counts.bkg_starved_by_neighbour;
}
// A ring the NEIGHBOURS starved still holds readable pixels - only the mask set them aside.
// Most of those neighbours are predictions in the tail of their rocking curve, which put a
// few percent of their flux on this frame: on a finely sliced, dense pattern they fill the
// whole ring while the frame shows no spot there. Dropping the reflection loses a measurement
// to protect a background from flux that is mostly not there, so the ring is taken whole
// instead, and the high-side clip below removes the neighbour cores that are.
const bool ring_unmasked = keep_partial && n_bkg <= 5 && n_bkg + n_bkg_neighbour_valid > 5;
if (ring_unmasked) {
bkg_sum += bkg_sum_neighbour;
n_bkg += n_bkg_neighbour_valid;
bkg_vals.insert(bkg_vals.end(), bkg_vals_neighbour.begin(), bkg_vals_neighbour.end());
}
if (keep_partial && n_bkg <= 5) ++counts.bkg_starved;
// Would every predicted neighbour, the tails included, leave the ring starved that the
// detector alone would not? That is the pattern's density, what the guard on a widened
// radius reads (BraggIntegrationCounts).
if (keep_partial && n_clean <= 5 && n_ring > 5) ++counts.bkg_starved_by_neighbour;
if (keep_partial && n_bkg > 5) {
out.bkg = bkg_sum / n_bkg;
if (bkg_trim_frac > 0.0 && bkg_vals.size() > 5
@@ -317,7 +301,7 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
for (int x = x0; x <= x1; ++x) {
const auto d = BraggStencilDistances(st, x - r.predicted_x, y - r.predicted_y);
if (!(d.inner >= r2_sq && d.outer < r3_sq)) continue;
if (refl_mask.Get(x, y) && !ring_unmasked) continue;
if (refl_mask.Get(x, y) == MASK_FLUX) continue;
const int32_t px = img[y * W + x];
if (!valid(px)) continue;
if (static_cast<double>(px) <= thr) {