From a1816e905e9fc130d767285b388dabdf196c4fe7 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 26 Sep 2026 14:01:17 +0200 Subject: [PATCH] ShadowFinder: add beam-stop holder arms that let part of the beam through After the existing mask is complete, a second step takes the pixels it left out that are significantly dimmer than their ring, joins them through a 6 px bridge and across module gaps of any width, and adds a piece whole when it holds >= 2000 dim pixels of which >= 200 are deep. The existing mask is never touched, so a sweep with no such piece keeps its mask bit for bit. Quick subset tests (battery --only, no model check) against the rc173 a0518abe6 full battery: 152 of 233 masks identical; 10 of 10 controls (including the sets where earlier shadow changes regressed through marginal decisions) give byte-identical merges. On the 21 sets that gain a piece the space group never changes, merge outlier rejections fall on 19, R_meas falls and ISa rises on 17, and the shell-scaled model R improves on 16 of 18; a transmitting arm with an over-subtracted background strip is recovered (R_meas 13.3 -> 12.0 %, ISa 13.4 -> 15.5). Known costs: on one set with background bumps at ring radii the low-resolution agreement with the model falls (CC 0.89 -> 0.85) while its internal statistics improve, and two sets cut slightly coarser (1.50 -> 1.55 A, 1.72 -> 1.80 A) with a better model R. Squashed from branch hq-beamstop (cfa080a59..60b6c6822), whose history records the variants tried and dropped: a straight port of the earlier arm finder moved every mask, a two-shape construction filled almost a whole sweep through hole filling, and a deep-fraction floor rejected a real arm that is dim along its whole length. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C --- docs/CHANGELOG.md | 4 ++ image_analysis/beam_stop/SHADOW_FINDER.md | 14 ++++ image_analysis/beam_stop/ShadowFinder.cpp | 82 ++++++++++++++++++++++- tests/ShadowFinderTest.cpp | 47 ++++++++++++- 4 files changed, 145 insertions(+), 2 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index d34d283b3..81186c6dc 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -1,6 +1,10 @@ # Changelog ## 1.0.0 +### 1.0.0-rc.174 + +* Rugnux adds beam-stop holder arms that let part of the beam through to the beam-stop mask, without changing the mask of a sweep that has none. + ### 1.0.0-rc.173 * Rugnux corrects the background of reflections on a smooth powder or ice ring for the ring's radial profile by default (`--background-radial=auto`); `--background-radial=off` restores the flat background ring. diff --git a/image_analysis/beam_stop/SHADOW_FINDER.md b/image_analysis/beam_stop/SHADOW_FINDER.md index 69df8e270..3e446ef10 100644 --- a/image_analysis/beam_stop/SHADOW_FINDER.md +++ b/image_analysis/beam_stop/SHADOW_FINDER.md @@ -93,6 +93,19 @@ own radius**, not to a global threshold. the disk interior. 10. **Carve reflections last.** Remove any lit pixel from the final mask (no fill afterwards) so a spot the geometry still covered is given back rather than re-enclosed. +11. **Arms that let part of the beam through.** A thin holder arm, or one blurred by its distance + from the detector, sits at 40-70 % of the background along most of its length and crosses + `shadow_ratio` only in places, so step 6 keeps only deep fragments, each below + `min_shadow_pixels`; under a bright background step 10 also gives back its attenuated pixels, + which reach `min_reflection`. So, after step 10, the **dim** pixels the mask left out + (`ratio < penumbra_ratio && sigma > min_deficit_sigma`) are joined through a `bridge_px`-dilated + copy and across any module gap whose two sides both belong to them, however wide the gap, and a + piece with at least `min_shadow_pixels` dim pixels of which `min_core_pixels` are core is added + whole. Nothing is given back inside it: the reflections behind transmitting hardware are + recorded attenuated. The step only ever adds such pieces - the dim edge of an ordinary stop is + left as steps 6-10 drew it, so a sweep without transmitting hardware keeps its mask bit for bit + (an earlier version redrew every stop's edge from the dim pixels and moved every mask in the + corpus by 0.01-0.7 % of the detector). Three findings are baked in: the **mean** ratio (not the max) does the detecting — the max projection is a great *picture* but a noisier detector; **size and counting statistics**, not a @@ -108,6 +121,7 @@ just safe headroom. | `shadow_ratio` | 0.50 | core threshold: mean below this fraction of the radial baseline | | `min_deficit_sigma` | 6 | … and the deficit significant to this many Poisson standard deviations | | `min_shadow_pixels` | 2000 | smallest region the per-pixel test may return | +| `min_core_pixels` | 200 | core pixels a dim piece of step 11 must hold | | `penumbra_ratio` | 0.75 | soft boundary grows out to this ratio … | | `penumbra_max_px` | 30 | … but no further than this from the core | | `bridge_px` | 6 | bridge module gaps / breaks the arm crosses | diff --git a/image_analysis/beam_stop/ShadowFinder.cpp b/image_analysis/beam_stop/ShadowFinder.cpp index 6a6816173..944e965e4 100644 --- a/image_analysis/beam_stop/ShadowFinder.cpp +++ b/image_analysis/beam_stop/ShadowFinder.cpp @@ -28,7 +28,8 @@ constexpr float SHADOW_RATIO = 0.50f; constexpr float PENUMBRA_RATIO = 0.75f; constexpr int PENUMBRA_MAX_PX = 30; -// Bridge module gaps and small breaks that the holder arm crosses. +// Bridge small breaks along the holder arm. A module gap wider than this is bridged separately for +// the arm search (see bridge_gaps). constexpr int BRIDGE_PX = 6; // A pixel whose maximum reaches this recorded a real reflection and is never masked - a @@ -48,6 +49,12 @@ constexpr double MIN_DEFICIT_SIGMA = 6.0; // This is what keeps the test specific now that a shadow no longer has to touch the direct beam. constexpr int MIN_SHADOW_PIXELS = 2000; +// ... and of those, how many must be deep (below SHADOW_RATIO) for a region of merely DIM pixels - +// hardware that lets part of the beam through - to count. A thin holder arm is dim along most of +// its length and deep in places; the background drifting over a detector's edge is dim everywhere +// and deep nowhere. +constexpr int MIN_CORE_PIXELS = 200; + // Side of the box the background is pooled over before testing. Its area is how many pixels back // a ring's countability test, which decides where an azimuthal comparison is possible at all. constexpr int POOL_PX = 5; @@ -146,6 +153,31 @@ std::vector erode(const std::vector &in, int W, int H, int r, size_t return out; } +// Join a region across the module gaps it crosses: every run of gap pixels along a row or a column +// whose two ends both touch the region becomes region. A gap carries no data, so a shadow that +// continues on both sides of it is one shadow - but a gap can be wider than BRIDGE_PX reaches (17 px +// between the rows of PILATUS modules), and an arm crossing one fell apart into pieces each too small +// to be believed. +std::vector bridge_gaps(const std::vector ®ion, const std::vector &valid, + int W, int H) { + std::vector out = region; + auto walk = [&](int n_lines, int len, auto index) { + for (int line = 0; line < n_lines; line++) { + int k = 0; + while (k < len) { + if (valid[index(line, k)]) { k++; continue; } + const int start = k; + while (k < len && !valid[index(line, k)]) k++; + if (start > 0 && k < len && region[index(line, start - 1)] && region[index(line, k)]) + for (int j = start; j < k; j++) out[index(line, j)] = 1; + } + } + }; + walk(H, W, [W](int y, int x) { return static_cast(y) * W + x; }); + walk(W, H, [W](int x, int y) { return static_cast(y) * W + x; }); + return out; +} + // Fill holes: background not reachable from the image border becomes region. // // The flood is run over the bounding box of `region` grown by one, not the whole detector. Outside @@ -800,5 +832,53 @@ std::vector ShadowFinder::GetMask(size_t nthreads) const { for (int i = 0; i < n_pixels; i++) mask[i] = region[i] ? 1 : 0; + + // Hardware that lets part of the beam through - a thin holder arm, or one whose shadow is blurred + // by its distance from the detector - sits between PENUMBRA_RATIO and SHADOW_RATIO along most of + // its length and crosses SHADOW_RATIO only in places, so the region test above keeps only the + // fragments of it that are deep, each too small to be believed; and where the background is + // bright, the attenuated pixels it does keep reach MIN_REFLECTION and are given back as + // reflections. Such an arm is found here as what the mask above left out: the significantly + // DIM pixels, joined across small breaks and module gaps, in a piece as large as a shadow must be + // and deep in enough places. Nothing is given back inside it - the reflections behind it are + // recorded attenuated, and giving them back is what integrates them low. + // + // It only adds whole pieces. The dim pixels along the edge of an ordinary stop are left as the + // region above drew them: each such edge feeds decisions downstream that can sit at a margin, so + // a sweep with no transmitting hardware keeps its mask bit for bit. + std::vector dim(n_pixels, 0); + for (int i = 0; i < n_pixels; i++) + dim[i] = valid[i] && !region[i] && ratio[i] < PENUMBRA_RATIO && deficit[i] > MIN_DEFICIT_SIGMA; + const std::vector joined = bridge_gaps(dilate(dim, W, H, BRIDGE_PX, nthreads), valid, W, H); + std::vector seen(n_pixels, 0); + std::vector component; + std::queue q; + for (int start = 0; start < n_pixels; start++) { + if (!joined[start] || seen[start]) + continue; + component.clear(); + int n_dim = 0, n_low = 0; + seen[start] = 1; + q.push(start); + while (!q.empty()) { + const int i = q.front(); q.pop(); + component.push_back(i); + n_dim += dim[i]; + n_low += dim[i] && low[i]; + const int y = i / W, x = i % W; + for (int dy = -1; dy <= 1; dy++) + for (int dx = -1; dx <= 1; dx++) { + const int yy = y + dy, xx = x + dx; + if (yy < 0 || yy >= H || xx < 0 || xx >= W) + continue; + const int j = yy * W + xx; + if (joined[j] && !seen[j]) { seen[j] = 1; q.push(j); } + } + } + if (n_dim >= MIN_SHADOW_PIXELS && n_low >= MIN_CORE_PIXELS) + for (const int i : component) + if (dim[i]) + mask[i] = 1; + } return mask; } diff --git a/tests/ShadowFinderTest.cpp b/tests/ShadowFinderTest.cpp index 0efdb4c5f..0cae4a368 100644 --- a/tests/ShadowFinderTest.cpp +++ b/tests/ShadowFinderTest.cpp @@ -4,6 +4,7 @@ #include #include +#include #include #include #include @@ -103,10 +104,54 @@ TEST_CASE("ShadowFinder_FindsAnInjectedBeamStop", "[ShadowFinder]") { // Pinned from the serial implementation. A rewrite of the dilation, the hole fill or the ring // median that moves the mask by one pixel fails here, rather than in a merging statistic - // several stages downstream. + // several stages downstream. Nothing here lets part of the beam through, so the transmitting + // shape may redraw the stop's edge but must not add to the mask: the count is the opaque one. CHECK(std::count(mask.begin(), mask.end(), 1u) == 2612); } +// A holder arm that lets part of the beam through, on a background bright enough that every pixel +// reaches the reflection guard's count, crossing a module gap wider than the bridge. Each of the +// three hid the arm on its own: the partly transmitting stretch never reaches SHADOW_RATIO, the gap +// cut what did into pieces too small to be believed, and a count threshold read the attenuated +// background under the arm as recorded reflections and gave every pixel of it back. +TEST_CASE("ShadowFinder_FindsAnArmThatLetsPartOfTheBeamThrough", "[ShadowFinder]") { + constexpr int32_t BRIGHT = 50; // even the arm reaches MIN_REFLECTION + constexpr int ARM_HALF_WIDE = 15, GAP_X0 = 200, GAP_X1 = 216, OPAQUE_FROM_X = 232; + + const DiffractionExperiment x = TestExperiment(); + const PixelMask pixel_mask(x); + ShadowFinder finder(x, pixel_mask); + + std::vector> frames; + std::vector buffer; + for (int f = 0; f < NFRAMES; f++) { + frames.emplace_back(static_cast(W) * H, BRIGHT); + auto &frame = frames.back(); + for (int y = 0; y < H; y++) + for (int xi = 0; xi < W; xi++) { + const int dx = xi - C, dy = y - C; + if (dx * dx + dy * dy <= STOP_R * STOP_R) + frame[I(xi, y)] = 0; + else if (dx >= 0 && std::abs(dy) <= ARM_HALF_WIDE) + frame[I(xi, y)] = xi >= OPAQUE_FROM_X ? 0 : BRIGHT * 6 / 10; + if (xi >= GAP_X0 && xi <= GAP_X1) + frame[I(xi, y)] = INT32_MIN; // no data + } + DataMessage msg{}; + msg.image = CompressedImage(frame, W, H); + finder.AddImage(msg, buffer); + } + + const auto mask = finder.GetMask(); + CHECK(mask[I(C, C)] == 1); // the stop + CHECK(mask[I(GAP_X0 - 10, C)] == 1); // the arm where it transmits, before the gap + CHECK(mask[I(GAP_X1 + 10, C + ARM_HALF_WIDE - 3)] == 1); // ... and after it + CHECK(mask[I(W - 3, C)] == 1); // where it is opaque + CHECK(mask[I(GAP_X1 + 10, C + ARM_HALF_WIDE + 25)] == 0); // and nothing beside it + CHECK(mask[I(0, 0)] == 0); + CHECK(mask[I(C - 60, C)] == 0); +} + // The per-pixel passes are split across threads, so where the split falls must not be visible in the // answer. The x pass and the y pass of the dilation and of the pooled sum are written differently - // one a plain scan, the other blocked by column - so an x/y asymmetry is the plausible regression.