From dd95704d7b1aa3fbb5f0a66af45b42985fbf7996 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Wed, 9 Sep 2026 16:22:36 +0200 Subject: [PATCH] beam stop: the rings are drawn about the centre the data measures, not the one the file claims Every pixel is compared against the ring it sits on, and the rings were drawn about the beam centre in the file. Displacing that centre costs nothing for tens of pixels and a great deal beyond: on clean sweeps 50 px leaves the mask where it was, 150 px returns a tenth of the detector as shadow with nothing blocking it. Header centres are wrong by that much - one sweep in the corpus states one 351 px from where the data put it, on a run that otherwise indexes every frame and merges at CC1/2 0.999 - so a shadow search that trusts the header is a live hazard now that an empty mask is no longer the fail-safe it used to be. The centre is therefore fitted from the projection BEFORE the mask is read, and named to the finder with BeamCenter(). It costs no frame of its own: the fit reads the same per-pixel mean the mask is computed from. Measured on the phase boundary, the pre-scan grows by 0.4-0.6 s on a 2.5 Mpx detector and by 3-4 s on an 18 Mpx one. The other half of the circle is that this fit is itself biased by a shadow it has not yet masked - on a sweep with a third of the detector behind a pin it put the centre 5 px from the truth and called it 0.40 px, ten of its own sigmas out. It does not have to be unbiased here. The ring comparison does not notice tens of pixels and the bias is a few, and the centre the run REPORTS and consumes is not this one: it is the fit that already ran after the mask was loaded, unchanged, with the shadow out of the way. The order is fit, mask, fit, and only the second answer leaves the function. Two rounds are enough, measured rather than assumed: on four clean sweeps the fit before the mask and the fit after it agree to 0.03 px, so a third round would draw the same rings. The centre estimator is not made robust to a shadow either - it already robustifies across azimuthal sectors, three IRLS rounds on the per-sector shifts, and that is what returned 5 px at 0.40 px, because a third of the azimuth is a second population and not an outlier. And no guard refuses to mask when the fitted centre disagrees with the header: a header 351 px out is precisely the case this has to survive. Mask, before -> after (pixels, and per cent of the detector): clean, 2.5 Mpx 20851 (0.82%) -> 20670 (0.82%) clean, 2.5 Mpx 33308 (1.32%) -> 32730 (1.29%) clean, 18 Mpx, low bkg 173804 (0.96%) -> 170031 (0.94%) clean, 18 Mpx 250196 (1.38%) -> 249915 (1.38%) a third behind a pin 818184 (32.32%) -> 815699 (32.22%) header 351 px out 54183 (0.87%) -> 314592 (5.05%) The clean sweeps do not move; on all four the measured centre is 1-9 px from the file's and the mask shrinks by under 3 %. The pinned sweep does not move either - its header centre happens to be right, so the rings were already where they belong - and its merge is unchanged to the third decimal. The last row goes the other way and is not the ordering: with the rings finally about the beam, this geometry reaches 2 theta = 68 deg, where the polarization of the source modulates the background around a ring by a factor of three. That is four times the threshold this test cuts at, so the horizontal lobes read as shadow. The commit that follows removes that confound; with it the same row reads 39 k px (0.63 %), which is the holder arm and nothing else. Even uncorrected the run's answer is unchanged - same space group, same cell to the third decimal, 12658 unique reflections either way, R_meas 9.06 -> 9.10 %, and 1 % of the multiplicity lost. Co-Authored-By: Claude Opus 5 (1M context) --- image_analysis/beam_stop/SHADOW_FINDER.md | 16 +++++--- image_analysis/beam_stop/ShadowFinder.cpp | 6 +++ image_analysis/beam_stop/ShadowFinder.h | 11 ++++- rugnux/Rugnux.cpp | 45 +++++++++++++++------ tests/ShadowFinderTest.cpp | 49 +++++++++++++++++++++++ 5 files changed, 108 insertions(+), 19 deletions(-) diff --git a/image_analysis/beam_stop/SHADOW_FINDER.md b/image_analysis/beam_stop/SHADOW_FINDER.md index ebcf4984c..32ea60734 100644 --- a/image_analysis/beam_stop/SHADOW_FINDER.md +++ b/image_analysis/beam_stop/SHADOW_FINDER.md @@ -19,6 +19,7 @@ ShadowFinder finder(experiment); // geometry (beam centre, conve std::vector scratch; for (int i : first_N_frames) // ~20–50 spread frames is plenty (see below) finder.AddImage(read_frame(i), scratch); // DataMessage with .image, like DarkMaskAnalysis +finder.BeamCenter(x, y); // optional: the measured centre, not the file's std::vector shadow = finder.GetMask(); // 1 = shadow, 0 = keep; converted geometry ``` @@ -46,6 +47,11 @@ own radius**, not to a global threshold. 2. **Robust radial baseline.** Take the **median of the mean-projection in each 1-pixel ring** about the beam centre. The median ignores the shadow (a minority of each ring) and the spots. Iterate 3×, excluding the shadow it finds, so the baseline is not pulled down by the shadow. + The rings are drawn about the centre the caller names with `BeamCenter()`, which defaults to the + file's. Tens of pixels of error cost nothing, but a couple of hundred draws the rings across the + background's own fall-off instead of along it and a large region then reads as shadow for + reasons that have nothing to do with hardware — so `rugnux` fits the centre from this same + projection first and names that one (see below). 3. **Ratio.** `ratio = mean / baseline(r)`. Shadowed pixels sit far below 1; the smooth halo and symmetric ice rings cancel to ≈1 and are **not** flagged. 4. **Counting statistics.** A ratio on its own says nothing when the background it is made of is a @@ -148,11 +154,11 @@ beam-stop mask is an **offline** (Rugnux + viewer) feature and never goes to the - **Off-centre disk**: the low-res disk guard keys on the *stored* beam centre. If that is off the physical stop (seen on one stills detector), a small already-dead disk gets no extra circular guard — a data-driven disk centre would close it. -- **A grossly wrong beam centre**: the rings are drawn about the centre in the file. A centre off by - tens of pixels costs nothing, but one off by a couple of hundred draws the rings across the - background's own fall-off rather than along it, and a large region then falls below the ratio for - reasons that have nothing to do with hardware. Measuring the centre before the shadow, rather than - after it, would close this. +- **A grossly wrong beam centre**: closed in `rugnux` by ordering. The centre is fitted from the + projection *before* the mask is read, and named with `BeamCenter()`; the shadow that biases that + fit moves it by a few pixels, three orders below what the ring comparison notices. What remains + is the case where the fit declines to place a centre at all, where the file's is used as before. + Another caller that reads the mask at the file's centre keeps the old exposure. - **Unsigned saturation**: for unsigned images a detector-saturating reflection collides with the no-data sentinel and is skipped; harmless in practice (such reflections do not sit under the stop), and absent entirely on the signed/preprocessed path. diff --git a/image_analysis/beam_stop/ShadowFinder.cpp b/image_analysis/beam_stop/ShadowFinder.cpp index d19422585..94b1af50b 100644 --- a/image_analysis/beam_stop/ShadowFinder.cpp +++ b/image_analysis/beam_stop/ShadowFinder.cpp @@ -305,6 +305,12 @@ ShadowFinder::ShadowFinder(const DiffractionExperiment &experiment, const PixelM #endif } +void ShadowFinder::BeamCenter(float x, float y) { + std::unique_lock ul(m); + beam_x = x; + beam_y = y; +} + // A shard's accumulators are allocated when a frame is first added to it, not here: with a GPU they // are never used at all, and on a 16 Mpx detector eight of them are 2.9 GB to allocate and clear - // which measured 0.8 s of the pre-scan, all of it wasted. diff --git a/image_analysis/beam_stop/ShadowFinder.h b/image_analysis/beam_stop/ShadowFinder.h index 4bcb99300..a337ac0a2 100644 --- a/image_analysis/beam_stop/ShadowFinder.h +++ b/image_analysis/beam_stop/ShadowFinder.h @@ -42,8 +42,8 @@ class ShadowFinder { const int width; const int height; - const float beam_x; - const float beam_y; + float beam_x; + float beam_y; std::vector pixel_mask; // pixels already masked carry no background to test @@ -85,6 +85,13 @@ class ShadowFinder { public: ShadowFinder(const DiffractionExperiment &experiment, const PixelMask &mask); + // The centre the rings are drawn about. It starts as the file's, which is the only one there + // is when the finder is built; a caller that has measured one replaces it before reading the + // mask. A centre far from the truth draws the rings across the background's own radial + // fall-off instead of along it, and the comparison then describes the fall-off rather than the + // hardware. The projection is not centred on anything, so this may be set after the frames. + void BeamCenter(float x, float y); + // Give each worker a shard to accumulate into. Must be called before the first AddImage, // and costs 20 bytes per pixel per shard. void SetShardCount(size_t n); diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index 9f6fc2ef5..a95a76bf1 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -1155,19 +1155,39 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru } if (want_shadow) { + // Every pixel is compared against the ring it sits on, so the rings have to be drawn about + // the beam. A centre far enough out draws them across the background's own radial fall-off + // instead of along it, and what comes back describes the fall-off rather than the hardware: + // displacing a centre by 50 px on a clean sweep leaves the mask where it was, by 150 px + // masks a tenth of the detector. A centre in a file can be that wrong - one sweep here + // states one 351 px from where the data put it - so the rings are drawn about the centre + // this same projection measures rather than the one the file claims. That fit costs no + // frame of its own, and an unmasked shadow biases it by a few pixels at most, which is + // three orders below what this comparison would notice. The centre the run reports and + // consumes is the one fitted below instead, with the shadow out of the way. + if (const auto ring_center = FindBeamCenterFromBackground(experiment_, pixel_mask_, + finder.GetMeanProjection())) { + finder.BeamCenter(ring_center->beam_x_pxl, ring_center->beam_y_pxl); + logger.Info("Beam stop: comparing each pixel against its ring about the measured " + "centre ({:.2f},{:.2f}) +- {:.2f} px, {:.2f} px from the file's", + ring_center->beam_x_pxl, ring_center->beam_y_pxl, ring_center->sigma_pxl, + std::hypot(ring_center->beam_x_pxl - experiment_.GetBeamX_pxl(), + ring_center->beam_y_pxl - experiment_.GetBeamY_pxl())); + } else { + logger.Info("Beam stop: the background does not place a centre on this run, so the " + "rings are drawn about the file's ({:.2f},{:.2f})", + experiment_.GetBeamX_pxl(), experiment_.GetBeamY_pxl()); + } const auto shadow = finder.GetMask(config_.nthreads); const auto shadowed = std::count(shadow.begin(), shadow.end(), 1u); pixel_mask_.LoadBeamStopMask(experiment_, shadow); - // Not "this beamline has no beam stop". Every pixel is compared against the ring it sits on, - // and the rings are drawn about the beam centre in the file: a centre far enough out draws - // them across the background's own fall-off instead of along it, and what comes back stops - // describing the hardware. That reads both ways - it has been seen as an empty mask, and on - // a deliberately displaced centre as a large spurious one - so an unexpected answer here is - // worth reading as a statement about the centre first. + // Not "this beamline has no beam stop": it is also what a comparison drawn about the wrong + // point returns, so an unexpected answer here is worth reading against the centre named + // above first. if (shadowed == 0) - logger.Warning("Beam stop: no shadow found in {} images. The comparison is made around " - "the beam centre in the file, so this may point at that centre rather " - "than at the experiment; --estimate-beam-center measures it", + logger.Warning("Beam stop: no shadow found in {} images. The comparison is made about " + "the centre named above, so this may point at that centre rather than " + "at the experiment", finder.GetFrameCount()); else logger.Info("Beam stop shadow: {} pixels ({:.2f}% of the detector) found in {} images", @@ -1175,9 +1195,10 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru finder.GetFrameCount()); } - // The beam centre comes after, so that the shadow is in the mask by the time it is placed and - // the holder arm - the largest azimuthal asymmetry on the detector - is out of the way of the - // background the centre is read from. + // The beam centre is fitted again here, so that the shadow is in the mask by the time it is + // placed and the holder arm - the largest azimuthal asymmetry on the detector - is out of the + // way of the background the centre is read from. This is the answer the run reports and + // consumes; the fit above only chose where to draw the rings, and is thrown away. // What the scattered background makes the centre, measured on EVERY run and committed on none. // The fit reads the projection the beam-stop pre-pass has just built and no frame of its own, so diff --git a/tests/ShadowFinderTest.cpp b/tests/ShadowFinderTest.cpp index 8b91cb1e5..da4c5bd94 100644 --- a/tests/ShadowFinderTest.cpp +++ b/tests/ShadowFinderTest.cpp @@ -230,3 +230,52 @@ TEST_CASE("ShadowFinder_FindsNothingOnAnUnobstructedScene", "[ShadowFinder]") { const auto mask = finder.GetMask(); CHECK(std::count(mask.begin(), mask.end(), 1u) == 0); } + +// The rings are drawn about a beam centre, and a centre in a file can be a long way from the truth. +// On a background that falls with radius, rings drawn about the wrong point cut across that +// fall-off, and pixels that are simply further out than the ring's median read as shadow: a large +// part of the detector comes back masked with nothing blocking it. The caller therefore names the +// centre it has measured, and that is the one the comparison has to use. +TEST_CASE("ShadowFinder_TheRingsFollowTheCentreTheCallerNames", "[ShadowFinder]") { + // Far enough out that the ring about the file's centre spans a wide range of true radii. + constexpr int OFFSET = 100; + // Opaque, well clear of the beam, and larger than the smallest region the test may return. + constexpr int PATCH_X0 = 175, PATCH_X1 = 245, PATCH_Y0 = 30, PATCH_Y1 = 90; + constexpr int PATCH_PX = (PATCH_X1 - PATCH_X0 + 1) * (PATCH_Y1 - PATCH_Y0 + 1); + + DiffractionExperiment x = TestExperiment(); + x.BeamX_pxl(static_cast(C + OFFSET)); // what the file claims + const PixelMask pixel_mask(x); + ShadowFinder finder(x, pixel_mask); + + // A background falling with the true radius, kept under MIN_REFLECTION so no pixel is exempt. + std::vector> frames; + std::vector buffer; + for (int f = 0; f < NFRAMES; f++) { + frames.emplace_back(static_cast(W) * H, 0); + for (int y = 0; y < H; y++) + for (int xi = 0; xi < W; xi++) { + const float r = std::hypot(static_cast(xi - C), static_cast(y - C)); + const bool blocked = xi >= PATCH_X0 && xi <= PATCH_X1 && y >= PATCH_Y0 && y <= PATCH_Y1; + frames.back()[I(xi, y)] = blocked ? 0 : std::lround(24.0f * std::exp(-r / 80.0f)); + } + DataMessage msg{}; + msg.image = CompressedImage(frames.back(), W, H); + finder.AddImage(msg, buffer); + } + + const auto at_file = finder.GetMask(); + finder.BeamCenter(static_cast(C), static_cast(C)); + const auto at_measured = finder.GetMask(); + + // The patch is shadow either way - it is opaque, and no centre makes it look lit. + CHECK(at_file[I((PATCH_X0 + PATCH_X1) / 2, (PATCH_Y0 + PATCH_Y1) / 2)] == 1); + CHECK(at_measured[I((PATCH_X0 + PATCH_X1) / 2, (PATCH_Y0 + PATCH_Y1) / 2)] == 1); + + // About the true centre the rings are flat and only the patch and its penumbra come back; + // about the file's they cut across the fall-off and a large part of the detector does. + const auto masked_at_file = std::count(at_file.begin(), at_file.end(), 1u); + const auto masked_at_measured = std::count(at_measured.begin(), at_measured.end(), 1u); + CHECK(masked_at_measured < 2 * PATCH_PX); + CHECK(masked_at_file > 3 * PATCH_PX); +}