mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 67 Wiki Activity

Beam-stop shadow detection, and a low-resolution limit for scaling

Browse Source 
rugnux finds the beam stop and its holder in a projection of 60 images and
marks them in the pixel mask as bit 9 (--detect-beam-stop[=N|off], on by
default). Reflections behind the stop are attenuated but not flagged, so they
integrate low with a plausible sigma and nothing downstream catches them: the
signal-box gate requires 100% valid pixels and shadow pixels are valid, the
background clip is high-side only, and the |zeta| cut applies only to the
space-group search merge.

The detection compares each pixel's background against the typical background
at the same radius on two channels. An azimuthal one (the ring median) finds
the holder arm, which is a minority of its ring; a radial one (the background
just outside) finds the disk, which the ring median cannot see because inside a
fully blocked ring the median is the shadow itself. Pixels are pooled over a
5x5 box and tested only where the background has actually been counted, so
low-background data no longer masks the whole detector. Recorded reflections
are carved back out - a beam stop cannot block a reflection that was measured.

Bit 9 belongs to the run that found it, not to the dataset: it is cleared when
a run starts, so a mask read back from a file that carries one starts clear.
The user mask (bit 8) is left alone.

Scaling and merging gain a low-resolution limit, default 50 A
(--scaling-low-resolution <num>, 0 removes it), applied per observation before
scaling so it also protects the per-frame scale fit and the space-group search.
50 A is the value XDS configurations use; rugnux_vs_xds.py now matches both of
XDS's resolution limits instead of only the high one, so the lowest shell is
the same shell in the two programs.

The viewer draws the detected shadow in coral with a "Show beam stop" switch in
the side panel, exposes the low-resolution limit in the settings dock, and
offers detection in its processing jobs. Adding an image marker meant giving
the reader a MIN_REAL_PXL_VALUE, because several places classify a pixel by
range rather than by equality and would otherwise read the new marker as a very
negative intensity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-08-09 01:05:31 +02:00
co-authored by Claude Opus 5
parent c673521b76
commit a29c36600f
45 changed files with 562 additions and 196 deletions
Download Patch File Download Diff File Expand all files Collapse all files
image_analysis/beam_stop/ShadowFinder.cpp
+167 -121
View File
@@ -11,15 +11,38 @@
#include "../../common/JFJochException.h"
// ---------------------------------------------------------------------------------
// Small binary-image helpers on a width*height frame stored row-major as char (0/1).
// All run once, at GetMask() time. The BFS forms keep them O(pixels) rather than
// O(pixels * radius), so a radius-14 dilation is still a single sweep.
// ---------------------------------------------------------------------------------
// A pixel is shadow when its background is below this fraction of the background it is
// compared against.
constexpr float SHADOW_RATIO = 0.35f;
// The boundary grows outward into partially shadowed pixels down to this fraction, but no
// further than PENUMBRA_MAX_PX from the core.
constexpr float PENUMBRA_RATIO = 0.72f;
constexpr int PENUMBRA_MAX_PX = 14;
// Bridge module gaps and small breaks that the holder arm crosses.
constexpr int BRIDGE_PX = 6;
// A pixel whose maximum reaches this recorded a real reflection and is never masked - a
// beam stop cannot block a reflection that was measured.
constexpr int64_t MIN_REFLECTION = 25;
// Counts the background must have accumulated over the frames and the pooled pixels before
// a dip in it is believable. Below this a Poisson hole is indistinguishable from a shadow,
// and testing anyway masks whole detectors on low-background data.
constexpr double MIN_EXPECTED_COUNTS = 60;
// Side of the box the background is pooled over before testing, and how far out the radial
// comparison looks for unshadowed background.
constexpr int POOL_PX = 5;
constexpr float ENVELOPE_MM = 6.0f;
// Binary-image helpers on a width*height frame stored row-major as char (0/1). All run once,
// at GetMask() time; the BFS forms keep them O(pixels) rather than O(pixels * radius).
namespace {
// 8-connected dilation by `r` pixels (Chebyshev), via a multi-source BFS.
std::vector<char> Dilate(const std::vector<char> &in, int W, int H, int r) {
std::vector<char> dilate(const std::vector<char> &in, int W, int H, int r) {
if (r <= 0)
return in;
std::vector<int> dist(in.size(), -1);
@@ -46,12 +69,12 @@ std::vector<char> Dilate(const std::vector<char> &in, int W, int H, int r) {
return out;
}
// Erosion by `r` = dilation of the complement (image border counts as outside).
std::vector<char> Erode(const std::vector<char> &in, int W, int H, int r) {
// Erosion by `r` = dilation of the complement; outside the frame counts as complement.
std::vector<char> erode(const std::vector<char> &in, int W, int H, int r) {
std::vector<char> comp(in.size());
for (size_t i = 0; i < in.size(); i++)
comp[i] = !in[i];
const auto grown = Dilate(comp, W, H, r);
const auto grown = dilate(comp, W, H, r);
std::vector<char> out(in.size());
for (size_t i = 0; i < out.size(); i++)
out[i] = !grown[i];
@@ -59,13 +82,11 @@ std::vector<char> Erode(const std::vector<char> &in, int W, int H, int r) {
}
// Pixels of `passable` reachable from any of `seeds` (8-connected flood).
std::vector<char> Flood(const std::vector<char> &passable, int W, int H, const std::vector<int> &seeds) {
std::vector<char> flood(const std::vector<char> &passable, int W, int H, const std::vector<int> &seeds) {
std::vector<char> visited(passable.size(), 0);
std::queue<int> q;
for (const int s : seeds)
if (s >= 0 && s < static_cast<int>(passable.size()) && passable[s] && !visited[s]) {
visited[s] = 1; q.push(s);
}
if (passable[s] && !visited[s]) { visited[s] = 1; q.push(s); }
while (!q.empty()) {
const int i = q.front(); q.pop();
const int y = i / W, x = i % W;
@@ -82,7 +103,7 @@ std::vector<char> Flood(const std::vector<char> &passable, int W, int H, const s
}
// Fill holes: background not reachable from the image border becomes region.
std::vector<char> FillHoles(const std::vector<char> &region, int W, int H) {
std::vector<char> fill_holes(const std::vector<char> &region, int W, int H) {
std::vector<char> bg_visited(region.size(), 0);
std::queue<int> q;
auto push = [&](int i) { if (!region[i] && !bg_visited[i]) { bg_visited[i] = 1; q.push(i); } };
@@ -107,9 +128,36 @@ std::vector<char> FillHoles(const std::vector<char> &region, int W, int H) {
return out;
}
// Sum of `in` over the k x k box centred on each pixel, zero outside the frame.
std::vector<double> box_sum(const std::vector<double> &in, int W, int H, int k) {
const int half = k / 2;
std::vector<double> row(in.size(), 0.0), out(in.size(), 0.0);
for (int y = 0; y < H; y++) {
double s = 0;
for (int x = 0; x <= std::min(half, W - 1); x++)
s += in[y * W + x];
for (int x = 0; x < W; x++) {
row[y * W + x] = s;
if (x + half + 1 < W) s += in[y * W + x + half + 1];
if (x - half >= 0) s -= in[y * W + x - half];
}
}
for (int x = 0; x < W; x++) {
double s = 0;
for (int y = 0; y <= std::min(half, H - 1); y++)
s += row[y * W + x];
for (int y = 0; y < H; y++) {
out[y * W + x] = s;
if (y + half + 1 < H) s += row[(y + half + 1) * W + x];
if (y - half >= 0) s -= row[(y - half) * W + x];
}
}
return out;
}
// Median of `values` per integer radius, over the pixels flagged in `use`.
std::vector<float> RingMedian(const std::vector<float> &values, const std::vector<char> &use,
const std::vector<int> &radius, int max_radius) {
std::vector<float> ring_median(const std::vector<float> &values, const std::vector<char> &use,
const std::vector<int> &radius, int max_radius) {
std::vector<std::vector<float>> bins(max_radius + 1);
for (size_t i = 0; i < values.size(); i++)
if (use[i])
@@ -126,39 +174,42 @@ std::vector<float> RingMedian(const std::vector<float> &values, const std::vecto
return median;
}
// Fraction of each integer-radius ring that is flagged in `blocked`.
std::vector<float> RingFraction(const std::vector<char> &blocked, const std::vector<int> &radius, int max_radius) {
std::vector<int64_t> num(max_radius + 1, 0), den(max_radius + 1, 0);
for (size_t i = 0; i < blocked.size(); i++) {
den[radius[i]]++;
if (blocked[i]) num[radius[i]]++;
// Largest baseline over [r, r + win] - the background just outside radius r.
std::vector<float> outer_envelope(const std::vector<float> &baseline, int win) {
const int n = static_cast<int>(baseline.size());
std::vector<float> out(n, 0.0f);
for (int r = 0; r < n; r++) {
float v = baseline[r];
for (int k = 1; k <= win; k++)
v = std::max(v, baseline[std::min(r + k, n - 1)]);
out[r] = v;
}
std::vector<float> frac(max_radius + 1, 0.0f);
for (int r = 0; r <= max_radius; r++)
frac[r] = den[r] ? static_cast<float>(num[r]) / static_cast<float>(den[r]) : 0.0f;
return frac;
return out;
}
} // namespace
// ---------------------------------------------------------------------------------
ShadowFinder::ShadowFinder(const DiffractionExperiment &experiment, ShadowFinderSettings in_settings)
ShadowFinder::ShadowFinder(const DiffractionExperiment &experiment, const PixelMask &mask)
: width(static_cast<int>(experiment.GetXPixelsNumConv())),
height(static_cast<int>(experiment.GetYPixelsNumConv())),
beam_x(experiment.GetBeamX_pxl()),
beam_y(experiment.GetBeamY_pxl()),
settings(in_settings),
envelope_px(std::max(4, static_cast<int>(std::lround(ENVELOPE_MM / experiment.GetPixelSize_mm())))),
pixel_mask(mask.GetMask(experiment)),
max_value(static_cast<size_t>(width) * height, 0),
sum_value(static_cast<size_t>(width) * height, 0),
valid_count(static_cast<size_t>(width) * height, 0) {}
valid_count(static_cast<size_t>(width) * height, 0) {
if (pixel_mask.size() != max_value.size())
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"ShadowFinder: pixel mask does not match the detector");
}
template<class T>
void ShadowFinder::Add(const T *ptr) {
// The pixel type's sentinel extreme marks "no data" (module gap / masked): the
// preprocessor/writer stores INT*_MIN for signed and UINT*_MAX for unsigned. For
// signed types the opposite extreme (INT*_MAX) is a genuine saturated value and is
// kept, so a saturated reflection still registers as bright.
// preprocessor/writer stores INT*_MIN for signed and UINT*_MAX for unsigned. For signed
// types the opposite extreme is a genuine saturated value and is kept, so a saturated
// reflection still registers as bright.
T masked;
if constexpr (std::is_signed_v<T>)
masked = std::numeric_limits<T>::min();
@@ -170,7 +221,7 @@ void ShadowFinder::Add(const T *ptr) {
const T v = ptr[i];
if (v == masked)
continue;
const int32_t vi = static_cast<int32_t>(v);
const int64_t vi = static_cast<int64_t>(v);
if (valid_count[i] == 0 || vi > max_value[i])
max_value[i] = vi;
sum_value[i] += vi;
@@ -207,70 +258,96 @@ std::vector<uint32_t> ShadowFinder::GetMask() const {
std::unique_lock ul(m);
const int W = width, H = height;
const int N = W * H;
const ShadowFinderSettings &S = settings;
const int n_pixels = W * H;
std::vector<uint32_t> mask(N, 0);
std::vector<uint32_t> mask(n_pixels, 0);
if (frames == 0)
return mask;
// --- mean projection, per-pixel validity and radius from the beam centre ---
std::vector<float> mean(N, 0.0f);
std::vector<char> valid(N, 0);
std::vector<int> radius(N, 0);
// mean projection, usable pixels and radius from the beam centre
std::vector<float> mean(n_pixels, 0.0f);
std::vector<char> valid(n_pixels, 0);
std::vector<int> radius(n_pixels, 0);
int max_radius = 0;
for (int y = 0; y < H; y++)
for (int x = 0; x < W; x++) {
const int i = y * W + x;
if (valid_count[i] > 0) {
if (valid_count[i] > 0 && pixel_mask[i] == 0) {
mean[i] = static_cast<float>(static_cast<double>(sum_value[i]) / valid_count[i]);
valid[i] = 1;
}
const double dx = x - beam_x, dy = y - beam_y;
const int r = static_cast<int>(std::lround(std::sqrt(dx * dx + dy * dy)));
radius[i] = r;
if (r > max_radius) max_radius = r;
const float dx = x - beam_x, dy = y - beam_y;
radius[i] = static_cast<int>(std::lround(std::sqrt(dx * dx + dy * dy)));
max_radius = std::max(max_radius, radius[i]);
}
// --- robust radial baseline; iterate to keep the shadow out of its own baseline ---
std::vector<float> ratio(N, 1.0f);
std::vector<char> excluded(N, 0);
// Pool the background over a small box before testing it. A background of a fraction of
// a count per pixel per frame gives no single pixel enough counts to tell a shadow from
// a Poisson hole; the stop and its arm are wider than the box, so pooling costs no
// resolution that matters and multiplies the statistics by the pixels in the box.
std::vector<double> num(n_pixels), den(n_pixels);
for (int i = 0; i < n_pixels; i++) {
num[i] = valid[i] ? mean[i] : 0.0;
den[i] = valid[i] ? 1.0 : 0.0;
}
const auto pooled_sum = box_sum(num, W, H, POOL_PX);
const auto pooled_count = box_sum(den, W, H, POOL_PX);
std::vector<float> pooled(n_pixels, 0.0f);
for (int i = 0; i < n_pixels; i++)
if (pooled_count[i] > 0)
pooled[i] = static_cast<float>(pooled_sum[i] / pooled_count[i]);
// Azimuthal comparison: the median of the ring, iterated so the shadow stays out of the
// baseline it is measured against.
std::vector<float> ratio(n_pixels, 1.0f);
std::vector<char> excluded(n_pixels, 0);
std::vector<float> baseline;
for (int iter = 0; iter < 3; iter++) {
std::vector<char> use(N);
for (int i = 0; i < N; i++)
std::vector<char> use(n_pixels);
for (int i = 0; i < n_pixels; i++)
use[i] = valid[i] && !excluded[i];
const auto baseline = RingMedian(mean, use, radius, max_radius);
for (int i = 0; i < N; i++)
baseline = ring_median(pooled, use, radius, max_radius);
for (int i = 0; i < n_pixels; i++)
if (valid[i])
ratio[i] = mean[i] / std::max(baseline[radius[i]], 1e-6f);
for (int i = 0; i < N; i++)
excluded[i] = valid[i] && ratio[i] < S.shadow_ratio;
ratio[i] = pooled[i] / std::max(baseline[radius[i]], 1e-6f);
for (int i = 0; i < n_pixels; i++)
excluded[i] = valid[i] && ratio[i] < SHADOW_RATIO;
}
// --- shadow core: low-ratio pixels connected to the beam centre (bridging gaps) ---
std::vector<char> low(N);
for (int i = 0; i < N; i++)
low[i] = valid[i] && ratio[i] < S.shadow_ratio;
// Radial comparison: the background just outside this radius. The disk blocks its rings
// completely, so their median is the shadow itself and only this comparison sees it.
const auto envelope = outer_envelope(baseline, envelope_px);
std::vector<float> ratio_radial(n_pixels, 1.0f);
for (int i = 0; i < n_pixels; i++)
if (valid[i])
ratio_radial[i] = pooled[i] / std::max(envelope[radius[i]], 1e-6f);
const std::vector<char> grown = Dilate(low, W, H, S.bridge_px);
std::vector<int> seeds; // a small disk at the beam centre
for (int y = 0; y < H; y++)
for (int x = 0; x < W; x++) {
const double dx = x - beam_x, dy = y - beam_y;
if (dx * dx + dy * dy < 4.0 * 4.0)
seeds.push_back(y * W + x);
}
const std::vector<char> connected = Flood(grown, W, H, seeds);
std::vector<char> core(N);
for (int i = 0; i < N; i++)
core[i] = low[i] && connected[i];
// A dip counts only where the background it is compared against was actually counted.
std::vector<char> low(n_pixels, 0);
for (int i = 0; i < n_pixels; i++) {
if (!valid[i])
continue;
const double counted = frames * pooled_count[i];
low[i] = (ratio[i] < SHADOW_RATIO && baseline[radius[i]] * counted >= MIN_EXPECTED_COUNTS)
|| (ratio_radial[i] < SHADOW_RATIO && envelope[radius[i]] * counted >= MIN_EXPECTED_COUNTS);
}
// --- real reflections: any pixel that recorded signal is never masked. Require a
// small cluster so a single-frame zinger does not count as a reflection. ---
std::vector<char> lit(N, 0);
for (int i = 0; i < N; i++)
lit[i] = (valid_count[i] > 0) && (max_value[i] >= static_cast<int32_t>(S.min_reflection));
std::vector<char> reflection(N, 0);
// The shadow is the low region connected to the beam centre, bridging the gaps it crosses.
const std::vector<char> bridged = dilate(low, W, H, BRIDGE_PX);
std::vector<int> seeds;
for (int i = 0; i < n_pixels; i++)
if (radius[i] < 4)
seeds.push_back(i);
const std::vector<char> connected = flood(bridged, W, H, seeds);
std::vector<char> region(n_pixels);
for (int i = 0; i < n_pixels; i++)
region[i] = low[i] && connected[i];
// Recorded reflections. A small cluster is required so a single-frame zinger does not count.
std::vector<char> lit(n_pixels, 0);
for (int i = 0; i < n_pixels; i++)
lit[i] = (valid_count[i] > 0) && (max_value[i] >= MIN_REFLECTION);
std::vector<char> reflection(n_pixels, 0);
for (int y = 0; y < H; y++)
for (int x = 0; x < W; x++) {
const int i = y * W + x;
@@ -285,54 +362,23 @@ std::vector<uint32_t> ShadowFinder::GetMask() const {
reflection[i] = (neighbours >= 2);
}
// --- central low-res disk: the fully-blocked region about the beam centre. Sized by
// the azimuthal blocked fraction (a disk blocks ~every azimuth; a thin arm or
// gap does not), and capped just inside the innermost reflection. ---
std::vector<char> blocked(N);
for (int i = 0; i < N; i++)
blocked[i] = (valid_count[i] == 0) || low[i];
const auto blocked_frac = RingFraction(blocked, radius, max_radius);
int disk_radius = 0;
{
float head = 0.0f; int head_n = 0;
for (int r = 0; r <= std::min(5, max_radius); r++) { head += blocked_frac[r]; head_n++; }
if (head_n > 0 && head / head_n >= 0.65f) { // the beam centre is behind a disk
disk_radius = max_radius;
for (int r = 1; r <= max_radius; r++)
if (blocked_frac[r] < 0.55f) { disk_radius = r; break; }
}
}
int reflection_radius = max_radius + 1; // innermost reflection (ignore the very centre)
for (int i = 0; i < N; i++)
if (reflection[i] && radius[i] > 12 && radius[i] < reflection_radius)
reflection_radius = radius[i];
if (disk_radius > reflection_radius - 4)
disk_radius = reflection_radius - 4;
if (disk_radius < 0)
disk_radius = 0;
// --- assemble: core + disk, grow the soft penumbra, round, fill the disk interior ---
std::vector<char> region(N);
for (int i = 0; i < N; i++)
region[i] = core[i] || (disk_radius > 0 && radius[i] < disk_radius);
const std::vector<char> near = Dilate(region, W, H, S.penumbra_max_px);
for (int i = 0; i < N; i++)
if (near[i] && valid[i] && ratio[i] < S.penumbra_ratio)
// Grow the soft boundary, round it and fill the disk interior.
const std::vector<char> penumbra = dilate(region, W, H, PENUMBRA_MAX_PX);
for (int i = 0; i < n_pixels; i++)
if (penumbra[i] && valid[i] && std::min(ratio[i], ratio_radial[i]) < PENUMBRA_RATIO)
region[i] = 1;
region = Erode(Dilate(region, W, H, 2), W, H, 2); // close: round the boundary
region = FillHoles(region, W, H);
region = erode(dilate(region, W, H, 2), W, H, 2);
region = fill_holes(region, W, H);
// Expose recorded reflections - done last, with no fill afterwards, so a spot the
// geometry still covered is given back rather than re-enclosed.
const std::vector<char> reflection_grown = Dilate(reflection, W, H, 1);
for (int i = 0; i < N; i++)
// Expose recorded reflections - done last, with no fill afterwards, so a spot the shadow
// still covered is given back rather than re-enclosed.
const std::vector<char> reflection_grown = dilate(reflection, W, H, 1);
for (int i = 0; i < n_pixels; i++)
if (reflection_grown[i])
region[i] = 0;
for (int i = 0; i < N; i++)
for (int i = 0; i < n_pixels; i++)
mask[i] = region[i] ? 1 : 0;
return mask;
}