Jungfraujoch/image_analysis/beam_stop/SHADOW_FINDER.md

Beam-stop shadow finder

ShadowFinder detects the beam-stop shadow — the central disk plus its holder arm — from a small number of images, and returns a per-pixel mask of the shadowed region. It exists so the pipeline can mask the beam stop automatically instead of relying on a hand-drawn user mask.

It is written in the accumulate-then-finalize shape of DarkMaskAnalysis: feed frames with AddImage(), then read the mask once with GetMask(). It is not yet wired into any workflow — see Wiring plan below. The class is self-contained (common/ only) and unit-testable.

API

ShadowFinder finder(experiment);                 // geometry (beam centre, converted size)
std::vector<uint8_t> scratch;
for (int i : first_N_frames)                      // ~20–50 spread frames is plenty (see below)
    finder.AddImage(read_frame(i), scratch);      // DataMessage with .image, like DarkMaskAnalysis
std::vector<uint32_t> shadow = finder.GetMask();  // 1 = shadow, 0 = keep; converted geometry

• Input (AddImage): a full converted-geometry image inside a DataMessage, dispatched by pixel type exactly like DarkMaskAnalysis::AnalyzeImage. A pixel equal to its type's sentinel extreme (INT*_MIN for signed, UINT*_MAX for unsigned) is treated as no data (module gap / masked) and skipped. For signed types the opposite extreme (INT*_MAX, a genuine saturation clamp) is kept, so saturated reflections still register as bright. Thread-safe.
• Output (GetMask): a 0/1 vector of size GetPixelsNumConv(), ready to feed PixelMask::LoadMask(..., bit). Recomputed from the accumulators on each call — call it once at the end.
• Settings (ShadowFinderSettings): plain struct with defaults (below). When wired, this can move onto DiffractionExperiment the way DarkMaskSettings does.

Algorithm

The shadow is a stationary, azimuthal anomaly: air scatter makes the background fall off smoothly with radius but stay roughly constant around each ring, so the beam stop is a localized dip at a fixed place. Detection therefore compares each pixel to the typical background at its own radius, not to a global threshold.

1. Projection. Per pixel accumulate the max and the mean (sum + count) over the added frames, skipping the no-data sentinel. The stop is stationary, so it stays dark while spots and the air-scatter halo build up around it.
2. Robust radial baseline. Take the median of the mean-projection in each 1-pixel ring about the beam centre. The median ignores the shadow (a minority of each ring) and the spots. Iterate 3×, excluding the shadow it finds, so the baseline is not pulled down by the shadow.
3. Ratio. ratio = mean / baseline(r). Shadowed pixels sit far below 1; the smooth halo and symmetric ice rings cancel to ≈1 and are not flagged.
4. Core + connectivity. core = ratio < shadow_ratio, then keep only the part connected to the beam centre (flood from a small central seed through a bridge_px-dilated copy, so the arm crosses module gaps). The centre anchor is what makes it specific to the beam stop rather than any random low patch.
5. Reflection guard. A pixel whose max-projection reached min_reflection recorded a real reflection — a beam stop cannot block a reflection that was measured — so it is never masked (a small-cluster test drops single-pixel zingers).
6. Central disk (low-res guard). From the azimuthal blocked fraction about the beam centre, mask a full disk out to the radius where nearly every azimuth is blocked (a disk blocks all directions; a thin arm/gap does not). Returns nothing when the centre is not behind a disk, and is capped just inside the innermost reflection so a real edge spot is never swallowed. This guards the low-resolution zone the stop makes un-measurable, including a disk the raw data has already voided.
7. Soft boundary. Grow the core+disk into partially-shadowed pixels (ratio < penumbra_ratio) within penumbra_max_px, then a morphological close rounds the contour and hole-filling closes the disk interior.
8. Carve reflections last. Remove any lit pixel from the final mask (no fill afterwards) so a spot the geometry still covered is given back rather than re-enclosed.

Three findings from the prototype are baked in: the mean ratio (not the max) does the detecting — the max projection is a great picture but a noisier detector; the centre anchor is what distinguishes the stop from stray low patches; and a lit pixel is never masked. Detection converges in ~2–3 frames (the shadow cuts the halo present in every frame); 20–50 is just safe headroom.

Parameters (ShadowFinderSettings)

field	default	meaning
shadow_ratio	0.35	core threshold: mean below this fraction of the radial baseline
penumbra_ratio	0.72	soft boundary grows out to this ratio …
penumbra_max_px	14	… but no further than this from the core
bridge_px	6	bridge module gaps / breaks the arm crosses
min_reflection	25	max-projection value at/above which a pixel is a protected reflection

Complexity & memory

Per AddImage: O(pixels). GetMask: a handful of O(pixels) sweeps (three ring-median passes and a few BFS-based morphology ops), i.e. a couple of seconds on a full ~18 MP detector — a one-time finalize. Accumulators are int32 + int64 + uint32 per pixel (~0.3 GB at 18 MP); the finalize adds a few transient float/char buffers. Fine for the offline (Rugnux/viewer) context it targets; not intended for the online per-frame hot path.

Wiring plan (deferred — implement later)

The detector is deliberately reader-agnostic and bit-agnostic: one algorithm, fed frames by each caller, with the caller deciding where the resulting mask lands. Two intended consumers:

1. Build. Add beam_stop/ShadowFinder.cpp + .h to the JFJochImageAnalysis source list in image_analysis/CMakeLists.txt (mirroring dark_mask_analysis/DarkMaskAnalysis.* at lines 27–28).

2. Mask bit + loader (common/PixelMask.{h,cpp}): add BeamStopShadowBit = 9 (bit 9 is free; PixelMask.h:31-36), a LoadBeamStopMask mirroring LoadDarkBadPixelMask (.cpp:298-315), a beam_stop_pixel counter in PixelMaskStatistics / GetStatistics (.cpp:186-205), and optionally GetBeamStopMask() (mirroring GetUserMask). Leave GetUserMask untouched (keeps the user-mask channel clean). Application is then automatic: the CPU/GPU preprocessor masks any pixel with a non-zero bit (ImagePreprocessorCPU.cpp:10, ImagePreprocessorGPU.cu:104), and HDF5 write/read + CBOR already carry the full bitfield, so bit 9 persists to _process.h5 and reloads with no extra code.

3. Rugnux offline processing → bit 9 (dataset-specific, per run). Add a detect_beam_stop flag to ProcessConfig (rugnux/Rugnux.h) and a --detect-beam-stop CLI flag mirroring --detect-ice-rings in rugnux/rugnux_cli.cpp. In Rugnux::Run(), after geometry/goniometer setup and before the azimuthal mapping (Rugnux.cpp:281) and the start_message.pixel_mask persist (:296), run a ShadowFinder pre-pass over a spread sample (select_equally_spaced_image_ordinals

• reader_.GetRawImage, as RefineStillsGeometry does at :96-157) and pixel_mask_.LoadBeamStopMask(...). Everything downstream then applies and persists it for free.

4. Viewer standalone → user mask (bit 8) and a viewer preview. A "Find beam stop" action reads the first N frames via the worker's file_reader, runs ShadowFinder, and holds the result as bit 9 for a distinct-coloured, toggleable overlay; a "Transfer to user mask" action ORs it into the user mask through the existing chokepoint JFJochImageReadingWorker::UpdateUserMask_i (.cpp:783, same idiom as LoadUserMaskTIFF add-mode :851-858). The distinct colour needs a ColorScaleSpecial::BeamStop + sentinel branch in JFJochReaderImage.cpp:117-124; the on/off toggle mirrors "Show saturated pixels".

5. Docs. Add a "Bit 9 – beam-stop shadow" row to the bit table in docs/PIXEL_MASK.md:9-21.

Not needed: the online broker / receiver / FPGA / REST-API / React-frontend paths — the auto beam-stop mask is an offline (Rugnux + viewer) feature and never goes to the detector.

Known limitations / future refinements

• Boundary smoothing uses a morphological close; the prototype used a Gaussian level-set for a slightly smoother edge. Upgrade if the edge looks blocky.
• Off-centre disk: the low-res disk guard keys on the stored beam centre. If that is off the physical stop (seen on one stills detector), a small already-dead disk gets no extra circular guard — a data-driven disk centre would close it.
• Unsigned saturation: for unsigned images a detector-saturating reflection collides with the no-data sentinel and is skipped; harmless in practice (such reflections do not sit under the stop), and absent entirely on the signed/preprocessed path.