Files
Jungfraujoch/image_analysis/bragg_integration/BraggIntegrationEngine.h
leonarski_fandClaude Opus 5 f0cdb027e1 Ice: default the merge mask off, gate the radial background on smooth ice, and pick detection by geometry
Three defaults, each settled by measurement rather than by argument. The
arbiter throughout is structure-referenced - anomalous peak height where a
crystal can carry it, and otherwise the agreement of the ice bands with a fixed
external model against resolution-matched DECOY bands carrying no ice. The
band-versus-decoy contrast is used because R-free here tracks completeness, and
every one of these switches moves completeness.

The damage is real and it localizes: over the rotation battery the ice bands'
excess amplitude reaches +9.6% on a smooth-ice crystal and +35% on the worst,
while a clean control sits at +0.6% (z +0.45). On the worst crystal, nine of the
ten largest excess peaks in a q scan land on hexagonal ring positions. Turning
ice handling off leaves the contrast unchanged and forcing it on a clean crystal
does not create one, so it is the ice and not the machinery.

MERGE-TIME RING MASK -> OFF. It deletes reflections, which no other program does
by default - AIMLESS, DIALS, xia2, XDS and CrystFEL all keep ice-band
reflections in the merge and exclude them only from the model fit; autoPROC is
the sole exception. On the one battery crystal where the mask fires and an
anomalous arbiter can score it, dropping the band moved the mean peak height at
the known sites by -0.001 +- 0.018 sigma, 2% of the site height, while removing
1149 unique reflections whose mean I/sigma was 3.62 against the dataset's own
3.05 - better than average data - and costing 17 completeness points in that
shell. It fires on 5 of 37 crystals, changes no space group, and those 5
disagree in sign: it clearly helps the two most heavily iced, is a wash on two
and costs a third. So it stays as a switch, worth setting by hand on a badly
iced crystal where it shows in the high shell, but it is not a default.

RADIAL BACKGROUND -> AUTO, gated per image. The correction models the background
as a function of radius alone, and that is exactly when it works. On a crystal
with pure smooth powder ice it removes 43% of the bands' excess amplitude, with
the improvement 7x larger inside the bands than outside; on a crystal whose ice
is discrete crystallite spots - no smooth ring to model - the excess amplitude
GREW by half; on clean data it is inert to four decimals. The two ice channels
already separate those morphologies, so --background-radial takes on|off|auto
and auto applies it to an image when that image's peak-excluded score reaches
--ice-min-score. Auto never engages without such a score, because the plain
profile carries the Bragg peaks and cannot support an absolute threshold.

Per image rather than per run, and that was tested rather than assumed: the
gate fires on 100% and 94% of frames on the two crystals that want it, and on
1.5% of frames - 32 blocks, 23 of them single frames - on the textured-ice
crystal. A seam statistic against off + f*(on - off) is null on both mixed runs,
every merge statistic is bracketed by the pure arms, and the textured crystal's
auto arm lands on `off` rather than on `on`'s harm. A run-level gate would need
the score before the pass that integrates, i.e. rotation-only plumbing, and buys
nothing measurable.

The kernel was already built unconditionally, so flipping the flag per image is
free - except on the GPU, where the launches were gated on a construction-time
n_rad. That is why the buffers are now allocated whenever the correction could
run, and Run() decides per image.

DETECTION -> the geometry's default when the file is silent: on for rotation,
off for stills, with the command line and then the file taking precedence. A
rotation sweep sits on the same rings for the whole run, so ice there is a
coherent systematic and the presence gate keeps it inert on a clean crystal; a
serial stills run has too few spots per image to spend any on flagging. The
master file's key is kept as written rather than collapsed to a bool, so "the
file said nothing" is distinguishable from "the file said no" - it used to fall
silently to off, taking the exclusion from the scale fit with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 19:06:30 +02:00

142 lines
8.0 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
// =============================================================================
// BraggIntegrationEngine — box-sum + profile-fitting 2D integrator, GPU-ready
// =============================================================================
//
// A reimplementation of BraggIntegrate2D (box sum) and ProfileIntegrate2D (Kabsch profile
// fit) under one roof, following the AzIntEngine / ROIIntegration pattern: a base class that
// extracts the fixed per-experiment configuration, a plain-C++ CPU engine (the fallback and the
// numeric oracle), and a CUDA engine (BraggIntegrationEngineGPU) that reaches the same result up
// to floating-point precision.
//
// Unlike BraggIntegrate2D/ProfileIntegrate2D, which read the raw CompressedImage per pixel type
// and reject the special/saturation +/-1 band, this engine reads the already-preprocessed int32
// image held in an ImagePreprocessorBuffer (the same buffer AzIntEngineGPU/ROIIntegrationGPU
// consume): masked/bad pixels are INT32_MIN and saturated pixels INT32_MAX, so bad-pixel identity
// is owned by the preprocessor and a pixel is valid iff v != INT32_MIN && v != INT32_MAX.
//
// The integrator is selected by BraggIntegrationSettings::Integrator:
// BoxSum -> BraggIntegrate2D equivalent (rough disk sum minus ring-mean background)
// ProfileGaussian -> per-reflection measured-width Gaussian profile fit (the default)
// ProfileEmpirical-> per-shell learned empirical profile fit
// The box sum is also the seed pass (Pass A) of the two profile modes, so it always runs.
//
// This is the Bragg integrator used by the pipeline (bound in MXAnalysisWithoutFPGA: the GPU
// engine when a device is present, otherwise the CPU engine). It takes a preprocessed image +
// the predicted reflections and returns the vector<Reflection> (I, sigma, bkg, partiality, ...)
// that the downstream scaling/merge consumes unchanged.
// =============================================================================
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <optional>
#include <vector>
#include "../../common/BraggIntegrationSettings.h"
#include "../../common/DiffractionExperiment.h"
#include "../../common/DiffractionGeometry.h"
#include "../../common/Reflection.h"
#include "../image_preprocessing/ImagePreprocessorBuffer.h"
namespace bragg_engine {
// Shared with both engines so the CPU and GPU paths stay numerically aligned.
constexpr int N_SHELL = 6; // resolution shells for per-shell profile learning
constexpr double STRONG_I_OVER_SIGMA = 5.0; // strong-spot threshold that seeds the profile
constexpr int MIN_STRONG_PER_SHELL = 30; // below this a shell falls back to the global profile
constexpr double C_CAPTURE = 2.5; // weak-spot radial capture term (monochromatic only)
// Per-pixel variance floor for the Kabsch fit weights (v = floor + signal). The detector noise floor is
// the quantization noise from rounding the charge-spread deposited energy to an integer: a uniform
// rounding error has variance 1/12. Electronic noise is far below this for both EIGER and JUNGFRAU. A
// larger floor (the previous 1.0) silently over-regularizes — it inflates weak-reflection sigma and
// pins the scaling error model's `a` term at its floor.
constexpr double PIXEL_VARIANCE_FLOOR = 1.0 / 12.0;
// Guard against profile-fit runaways: on a weak / near-zero reflection the reweighted Kabsch iteration
// has no real peak to lock onto and can manufacture intensity the box sum never sees. Fall back to the
// summation (box-sum) intensity when the profile result disagrees with the summation seed by more than
// this many box-sum sigmas (a real fit agrees within counting noise, so the margin is generous).
constexpr double PROFILE_SUMMATION_MAX_NSIGMA = 10.0;
} // namespace bragg_engine
// One reflection's extracted intensity, produced by the derived engine and turned into a
// Reflection by Finalize() (which owns the polarization correction and scale bookkeeping).
struct BraggFitResult {
float I = 0.0f;
float sigma = NAN;
float bkg = 0.0f;
float observed_x = 0.0f; // intensity-weighted centroid (BoxSum mode only)
float observed_y = 0.0f;
bool ok = false;
bool has_observed = false;
};
class BraggIntegrationEngine {
protected:
// --- fixed configuration extracted from the experiment (see ProfileIntegrate2D) ---
IntegratorMode mode;
bool empirical; // ProfileEmpirical (vs ProfileGaussian)
size_t xpixel, ypixel, npixel;
float r1_sq;
float r2, r2_sq;
float r3, r3_sq;
float min_sigma_ratio;
int R, G, GG; // profile-grid half-size, edge (2R+1) and area (G*G)
bool broadband; // a set bandwidth (stills) vs monochromatic (rotation)
double bw_sigma; // bandwidth sigma [dimensionless, * Rpx -> px]
float bkg_clip_nsigma; // high-outlier background sigma-clip multiplier (0 = no clip)
bool use_ellipse; // radially elongate the per-reflection Gaussian
double c_radial; // radial variance coefficient of tan^2(2theta): parallax + capture
double F_px; // detector distance expressed in pixels
float beam_x, beam_y;
// Effective symmetric trimmed-mean background fraction (BraggIntegrationSettings): the configured
// fraction for monochromatic (rotation) data, forced to 0 for broadband (stills, which keep their
// high-side sigma-clip). 0 = plain ring mean. Read by both the CPU and GPU engines.
float bkg_trim = 0.0f;
// --- radial background curvature correction (BraggIntegrationSettings) ---
// The disk and the annulus are concentric, so any background LINEAR in position cancels between
// them; what survives is the curvature of the radial background. Every reflection uses the same
// stencil, so mean_annulus(B) - mean_disk(B) of a radial B is a FIXED kernel over radial offset:
// bkg_error = sum_k k_diff[k] * B(r0 + k - k_off)
// That is one short dot product per reflection and reads no pixels. Built in the constructor.
// The kernel is built whatever the setting says, so bkg_radial can be flipped between images at
// no cost - which is what the auto mode does, applying the correction only to the images whose
// background really is a smooth function of radius (see BackgroundRadial below).
bool bkg_radial = false;
bool bkg_radial_auto = false; // settings left it unset: decide per image from the ice score
int k_off = 0; // index of offset 0 in k_diff
std::vector<float> k_diff; // annulus-minus-disk weight per integer radial offset
DiffractionGeometry geom; // kept for the per-reflection polarization correction
std::optional<float> polarization;
// Assemble output reflections from the per-reflection fit results (polarization + scale corr).
std::vector<Reflection> Finalize(const std::vector<Reflection> &predicted, size_t npredicted,
const std::vector<BraggFitResult> &results,
int64_t image_number) const;
public:
explicit BraggIntegrationEngine(const DiffractionExperiment &experiment);
virtual ~BraggIntegrationEngine() = default;
// predicted[0..npredicted) are the reflections to extract; image is the preprocessed int32
// frame (image.size() == npixel). Returns only the observed reflections.
virtual std::vector<Reflection> Run(const ImagePreprocessorBuffer &image,
const std::vector<Reflection> &predicted, size_t npredicted,
int64_t image_number) = 0;
// Turn the radial background correction on or off for the images that follow. The caller owns
// the decision; in the auto mode the analysis sets it per image from that image's ice score.
void BackgroundRadial(bool on) { bkg_radial = on; }
[[nodiscard]] bool IsBackgroundRadialAuto() const { return bkg_radial_auto; }
};