The profile fit weights each pixel by 1/v with v = max(bkg, floor) + max(0, I)*P, where I is the fit's own current estimate. Rectifying it means that at true zero the plug-in is E[max(0,I)] = 0.4*sigma rather than 0, and with sum(P^3)/sum(P^2)^2 = 4/3 for a Gaussian the reported sigma comes out about 0.2 counts too large - always, additively. That is nothing at sigma ~ 7 counts and 11% at sigma ~ 2, so it only shows on data measured against roughly one background count. Clamp the whole weight instead of the intensity: v = max(bkg + I*P, bkg/2). Simulation of the real integrator gives claimed/true sigma 0.92-1.01 at zero intensity across backgrounds 0.02-2.0 ct/px and 1.000-1.007 above I = 30, where the clamp never binds. Dropping the signal term entirely instead (v = max(bkg, floor)) is exact at zero and wrong everywhere else - 1.91 at I = 5, 4.29 at I = 30, 13.3 at I = 300 - and a test built on systematically absent reflections cannot see that, because it only measures zero. Removing the clamp altogether overshoots and biases the intensity, since a downward fluctuation shrinks v at the peak and over-weights it. The pixel variance floor was 1/12, documented as the rounding of a continuous energy. That does not describe a photon counter: measured on raw frames at 0.065-0.082 ct/px, var/mean is 1.042-1.045, i.e. Poisson with no digitisation term, and a digitisation term would be additive rather than a floor. What the floor really protects is the background estimate, which a small ring can read as exactly zero, so it belongs at the resolution of that estimate, ~1/n_bkg. At 1/12 it multiplied the reported variance by floor/bkg below 0.083 ct/px - a factor of two at 0.04. Set to 0.01. Measured on systematically absent reflections, whose true intensity is zero, as std(I)/rms(sigma) binned by background - not std(I/sigma), which is deflated by the correlation between the plug-in sigma and the reflection's own fluctuation. On 2.78 M absent observations at 0.16-3 ct/px the ratio goes 1.04-1.07 to 0.99-1.00. On 2.58 M at 0.005-0.6 ct/px, decomposed: the clamp carries it above 0.08 ct/px, the floor below it. Intensities move 0.4%; this changes sigma, not I. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
154 lines
9.1 KiB
C++
154 lines
9.1 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
// =============================================================================
|
|
// BraggIntegrationEngine — box-sum + profile-fitting 2D integrator, GPU-ready
|
|
// =============================================================================
|
|
//
|
|
// A reimplementation of BraggIntegrate2D (box sum) and ProfileIntegrate2D (Kabsch profile
|
|
// fit) under one roof, following the AzIntEngine / ROIIntegration pattern: a base class that
|
|
// extracts the fixed per-experiment configuration, a plain-C++ CPU engine (the fallback and the
|
|
// numeric oracle), and a CUDA engine (BraggIntegrationEngineGPU) that reaches the same result up
|
|
// to floating-point precision.
|
|
//
|
|
// Unlike BraggIntegrate2D/ProfileIntegrate2D, which read the raw CompressedImage per pixel type
|
|
// and reject the special/saturation +/-1 band, this engine reads the already-preprocessed int32
|
|
// image held in an ImagePreprocessorBuffer (the same buffer AzIntEngineGPU/ROIIntegrationGPU
|
|
// consume): masked/bad pixels are INT32_MIN and saturated pixels INT32_MAX, so bad-pixel identity
|
|
// is owned by the preprocessor and a pixel is valid iff v != INT32_MIN && v != INT32_MAX.
|
|
//
|
|
// The integrator is selected by BraggIntegrationSettings::Integrator:
|
|
// BoxSum -> BraggIntegrate2D equivalent (rough disk sum minus ring-mean background)
|
|
// ProfileGaussian -> per-reflection measured-width Gaussian profile fit (the default)
|
|
// ProfileEmpirical-> per-shell learned empirical profile fit
|
|
// The box sum is also the seed pass (Pass A) of the two profile modes, so it always runs.
|
|
//
|
|
// This is the Bragg integrator used by the pipeline (bound in MXAnalysisWithoutFPGA: the GPU
|
|
// engine when a device is present, otherwise the CPU engine). It takes a preprocessed image +
|
|
// the predicted reflections and returns the vector<Reflection> (I, sigma, bkg, partiality, ...)
|
|
// that the downstream scaling/merge consumes unchanged.
|
|
// =============================================================================
|
|
|
|
#include <cmath>
|
|
#include <cstddef>
|
|
#include <cstdint>
|
|
#include <optional>
|
|
#include <vector>
|
|
|
|
#include "../../common/BraggIntegrationSettings.h"
|
|
#include "../../common/DiffractionExperiment.h"
|
|
#include "../../common/DiffractionGeometry.h"
|
|
#include "../../common/Reflection.h"
|
|
#include "../image_preprocessing/ImagePreprocessorBuffer.h"
|
|
|
|
namespace bragg_engine {
|
|
// Shared with both engines so the CPU and GPU paths stay numerically aligned.
|
|
constexpr int N_SHELL = 6; // resolution shells for per-shell profile learning
|
|
constexpr double STRONG_I_OVER_SIGMA = 5.0; // strong-spot threshold that seeds the profile
|
|
constexpr int MIN_STRONG_PER_SHELL = 30; // below this a shell falls back to the global profile
|
|
constexpr double C_CAPTURE = 2.5; // weak-spot radial capture term (monochromatic only)
|
|
// Lower bound on the background term of the Kabsch fit weights (v = max(bkg, floor) + signal). It
|
|
// guards the background ESTIMATE, not the detector: the r2..r3 ring mean of a high-angle reflection
|
|
// can come out exactly zero, and v = 0 makes the weights P^2/v diverge. A ring of n pixels cannot
|
|
// resolve a background below ~1/n (0.005..0.02 for the default r2=6/r3=10 stencil), so that is the
|
|
// scale the floor has to work at. Anything larger over-regularizes: the floor multiplies the reported
|
|
// variance by floor/bkg for every pixel below it, so the previous 1/12 inflated sigma by 1.3x at
|
|
// 0.05 ct/px and 1.7x at 0.03 - exactly where the weakest high-resolution data live. Digitisation
|
|
// noise, where a detector has it, is additive on top of the background and does not belong here.
|
|
constexpr double PIXEL_VARIANCE_FLOOR = 0.01;
|
|
// The plug-in signal term of the fit weights may lower the per-pixel variance as well as raise it,
|
|
// but not below this fraction of the background. Half-wave rectifying it (max(0, I)) instead makes
|
|
// the weights - and so the reported 1/den - respond only to upward fluctuations of a noisy intensity
|
|
// estimate, which adds ~0.4*sigma*sum(P^3)/sum(P^2)^2 to every sigma whatever the count rate.
|
|
constexpr double WEIGHT_VARIANCE_MIN_FRACTION = 0.5;
|
|
|
|
// Guard against profile-fit runaways: on a weak / near-zero reflection the reweighted Kabsch iteration
|
|
// has no real peak to lock onto and can manufacture intensity the box sum never sees. Fall back to the
|
|
// summation (box-sum) intensity when the profile result disagrees with the summation seed by more than
|
|
// this many box-sum sigmas (a real fit agrees within counting noise, so the margin is generous).
|
|
constexpr double PROFILE_SUMMATION_MAX_NSIGMA = 10.0;
|
|
} // namespace bragg_engine
|
|
|
|
// One reflection's extracted intensity, produced by the derived engine and turned into a
|
|
// Reflection by Finalize() (which owns the polarization correction and scale bookkeeping).
|
|
struct BraggFitResult {
|
|
float I = 0.0f;
|
|
float sigma = NAN;
|
|
float bkg = 0.0f;
|
|
float observed_x = 0.0f; // intensity-weighted centroid (BoxSum mode only)
|
|
float observed_y = 0.0f;
|
|
// Variance of everything in `sigma` that is NOT the reflection's own Poisson signal (the
|
|
// background and the error of its estimate). The merge rebuilds each partial's variance at the
|
|
// pooled intensity and needs this term; back-deriving it as sigma^2 - I only works while that
|
|
// identity holds exactly, which it does not for a profile fit.
|
|
float var_bkg = 0.0f;
|
|
bool ok = false;
|
|
bool has_observed = false;
|
|
};
|
|
|
|
class BraggIntegrationEngine {
|
|
protected:
|
|
// --- fixed configuration extracted from the experiment (see ProfileIntegrate2D) ---
|
|
IntegratorMode mode;
|
|
bool empirical; // ProfileEmpirical (vs ProfileGaussian)
|
|
|
|
size_t xpixel, ypixel, npixel;
|
|
|
|
float r1_sq;
|
|
float r2, r2_sq;
|
|
float r3, r3_sq;
|
|
int R, G, GG; // profile-grid half-size, edge (2R+1) and area (G*G)
|
|
|
|
bool broadband; // a set bandwidth (stills) vs monochromatic (rotation)
|
|
double bw_sigma; // bandwidth sigma [dimensionless, * Rpx -> px]
|
|
float bkg_clip_nsigma; // high-outlier background sigma-clip multiplier (0 = no clip)
|
|
bool use_ellipse; // radially elongate the per-reflection Gaussian
|
|
|
|
double c_radial; // radial variance coefficient of tan^2(2theta): parallax + capture
|
|
double F_px; // detector distance expressed in pixels
|
|
float beam_x, beam_y;
|
|
// Effective symmetric trimmed-mean background fraction (BraggIntegrationSettings): the configured
|
|
// fraction for monochromatic (rotation) data, forced to 0 for broadband (stills, which keep their
|
|
// high-side sigma-clip). 0 = plain ring mean. Read by both the CPU and GPU engines.
|
|
float bkg_trim = 0.0f;
|
|
|
|
// --- radial background curvature correction (BraggIntegrationSettings) ---
|
|
// The disk and the annulus are concentric, so any background LINEAR in position cancels between
|
|
// them; what survives is the curvature of the radial background. Every reflection uses the same
|
|
// stencil, so mean_annulus(B) - mean_disk(B) of a radial B is a FIXED kernel over radial offset:
|
|
// bkg_error = sum_k k_diff[k] * B(r0 + k - k_off)
|
|
// That is one short dot product per reflection and reads no pixels. Built in the constructor.
|
|
// The kernel is built whatever the setting says, so bkg_radial can be flipped between images at
|
|
// no cost - which is what the auto mode does, applying the correction only to the images whose
|
|
// background really is a smooth function of radius (see BackgroundRadial below).
|
|
bool bkg_radial = false;
|
|
bool bkg_radial_auto = false; // settings left it unset: decide per image from the ice score
|
|
int k_off = 0; // index of offset 0 in k_diff
|
|
std::vector<float> k_diff; // annulus-minus-disk weight per integer radial offset
|
|
|
|
DiffractionGeometry geom; // kept for the per-reflection polarization correction
|
|
std::optional<float> polarization;
|
|
|
|
// Assemble output reflections from the per-reflection fit results (polarization + scale corr).
|
|
std::vector<Reflection> Finalize(const std::vector<Reflection> &predicted, size_t npredicted,
|
|
const std::vector<BraggFitResult> &results,
|
|
int64_t image_number) const;
|
|
|
|
public:
|
|
explicit BraggIntegrationEngine(const DiffractionExperiment &experiment);
|
|
virtual ~BraggIntegrationEngine() = default;
|
|
|
|
// predicted[0..npredicted) are the reflections to extract; image is the preprocessed int32
|
|
// frame (image.size() == npixel). Returns only the observed reflections.
|
|
virtual std::vector<Reflection> Run(const ImagePreprocessorBuffer &image,
|
|
const std::vector<Reflection> &predicted, size_t npredicted,
|
|
int64_t image_number) = 0;
|
|
|
|
// Turn the radial background correction on or off for the images that follow. The caller owns
|
|
// the decision; in the auto mode the analysis sets it per image from that image's ice score.
|
|
void BackgroundRadial(bool on) { bkg_radial = on; }
|
|
[[nodiscard]] bool IsBackgroundRadialAuto() const { return bkg_radial_auto; }
|
|
};
|