Three defaults, each settled by measurement rather than by argument. The arbiter throughout is structure-referenced - anomalous peak height where a crystal can carry it, and otherwise the agreement of the ice bands with a fixed external model against resolution-matched DECOY bands carrying no ice. The band-versus-decoy contrast is used because R-free here tracks completeness, and every one of these switches moves completeness. The damage is real and it localizes: over the rotation battery the ice bands' excess amplitude reaches +9.6% on a smooth-ice crystal and +35% on the worst, while a clean control sits at +0.6% (z +0.45). On the worst crystal, nine of the ten largest excess peaks in a q scan land on hexagonal ring positions. Turning ice handling off leaves the contrast unchanged and forcing it on a clean crystal does not create one, so it is the ice and not the machinery. MERGE-TIME RING MASK -> OFF. It deletes reflections, which no other program does by default - AIMLESS, DIALS, xia2, XDS and CrystFEL all keep ice-band reflections in the merge and exclude them only from the model fit; autoPROC is the sole exception. On the one battery crystal where the mask fires and an anomalous arbiter can score it, dropping the band moved the mean peak height at the known sites by -0.001 +- 0.018 sigma, 2% of the site height, while removing 1149 unique reflections whose mean I/sigma was 3.62 against the dataset's own 3.05 - better than average data - and costing 17 completeness points in that shell. It fires on 5 of 37 crystals, changes no space group, and those 5 disagree in sign: it clearly helps the two most heavily iced, is a wash on two and costs a third. So it stays as a switch, worth setting by hand on a badly iced crystal where it shows in the high shell, but it is not a default. RADIAL BACKGROUND -> AUTO, gated per image. The correction models the background as a function of radius alone, and that is exactly when it works. On a crystal with pure smooth powder ice it removes 43% of the bands' excess amplitude, with the improvement 7x larger inside the bands than outside; on a crystal whose ice is discrete crystallite spots - no smooth ring to model - the excess amplitude GREW by half; on clean data it is inert to four decimals. The two ice channels already separate those morphologies, so --background-radial takes on|off|auto and auto applies it to an image when that image's peak-excluded score reaches --ice-min-score. Auto never engages without such a score, because the plain profile carries the Bragg peaks and cannot support an absolute threshold. Per image rather than per run, and that was tested rather than assumed: the gate fires on 100% and 94% of frames on the two crystals that want it, and on 1.5% of frames - 32 blocks, 23 of them single frames - on the textured-ice crystal. A seam statistic against off + f*(on - off) is null on both mixed runs, every merge statistic is bracketed by the pure arms, and the textured crystal's auto arm lands on `off` rather than on `on`'s harm. A run-level gate would need the score before the pass that integrates, i.e. rotation-only plumbing, and buys nothing measurable. The kernel was already built unconditionally, so flipping the flag per image is free - except on the GPU, where the launches were gated on a construction-time n_rad. That is why the buffers are now allocated whenever the correction could run, and Run() decides per image. DETECTION -> the geometry's default when the file is silent: on for rotation, off for stills, with the command line and then the file taking precedence. A rotation sweep sits on the same rings for the whole run, so ice there is a coherent systematic and the presence gate keeps it inert on a clean crystal; a serial stills run has too few spots per image to spend any on flagging. The master file's key is kept as written rather than collapsed to a bool, so "the file said nothing" is distinguishable from "the file said no" - it used to fall silently to off, taking the exclusion from the scale fit with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
142 lines
8.0 KiB
C++
142 lines
8.0 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
// =============================================================================
|
|
// BraggIntegrationEngine — box-sum + profile-fitting 2D integrator, GPU-ready
|
|
// =============================================================================
|
|
//
|
|
// A reimplementation of BraggIntegrate2D (box sum) and ProfileIntegrate2D (Kabsch profile
|
|
// fit) under one roof, following the AzIntEngine / ROIIntegration pattern: a base class that
|
|
// extracts the fixed per-experiment configuration, a plain-C++ CPU engine (the fallback and the
|
|
// numeric oracle), and a CUDA engine (BraggIntegrationEngineGPU) that reaches the same result up
|
|
// to floating-point precision.
|
|
//
|
|
// Unlike BraggIntegrate2D/ProfileIntegrate2D, which read the raw CompressedImage per pixel type
|
|
// and reject the special/saturation +/-1 band, this engine reads the already-preprocessed int32
|
|
// image held in an ImagePreprocessorBuffer (the same buffer AzIntEngineGPU/ROIIntegrationGPU
|
|
// consume): masked/bad pixels are INT32_MIN and saturated pixels INT32_MAX, so bad-pixel identity
|
|
// is owned by the preprocessor and a pixel is valid iff v != INT32_MIN && v != INT32_MAX.
|
|
//
|
|
// The integrator is selected by BraggIntegrationSettings::Integrator:
|
|
// BoxSum -> BraggIntegrate2D equivalent (rough disk sum minus ring-mean background)
|
|
// ProfileGaussian -> per-reflection measured-width Gaussian profile fit (the default)
|
|
// ProfileEmpirical-> per-shell learned empirical profile fit
|
|
// The box sum is also the seed pass (Pass A) of the two profile modes, so it always runs.
|
|
//
|
|
// This is the Bragg integrator used by the pipeline (bound in MXAnalysisWithoutFPGA: the GPU
|
|
// engine when a device is present, otherwise the CPU engine). It takes a preprocessed image +
|
|
// the predicted reflections and returns the vector<Reflection> (I, sigma, bkg, partiality, ...)
|
|
// that the downstream scaling/merge consumes unchanged.
|
|
// =============================================================================
|
|
|
|
#include <cmath>
|
|
#include <cstddef>
|
|
#include <cstdint>
|
|
#include <optional>
|
|
#include <vector>
|
|
|
|
#include "../../common/BraggIntegrationSettings.h"
|
|
#include "../../common/DiffractionExperiment.h"
|
|
#include "../../common/DiffractionGeometry.h"
|
|
#include "../../common/Reflection.h"
|
|
#include "../image_preprocessing/ImagePreprocessorBuffer.h"
|
|
|
|
namespace bragg_engine {
|
|
// Shared with both engines so the CPU and GPU paths stay numerically aligned.
|
|
constexpr int N_SHELL = 6; // resolution shells for per-shell profile learning
|
|
constexpr double STRONG_I_OVER_SIGMA = 5.0; // strong-spot threshold that seeds the profile
|
|
constexpr int MIN_STRONG_PER_SHELL = 30; // below this a shell falls back to the global profile
|
|
constexpr double C_CAPTURE = 2.5; // weak-spot radial capture term (monochromatic only)
|
|
// Per-pixel variance floor for the Kabsch fit weights (v = floor + signal). The detector noise floor is
|
|
// the quantization noise from rounding the charge-spread deposited energy to an integer: a uniform
|
|
// rounding error has variance 1/12. Electronic noise is far below this for both EIGER and JUNGFRAU. A
|
|
// larger floor (the previous 1.0) silently over-regularizes — it inflates weak-reflection sigma and
|
|
// pins the scaling error model's `a` term at its floor.
|
|
constexpr double PIXEL_VARIANCE_FLOOR = 1.0 / 12.0;
|
|
|
|
// Guard against profile-fit runaways: on a weak / near-zero reflection the reweighted Kabsch iteration
|
|
// has no real peak to lock onto and can manufacture intensity the box sum never sees. Fall back to the
|
|
// summation (box-sum) intensity when the profile result disagrees with the summation seed by more than
|
|
// this many box-sum sigmas (a real fit agrees within counting noise, so the margin is generous).
|
|
constexpr double PROFILE_SUMMATION_MAX_NSIGMA = 10.0;
|
|
} // namespace bragg_engine
|
|
|
|
// One reflection's extracted intensity, produced by the derived engine and turned into a
|
|
// Reflection by Finalize() (which owns the polarization correction and scale bookkeeping).
|
|
struct BraggFitResult {
|
|
float I = 0.0f;
|
|
float sigma = NAN;
|
|
float bkg = 0.0f;
|
|
float observed_x = 0.0f; // intensity-weighted centroid (BoxSum mode only)
|
|
float observed_y = 0.0f;
|
|
bool ok = false;
|
|
bool has_observed = false;
|
|
};
|
|
|
|
class BraggIntegrationEngine {
|
|
protected:
|
|
// --- fixed configuration extracted from the experiment (see ProfileIntegrate2D) ---
|
|
IntegratorMode mode;
|
|
bool empirical; // ProfileEmpirical (vs ProfileGaussian)
|
|
|
|
size_t xpixel, ypixel, npixel;
|
|
|
|
float r1_sq;
|
|
float r2, r2_sq;
|
|
float r3, r3_sq;
|
|
float min_sigma_ratio;
|
|
int R, G, GG; // profile-grid half-size, edge (2R+1) and area (G*G)
|
|
|
|
bool broadband; // a set bandwidth (stills) vs monochromatic (rotation)
|
|
double bw_sigma; // bandwidth sigma [dimensionless, * Rpx -> px]
|
|
float bkg_clip_nsigma; // high-outlier background sigma-clip multiplier (0 = no clip)
|
|
bool use_ellipse; // radially elongate the per-reflection Gaussian
|
|
|
|
double c_radial; // radial variance coefficient of tan^2(2theta): parallax + capture
|
|
double F_px; // detector distance expressed in pixels
|
|
float beam_x, beam_y;
|
|
// Effective symmetric trimmed-mean background fraction (BraggIntegrationSettings): the configured
|
|
// fraction for monochromatic (rotation) data, forced to 0 for broadband (stills, which keep their
|
|
// high-side sigma-clip). 0 = plain ring mean. Read by both the CPU and GPU engines.
|
|
float bkg_trim = 0.0f;
|
|
|
|
// --- radial background curvature correction (BraggIntegrationSettings) ---
|
|
// The disk and the annulus are concentric, so any background LINEAR in position cancels between
|
|
// them; what survives is the curvature of the radial background. Every reflection uses the same
|
|
// stencil, so mean_annulus(B) - mean_disk(B) of a radial B is a FIXED kernel over radial offset:
|
|
// bkg_error = sum_k k_diff[k] * B(r0 + k - k_off)
|
|
// That is one short dot product per reflection and reads no pixels. Built in the constructor.
|
|
// The kernel is built whatever the setting says, so bkg_radial can be flipped between images at
|
|
// no cost - which is what the auto mode does, applying the correction only to the images whose
|
|
// background really is a smooth function of radius (see BackgroundRadial below).
|
|
bool bkg_radial = false;
|
|
bool bkg_radial_auto = false; // settings left it unset: decide per image from the ice score
|
|
int k_off = 0; // index of offset 0 in k_diff
|
|
std::vector<float> k_diff; // annulus-minus-disk weight per integer radial offset
|
|
|
|
DiffractionGeometry geom; // kept for the per-reflection polarization correction
|
|
std::optional<float> polarization;
|
|
|
|
// Assemble output reflections from the per-reflection fit results (polarization + scale corr).
|
|
std::vector<Reflection> Finalize(const std::vector<Reflection> &predicted, size_t npredicted,
|
|
const std::vector<BraggFitResult> &results,
|
|
int64_t image_number) const;
|
|
|
|
public:
|
|
explicit BraggIntegrationEngine(const DiffractionExperiment &experiment);
|
|
virtual ~BraggIntegrationEngine() = default;
|
|
|
|
// predicted[0..npredicted) are the reflections to extract; image is the preprocessed int32
|
|
// frame (image.size() == npixel). Returns only the observed reflections.
|
|
virtual std::vector<Reflection> Run(const ImagePreprocessorBuffer &image,
|
|
const std::vector<Reflection> &predicted, size_t npredicted,
|
|
int64_t image_number) = 0;
|
|
|
|
// Turn the radial background correction on or off for the images that follow. The caller owns
|
|
// the decision; in the auto mode the analysis sets it per image from that image's ice score.
|
|
void BackgroundRadial(bool on) { bkg_radial = on; }
|
|
[[nodiscard]] bool IsBackgroundRadialAuto() const { return bkg_radial_auto; }
|
|
};
|