Three defaults, each settled by measurement rather than by argument. The arbiter throughout is structure-referenced - anomalous peak height where a crystal can carry it, and otherwise the agreement of the ice bands with a fixed external model against resolution-matched DECOY bands carrying no ice. The band-versus-decoy contrast is used because R-free here tracks completeness, and every one of these switches moves completeness. The damage is real and it localizes: over the rotation battery the ice bands' excess amplitude reaches +9.6% on a smooth-ice crystal and +35% on the worst, while a clean control sits at +0.6% (z +0.45). On the worst crystal, nine of the ten largest excess peaks in a q scan land on hexagonal ring positions. Turning ice handling off leaves the contrast unchanged and forcing it on a clean crystal does not create one, so it is the ice and not the machinery. MERGE-TIME RING MASK -> OFF. It deletes reflections, which no other program does by default - AIMLESS, DIALS, xia2, XDS and CrystFEL all keep ice-band reflections in the merge and exclude them only from the model fit; autoPROC is the sole exception. On the one battery crystal where the mask fires and an anomalous arbiter can score it, dropping the band moved the mean peak height at the known sites by -0.001 +- 0.018 sigma, 2% of the site height, while removing 1149 unique reflections whose mean I/sigma was 3.62 against the dataset's own 3.05 - better than average data - and costing 17 completeness points in that shell. It fires on 5 of 37 crystals, changes no space group, and those 5 disagree in sign: it clearly helps the two most heavily iced, is a wash on two and costs a third. So it stays as a switch, worth setting by hand on a badly iced crystal where it shows in the high shell, but it is not a default. RADIAL BACKGROUND -> AUTO, gated per image. The correction models the background as a function of radius alone, and that is exactly when it works. On a crystal with pure smooth powder ice it removes 43% of the bands' excess amplitude, with the improvement 7x larger inside the bands than outside; on a crystal whose ice is discrete crystallite spots - no smooth ring to model - the excess amplitude GREW by half; on clean data it is inert to four decimals. The two ice channels already separate those morphologies, so --background-radial takes on|off|auto and auto applies it to an image when that image's peak-excluded score reaches --ice-min-score. Auto never engages without such a score, because the plain profile carries the Bragg peaks and cannot support an absolute threshold. Per image rather than per run, and that was tested rather than assumed: the gate fires on 100% and 94% of frames on the two crystals that want it, and on 1.5% of frames - 32 blocks, 23 of them single frames - on the textured-ice crystal. A seam statistic against off + f*(on - off) is null on both mixed runs, every merge statistic is bracketed by the pure arms, and the textured crystal's auto arm lands on `off` rather than on `on`'s harm. A run-level gate would need the score before the pass that integrates, i.e. rotation-only plumbing, and buys nothing measurable. The kernel was already built unconditionally, so flipping the flag per image is free - except on the GPU, where the launches were gated on a construction-time n_rad. That is why the buffers are now allocated whenever the correction could run, and Run() decides per image. DETECTION -> the geometry's default when the file is silent: on for rotation, off for stills, with the command line and then the file taking precedence. A rotation sweep sits on the same rings for the whole run, so ice there is a coherent systematic and the presence gate keeps it inert on a clean crystal; a serial stills run has too few spots per image to spend any on flagging. The master file's key is kept as written rather than collapsed to a bool, so "the file said nothing" is distinguishable from "the file said no" - it used to fall silently to off, taking the exclusion from the scale fit with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
110 lines
7.1 KiB
C++
110 lines
7.1 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <optional>
|
|
|
|
// Spot-intensity extraction method used by the Bragg integration engine. ProfileGaussian (default)
|
|
// profile-fits with a measured-width Gaussian (Kabsch-style) - more accurate intensities than the
|
|
// classical uniform BoxSum; validated on anomalous data (stronger S/Cl peaks vs box-sum). BoxSum is
|
|
// the simpler, faster fallback. ProfileEmpirical learns the profile per resolution shell from strong
|
|
// spots - see docs/CPU_DATA_ANALYSIS.md (Bragg integration).
|
|
enum class IntegratorMode { BoxSum, ProfileGaussian, ProfileEmpirical };
|
|
|
|
// The hkl half-width the broker bootstraps when a config carries no bragg_integration block. Matches
|
|
// the max_hkl default in broker/jfjoch_api.yaml, so an omitting client and an omitting config agree.
|
|
constexpr int BRAGG_ONLINE_DEFAULT_MAX_HKL = 100;
|
|
|
|
class BraggIntegrationSettings {
|
|
IntegratorMode integrator_mode = IntegratorMode::ProfileGaussian;
|
|
float r_1 = 4;
|
|
float r_2 = 6;
|
|
float r_3 = 10;
|
|
// Integration/prediction resolution limit. Unset means "as far as the detector reaches", resolved
|
|
// from the geometry where it is used. The predictor independently rejects any reflection that misses
|
|
// the detector, so this is a bound on how far the lattice walk goes rather than a second opinion on
|
|
// what is measurable - a fixed default simply truncated every experiment whose detector reached
|
|
// past it.
|
|
std::optional<float> d_min_limit_A;
|
|
std::optional<float> fixed_profile_radius;
|
|
float minimum_sigma_in_regards_to_i = 0.02;
|
|
// The r2..r3 background ring is estimated with ONE of two robust means, never both: a high-side
|
|
// sigma-clip (bkg_clip_nsigma, the default) or a symmetric trimmed mean (bkg_trim_fraction). Setting
|
|
// either through its setter clears the other, so whichever was asked for last is the one in force;
|
|
// with both at 0 the ring is a plain mean.
|
|
//
|
|
// Symmetric trimmed-mean fraction: drop the lowest and highest this fraction of ring pixels before
|
|
// averaging. Robust to the high-side contamination (neighbour-spot wings, tails, zingers) that
|
|
// biases a plain ring mean up, but a symmetric trim is NOT a consistent estimator of the mean of a
|
|
// right-skewed (Poisson) sample - it sits ~0.1 ct/px low at every level, which with ~50 ring pixels
|
|
// adds ~5 counts to every partial. Kept reachable (rugnux --background-trim) for back compatibility;
|
|
// 0.10 was the shipped value.
|
|
float bkg_trim_fraction = 0.0f;
|
|
// High-side-only sigma clip: reject ring pixels above mean + this many sqrt(mean). Rejects the same
|
|
// contamination as the trim - measurably better, in fact - without cutting the low side, so it does
|
|
// not carry the trim's skew bias. Measured empty-aperture pedestal, counts: plain mean -0.03..-0.20,
|
|
// 10% symmetric trim +5.05..+6.34, 4 sigma clip +0.02..+0.54. Applied to monochromatic data
|
|
// (rugnux --background-clip); broadband (non-zero bandwidth) data always clip, at their tuned 3 sigma.
|
|
float bkg_clip_nsigma = 4.0f;
|
|
// Radial background curvature correction. The signal disk and the background annulus are
|
|
// concentric, so for ANY background linear in position their means are equal - a plane fit buys
|
|
// nothing and the leading error is the CURVATURE of the radial background, which the flat annulus
|
|
// mean is structurally blind to. Sitting on an ice ring that reaches +26 counts on a single
|
|
// reflection. When on, a radial background curve is accumulated per image from the annulus pixels
|
|
// that are already read, and each reflection's background is corrected by
|
|
// mean_annulus(B) - mean_disk(B), evaluated as a fixed kernel over radial offset (O(1), no extra
|
|
// pixel reads). Measured empty-aperture bias over 9 bands on 3 crystals: 4.33 -> 0.79 counts mean
|
|
// |bias|, scatter unchanged.
|
|
//
|
|
// Unset means AUTO, which is the default: apply it per image where that image's peak-excluded ice
|
|
// score says a SMOOTH powder ring is present, and not otherwise. The correction models the
|
|
// background as a function of radius alone, so it helps exactly where that is true and not
|
|
// elsewhere. Measured against a fixed external model, band-versus-decoy-band: on a crystal with
|
|
// pure smooth ice it removes 43% of the ice bands' excess amplitude, with the effect 7x stronger
|
|
// inside the bands than outside; on a crystal whose ice is discrete crystallite SPOTS - no smooth
|
|
// radial ring to model - the excess amplitude instead GREW by half; on clean data it is inert to
|
|
// four decimal places. The ice score's two channels separate those two morphologies, so the
|
|
// correction is gated on the smooth one. Auto only engages where a peak-excluded score exists
|
|
// (adaptive spot finding); the plain profile carries the Bragg peaks and cannot support a
|
|
// threshold, so without it auto stays off.
|
|
std::optional<bool> bkg_radial_correction;
|
|
// Half-width of the hkl cube the predictor walks: every reflection with |h|,|k|,|l| <= this is
|
|
// tested against the Ewald sphere, and nothing outside it can ever be predicted. An axis is
|
|
// truncated once a/d_min exceeds this, and the GPU cost is the cube (2n+1)^3 of candidates, so
|
|
// neither a small nor a large fixed value is right for every crystal.
|
|
//
|
|
// Unset (the default) means "take it from the refined cell", which is exact: the predictor keeps
|
|
// only |q| <= 1/d_min and h = a.q, so no reflection can have |h| > a/d_min. See MaxHKLForCell.
|
|
// Offline that is what is wanted. ONLINE it is not: the broker bootstraps a concrete value
|
|
// (BRAGG_ONLINE_DEFAULT_MAX_HKL) so per-image cost stays predictable across samples.
|
|
std::optional<int> max_hkl;
|
|
|
|
public:
|
|
BraggIntegrationSettings& R1(float input);
|
|
BraggIntegrationSettings& R2(float input);
|
|
BraggIntegrationSettings& R3(float input);
|
|
BraggIntegrationSettings& DMinLimit_A(std::optional<float> input);
|
|
BraggIntegrationSettings& FixedProfileRadius_recipA(std::optional<float> input);
|
|
BraggIntegrationSettings& Integrator(IntegratorMode input);
|
|
BraggIntegrationSettings& BackgroundTrimFraction(float input);
|
|
BraggIntegrationSettings& BackgroundClipNSigma(float input);
|
|
BraggIntegrationSettings& BackgroundRadialCorrection(std::optional<bool> input);
|
|
BraggIntegrationSettings& MaxHKL(std::optional<int> input);
|
|
|
|
|
|
[[nodiscard]] IntegratorMode GetIntegrator() const;
|
|
[[nodiscard]] float GetR1() const;
|
|
[[nodiscard]] float GetR2() const;
|
|
[[nodiscard]] float GetR3() const;
|
|
[[nodiscard]] std::optional<float> GetFixedProfileRadius_recipA() const;
|
|
[[nodiscard]] std::optional<float> GetDMinLimit_A() const;
|
|
|
|
[[nodiscard]] float GetMinimumSigmaInRegardsToI() const;
|
|
[[nodiscard]] float GetBackgroundTrimFraction() const;
|
|
[[nodiscard]] float GetBackgroundClipNSigma() const;
|
|
// Unset = auto (gate per image on the smooth-ice score); see bkg_radial_correction.
|
|
[[nodiscard]] std::optional<bool> GetBackgroundRadialCorrection() const;
|
|
[[nodiscard]] std::optional<int> GetMaxHKL() const;
|
|
};
|