Bragg integration: drop the 2% sigma floor and carry the background variance
Two changes to the same variance chain; they are in one commit because the second exists to remove an assumption the first was breaking, and separating them leaves a tree that is correct only by luck. The reported sigma was floored at 2% of the intensity, a per-partial I/sigma cap of 50. It applied only to the box-sum seed, never to the profile fit, so the shipped default was unaffected - but the combine back-derives each partial's non-signal variance as sigma^2 - I, and a floored sigma makes that quantity mean nothing. It then read corr^2 * (0.0004 I^2 - I), which is not a background variance. Measured on --integrator boxsum: the reported sigma understated the true scatter by up to 16x at I ~ 21000 counts per partial, and pooled_I amplified a 1 ct/px background drift into an 11.5% intensity error on the strongest reflections. What the floor stood in for - that at high intensity the error is systematic rather than counting - is already carried downstream, twice: the fitted b in v = a*sigma^2 + (b*I)^2, measured from the data rather than assumed, and SigmaWithSystematicFloor on the merged sigma. The floor was that idea applied one level too early with a hardcoded b of 0.02. It arrived without a test or a setter and was unreachable from the CLI, the API and the config. The merge now takes the non-signal variance the integrator actually measured instead of inverting sigma^2 = I + N. That identity is exact for a box sum once the floor is gone and was never exact for a profile fit, whose sigma^2 = 1/den + (wsum/den)^2 * bkg_var is formed against a fitted intensity. The value is carried through BraggFitResult, Reflection and Obs, both engines, both merges, and the process-file round trip; files written before this change are read with the term absent, which is what they had. Battery, 37 crystals, paired: space groups unchanged, reflection sets unchanged, median delta zero on R_meas and CC1/2. --integrator boxsum on the reference crystal goes ISa 8.9 -> 20.2 with a 0.947 -> 1.032. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -82,7 +82,7 @@ void MergeOnTheFly::AddImage(const IntegrationOutcome &outcome, int64_t image_id
|
||||
continue;
|
||||
auto hkl = generator(r);
|
||||
auto hkl_key = hkl.pack();
|
||||
sigma_corr = CorrectedSigma(I_corr, sigma_corr, r.image_scale_corr, hkl_key);
|
||||
sigma_corr = CorrectedSigma(I_corr, sigma_corr, r.image_scale_corr, r.var_bkg, hkl_key);
|
||||
|
||||
// Robust outlier rejection: drop this observation if it sits more than
|
||||
// reject_nsigma error-model sigmas from the reflection's median. Needs the active
|
||||
@@ -121,6 +121,7 @@ void MergeOnTheFly::AddImage(const IntegrationOutcome &outcome, int64_t image_id
|
||||
}
|
||||
|
||||
float MergeOnTheFly::CorrectedSigma(float I_corr, float sigma_corr, float image_scale_corr,
|
||||
float var_bkg,
|
||||
uint64_t hkl_key) const {
|
||||
if (!error_model_active)
|
||||
return sigma_corr;
|
||||
@@ -144,8 +145,7 @@ float MergeOnTheFly::CorrectedSigma(float I_corr, float sigma_corr, float image_
|
||||
// observations this correction exists to stop being mistreated.
|
||||
double a_var = static_cast<double>(sigma_corr) * sigma_corr;
|
||||
if (scaling_settings.GetExpectedVarianceMerge()) {
|
||||
const double bkg_var = std::max(0.0, a_var - static_cast<double>(image_scale_corr)
|
||||
* std::max(0.0, static_cast<double>(I_corr)));
|
||||
const double bkg_var = static_cast<double>(image_scale_corr) * image_scale_corr * var_bkg;
|
||||
const double base = bkg_var + static_cast<double>(image_scale_corr) * std::max(0.0, I_for_b);
|
||||
if (base > 0.0)
|
||||
a_var = base;
|
||||
|
||||
Reference in New Issue
Block a user