Reader: reconstruct the background variance a legacy file does not store

A _process.h5 written before background_variance existed was read with var_bkg = 0, on the reasoning
that zero leaves the combine with the signal term alone, "which is what it had before". It does not.
Before, the combine back-derived the non-signal variance from sigma itself, and on a weak reflection
that is essentially the whole of sigma^2; zero deletes the dominant term and weights the reflection by
roughly 1/I instead of 1/sigma^2.

Recover it from the integrator's own identity, sigma^2 = I + var_bkg, when the dataset is absent.

Measured by re-scaling a stills _process.h5 with the dataset deleted, against the same file with it
intact: the automatic resolution cutoff was reading 1.66 A where the intact file reads 1.81, with
14731 unique reflections against 11508 - i.e. the zeroed file looked good enough to merge 0.15 A past
its own limit. Reconstructed, it reads 1.78 A and 11926, within 0.03 A of the intact file. The
residue is the profile-fit path, where var_bkg is not exactly sigma^2 - I and only the box-sum
identity is exact; that is recoverable to a closer approximation only by storing it, which is what
files written from now on do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-10 05:52:56 +02:00
co-authored by Claude Opus 5
parent a7c5d7a89b
commit 52ea727650
3 changed files with 11 additions and 3 deletions
+6 -3
View File
@@ -179,9 +179,12 @@ bool ReadReflectionsFromGroup(HDF5Object &file,
if (zeta.size() > i)
zeta_val = zeta[i];
// A file written before the merge carried this cannot say what the non-signal variance was;
// 0 leaves the combine with the signal term alone, which is what it had before.
float var_bkg_val = 0.0f;
// A file written before this dataset existed has to have the non-signal variance reconstructed,
// not zeroed. The combine takes var_bkg as the authoritative non-signal term, so a zero would
// leave it the signal alone and weight a weak reflection by ~1/I instead of ~1/sigma^2 - orders
// of magnitude too high, and worst exactly where the reflection is weakest. The integrator's own
// identity sigma^2 = I + var_bkg inverts to recover what the file does not store.
float var_bkg_val = std::max(0.0f, int_err.at(i) * int_err.at(i) - int_sum.at(i));
if (var_bkg.size() > i)
var_bkg_val = var_bkg[i];