Merging: do not floor the merged sigma at the systematic term
The merged sigma was floored at b*|I|, so I/sigma could never exceed the reported ISa. On one dataset every merged reflection came out at I/sigma <= 12.96 with a 99th percentile of 12.77 in every resolution shell alike, while the scatter of the observations implied about 44 and XDS reported 58. The floor is wrong in principle. `b` is fitted from the scatter BETWEEN a reflection's symmetry equivalents, i.e. from the part that is not common to them, so it averages down with multiplicity exactly like the counting term. 1/sqrt(sum_w) with the b-inflated per-observation sigma already gives b*I/sqrt(n); flooring at b*|I| puts the sqrt(n) back. That is the whole effect: 12.96 * sqrt(21.6) = 60, against XDS's 58. It was introduced on a comparison of our MERGED I/sigma against XDS's UNMERGED I/sigma. XDS's own merged low-resolution I/sigma exceeds its reported ISa on 30 of the 39 reference datasets here, median ratio 1.78 and up to 4.23. Merged low-shell I/sigma now lands where XDS's does: 22.4 -> 46.2 against 46.2 on one crystal, 26.7 -> 115.7 against 96.6 on another, 12.5 -> 45.0 against 58.0 on a third. Over the 38-crystal battery the space groups, the merged reflection sets, R_meas and CC1/2 are all unchanged - every one of them is sigma-independent, which is what makes them the right control - and <I/sigma> rises on 35 crystals with none worse. The asymptotic estimator that fed the floor stays, for the reported ISa only, and is repaired in the process: it subtracts a*sigma^2 rather than the raw sigma^2 (at a < 1 the difference is the same size as the b^2 being measured, which is what made it flip between 10.9 and 62.7 on consecutive passes of the same data), it rescales each group's variance median-unbiased before subtracting an unbiased counting term, its I/sigma gate uses the same convention, and it is bounded by the whole-range b - an asymptote exists to refine 1/b upward, not to report 0.3 because "strong" was selected on a sigma scale the fit itself rejects. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -76,8 +76,8 @@ namespace {
|
||||
// model's b matching the systematic scatter that is actually there.
|
||||
{"sigmas 1.7x too small", 1.7, 0.05, std::nullopt},
|
||||
// The other way a fitted error model misses: the statistical sigmas come out somewhat too
|
||||
// LARGE while b - the asymptotic I/sigma ceiling, ISa = 1/b - is fitted 3x too optimistic, so
|
||||
// the systematic scatter present is 3x what the merged sigmas' floor admits.
|
||||
// LARGE while b - the asymptotic per-observation I/sigma, ISa = 1/b - is fitted 3x too
|
||||
// optimistic, so the systematic scatter present is 3x what the merged sigmas admit.
|
||||
{"ISa 3x too optimistic", 0.6, 0.02, 0.06},
|
||||
};
|
||||
|
||||
@@ -259,8 +259,8 @@ TEST_CASE("SearchSpaceGroup on a perfect merohedral twin returns one of the two
|
||||
}
|
||||
}
|
||||
|
||||
// THE property this harness exists for. Multiplicity changes only the sigmas - the random part of a
|
||||
// merged sigma averages down as 1/sqrt(n) while the systematic floor b*|I| does not - so it changes
|
||||
// THE property this harness exists for. Multiplicity changes only the sigmas - a merged sigma averages
|
||||
// down as 1/sqrt(n) while the systematic error the crystal carries does not - so it changes
|
||||
// how well the SAME crystal is measured, never what its symmetry is. A symmetry decision that moves
|
||||
// when the same crystal is merged 2x instead of 18x is a defect of the criterion, not a property of
|
||||
// the data.
|
||||
|
||||
Reference in New Issue
Block a user