The error model's systematic term b is identified only by the spread of I^2/sigma^2
across the intensity bins the fit uses, and those bins hold equal COUNTS. So when fewer
reflections are strong than one bin holds - a sixteenth of the pool - the top bin's
median sits at an intensity where b cannot be measured at all, and the fit hands it the
bins' own noise-selection slope instead: sorting noise by its group mean squared makes
dev2 rise with I2 even when the true b is zero, and with no strong bin to out-vote it
that slope becomes b.
The result is not a small error. On the battery's weakest crystal, 2.2% of whose fulls
reach I/sigma 2, the fit returns b = 5.6 - sigma -> 2*I at the strong end - and since
corrected_sigma applies b at the GROUP MEAN, sigma^2 = a*sigma^2 + (b*mean)^2 is a
per-group constant that caps merged |I/sigma| at sqrt(n)/b. The cap lands at 2.3, so 98.8%
of merged reflections come out below 3 and the reported ISa is 0.50, on data whose CC1/2
is 99.3% at multiplicity 18.7. XDS fits 6.13 from the same images. Feeding XDS's own
scaled observations through this estimator returns 0.84, so it is the estimator and not
the data; synthetic data built with b = 0 and 1.8% strong reproduces a = 0.51 and ISa 0.50
to two digits, and recovers the truth as soon as the strong fraction passes one bin.
So refuse to report what was not measured: when the strongest bin's own (I/sigma)^2 is
below 4, fit a alone, hold b at zero and warn that ISa is unmeasured. The threshold is not
delicate - the two crystals it fires on sit at 0.22 and 0.84 while the next crystal in the
battery is at 31.7 and a healthy one at 342, so anything from 4 to 25 selects the same two.
Full 38-crystal rotation battery: it fires on those two crystals and no others, and space
groups are unchanged at 35/38. Dropping the spurious term also fixes the merge weights it
had been distorting - on the worse of the two, R_meas 19.2 -> 13.6%, low-resolution R_meas
13.8 -> 6.9% against XDS's 14.1%, CC1/2 98.7 -> 100.0%, with chi2 1.11 on the
one-parameter model. Two further crystals move slightly; the guard never fires on either,
and they are marginal crystals of the kind whose two-pass branch any recompilation can
shift.
This reports the parameter as unmeasured rather than clamping it to something plausible,
because the honest statement is that the data do not reach far enough for a systematic
error to be seen - not that there is none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
leonarski_fandClaude Opus 5
7bbf072ad2