RefineAbsorption indexes its surface by the diffracted direction
de-rotated into the crystal frame, deliberately without a time axis, and
RefineModulation indexes its by detector position, also without one.
Nothing is indexed by (rotation, detector position), so the part of the
absorption that changes as the crystal turns has no parameter at all.
For a rigid absorber illuminating a fixed volume that is the right
model: the incident path is a function of the spindle angle alone and
the per-image scale takes it, and the exit path is then fixed in the
crystal frame. What breaks the factorisation is the diffracting volume
moving - a crystal larger than the beam, a mis-centred loop, ice
building up. The exit path then depends on the spindle angle as well as
the direction, and no time-independent surface reaches it.
Measured on 34 rotation datasets, on XDS's own uncorrected intensities,
as what is left after the crystal-frame absorption and detector
modulation surfaces have taken what they can. The cross-validation gate
lets the surface engage on 22 of them. Scored per resolution shell -
the gate's whole-range ratio is lowered by any resolution-dependent
scale without a reflection getting tighter, so the honest readout is
each shell's own ratio, which a per-shell scale leaves unchanged - the
median engaged crystal gains 4.1 %, the set gains 117 % summed against
19 % of damage, and 3 of the 22 are hurt.
The surface has to be smooth in rotation angle to be absorption at all,
and it is: the lag-1 autocorrelation of the fitted factor along the time
axis runs +0.32 to +0.71 on the crystals it engages, against -0.08 for
the same surface with its time bins shuffled. Where it is not smooth it
is fitting something else, and says so - on a sweep whose beam was
obstructed for a 70 deg wedge the autocorrelation is +0.16 and the
profile is a cliff at the wedge, not a turn.
Two null controls. Assign every observation a random cell and the gate
refuses it (-1.2 % to -3.8 %). Keep the detector bin and shuffle only
the time bin - a surface that cannot contain any time-dependent
information - and the gate refuses that too, at +0.04 %, -0.60 % and
+0.33 % on three crystals. Against the real surface's +3.7 % to
+14.8 % on the same three.
12 time bins x a 10 x 10 detector grid = 1200 factors. On the per-shell
score the median gain moves only between 3.1 % and 4.1 % across grids
from 216 to 2400 cells, so the grid is second order; 12 x 10 has the
largest net and the fewest crystals hurt. Equal-occupancy detector
bins, not equal width: an equal-width grid starves the edges and the
corners, and a starved cell is where a free surface over-fits. Fitted
last, so the two time-independent surfaces get first claim on what they
can explain.
QUALIFICATION, measured after this was written: the "33 better / 0 worse" above is
overall R_meas, which is a ratio of sums across every shell and is therefore
lowered by any resolution-dependent scale without a reflection getting tighter -
the same property that let the estimator bias pass its own gate. Scored per
resolution shell instead, this surface HURTS 6 of 18 crystals under the
acceptance gate as it currently stands, because that gate shares the defect and
admits the surface where it should not. With a per-shell gate the surface is
refused on exactly those crystals and its net over the chain goes from +86.3 to
+159.5 per cent with none worse. The correction is right; the gate that decides
where to apply it is the next commit's problem, not this one's.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Full 38-crystal rotation battery against its own matched baseline - the same
binary with the corrected estimator and without this surface:
R_meas better 33 / worse 0, -78.7
R_meas_lo better 24 / worse 2, -20.7
CC1/2 better 8 / worse 0, +9.6
ISa better 30 / worse 2, +99.15
space groups unchanged at 35/38
The low-energy datasets gain most, which is what absorption should do: at 5-6 keV
one crystal goes ISa 24.68 -> 37.42 and another 14.21 -> 22.08, while the same
protein measured at 13 keV moves 13.41 -> 14.83.
Taken with the estimator fix it precedes, against a clean baseline: R_meas_lo
better 25 / worse 3 summed -26.0, ISa better 31 / worse 2 summed +113.1,
outer-shell CC1/2 +83.0, observations +25 880 on 36 crystals of 38, CC1/2 flat at
-1.4 and no space group moved. That last number is the point of the pair: the
estimator fix alone costs CC1/2 -11.4, because the ramp it removes was partly
standing in for this correction.
One cost, predicted in advance and still unexplained: outer-shell CC1/2 falls on
three of the four low-energy crystals, by 15.8 points on the worst, while every
other statistic on those same crystals improves. The fourth goes up. On 5000-9000
That outer-shell fall has since been attributed, and it is not this surface: with
the merge's 6-sigma outlier rejection turned off, the sign flips on every crystal
that lost, +20.5 and +20.7 where it read -15.8 and -8.0. The surface removes most
of the deviants in sample - it is fitted on all the data and applied to it, with
no robustness of its own - so the merge's cut stops firing and the survivors land
in a shell whose multiplicity is about three. Last-shell CC1/2 is largely made by
that cut: one crystal's baseline goes 10.4 to 92.7 purely by dropping 19 per cent
of the shell.
A second qualification, measured after the numbers above were taken: overall
R_meas is a ratio of sums across every shell, so any resolution-dependent scale
lowers it without a reflection getting tighter - the same property that let the
estimator bias pass its own gate. Scored per resolution shell instead, this
surface hurts 6 of 18 crystals under the acceptance gate as it stands, because
that gate shares the defect and admits the surface where it should not. Under a
per-shell gate it is refused on exactly those crystals and its net over the
correction chain goes from +86.3 to +159.5 per cent with none worse. The
correction is right; where to apply it is the gate's problem.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
leonarski_fandClaude Opus 5
2222a4085c
