T

leonarski_fandClaude Opus 5 8661193712 merge: balanced half-sets by rank, and CCanom read off them

Two statistics and the machinery they share.

THE SPLIT. CC1/2 correlates two half-set means, so a reflection whose
observations all land in one half has no second mean and contributes
nothing. The half was a hash of the image index alone, which does that to
2^(1-n) of the reflections at multiplicity n: half of the doubly measured
ones, a quarter of the triples. Measured on in-house rotation data that is
0% of a 26-fold redundant sweep but 6-30% of a 3-to-5-fold one, and 35-60%
of its outer shell - the shell the automatic resolution cutoff is read
from.

An observation's half is now the PARITY OF ITS RANK among its
reflection's observations, ordered by a key built from the raw Miller
index and the peak frame. That is exactly balanced - floor(n/2) against
ceil(n/2), the split cctbx's compute_cc_one_half uses and
phenix.merging_statistics through it - and, because a rank is a property
of the set rather than of the order it is walked in, it is the same on
every path. A sequential "put it wherever the counts are more even" rule
is balanced too but not that: the device holds the fulls in emit order and
the host in frame order, and on three test crystals that alone moved
CC_HALF between a CUDA and a JFJOCH_USE_CUDA=OFF build by up to 0.0077,
with 4 to 12 shell rows differing. Both now agree bit for bit, on the
whole scaling-and-merging section, while the radiation-damage B in the
same report still differs between the builds - so the agreement is a
property of the split, not of the two pipelines being identical. It is
also stable across repeated runs and across -N 1, 4 and 12.

Ranking is quadratic in a reflection's multiplicity - tens - so it is one
multiplicity-weighted sweep of the observations, assigned once per pass
and read by both the host loop and MergeAccumKernel. The kernel no longer
decides anything, which is why the two cannot drift apart.

CCANOM. SigAno was the only anomalous quality statistic reported, and it
is a ratio against the error model: an optimistic sigma raises it, and it
cannot separate real anomalous signal from an underestimated sigma. CCanom
carries no sigma. For each acentric reflection the anomalous difference is
formed twice, once per half, and the two are correlated over every pair
where both hands split into two non-empty halves - per shell, and overall
as one correlation rather than a mean of shells. The halves for it are
balanced within each MATE, not within the reflection: a mate left entirely
in one half loses the whole pair, and balancing per pair instead measured
0.223 where per mate gives 0.260.

It hangs on the anomalous accumulation that already existed for the
I(+)/I(-) export, which runs whether or not the merge is Friedel-averaged
- so CCanom is reported on a default run too, and -A only changes the
counting basis. The whole-mate sums SigAno and the reflection files read
are the two halves added back together, so neither moves.

Against external programs on the same observations: AIMLESS 0.264 and
cctbx 0.266 where this gives 0.260, per shell within about a point. XDS's
CORRECT.LP has a column named `Anomal Corr` which is NOT this quantity -
two to three times larger in the low shells, with a total below every one
of its own shells - so the report says so where it describes the key. On
data with no anomalous signal and around two observations per mate the
statistic is unstable and goes strongly negative; phenix reads
-0.83/-0.45/-0.42 where this reads +0.12/-0.33/-0.41 on the same file, so
that is the statistic, not this implementation, and it is reported as
measured.

REPORTING. A shell prints `-` where it could not measure a quantity
instead of `nan` - which every run printed for CCref, and a multiplicity-1
shell printed for most of the row - and CC_ANOM is omitted from the report
rather than written as a placeholder when no pair could be split in both
hands. A dash and an absent key are the same statement; a measured value
always prints, including a negative one.

Reported CC1/2 moves, so baselines keyed on CC_HALF need regenerating, and
the automatic resolution cutoff reads the same statistic and can move a
shell edge with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

2026-09-02 17:07:32 +02:00