T

leonarski_fandClaude Opus 5 e332aaf28c Space-group search: read the presence cut on counting significance

present_i_over_sigma is meant to say "this reflection is really there".
The merged I/sigma is not that quantity. The merge carries the error
model's intensity-proportional term, sigma^2 = a*sigma0^2 + (b*I)^2, so
I/sigma saturates - at ISa*sqrt(n) for a reflection observed n times, and
at ISa exactly for one observed once. Above that knee it stops rising
with the intensity.

Measured over the battery's 76 search merges: the median I/sigma of the
top E^2 decile sits at 0.81 of ISa, and on the weakest merge the decile
profile reads 1.58 1.61 1.63 1.64 1.64 1.65 1.65 against an ISa of 1.70 -
flat to three digits, log-log slope against E^2 of 0.008. Reflections an
order of magnitude apart in real intensity read the same number. So one
constant demands between 2.16 and 11.91 in counting significance
depending on the crystal, and the fraction of a merge clearing it runs
from 3.1% to 98.8%.

Convert it once instead. For a reflection observed once
sigma_counting^2 = sigma^2 - (b*I)^2, so I/sigma_counting >= T is exactly
t >= T/sqrt(1+(T/ISa)^2). That cut lies strictly below ISa for every ISa,
so it is always reachable by the reflection the ceiling binds hardest,
and it is within 1% of T on any merge with ISa >= 21, so a healthy merge
is left where it was. Multiplicity is taken as 1 deliberately rather than
estimated, for the same reason.

The quantile fallback goes with it. Its comment claimed to be inert on
the battery; it fires on 4 of 76 arms, and on one of them it relaxes the
cut BELOW the nominal value because that merge is bimodal. A cap on a
statistic is not a substitute for measuring the right statistic.

Two comments in the tree still described a merged sigma floored at b|I|,
removed in 72efb75a8 before rc-161. Both corrected; that stale claim is
what made this look like a different bug.

Battery, both arms, against a control floor of exactly zero: space groups
35/38 before and after, the same three crystals differing from the
reference, all 380 shells identical and all 38 merged mmCIFs
byte-identical. It is live rather than dead code - the pass_cc population
moves on 45 of 47 arms and 212 of 314 operator correlations move, with 0
of 314 verdicts changed. The weakest crystal's single 2-fold goes from CC
0.489 on 1131 pairs to 0.597 on 4822, widening its margin over the
threshold by 57%.

Two things this does not fix, both quantified in the work behind it. The
conversion is monotone, so it corrects the threshold but cannot restore
the ranking the (b*I)^2 term already destroyed - that needs a
counting-only merged sigma and touches the CUDA merge kernel. And one
crystal's gate still admits 98.8% of its merge with a slope of 0.037,
discriminating nothing; the instrument for that is E^2, which needs
min_operator_cc re-derived first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CHMmeM1d489zvNFT7ZMN2P

2026-08-26 12:43:04 +02:00