T

leonarski_fandClaude Opus 5 7040987125 Rotation scaling: do not trust a per-frame scale that has collapsed toward zero

The per-frame scale enters every intensity as 1/G, and SolveScaleIRLS floors G
at zero and nothing else. A frame whose fit is not determined by its data can
return G ~ 0.002 against a run median of 0.865, and every observation it
carries is then multiplied by ~500 - sigma by the identical factor, which is
why no sigma-based outlier test can see it and why this looked for a long
time like a partiality problem. (The 1/partiality path is in fact guarded:
min_captured_fraction floors it at 0.7 by default on rotation.)

The window smoothing that should have absorbed such a frame instead made it
permanent. It averages log G over a window, so a scale collapsing toward zero
does not merely corrupt its own frame - its logarithm drags the whole window
down. Worse, where a run has a stretch of frames too sparse to fit at all, the
only FITTED frames in a window can be the collapsed ones, and the geometric
mean then averages the fault with itself. Measured on a multi-lattice dataset:
frames 816 and 818 fitted G = 0.0023 and 0.0014 with every neighbour from 800
to 839 unfitted, so smoothing set G = 0.0018 across the whole neighbourhood -
a 546x amplification. About 500 observations of 152000 (0.66%) then carried
99% of sum(I^2), and the merged CC1/2 read 17.2% where the same data with the
classic finder read 93.7%.

Treat a fitted scale far below the run's median as what it is - an
undetermined scale, exactly like the too-few-reflections case the code already
handles - rather than as a successful fit. Such frames no longer contribute to
the smoothing mean, and a frame whose own scale is not credible takes the
neighbourhood's, or the run's typical scale when the neighbourhood holds
nothing credible either.

The bound is a RATIO to the run's own median because the rotation per-frame G
is not gauge-fixed: G and the group means have an exact global multiplicative
degeneracy, and the fitted median drifts over 0.745-1.358 across the battery.
An absolute floor would reject everything in a run that drifted low.
MIN_CREDIBLE_SCALE_RATIO = 0.02 was chosen from measurement over 12 crystals
in the default configuration, where the smallest legitimate min(G)/median(G)
is 0.070; the failing case sat at 0.0017. It is 3.5x below anything real and
12x above the failure.

Effect on the intensity tail of the failing case: max I 10224 -> 438, and the
top 1000 observations' share of sum(I^2) 0.990 -> 0.421 (the classic-finder
reference is 0.632, so the tail is now cleaner than the run this was compared
against). Rotation battery, 33 crystals in the default configuration: ZERO
crystals differ - no space group, CC1/2, high-shell CC or ISa change anywhere.
The guard fires only on the pathology.

It does NOT rescue that dataset: with the amplification gone its CC1/2 is
26.2% and R_meas 49.2% against the classic finder's 93.7% and 27.8%. Adaptive
detection degrades those intensities for a second, independent reason that is
still open. This commit removes a latent hazard for any run with a sparse
stretch of frames; it is not the fix for that dataset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-27 22:43:58 +02:00