T

leonarski_fandClaude Opus 5 4895dc1018 Rotation: let --min-image-cc drop frames that disagree with the merged reference

The flag was accepted on rotation data and did nothing - it is read only by the
stills merge (Merge.cpp), and the CLI warned about that rather than fixing it.
Meanwhile RotationScaleMerge already COMPUTES a per-frame correlation against
the merged reference and writes it to the per-image table; nothing acted on it.

Wire the two together. A rejected frame has its partials' corr set to 0, which
is how a frame already leaves the pipeline - every consumer requires corr > 0,
so the combine, the merge and the error model all drop it together. The GPU
path reuses the SmoothCorr kernel with a ratio of 0, so one implementation
covers both. Off by default (0), and verified bit-identical to the previous
binary when off.

What it catches, on the two rotation datasets that have a population to catch:

  a two-lattice crystal - two lattices in two physical AREAS of the sample, so
  the sweep passes from one to the other and whole blocks of frames measure a
  different crystal from the one being merged (frames 500-700 index perfectly
  well at a per-frame CC of 0.22 against 0.47-0.56 either side, in 11 contiguous
  runs). R_meas 28.6 -> 24.6%, CC1/2 93.6 -> 95.1, high-shell CC 23.4 -> 38.3.

  a second dataset with 9.5% of frames below CC 0.30: R_meas 24.3 -> 23.4%,
  CC1/2 92.6 -> 93.4.

The criterion is "this frame disagrees with the merged reference", NOT "this
frame is off-crystal". It happens to catch both, because a frame that measures
nothing and a frame that measures a DIFFERENT crystal fail the same test, and it
does not need to know which. For the two-area case that is a workaround, not a
treatment: it recovers one crystal by discarding the other, where processing the
two as separate sweeps would keep both. The frame-block structure is clean
enough that such a split could be detected automatically.

WHY THERE IS NO DEFAULT. The per-frame CC is not comparable between datasets -
it is as much a measure of data quality as of frame validity. Measured medians
across the battery run from 0.30 to 0.81, so one absolute bound removes 13
frames from one dataset and 584 of 1800 from another:

  battery at --min-image-cc 30, 33 crystals: no point group changed (30/33),
  four crystals clearly better (one +5.4 CC1/2 points, the two-lattice case
  above, and ISa gains of 1.3-4.6 on three others) - and one healthy crystal
  lost a third of its frames and with them its high-resolution shell
  (CC1/2_hi 26.2 -> 2.0).

This is the same trap as an absolute bound on any per-operator or per-frame
agreement statistic, and the same one the per-frame scale guard avoids by
measuring against the run's own median. A principled version would cut on the
SHAPE of the per-frame CC distribution - a dataset with a bad subpopulation is
bimodal, a uniformly weak one is not - rather than on an absolute value. Until
that exists this stays opt-in, and the per-image CC it keys on is already in the
_image.dat table for anyone choosing a value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-28 11:45:49 +02:00