T

jungfrauandClaude Opus 5 2308bbad8c Stop scaling and merging what the resolution range excludes

A crystal integrated to the detector corner but merged well short of it carries
observations through the whole merge that the merge then discards. On the heaviest
dataset in the rotation test set that is 63.3 M partials of which 6.4 M are ever
used: the other nine tenths are sorted, uploaded, scaled, combined, error-modelled
and post-refined before anything looks at their resolution. Ingest copied every one
of them unconditionally, and the d_min limit was first applied far downstream, in
the ASU grouping.

They are now dropped at ingest, immediately after the one big sort:

- WHOLE raw-hkl runs are dropped, on the same rawrun_d the ASU grouping already
  tests. A per-observation test is not equivalent - a run is in or out today by one
  member's d - and using a different rule here would put the two out of step.
- The drop happens AFTER the flux meter, which takes each frame's mean background
  over every reflection on it, and after the sort, so neither changes.
- The compaction runs in index order, so a frame's observations stay contiguous and
  keep their order, and every per-frame sum keeps its sequence of roundings.

Post-refinement reads the integration outcomes rather than the merge arrays, so it
still sees every reflection. That is the point: --integration-high-resolution buys
the same time by never integrating the reflections, and pays for it in the per-frame
geometry, which wants them.

Two more passes over the observation array go with it. The incident-flux divide was
15.8 % of all user cycles to read one int and divide one float across 5 GB; the
per-frame mean it needs is now accumulated by the ingest fill loop - one frame, one
thread, same order, so bit-exact - and the divide rides on the finiteness pass that
already touches that field. And the geometry post-refinement is fitted on a bounded
sample of partials, selected by a hash of the raw hkl so whole rocking events are
kept or dropped together and the sweep and the detector are thinned uniformly.

The sample size is 8 M and the reason it is not smaller is measured. Over a 126x
thinning the fitted rotation scale is flat to 2e-5 and the beam centre moves 0.03 px,
but the CELL scale breaks between 8 M and 4 M: the axis step keeps the 20 000
strongest events, so once the pool approaches that size it starts fitting weaker ones
and the second pass's cell shifts by ~0.1 %.

Measured on the heaviest crystal, three A/B pairs with the order alternated:
68.4 s -> 37.3 s wall, 530 s -> 221 s of CPU. Whole battery 8m07s -> 7m15s, space
group 21/24 with the same three disagreements as before, no failures. Bit-identical
is not available on the GPU path - the resident reductions and the fulls emit order
depend on array length - so what is shown is that every difference sits inside the
spread the unmodified binary has against itself between two runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-17 18:04:55 -04:00