Fit the modulation surface on a grid that spans the detector

The detector-frame modulation correction takes its 16x16 grid extent from a pass
over every full, but the surface is fitted only on the fulls that belong to an ASU
group. Those are two different populations, and the gap between them is whatever
was integrated past the resolution the merge uses.

That made the correction's fate depend on how far integration reached. Cut it back
and the grid contracts onto the merged disc while the cell count stays the same, so
each cell holds too few reflections, the surface over-fits, and cross-validation
throws it away - correctly, on a surface that should never have been fitted at that
scale. Varying only the integration limit on one rotation dataset, merged R_meas
came out 28.4 / 33.1 / 29.0 / 32.8 / 31.7 %, and the four-point spread is entirely
the correction switching on and off: every low value is a run where it was applied,
every high value one where it was refused, with no exceptions. Nothing else moved.

The grid now spans the detector. Cells with no observations in them keep a factor of
1 and cost nothing, and with integration running to the detector corner - the default
- the grid is the one it always was, so the common case is unchanged. On a crystal
carrying no resolution limit at all it takes merged R_meas from 39.7 % to 35.5 %.

This is a correctness fix in its own right. It also has to come first: without it, any
change that narrows the integrated resolution range trips the same over-fit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jungfrau
2026-08-17 18:04:23 -04:00
co-authored by Claude Opus 5
parent 088ba1cff8
commit 20f1827550
2 changed files with 15 additions and 10 deletions
+1
View File
@@ -1,6 +1,7 @@
# Changelog
## 1.0.0
### Unreleased
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector rather than the reflections that happen to be present, so whether it is applied no longer depends on how far integration reached. On a crystal where it was being refused, merged R_meas improves by 4 percentage points.
* rugnux: the beam-stop pre-scan reads and accumulates its frames on several threads instead of one. On a 16M-pixel rotation dataset it drops from 7.9 s to 4.8 s; the shadow it finds is unchanged.
### 1.0.0-rc.161