This reverts commit f257d51ea.
The block-wise moving-shadow mask fixed its one target (a half-image
lysozyme set whose pin turns with the sample) but fired on 47 sets of
the full battery: on one small-molecule set the mask covered 51 % of
the detector and SHELXL R1 doubled (0.10 -> 0.20), on another the mask
grew from 8 to 21 %. The pre-scan cost rose by +87 s on the HDF5 sets
and +499 s on the CBF sets (the commit claimed +0.1 s / +1-5 s per set).
A narrowed version (regr2-mask) still costs +25-45 s battery-wide, about
+0.3 s per HDF5 set, for one difficult set - more than the owner's
budget allows. It is parked for rc176, where a moved shadow is to be
found by a far-end check of the early decisions instead.
The documentation paragraph the commit added goes with it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
A pin, loop or capillary on the goniometer shadows one place for part of the rotation only. The
beam-stop mask is one for every frame and is read off the mean over the pre-scan sample, so a pixel
such hardware covers on fewer than about half the sampled frames is kept and integrated attenuated
on the frames it is covered on. Since the pre-scan samples only the first part of the sweep (part B,
5b9144410), a sweep whose moving shadow is short in the first half and long in the second loses it:
on the in-house half-image lysozyme sweep the mask fell from 32.8% to 21.6% of the detector, the
second half of the sweep was integrated through the shadow, ISa dropped 6.9 -> 4.1, R_meas 116 ->
240%, the cell refined a != b, and the attenuated intensities erased the a-axis screw evidence
(merged rc175: P4_1 2 2 instead of P4_1 2_1 2).
The sample is now also read in blocks of 10 consecutive frames (rotation only); each block's shadow
is found on its own with the rings about the whole sample's centre, and the blocks' pieces that
reach beyond the 30 px band the whole sample's edge can grow over (PENUMBRA_MAX_PX) are added whole.
A pixel shadowed on most frames of any longer stretch is shadowed on most frames of one block, so
nothing finer is needed; pieces within the band are a block redrawing a static stop's edge and are
left out, so a sweep without moving hardware keeps its mask bit for bit. Part B's principle is kept:
everything is still decided on the first part of the sweep.
Measured (GPU, against merged rc175 eac0833c8, 19 sets): half-image lysozyme P4_1 2_1 2, ISa 4.1 ->
9.7, R_meas 240 -> 113%, CC1/2 0.955 -> 0.979, R-free(model) 0.331 -> 0.282 (last night 6.9 / 116%).
p.mtz md5-identical on 14 of 19 (8v4o, 9rci, 5ky6, 6yqf, 6ze4, 5reo, 9qw8, insu_I low-ISa,
insu_H notwin, myob powder 1/2, myob split, nothing_1/2); changed: 5nw5 (a dim crescent at the stop
edge now masked: ISa 8.4 -> 8.9, CC1/2 0.937 -> 0.980, R-free 0.462 -> 0.429), 7atg (0.85% added at
the detector edge, statistics unchanged), 9min (0.03%, fail -> fail unchanged), 5jk4 (<0.01%,
unchanged). Cost: pre-scan +0.1 s on HDF5 sweeps, +1-5 s on large file-per-image (CBF, marCCD)
sweeps, where the blocks are re-read and their masks computed on the host.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
The pre-scan sample (beam-stop shadow, defective pixels, spot width, the background beam centre)
and the first pass's frames (the spread and wedge schemes, the validation frames, the prefetch) now
come from [start, PrepassEnd) - the part of the sweep the geometry pre-pass integrates - instead of
the whole sweep. Rugnux::PrepassEnd is the one place that range is defined, for the pre-pass and for
these decisions alike, so any change to the pre-pass range (such as a minimum rotation) moves both.
Stills keep the whole range.
With rugnux following a collection while it is written, nothing before pass 2 then waits for the
second part of the sweep: the pre-scan, both first passes and the pre-pass run while it is
collected, and pass 2 starts reading as soon as they are done. An offline run takes the same
decisions from the same frames, so the streamed result still equals the offline one by md5 (in-house
JUNGFRAU 4M thaumatin replayed at one data file per second, also at -N 8 and with the replay paused
and resumed): 2.4 s from the last data file to the result, against 8.0 s for the offline run.
Result-changing. p.mtz md5 at af3ea9cca -> here (with the 120 deg minimum of the previous commit),
GPU: myob de2ab820 -> 9bdd6916, cytc 3e5d2f91 -> 62e744a1, thau bc5e3de3 -> 3ec88c34,
kdp cf83ee5d -> cdc2c068, 8a1a 50ab659e -> 309ea9a8, 8qaw d3287922 -> c2849d52; CPU: myob 3f9d12d8 ->
18c7ac2f, cytc d30c9969 -> 40e17d97, thau 4cdbdd94 -> f14e52da, kdp f1d801f3 -> eb930714. Sweeps of
120 deg or less are unchanged by this commit (their early range is the whole sweep). Needs the
battery before it is merged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
A pixel that reads at most half of its own sector's level on average, by more
than the bound a hot pixel's excess has to clear, is now masked as dead - the
mirror of the hot rule. Nothing but a defect keeps a pixel dark frame after
frame, and every reflection whose disk covers it is integrated short.
The finder now also runs on CCD sweeps (no sensor depth): a dead CCD pixel
reads its offset, far below any ring with background, and the hot rule's
ten-fold ratio does not depend on Poisson counts either.
Found on 5jk4 (ADSC Q4, 2304x2304): the last row and the first two columns
read the offset (~20 ADU against ~60-100 around them) and were unmasked.
Reflections touching them merged to large negative intensities, and one
observation at y=2298 with sigma 10334 swamped the outer shell's half-set
CC1/2 (3.9 %) and the overall CC1/2 (0.786; 0.998 at rc.174). With the
6630 dead pixels masked the fall-off is clean: CC1/2 0.998, cut 0.96 A
(fit 1.01 A), at --prepass-fraction 0.5 and 1 alike. Counting-detector
sets with low background are untouched (myoglobin X06DA: 0 dead).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
leonarski_fandClaude Opus 5.5