SweepLayout rejected any inter-frame difference below half the header's
oscillation width as read-back jitter. A series that steps by less than
it exposes - overlapping wedges, or a writer that puts the whole sweep's
range in that field - then had every difference rejected, no step was
found, and the files fell into the "series that never turns" branch,
which lays them out end to end and, unlike the gapped path, said
nothing. The jitter bound is now absolute (1e-4 deg, below any step an
instrument makes and above 32-bit float read-back noise), and the
end-to-end fallback warns when the angles do span a rotation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
A 2700 frame, 540 degree sweep was refused at load as "screening images taken at
scattered angles". Nothing about it is irregular: every consecutive difference is
a clean step, and the headers simply write the start angle modulo 360, so the
second and third revolutions repeat angles the first already had.
The unwrap handled that correctly. What failed was the STEP. It was taken to be
the smallest difference between two adjacent frames - a single recorded value, and
of all the samples the one biased furthest low by read-back noise. These headers
are written from 32-bit floats, so that one sample was 1.3e-4 low, and placing a
frame by dividing its distance from the FIRST frame by that step multiplied the
error by the frame number: by frame 1924 the drift had reached a quarter of a step
and the on-grid check refused the rest of the sweep.
The fix separates the two jobs the step was doing. The smallest difference is now
only a guess, used to count how many steps apart each pair of NEIGHBOURS is; a
local difference carries the noise of one reading and nothing accumulates, however
long the sweep. The step itself is then a straight line fitted through (steps,
angle) over every frame, so the noise averages out instead of one unlucky pair
setting the scale for thousands of frames. Over the whole corpus this changes the
step for 4 of 82 numbered series and leaves the other 78 bit-identical; of the 4,
three were drifting by 0.17 to 0.35 of a step across the sweep and one by 1e-8.
Counting steps between neighbours is also what carries a sweep past a full turn:
each fold is one step forward, so the count keeps climbing through 360 and a frame
taken on the second revolution lands beyond the first rather than on top of it. A
series that genuinely goes BACK - two sweeps of one crystal concatenated - folds to
a large negative count in a single pair and still collides, so that refusal keeps
working; its message now says what it means.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
A series of one file per image was laid out end to end: the rotation start came
from the first file and the step from the difference between the first two, so a
series with frames missing came out compressed - one 179.8 degree deposited sweep
of 1108 files out of 1800 was read as 111 degrees, and every frame past the first
gap was analysed at the wrong spindle angle. Indexing then found a lattice that
took 4% of the validation spots, and two other gapped series aborted outright
with "it is not this crystal's lattice".
Every one of these formats writes each image's own start angle in its own header,
so the sweep is fully recoverable. The new reader/SweepLayout places each frame at
the slot its own angle puts it in and leaves a missing frame as a gap - a slot with
no file, which ReadRawImage reports as nothing to read, which every image loop in
the pipeline already passes over. The goniometer's start + increment * image_number
is then the true angle of every image, and the sweep range, the per-10-degree
delta-CC1/2 batches and the sweep-quality ledger all read the rotation the headers
describe. The rotation step is the smallest move between two frames that really are
adjacent, not the first pair.
The three readers shared this code by duplication; it is now written once. The same
place refuses what is not a sweep rather than averaging it into one: headers that
disagree about the detector distance, the beam centre, the wavelength or the
oscillation width, angles that do not sit on a single step (a folder of screening
shots), or two frames claiming the same angle - each naming the frames. A series
that does not turn at all is left exactly as it was.
A directory holding fewer files than its own numbering spans is also reported, with
both counts: that is the signal that a sweep was not unpacked or copied whole, which
otherwise shows up only as a resolution nobody can explain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
leonarski_fandClaude Opus 5