Ingest wrote every observation's 24-byte SortKey into an index-order array and then scattered that
array into its h buckets - two key arrays alive together at the top of the ingest. Count the
buckets per chunk of frames straight from the source reflections and scatter from them instead:
a frame's reflections are consecutive in the key numbering, so the chunks are consecutive index
ranges and each bucket lands in the same index order as before. The key array is byte-identical.
BuildInRangeObservations also hands back the remap temporaries (old runs, per-run maps, new_idx)
before the observation array is built rather than at the end of the function.
Measured on a 3600-frame long-axis rotation set (95.7 M partials in the geometry pre-pass): the
pre-pass ingest high-water drops 15.45 -> 14.1 GB (sampled RSS, -N 6); merged MTZ/HKL/CIF, the P1
cross-check MTZ and the per-image table are byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
leonarski_fandClaude Opus 5
c58510edba