Post-refinement fits eight numbers, and it selects the twenty thousand best-recorded events to fit
them from. Before it can select, it copies every integrated partial into an array of its own and
sorts it. On a large cell that is 63 million of them, and the phase took 8.5 s of a 56 s run.
Almost none of that was the sort. `std::vector<Partial> pts(n)` value-initialises: one thread writes
3.5 GB of zeroes, page by page, before the parallel fill overwrites every byte of it - and being the
first touch, it also decides where the pages live, so the whole array lands on one NUMA node and
every later pass over it runs at one node's bandwidth. The same again for the sorted copy. Allocate
the storage without initialising it and let the parallel fill be the first touch.
The record itself carried more than the sort reads. `angle_rad` is a function of the image number
that the goniometer can give back on demand, and the two observed positions are wanted only by the
distance step, and only for the twenty thousand it keeps. Storing what is read - and as the floats
the fields already were, since widening a float to a double is exact - takes the record from 56
bytes to 32, which is a third off the fill and half off the sort's element moves.
Then three passes that walked the whole array to no purpose. The h range is now taken in the count
pass, which reads the same reflections anyway; the bucket histogram in the fill pass, which already
has h in hand. The event split walked serially and grew its output by doubling - about a gigabyte of
pure copying - although h is the leading sort key, so a rocking event never crosses an h bucket:
count per bucket, prefix, fill in parallel, and the events come out in the order the serial walk
produced them. And the copy of the whole event list, made only so that nth_element could destroy the
original, is now an index array.
Every one of these is the same arithmetic in the same order. Measured on a large-cell rotation set,
with the two commits that follow: 52.6 s -> 46.1 s, and the merged .hkl, .mtz and .cif are
byte-identical. `part_less` is deliberately left as it was, not a total order: what makes it
reproducible is that each bucket reaches the sort in gather order, and the new chunking is still a
contiguous span of that order.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011n8riB6X59oRjkrSHzNPAU
