The device correction-surface fit (gpu-tail2) keeps its per-term arrays - the
terms, their parity, the group CSR, the three subsets' orders and the per-term
weights, about 60 bytes a term - as engine members, and nothing freed them
after the last surface. They stayed on the card through the merge, where the
GPU Wilson outlier test (gpu-tail) then asks for ~96 bytes per full. On a
large rotation set (139.8 M partials, 13.35 M fulls, 16 GB card) the engine's
resident arrays left 1242 MiB free for the first Wilson test and 998 MiB for
the one after the automatic resolution cut, against 1222 MiB needed: the run
stopped with "Wilson outlier test ... run the CPU build" although nothing ran
beside it. The surfaces' buffers are released once the last surface is
fitted (the next pass's surfaces build their own), which gives back 905 MiB
there.
The up-front checks of the three dataset-sized tail steps (Wilson test, pair
sort, error-model fit) also read free memory as cudaMemGetInfo reports it,
which counts what the allocation pool holds with no buffer in it as taken.
They now hand that back and, only if it is still short, wait for GPU work
running beside the merge (GPUWorkBeside, the speculative probe) to end before
giving up - as the engine's own allocations already do. That changes when a
step runs, never what it computes; a set that does not fit still fails at
once with the same "run the CPU build" message.
p.mtz: the large set completes (and is identical with the Wilson test forced
onto the host); myob 9bdd6916 and 8a1a 309ea9a8 unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
FitErrorModelGPU puts the samples in the host's rank order with four stable
radix passes, so the equal-count bins hold the same samples; computes every
normalised deviation with the host's rounding, so every bin median is the
host's to the bit; and hands the per-bin terms to the host's own update step,
now ErrorModelUpdate (a pure extraction from FitErrorModel). The one thing
that differs is the order each bin's two sums are added in: a fixed tree on the
device, the order nth_element happened to leave the bin in on the host. a and
b^2 can therefore differ from the host fit in their last bits - measured on two
synthetic pools: relative difference 2e-16 and 3e-13 in a, 5e-16 in b^2. The
device fit is deterministic (same pool, same bits).
p.mtz md5 nevertheless unchanged on myob, cytc, 8a1a and 8qaw (GPU build): the
difference does not survive the float rounding of the merged sigmas there. A
knife-edge decision elsewhere could still see it, which is why this is kept as
its own commit.
The host fit took 0.4-0.6 s per fit on the large merges, about half of it the
initial equal-count split, with only 16-way parallelism in the iterations.
Device peak about 72 bytes per sample (the rank sort; each stage's buffers are
freed when it is done); the free memory is checked first and a device without
it stops with a message naming the CPU build.
Kept last on the branch so that it can be dropped on its own.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
leonarski_fandClaude Opus 5.5