fit and boxsum were 80% of GPU time on a crowded crystal - 78 s of it. Neither
was bandwidth- or occupancy-bound: both sat at about an eighth of the issue rate
the card can sustain, stalled.
What stalls them is the block-wide accumulations. Every one has all 128 lanes of
the block adding into one shared address, and a shared-memory atomicAdd on a
float or a 64-bit integer has no instruction on either Turing or Ada - it
compiles to a compare-and-swap retry loop. So those 128 lanes serialise into 128
retries, eighteen times per thread in fit. Summing across the warp first and
letting one lane do the atomic leaves four per block instead of 128.
That is the whole story: the arithmetic below was worth 2%, the atomics 5.6x.
The arithmetic is still worth having, and is what was expected to matter:
- compute_shell ran on all 128 threads of a block for a value that belongs to
the reflection. It is two software double-precision divisions, on a card
whose double throughput is a thirty-second (a sixty-fourth on the production
one) of its single. One thread does it now.
- The Kabsch inner loop divided by the same weight three times; the compiler
emits the whole correctly-rounded sequence each time. One reciprocal now.
Likewise the two Gaussian widths and the profile normalisation, which are
constant over a reflection's cells and were divided per cell.
- boxsum read the pixel before deciding whether it wanted it. The window is the
bounding box of an ellipse, so nearly half of it is neither the signal disk
nor the background ring, and those slots were fetching a cache line for
nothing.
Measured: fit 50.8 s -> 9.1 s, boxsum 27.5 s -> 12.1 s. A crowded crystal
2m22s -> 1m58s, a 16M-pixel one 39.5 s -> 37.2 s, the whole battery 12m30s ->
11m35s. Same space group on all 24 crystals, none failed.
The integer sums are unchanged - addition is associative. The float ones move in
their last bits and become more reproducible, since a fixed shuffle tree
replaces whatever order the atomics arrived in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
