The stills per-image loop spent 99% of its time indexing (28.4 of 28.7 ms per frame on a 16 Mpx
serial set; read, decode, spot finding and integration together 0.33 ms), and only four threads
were ever busy:
- The 16 image workers each block on IndexerThreadPool, which had the default 4 threads, while the
FFT candidate post-processing is CPU work. Stills now size the pool to min(-N, 8); 8 gets 94% of
what 16 does (83 / 125 / 133 frames/s at 4 / 8 / 16 with the changes below) at half the card
memory - each GPU FFT indexer holds ~0.5 GB, 16 of them took the run to 12.7 GB.
- FFTIndexer::ReduceResults (widened path, taken on ~2/3 of calls) deduplicated ~2900 candidate
cells per call by a linear is_close scan: 46% of all CPU cycles. The accepted cells are now kept
ordered by a and only the 2%-window of a is asked; the answer (any close cell) is the same.
- FilterFFTResults built a 16k-entry multimap per call (11% of cycles); now a partial sort of
indices in the multimap's reverse order (magnitude descending, later direction first on ties).
- calculate_fft_result reads magnitudes from a bin-major transposed copy (in the histogram buffer,
no new memory): neighbouring lanes no longer read a spectrum apart. Bit-identical results
(checked against the old kernel on 4000+ calls, 0 mismatches); not faster on an idle card, but
67 -> 98 frames/s with the pool's streams sharing it.
- The stills min-pix choice indexes a frame at 3/2/1 and then the winner again; the seed of
strongest spots is often the same list each time (31% of indexer calls repeated an earlier call
of the same frame). IndexAndRefine answers a repeated list from a per-frame FrameIndexerCalls
memo kept by MXAnalysisWithoutFPGA for the duration of one frame.
Measured (RTX 5080, 32 threads, data in page cache), first 2000 frames, ABAB: 35/35 -> 127/127
frames/s, p.hkl md5 identical. Full 24960-frame run: loop 1081 s -> 277 s, wall 1132 s -> 328 s,
p.hkl md5 identical (P41212, 33234 unique); with a pool of 16 the loop is 222 s (the full run has
more spots per frame than its first 2000 frames, so the CPU side weighs more there). With a known cell (ffbidx path) 715 -> 920 frames/s on
2000 frames, p.hkl identical. Tests: jfjoch_test [Indexing] pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
leonarski_fandClaude Opus 5.5
9849154081