CBFDecoderGPU: walk short parts from shared memory

The first device decoder walked 256-byte tiles, one thread each, straight from global memory: on a
PILATUS 6M frame (~6 MB of stream) that is ~24k threads walking alone, and the two walks took
~0.5 ms of the card per frame - with the CBF decode gone from the workers the card is what bounds
the 7qij loops (kernels 67-78 % busy). Now each thread walks a 68-byte part that its block has read
into shared memory together (68, not 64, so the threads of a warp do not read the same banks), the
scan runs over tiles of 8 parts whose maps are composed first, and the write kernel stages a block's
pixels in shared memory and writes them out coalesced.

Measured on 36 real frames of 9 CBF sets (nsys, average per frame): 0.41 ms against ~0.6 ms before.
Output unchanged: the same tests, and the same 36 frames identical to the host decoder.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
This commit is contained in:
2026-10-13 02:53:52 +02:00
co-authored by Claude Opus 5.5
parent dfd69fcb91
commit 11052bc376
2 changed files with 116 additions and 43 deletions
@@ -23,12 +23,12 @@ struct CBFTileMap {
// int32 pixels, and the same refusal of a stream that ends before the image does.
//
// The format is a running sum of deltas whose tokens are 1, 3, 7 or 15 bytes long, so where a token
// starts depends on every token before it. The stream is cut into tiles; for each tile and each of the
// 15 offsets a decode can enter it at, one thread works out where it leaves, how many pixels it emits
// and what they sum to (the walks from different offsets meet within a few bytes, after which they
// are the same walk). Composing those maps across tiles is a scan, which gives every tile its true
// entry offset, its first pixel and the running value there; a second walk of each tile then writes
// the pixels.
// starts depends on every token before it. The stream is cut into short parts; for each part and each
// of the 15 offsets a decode can enter it at, one thread works out where it leaves, how many pixels it
// emits and what they sum to (the walks from different offsets meet within a few bytes, after which
// they are the same walk). Composing those maps is a scan - over tiles of a few parts, whose maps are
// the parts' composed - which gives every part its true entry offset, its first pixel and the running
// value there; a second walk of each part then writes the pixels.
class CBFDecoderGPU {
std::shared_ptr<CudaStream> stream;
@@ -37,6 +37,7 @@ class CBFDecoderGPU {
CudaDevicePtr<uint8_t> gpu_compressed;
size_t compressed_capacity = 0;
CudaDevicePtr<CBFTileMap> gpu_part_maps;
CudaDevicePtr<CBFTileMap> gpu_maps;
CudaDevicePtr<CBFTileMap> gpu_entries; // exclusive scan of gpu_maps: the state each tile is entered in
CudaDevicePtr<uint8_t> gpu_scan_temp;