Decode a compressed frame into shared memory and preprocess it there

The image loop on a 16 Mpx detector is two thirds of the run and both cards are busy for essentially
all of it, so card time removed is wall time removed. Of the six milliseconds a frame costs, two and
a half were spent decompressing it - and not because the card was short of bandwidth. The LZ4 pass
moved 53 GB/s where the strong-pixel flagger, reading the same image and the same bin table, gets
276. It is latency, not bandwidth: the copy loop moves 32 bytes per warp iteration with a syncwarp
after each one, and for a match copy the source and the destination both derive from the same
pointer, so nothing pipelines. The warp spends its time waiting for global memory, one dependent
round trip at a time.

So decode where the waiting is cheap. One CUDA block now owns one bitshuffle block: its first warp
decodes the payload into shared memory, and the whole block then un-transposes and preprocesses out
of shared and writes finished pixels. A shared round trip is tens of cycles rather than hundreds,
and the 72 MB shuffled intermediate never reaches DRAM at all - the pair of kernels moved about 238
MB a frame and the fused one moves 93.

The parser is lifted into a device function that both kernels call over the same bytes, so the
standalone path and the fused one cannot decode a chunk differently. The statistics reduction had to
change with it: 48 bytes of static shared on top of a full bitshuffle block costs a whole resident
block per multiprocessor, so the counts now reduce through a warp shuffle and one integer atomic per
warp. Blocks larger than 16 kB keep the two-kernel path, and the beam stop's own decoder is
untouched.

What this costs is decoder parallelism: a block that holds 16 kB of shared is one of four resident
per multiprocessor on this card, where the old kernel fitted thirty-two warps each decoding on its
own. The trade is favourable here and should be better on the production cards, which have half
again as much shared memory per multiprocessor. Measured on a 16 Mpx rotation set at the production
GPU count, with the indexing work of the next commit: 37.2 s -> 32.9 s, and the merged output is
byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011n8riB6X59oRjkrSHzNPAU

image_analysis/image_preprocessing/BSLZ4DecoderGPU.h

@@ -16,15 +16,21 @@ struct BSLZ4BlockDesc {
     uint32_t nelem;     // elements in this block (the last one is usually shorter)
 };
 
-// What DecodeShuffled() leaves on the device: the LZ4 output, still bitshuffled, plus everything the
-// un-transpose needs to finish the image. The tail is the handful of elements bitshuffle stores
-// verbatim; it is already on the device inside the uploaded chunk, so it is handed over as a device
-// pointer rather than copied again from the host.
+// One chunk, on the device, with everything the rest of the decode needs to finish the image. The
+// tail is the handful of elements bitshuffle stores verbatim; it is already on the device inside the
+// uploaded chunk, so it is handed over as a device pointer rather than copied again from the host.
+//
+// DecodeShuffled() fills in `shuffled` - the LZ4 output, still bitshuffled. UploadCompressed()
+// leaves it null and hands over `compressed` and `status` instead, so the caller can run the LZ4
+// pass itself.
 struct BSLZ4ShuffledImage {
     const uint8_t *shuffled = nullptr;
+    const uint8_t *compressed = nullptr;   // the uploaded chunk; desc[].in_off indexes into it
     const BSLZ4BlockDesc *desc = nullptr;
+    uint32_t *status = nullptr;            // where a caller-run LZ4 pass flags a bad block
     int nblocks = 0;
     uint32_t elem_size = 0;
+    uint32_t block_bytes = 0;              // uncompressed bytes in a full bitshuffle block
     const uint8_t *tail_src = nullptr;
     uint32_t tail_elems = 0;
     uint32_t tail_elem0 = 0;    // index of the first tail element in the image
@@ -76,17 +82,36 @@ class BSLZ4DecoderGPU {
     void EnsureUncompressedCapacity(size_t bytes);
     void EnsureBlockCapacity(size_t nblocks);
 
+    // Scan the container and upload it, stopping short of decoding the blocks.
+    BSLZ4ShuffledImage PrepareChunk(const CompressedImage &image);
+
 public:
     BSLZ4DecoderGPU(size_t max_uncompressed_bytes, std::shared_ptr<CudaStream> stream);
 
     // True when this image can be decoded on the device. Everything else must go the host route.
     static bool Supports(const CompressedImage &image);
 
+    // The uncompressed size of one bitshuffle block, straight out of the chunk header, so a caller
+    // that wants to decode the blocks in shared memory can size that memory before it commits to
+    // the route. Zero when the chunk is too short to hold a header, which the decode then reports.
+    static uint32_t BlockBytes(const CompressedImage &image);
+
     // Locate the blocks, upload the chunk, and run the LZ4 pass. The result is still bitshuffled -
     // the caller finishes it, either with Decode()'s un-transpose or by fusing the un-transpose into
     // its own kernel. Work is queued on the decoder's stream and the caller synchronises.
     BSLZ4ShuffledImage DecodeShuffled(const CompressedImage &image);
 
+    // Upload the chunk and locate its blocks, and stop there. For a caller that runs the LZ4 pass
+    // in its OWN kernel, decoding each block into shared memory and consuming it there, so the
+    // bitshuffled bytes never reach device memory. Such a caller must call QueueDecodeStatus()
+    // once that kernel is queued.
+    BSLZ4ShuffledImage UploadCompressed(const CompressedImage &image);
+
+    // Queue the device-side failure flag back to the host. DecodeShuffled() does this itself; a
+    // caller that decodes the blocks in its own kernel does it after queueing that kernel, or the
+    // flag ThrowIfDecodeFailed() reads is the one from before the decode.
+    void QueueDecodeStatus();
+
     // Decode into gpu_out, which must hold image.GetUncompressedSize() bytes. The plain raw-bytes
     // path: DecodeShuffled() plus the un-transpose. Used by the tests and by any caller that wants
     // the decompressed image rather than a preprocessed one.