image_preprocessing: decode bitshuffle+LZ4 on the GPU

The pipeline decompressed each image on the host and uploaded the result. On
an 18 Mpx rotation dataset that made the host-to-device copy the bottleneck of
the whole per-image loop: nsys puts the copies at 78% of the loop against 39%
for every kernel combined - 3600 transfers of 72.4 MB - and they ran at only
12.5 GB/s of an available 27-28 because the host-side decompression was itself
saturating host memory bandwidth. The GPU was mostly waiting.

So the compressed chunk goes across instead, about 4 MB rather than 72 MB, and
is decoded on the device. That removes the transfer and the host decompression
that was throttling it, in one change. Measured on an idle machine, a run goes
from 45.11 s to 24.97 s - 1.81x - with the merged output unchanged.

THE APPROACH IS JON WRIGHT'S (ESRF): "Experiences with GPU decompression for
bitshuffle + LZ4 data", HDF5 User Group 2021, and github.com/jonwright/
bslz4decoders. The kernels here are ours, but the idea and the demonstration
that it is worth doing are his. Cited in docs/ACKNOWLEDGEMENT.md and in the new
section 0 of docs/CPU_DATA_ANALYSIS.md.

Two kernels mirror the CPU decoder. LZ4 runs one WARP per bitshuffle block:
every lane parses the same sequence stream (a broadcast read, no divergence)
and the literal and match copies are split across the 32 lanes so the stores
coalesce; an overlapping match is treated as a pattern of period offset sourced
from bytes that already precede the write position, which keeps it parallel
rather than a serial byte loop. One thread per block instead measured 13x
slower. The bitshuffle inverse then un-transposes each byte-plane through
shared memory and interleaves the planes back into elements.

Only BSHUF_LZ4 is decoded on the device. The zstd variants have no device
decoder, and neither has an uncompressed or float image; Supports() returns
false for those and the caller decompresses on the host exactly as before. The
fallback is explicit, so a format we cannot decode on the device is a slower
path and never a wrong answer.

Tests hold the device decoder against the CPU one byte for byte, on data from
the production compressor, for every element size the detectors emit -
including the 8-bit DECTRIS modes, which take bitshuf_decode_block's separate
elem_size == 1 branch - plus a many-block frame, the formats it must decline,
and malformed containers, which must throw rather than run off a buffer.

Battery: 37 crystals, no failures, identical to the host-decode run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

image_analysis/image_preprocessing/ImagePreprocessorGPU.cu

@@ -145,18 +145,58 @@ ImageStatistics ImagePreprocessorGPU::Analyze(ImagePreprocessorBuffer &processed
     }
 }
 
+bool ImagePreprocessorGPU::AnalyzeCompressed(ImagePreprocessorBuffer &processed_image,
+                                             const CompressedImage &image,
+                                             ImageStatistics &stats) {
+    if (!BSLZ4DecoderGPU::Supports(image))
+        return false;   // caller decompresses on the host and uses Analyze()
+    if (image.GetUncompressedSize() != npixels * image.GetByteDepth())
+        return false;
+
+    if (!bslz4_decoder)
+        bslz4_decoder = std::make_unique<BSLZ4DecoderGPU>(npixels * sizeof(uint32_t), stream);
+
+    // Straight into the buffer Analyze() would have filled by copying the decompressed frame across
+    // the bus; from here the two paths are the same code.
+    bslz4_decoder->Decode(image, gpu_decompressed_image.get());
+
+    switch (image.GetMode()) {
+        case CompressedImageMode::Int8:
+            stats = AnalyzeOnDevice<int8_t>(processed_image, INT8_MIN, INT8_MAX); return true;
+        case CompressedImageMode::Uint8:
+            stats = AnalyzeOnDevice<uint8_t>(processed_image, UINT8_MAX, UINT8_MAX); return true;
+        case CompressedImageMode::Int16:
+            stats = AnalyzeOnDevice<int16_t>(processed_image, INT16_MIN, INT16_MAX); return true;
+        case CompressedImageMode::Uint16:
+            stats = AnalyzeOnDevice<uint16_t>(processed_image, UINT16_MAX, UINT16_MAX); return true;
+        case CompressedImageMode::Int32:
+            stats = AnalyzeOnDevice<int32_t>(processed_image, INT32_MIN, INT32_MAX); return true;
+        case CompressedImageMode::Uint32:
+            stats = AnalyzeOnDevice<uint32_t>(processed_image, UINT32_MAX, UINT32_MAX); return true;
+        default:
+            return false;   // Supports() already excludes these; belt and braces
+    }
+}
+
 template<class T>
 ImageStatistics ImagePreprocessorGPU::Analyze(ImagePreprocessorBuffer &processed_image,
                                               const uint8_t *input,
                                               T err_value,
                                               T sat_value) {
-    if (sat_value > saturation_limit)
-        sat_value = static_cast<T>(saturation_limit);
-
     // On this engine's own stream, not the NULL stream: a NULL-stream copy implicitly synchronises with
     // every blocking stream in the process, which serialised all workers behind whichever one was
     // uploading. The stream is synchronised at the end of this function, so the ordering is unchanged.
     cudaMemcpyAsync(gpu_decompressed_image, input, npixels * sizeof(T), cudaMemcpyHostToDevice, *stream);
+    return AnalyzeOnDevice<T>(processed_image, err_value, sat_value);
+}
+
+// Everything after the image is on the device, shared by the host-upload and the device-decode
+// entry points so the two cannot drift apart.
+template<class T>
+ImageStatistics ImagePreprocessorGPU::AnalyzeOnDevice(ImagePreprocessorBuffer &processed_image,
+                                                      T err_value, T sat_value) {
+    if (sat_value > saturation_limit)
+        sat_value = static_cast<T>(saturation_limit);
 
     cpu_stats[0] = ImageStatistics{.max_value = INT64_MIN, .min_value = INT64_MAX};
     cudaMemcpyAsync(gpu_stats, cpu_stats.data(), sizeof(ImageStatistics), cudaMemcpyHostToDevice, *stream);