miniCBF frames are decoded on the device

The CBF reader decoded every frame's x-CBF_BYTE_OFFSET stream on the worker thread - 19-30 % of the
image-loop worker CPU on CBF sets (7qij, 4mtk) - and the 4-byte image was then uploaded. ReadRawImage
now hands over the stream as the file stores it, in the image's own buffer, as a new read-only
CompressionAlgorithm::CBF_BYTE_OFFSET (mode Int32, what it decodes to), and ImagePreprocessorGPU
decodes it on the card into the same int32 image and runs the same preprocessing kernel on it.

The device decode (CBFDecoderGPU) cuts the stream into 256-byte tiles. A token is 1, 3, 7 or 15 bytes,
so a tile can be entered at 15 offsets; for each one thread works out where the decode leaves the
tile, how many pixels it emitted and their sum modulo 2^32 (the walks from different entries meet
within a few bytes, after which they are one walk). An exclusive scan composing these maps gives each
tile its true entry, first pixel and running value, and a second walk writes the pixels. A stream
that ends before the image is refused as on the host.

The host decoder moves to compression/ (DecodeCBFByteOffset) and is reached through
JFJochDecompressPtr, so every host consumer - the CPU build, the azimuthal-integration worker, the
pre-scan's shadow and gain passes - decodes exactly as before, now from the stream. It is the CPU
path, not a fallback: a frame the device refuses is not retried on the host. A process file written
from CBF input stores the decoded int32 image, as it always did. .cbf.gz is still inflated on the host.

Tests: CBFDecoderGPU_MatchesHost (every token size, 0x80 inside payloads, escape runs across tiles,
trailing CIF text), CBFDecoderGPU_ShortStreamIsRefused, ImagePreprocessorGPU_StoredEncodingsMatchCPU.
Checked locally against the host decoder on 36 real frames of 9 open-arm CBF sets (signed 32-bit,
signed and unsigned 16-bit, plain and gzip): identical.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
This commit is contained in:
2026-10-12 22:37:37 +02:00
co-authored by Claude Opus 5.5
parent e8409ef608
commit f4252ad39d
17 changed files with 662 additions and 92 deletions
@@ -7,6 +7,7 @@
#include "ImagePreprocessor.h"
#include "BSLZ4DecoderGPU.h"
#include "CBFDecoderGPU.h"
#include "../indexing/CUDAMemHelpers.h"
#include "../indexing/CudaSharedTables.h"
@@ -33,6 +34,14 @@ class ImagePreprocessorGPU : public ImagePreprocessor {
// Built on first use: a decoder that can serve this engine's images, sized to the frame.
std::unique_ptr<BSLZ4DecoderGPU> bslz4_decoder;
std::unique_ptr<CBFDecoderGPU> cbf_decoder;
// Which decoder the last AnalyzeCompressed used, for GetLastDecompressionTime_s.
enum class LastDecode { None, BSLZ4, CBF } last_decode = LastDecode::None;
// The file encoding of the miniCBF reader (CompressionAlgorithmEnum.h): the stored bytes are
// uploaded, decoded into gpu_decompressed_image as the int32 image the host would have produced,
// and preprocessed by the same kernel as an uploaded int32 image.
ImageStatistics AnalyzeStoredInt32(ImagePreprocessorBuffer &processed_image, const CompressedImage &image);
template <class T> ImageStatistics Analyze(ImagePreprocessorBuffer &processed_image, const uint8_t *input, T err_value, T sat_value);
// Preprocess an image already sitting in gpu_decompressed_image (the host-upload path).