leonarski_fandClaude Opus 5.5
f4252ad39d
The CBF reader decoded every frame's x-CBF_BYTE_OFFSET stream on the worker thread - 19-30 % of the image-loop worker CPU on CBF sets (7qij, 4mtk) - and the 4-byte image was then uploaded. ReadRawImage now hands over the stream as the file stores it, in the image's own buffer, as a new read-only CompressionAlgorithm::CBF_BYTE_OFFSET (mode Int32, what it decodes to), and ImagePreprocessorGPU decodes it on the card into the same int32 image and runs the same preprocessing kernel on it. The device decode (CBFDecoderGPU) cuts the stream into 256-byte tiles. A token is 1, 3, 7 or 15 bytes, so a tile can be entered at 15 offsets; for each one thread works out where the decode leaves the tile, how many pixels it emitted and their sum modulo 2^32 (the walks from different entries meet within a few bytes, after which they are one walk). An exclusive scan composing these maps gives each tile its true entry, first pixel and running value, and a second walk writes the pixels. A stream that ends before the image is refused as on the host. The host decoder moves to compression/ (DecodeCBFByteOffset) and is reached through JFJochDecompressPtr, so every host consumer - the CPU build, the azimuthal-integration worker, the pre-scan's shadow and gain passes - decodes exactly as before, now from the stream. It is the CPU path, not a fallback: a frame the device refuses is not retried on the host. A process file written from CBF input stores the decoded int32 image, as it always did. .cbf.gz is still inflated on the host. Tests: CBFDecoderGPU_MatchesHost (every token size, 0x80 inside payloads, escape runs across tiles, trailing CIF text), CBFDecoderGPU_ShortStreamIsRefused, ImagePreprocessorGPU_StoredEncodingsMatchCPU. Checked locally against the host decoder on 36 real frames of 9 open-arm CBF sets (signed 32-bit, signed and unsigned 16-bit, plain and gzip): identical. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
69 lines
4.0 KiB
C++
69 lines
4.0 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <memory>
|
|
|
|
#include "ImagePreprocessor.h"
|
|
#include "BSLZ4DecoderGPU.h"
|
|
#include "CBFDecoderGPU.h"
|
|
#include "../indexing/CUDAMemHelpers.h"
|
|
#include "../indexing/CudaSharedTables.h"
|
|
|
|
class ImagePreprocessorGPU : public ImagePreprocessor {
|
|
std::shared_ptr<CudaStream> stream;
|
|
const bool copy_image_to_host;
|
|
int threads;
|
|
int blocks;
|
|
// Geometry-only, so one copy per GPU shared with every other engine on it (CudaSharedTables.h).
|
|
std::shared_ptr<CudaDevicePtr<uint8_t>> gpu_mask;
|
|
// Landing buffer for the HOST-upload path only. The device-decode path un-transposes straight
|
|
// into the preprocessed image, so it never needs this - and at 4 bytes per pixel it is worth a
|
|
// frame per worker, so it is allocated on first use rather than always.
|
|
CudaDevicePtr<uint8_t> gpu_decompressed_image;
|
|
CudaDevicePtr<ImageStatistics> gpu_stats;
|
|
|
|
PinnedVector<ImageStatistics> cpu_stats;
|
|
CudaRegisteredVector<uint8_t> input_reg; // page-locks the caller's decompression buffer
|
|
// The uncompressed-image counterpart of input_reg: the region PinInputRegion last page-locked.
|
|
const void *pinned_input = nullptr;
|
|
size_t pinned_input_bytes = 0;
|
|
|
|
std::vector<int32_t> cpu_image;
|
|
|
|
// Built on first use: a decoder that can serve this engine's images, sized to the frame.
|
|
std::unique_ptr<BSLZ4DecoderGPU> bslz4_decoder;
|
|
std::unique_ptr<CBFDecoderGPU> cbf_decoder;
|
|
// Which decoder the last AnalyzeCompressed used, for GetLastDecompressionTime_s.
|
|
enum class LastDecode { None, BSLZ4, CBF } last_decode = LastDecode::None;
|
|
|
|
// The file encoding of the miniCBF reader (CompressionAlgorithmEnum.h): the stored bytes are
|
|
// uploaded, decoded into gpu_decompressed_image as the int32 image the host would have produced,
|
|
// and preprocessed by the same kernel as an uploaded int32 image.
|
|
ImageStatistics AnalyzeStoredInt32(ImagePreprocessorBuffer &processed_image, const CompressedImage &image);
|
|
|
|
template <class T> ImageStatistics Analyze(ImagePreprocessorBuffer &processed_image, const uint8_t *input, T err_value, T sat_value);
|
|
// Preprocess an image already sitting in gpu_decompressed_image (the host-upload path).
|
|
template <class T> ImageStatistics AnalyzeOnDevice(ImagePreprocessorBuffer &processed_image, T err_value, T sat_value);
|
|
// Preprocess straight out of the bitshuffled bytes (the device-decode path). Same per-pixel
|
|
// decision and same statistics as AnalyzeOnDevice, with the un-transpose folded in.
|
|
template <class T, int ES> ImageStatistics UntransposeAndAnalyze(ImagePreprocessorBuffer &processed_image,
|
|
const BSLZ4ShuffledImage &shuffled,
|
|
T err_value, T sat_value);
|
|
public:
|
|
// copy_image_to_host copies the preprocessed image back after every frame. It is only needed when
|
|
// something on the CPU reads it - the GPU engines all work off the device buffer - and at 4 bytes
|
|
// per pixel it is the single largest transfer in the pipeline, so the caller says whether it wants it.
|
|
ImagePreprocessorGPU(const DiffractionExperiment &experiment, const PixelMask &mask, std::shared_ptr<CudaStream> stream,
|
|
bool copy_image_to_host = true);
|
|
~ImagePreprocessorGPU() override;
|
|
ImageStatistics Analyze(ImagePreprocessorBuffer &processed_image, const uint8_t *decompressed_image, CompressedImageMode image_mode) override;
|
|
bool AnalyzeCompressed(ImagePreprocessorBuffer &processed_image, const CompressedImage &image,
|
|
ImageStatistics &stats) override;
|
|
[[nodiscard]] float GetLastDecompressionTime_s() const override;
|
|
void PinInputBuffer(std::vector<uint8_t> &buffer, size_t size) override;
|
|
void PinInputRegion(const void *ptr, size_t bytes) override;
|
|
};
|
|
|