The CBF reader decoded every frame's x-CBF_BYTE_OFFSET stream on the worker thread - 19-30 % of the image-loop worker CPU on CBF sets (7qij, 4mtk) - and the 4-byte image was then uploaded. ReadRawImage now hands over the stream as the file stores it, in the image's own buffer, as a new read-only CompressionAlgorithm::CBF_BYTE_OFFSET (mode Int32, what it decodes to), and ImagePreprocessorGPU decodes it on the card into the same int32 image and runs the same preprocessing kernel on it. The device decode (CBFDecoderGPU) cuts the stream into 256-byte tiles. A token is 1, 3, 7 or 15 bytes, so a tile can be entered at 15 offsets; for each one thread works out where the decode leaves the tile, how many pixels it emitted and their sum modulo 2^32 (the walks from different entries meet within a few bytes, after which they are one walk). An exclusive scan composing these maps gives each tile its true entry, first pixel and running value, and a second walk writes the pixels. A stream that ends before the image is refused as on the host. The host decoder moves to compression/ (DecodeCBFByteOffset) and is reached through JFJochDecompressPtr, so every host consumer - the CPU build, the azimuthal-integration worker, the pre-scan's shadow and gain passes - decodes exactly as before, now from the stream. It is the CPU path, not a fallback: a frame the device refuses is not retried on the host. A process file written from CBF input stores the decoded int32 image, as it always did. .cbf.gz is still inflated on the host. Tests: CBFDecoderGPU_MatchesHost (every token size, 0x80 inside payloads, escape runs across tiles, trailing CIF text), CBFDecoderGPU_ShortStreamIsRefused, ImagePreprocessorGPU_StoredEncodingsMatchCPU. Checked locally against the host decoder on 36 real frames of 9 open-arm CBF sets (signed 32-bit, signed and unsigned 16-bit, plain and gzip): identical. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
103 lines
5.5 KiB
C++
103 lines
5.5 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <array>
|
|
#include <cstdint>
|
|
#include <optional>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "../common/CompressedImage.h"
|
|
|
|
// PILATUS miniCBF: an ASCII header, four separator bytes, then one byte-offset compressed image.
|
|
// No CIF parser and no libcbf are needed - every value is on a "# " comment line or a MIME line.
|
|
namespace minicbf {
|
|
|
|
// The four bytes that end the MIME header and begin the binary section.
|
|
inline constexpr unsigned char BINARY_SEPARATOR[4] = {0x0c, 0x1a, 0x04, 0xd5};
|
|
|
|
struct Header {
|
|
std::string detector; // "PILATUS3 6M, S/N 60-0136"
|
|
int64_t nx = 0; // fast dimension (columns)
|
|
int64_t ny = 0; // slow dimension (rows)
|
|
int64_t nelem = 0; // pixel count declared by the MIME header
|
|
double pixel_x_m = 0;
|
|
double pixel_y_m = 0;
|
|
double thickness_m = 0;
|
|
std::string material = "Si"; // NORMALISED: the file says "Silicon", the rest of the code wants "Si"
|
|
double distance_m = 0;
|
|
double beam_x_px = 0;
|
|
double beam_y_px = 0;
|
|
double wavelength_A = 0;
|
|
double start_angle_deg = 0;
|
|
double angle_increment_deg = 0;
|
|
double two_theta_deg = 0;
|
|
// Goniometer head angles and the per-image increment of each, in degrees. A writer spells "this
|
|
// head has no such axis" as -9999, which is a sentinel and not an angle, so an axis the header
|
|
// does not really carry is absent here rather than 9999 degrees away from zero.
|
|
std::optional<double> chi_deg;
|
|
std::optional<double> omega_deg;
|
|
double chi_increment_deg = 0;
|
|
double phi_increment_deg = 0;
|
|
double omega_increment_deg = 0;
|
|
double exposure_s = 0;
|
|
double period_s = 0;
|
|
int64_t count_cutoff = 0; // saturation
|
|
std::string axis_name = "omega";
|
|
bool byte_offset = false; // the only conversion supported
|
|
|
|
// The imgCIF axis table the CBF template block carries after the "# " lines, as the file states
|
|
// it: vectors in the imgCIF laboratory frame (Z from the sample towards the source, Y opposite
|
|
// gravity, X completing a right-handed set). Turning them into any other frame is the caller's
|
|
// business, not this format reader's. Most headers carry no such table and leave all of these
|
|
// empty, which is not the same as their stating the usual mounting.
|
|
std::optional<std::array<double, 3>> fast_direction; // laboratory direction of a +1 column step
|
|
std::optional<std::array<double, 3>> slow_direction; // ... of a +1 row step
|
|
std::optional<std::array<double, 3>> spindle_axis; // the base goniometer rotation axis
|
|
// The goniometer rotation axis carried by the base one, stated in the base axis' own frame. A
|
|
// head whose inner circle is INCLINED - a fixed-chi stage, a kappa arm - says so only here: its
|
|
// "# Chi" and "# Kappa" lines read zero because no such circle is driven. Empty where the table
|
|
// states one rotation axis only, which is every Eulerian head.
|
|
std::optional<std::array<double, 3>> inner_spindle_axis;
|
|
std::optional<std::array<double, 3>> detector_axis; // the detector's own rotation axis: a 2theta arm
|
|
// "# Oscillation_axis X.CW +SLOW". Some writers say which of the two image directions the spindle
|
|
// runs along instead of stating a vector. The axis NAME on that line is a poor discriminator (see
|
|
// ScansPhi, and note that the one header carrying both a name and a table says "X" where its table
|
|
// says Y) but this token is not: where both are present they agree.
|
|
bool spindle_along_slow = false;
|
|
};
|
|
|
|
// Whether the sweep turns PHI rather than the base (omega) axis. What moved is the axis with a
|
|
// non-zero increment - a header routinely carries a "# Phi" angle for a phi that stands still - so
|
|
// the name is consulted only when no increment is stated at all. It is a poor discriminator: some
|
|
// writers put a direction convention there ("X.CW") instead of an axis name.
|
|
bool ScansPhi(const Header &h);
|
|
|
|
// Byte offset of the binary section (just past the separator), or nothing if there is none.
|
|
std::optional<size_t> FindBinarySection(const uint8_t *data, size_t size);
|
|
|
|
// Parse the ASCII header. Pass the bytes BEFORE the separator; headers are not a fixed size (one
|
|
// Diamond I24 set carries 6335 bytes, well past a 4 kB guess), so never parse a fixed prefix.
|
|
Header ParseHeader(const char *data, size_t size);
|
|
|
|
// Header + pixels of one file, read from disk.
|
|
Header Read(const std::string &path, std::vector<int32_t> &out);
|
|
|
|
// The same, decoding into memory the caller already has (one read of the file, no extra copy).
|
|
// The compressed file is read through scratch, which the caller keeps between frames: a fresh
|
|
// vector per image is megabytes of allocation and of zeroing nothing ever reads.
|
|
// Throws if the image does not fit in capacity pixels.
|
|
Header ReadInto(const std::string &path, int32_t *out, size_t capacity, std::vector<uint8_t> &scratch);
|
|
|
|
// The whole file read into buf, header parsed and checked, without decoding the pixels: the byte-offset
|
|
// stream starts at binary_start. For a consumer that decodes it itself - on the device, where there is one.
|
|
Header ReadCompressed(const std::string &path, RawByteBuffer &buf, size_t &binary_start);
|
|
|
|
// Header only - reads just enough of the file to reach the separator. Cheap enough to call per
|
|
// frame for the rotation angles.
|
|
Header ReadHeader(const std::string &path);
|
|
|
|
} // namespace minicbf
|