Files
Jungfraujoch/reader/JFJochCBFReader.h
T
leonarski_fandClaude Opus 5.5 420750e0aa
Build Packages / Create release (push) Successful in 21s
Build Packages / build:viewer:macos-arm64:nocuda (push) Successful in 3m1s
Build Packages / build:rugnux:macos-arm64:nocuda (push) Successful in 2m18s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 8m34s
Build Packages / build:rugnux:linux-x86_64:cuda (push) Successful in 9m22s
Build Packages / build:viewer:linux-x86_64:nocuda (push) Successful in 10m35s
Build Packages / build:viewer:linux-x86_64:cuda (push) Successful in 11m40s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 16m1s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 16m31s
Build Packages / build:viewer:windows-x86_64:nocuda (push) Successful in 17m47s
Build Packages / build:viewer:windows-x86_64:cuda (push) Successful in 20m21s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 24m42s
Build Packages / build:rugnux:windows-x86_64:cuda (push) Successful in 11m12s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 20m31s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 20m10s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 20m59s
Build Packages / Build documentation (push) Successful in 1m27s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 18m10s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 18m20s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 14m31s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 11m3s
Build Packages / Unit tests (push) Successful in 1h20m31s
Reader: remove the background read-ahead
rugnux will mostly run on GPFS/Lustre-type storage. GPFS caches in its own
fixed-size pagepool, often smaller than a dataset, so streaming the files
ahead of the consumer can evict data before it is used; and choosing the
behaviour by the underlying filesystem is not wanted. This removes the
ReadAhead streamer of aa3fa6b9c: reader/ReadAhead.{h,cpp},
JFJochReader::StartReadAhead/NoteImageRead, the DataFiles() lists that only
it used (HDF5 reader, HDF5ImageSource, HDF5ImageLocator, CBF/marCCD/SMV),
and its start in rugnux_cli.cpp.

Kept from the same commit: the in-order CBF/marCCD/SMV header scan
(sweep::ForEachInOrder) and the 16 kB -> 256 kB CBF header probe, with its
test. They only change the order and size of reads the program makes
anyway, hold no memory and read less (cold 2400-frame CBF header scan
18 s -> 6 s on the measured HDD).

Output unchanged: myob p.hkl, p.mtz, p_P1.mtz and p_unmerged.mtz
byte-identical to the rc173 reference (17.1 s wall warm, 3.8 GB peak RSS).
Reader tests: [HDF5] 108 cases, CBF/marCCD/SMV/sweep/VDS/GetRawImage cases
all pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
2026-09-27 18:48:10 +02:00

62 lines
2.7 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <string>
#include <vector>
#include "JFJochReader.h"
#include "MiniCBF.h"
// Reads a rotation sweep straight from a directory of PILATUS miniCBF files - the form most
// facilities still archive - with no conversion step and no libcbf.
//
// The sweep's geometry comes from the first file's header; the rotation angle of each image comes
// from its own header, which costs only the bytes up to the binary separator. Images are decoded on
// demand, one per call, so any number of workers can read at once - there is no global lock as there
// is on the HDF5 path, HDF5 not being thread-safe.
//
// A raw CBF carries no analysis results, so this reader has no spots, no reflections and no snapshots;
// the dataset it builds is the geometry, the mask and nothing else, exactly as a plain DECTRIS file
// with no /entry/MX would give.
class JFJochCBFReader : public JFJochReader {
std::vector<std::string> files_;
std::shared_ptr<JFJochReaderDataset> dataset_;
minicbf::Header header0_;
bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
DataMessage &message,
std::vector<uint8_t> &buffer,
int64_t image_number,
bool update_dataset) override;
// Decodes one image into the caller's byte buffer (RawByteBuffer or std::vector<uint8_t>) and
// returns the image that points at it. The compressed file is read through scratch, which the
// caller keeps between frames.
template <class Buffer>
CompressedImage DecodeInto(int64_t image_number, Buffer &buffer, std::vector<uint8_t> &scratch) const;
public:
~JFJochCBFReader() override = default;
// True if the path names something this reader can open: a miniCBF file, or a directory holding
// at least one. Cheap - it reads a few kB at most.
static bool CanRead(const std::string &path);
// path is a directory of *.cbf, or one *.cbf inside the sweep to take the whole directory from.
void ReadFiles(const std::string &path);
[[nodiscard]] uint64_t GetNumberOfImages() const override;
void Close() override;
// Whether the sweep really has an image at this point. A deposited series can be missing frames,
// and they are kept as gaps so that every image keeps the spindle angle its own header states;
// ReadRawImage hands such a slot back as "nothing to read".
[[nodiscard]] bool HasImage(int64_t image_number) const;
bool ReadRawImage(int64_t image_number, JFJochReaderRawImage &image) override;
[[nodiscard]] std::vector<SpotToSave> ReadSpots(int64_t image) const override;
};