Build Packages / Create release (push) Successful in 21s
Build Packages / build:viewer:macos-arm64:nocuda (push) Successful in 3m1s
Build Packages / build:rugnux:macos-arm64:nocuda (push) Successful in 2m18s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 8m34s
Build Packages / build:rugnux:linux-x86_64:cuda (push) Successful in 9m22s
Build Packages / build:viewer:linux-x86_64:nocuda (push) Successful in 10m35s
Build Packages / build:viewer:linux-x86_64:cuda (push) Successful in 11m40s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 16m1s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 16m31s
Build Packages / build:viewer:windows-x86_64:nocuda (push) Successful in 17m47s
Build Packages / build:viewer:windows-x86_64:cuda (push) Successful in 20m21s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 24m42s
Build Packages / build:rugnux:windows-x86_64:cuda (push) Successful in 11m12s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 20m31s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 20m10s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 20m59s
Build Packages / Build documentation (push) Successful in 1m27s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 18m10s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 18m20s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 14m31s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 11m3s
Build Packages / Unit tests (push) Successful in 1h20m31s
rugnux will mostly run on GPFS/Lustre-type storage. GPFS caches in its own
fixed-size pagepool, often smaller than a dataset, so streaming the files
ahead of the consumer can evict data before it is used; and choosing the
behaviour by the underlying filesystem is not wanted. This removes the
ReadAhead streamer of aa3fa6b9c: reader/ReadAhead.{h,cpp},
JFJochReader::StartReadAhead/NoteImageRead, the DataFiles() lists that only
it used (HDF5 reader, HDF5ImageSource, HDF5ImageLocator, CBF/marCCD/SMV),
and its start in rugnux_cli.cpp.
Kept from the same commit: the in-order CBF/marCCD/SMV header scan
(sweep::ForEachInOrder) and the 16 kB -> 256 kB CBF header probe, with its
test. They only change the order and size of reads the program makes
anyway, hold no memory and read less (cold 2400-frame CBF header scan
18 s -> 6 s on the measured HDD).
Output unchanged: myob p.hkl, p.mtz, p_P1.mtz and p_unmerged.mtz
byte-identical to the rc173 reference (17.1 s wall warm, 3.8 GB peak RSS).
Reader tests: [HDF5] 108 cases, CBF/marCCD/SMV/sweep/VDS/GetRawImage cases
all pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
66 lines
2.5 KiB
C++
66 lines
2.5 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#ifndef JFJOCHIMAGEREADER_H
|
|
#define JFJOCHIMAGEREADER_H
|
|
|
|
#include <unordered_set>
|
|
#include <map>
|
|
#include <mutex>
|
|
|
|
#include "../common/JFJochMessages.h"
|
|
#include "../common/Plot.h"
|
|
#include "../common/ROIBox.h"
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "JFJochReaderDataset.h"
|
|
#include "JFJochReaderImage.h"
|
|
#include "JFJochReaderSpots.h"
|
|
|
|
class JFJochReader {
|
|
mutable std::mutex m;
|
|
std::shared_ptr<JFJochReaderDataset> dataset;
|
|
|
|
virtual bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
|
|
DataMessage& message,
|
|
std::vector<uint8_t> &buffer,
|
|
int64_t image_number,
|
|
bool update_dataset) = 0;
|
|
|
|
mutable std::mutex summation_mutex;
|
|
void SummationThread(int64_t image0,
|
|
int64_t n_image,
|
|
int64_t image_jump,
|
|
JFJochReaderImage &image);
|
|
protected:
|
|
void SetStartMessage(const std::shared_ptr<JFJochReaderDataset> &val);
|
|
DiffractionExperiment default_experiment;
|
|
public:
|
|
virtual ~JFJochReader() = default;
|
|
|
|
JFJochReader& Experiment(const DiffractionExperiment& experiment);
|
|
|
|
std::shared_ptr<const JFJochReaderDataset> GetDataset() const;
|
|
[[nodiscard]] virtual uint64_t GetNumberOfImages() const = 0;
|
|
|
|
virtual void Close() = 0;
|
|
|
|
std::shared_ptr<JFJochReaderImage> LoadImage(int64_t image_number, int64_t summation_factor = 1);
|
|
|
|
void UpdateGeomMetadata(const DiffractionExperiment& experiment);
|
|
void UpdateUserMask(const std::vector<uint32_t>& mask);
|
|
|
|
// Read one image into a raw image the caller owns, false where there is nothing to read. A
|
|
// worker that walks a whole sweep hands the same object back on every frame and so keeps its
|
|
// buffer: GetRawImage() allocates a fresh one per call, which on a 4-byte 16 Mpx frame is 72 MB
|
|
// of untouched pages the decoder then faults in one by one, measurably more than the decode.
|
|
virtual bool ReadRawImage(int64_t image_number, JFJochReaderRawImage &image) = 0;
|
|
|
|
// The same image in a freshly allocated raw image.
|
|
std::shared_ptr<JFJochReaderRawImage> GetRawImage(int64_t image_number);
|
|
|
|
virtual std::vector<SpotToSave> ReadSpots(int64_t image) const = 0;
|
|
std::shared_ptr<JFJochReaderSpots> ReadAllSpots(int64_t start_image, int64_t end_image, int64_t stride = 1) const;
|
|
};
|
|
|
|
#endif //JFJOCHIMAGEREADER_H
|