Reader: remove the background read-ahead
Build Packages / Create release (push) Successful in 21s
Build Packages / build:viewer:macos-arm64:nocuda (push) Successful in 3m1s
Build Packages / build:rugnux:macos-arm64:nocuda (push) Successful in 2m18s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 8m34s
Build Packages / build:rugnux:linux-x86_64:cuda (push) Successful in 9m22s
Build Packages / build:viewer:linux-x86_64:nocuda (push) Successful in 10m35s
Build Packages / build:viewer:linux-x86_64:cuda (push) Successful in 11m40s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 16m1s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 16m31s
Build Packages / build:viewer:windows-x86_64:nocuda (push) Successful in 17m47s
Build Packages / build:viewer:windows-x86_64:cuda (push) Successful in 20m21s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 24m42s
Build Packages / build:rugnux:windows-x86_64:cuda (push) Successful in 11m12s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 20m31s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 20m10s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 20m59s
Build Packages / Build documentation (push) Successful in 1m27s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 18m10s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 18m20s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 14m31s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 11m3s
Build Packages / Unit tests (push) Successful in 1h20m31s

rugnux will mostly run on GPFS/Lustre-type storage. GPFS caches in its own
fixed-size pagepool, often smaller than a dataset, so streaming the files
ahead of the consumer can evict data before it is used; and choosing the
behaviour by the underlying filesystem is not wanted. This removes the
ReadAhead streamer of aa3fa6b9c: reader/ReadAhead.{h,cpp},
JFJochReader::StartReadAhead/NoteImageRead, the DataFiles() lists that only
it used (HDF5 reader, HDF5ImageSource, HDF5ImageLocator, CBF/marCCD/SMV),
and its start in rugnux_cli.cpp.

Kept from the same commit: the in-order CBF/marCCD/SMV header scan
(sweep::ForEachInOrder) and the 16 kB -> 256 kB CBF header probe, with its
test. They only change the order and size of reads the program makes
anyway, hold no memory and read less (cold 2400-frame CBF header scan
18 s -> 6 s on the measured HDD).

Output unchanged: myob p.hkl, p.mtz, p_P1.mtz and p_unmerged.mtz
byte-identical to the rc173 reference (17.1 s wall warm, 3.8 GB peak RSS).
Reader tests: [HDF5] 108 cases, CBF/marCCD/SMV/sweep/VDS/GetRawImage cases
all pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
This commit is contained in:
2026-09-27 18:48:10 +02:00
co-authored by Claude Opus 5.5
parent 1479af2984
commit 420750e0aa
17 changed files with 0 additions and 211 deletions
-18
View File
@@ -4,8 +4,6 @@
#include "HDF5ImageLocator.h"
#include "../common/JFJochException.h"
#include <algorithm>
namespace {
// Coalesce consecutive single-image mappings into one contiguous range when the source and
// virtual images stay contiguous in the same file/dataset.
@@ -103,22 +101,6 @@ HDF5ImageLocator::Location HDF5ImageLocator::Resolve(int64_t global_image) const
return {layout_.master_file, static_cast<uint32_t>(global_image), layout_.master_filename};
}
std::vector<std::string> HDF5ImageLocator::DataFiles() const {
std::vector<std::string> ret;
if (layout_.format == FileWriterFormat::NXmxLegacy) {
for (const auto &f: layout_.legacy_files)
ret.push_back(f.path);
} else if (layout_.format == FileWriterFormat::NXmxVDS
&& layout_.data_layout == HDF5DataSetLayout::VIRTUAL) {
for (const auto &mapping: layout_.vds_mappings)
if (CoversFirstChannel(mapping)
&& std::find(ret.begin(), ret.end(), mapping.filename) == ret.end())
ret.push_back(mapping.filename);
} else if (!layout_.master_filename.empty())
ret.push_back(layout_.master_filename);
return ret;
}
std::vector<HDF5DataSourceMessage> HDF5ImageLocator::GetSourceMapping(uint64_t first_image,
std::optional<uint64_t> image_count,
uint64_t total_images,