mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 68 Wiki Activity

Ask HDF5 where an image is, then read it without the lock

Browse Source 
Two things every worker thread of an offline run did inside the global HDF5
mutex, per image.

It opened /entry/data/data and asked it for its dataspace, its datatype and its
creation plist, then asked those for the rank, the dimensions, the chunking and
the compression. All of that is a property of the file and identical for all of
its images, so it is now resolved once when the file is first touched.

And it read the pixels - megabytes of them, with the lock held, which is what
turned a worker per hardware thread into a queue. HDF5 can say where a chunk
lives instead - address and byte count, a lookup in the chunk index with no read
attached - so that is all it is asked for now, and the bytes are fetched after
the lock is dropped, with a positional read that any number of threads can make
through one handle at once. Chunk addresses count from the end of the user
block, so its size is added; zero for anything this project writes, not for
every file. A file that is not one chunk per image, or a chunk that was never
written and exists only as a fill value, still goes the old way - only HDF5
knows what those read as.

On a 16 Mpx rotation dataset with the process file being written, the per-image
loop at 48 workers goes 12.4 s -> 6.8 s, and stops getting slower as workers are
added: 8 workers were faster than 48 before, and are not now. Where no process
file is written the same loop only improves ~1%, because this machine has 1.5 TB
of RAM and held the whole 7 GB test set in page cache - the read was never the
expensive part here. It is where the cache is cold or the filesystem is remote.
Battery 9m45s, space group 21/24, no failures, unchanged.

The Windows path uses ReadFile with an OVERLAPPED offset for the same reason
pread is used elsewhere: it takes the offset as an argument rather than moving a
shared file position, so the viewer keeps building under MSVC and gets the same
concurrency.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jungfrau
2026-08-16 01:08:53 -04:00
co-authored by Claude Opus 5
parent 09cb9e2be9
commit f5b3193253
5 changed files with 225 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
reader/HDF5ImageLocator.cpp
+5 -3
View File
@@ -63,7 +63,8 @@ HDF5ImageLocator::Location HDF5ImageLocator::Resolve(int64_t global_image) const
if (layout_.format == FileWriterFormat::NXmxLegacy) {
const uint32_t file_id = global_image / layout_.images_per_file;
const uint32_t local_index = global_image % layout_.images_per_file;
return {OpenCached(layout_.legacy_files.at(file_id)), local_index};
const auto &path = layout_.legacy_files.at(file_id);
return {OpenCached(path), local_index, path};
}
if (layout_.format == FileWriterFormat::NXmxVDS
@@ -72,7 +73,8 @@ HDF5ImageLocator::Location HDF5ImageLocator::Resolve(int64_t global_image) const
for (const auto &mapping: layout_.vds_mappings) {
if (!mapping.ContainsVirtualImage(image))
continue;
return {OpenCached(mapping.filename), static_cast<uint32_t>(mapping.SourceImage(image))};
return {OpenCached(mapping.filename), static_cast<uint32_t>(mapping.SourceImage(image)),
mapping.filename};
}
throw JFJochException(JFJochExceptionCategory::HDF5,
"Image not covered by /entry/data/data VDS mappings");
@@ -81,7 +83,7 @@ HDF5ImageLocator::Location HDF5ImageLocator::Resolve(int64_t global_image) const
// Contiguous / integrated: pixels live in the master file at the global index.
if (!layout_.master_file)
throw JFJochException(JFJochExceptionCategory::HDF5, "Master file not loaded");
return {layout_.master_file, static_cast<uint32_t>(global_image)};
return {layout_.master_file, static_cast<uint32_t>(global_image), layout_.master_filename};
}
std::vector<HDF5DataSourceMessage> HDF5ImageLocator::GetSourceMapping(uint64_t first_image,