mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 73 Wiki Activity

reader: open a master whose beamline writes standards-correct NXmx differently

Browse Source 
Four defects, hit in sequence, that between them stopped eight masters from one beamline before any
geometry question was reached. The files are correct NeXus; the reader was assuming one writer's
conventions.

ReadScalar demanded rank 0. NXmx puts no rank on distance, saturation_value, two_theta or det_z, and
these files write them as shape (1,). Any dataspace holding exactly one element is now accepted; a
genuine vector is still refused.

frame_time was read unconditionally and NXmx does not require it.

A virtual-dataset source filename of "." was resolved as a relative path, giving <dir>/. - but "."
is HDF5's spelling for THIS file, and these masters compose /entry/data/data as a virtual dataset
over datasets in themselves that are external links to the data files.

The virtual source's DATASET PATH was parsed and then ignored in favour of a hardcoded
/entry/data/data, while these data files keep their images at the root. Fixing that exposed a fifth:
the positional-read fast path used the master's own path, but a dataset reached through an external
link lives in another file and a chunk address is an offset into THAT file.

An audit of all 38 masters in the non-SLS corpus finds exactly these eight need the change and the
other 30 need nothing. Corpus A/B over 73 dataset pairs, base and patched back to back: 64
byte-identical results, 9 identical failures, none differing. The blast radius is bounded by
construction - JFJochReader is linked only by rugnux and jfjoch_viewer, and the one shared header
this touches only ever accepts more, so nothing that opened before can read differently.

With it the eight files read completely: 12850 images, no decode errors, seven of them spanning two
source datasets through the master's own external links.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
This commit is contained in:
leonarski_f
2026-08-29 19:58:33 +02:00
co-authored by Claude Opus 5
parent 4ef6bd9952
commit 60c15d6931
7 changed files with 87 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
reader/HDF5ImageSource.cpp
+33 -15
View File
@@ -71,7 +71,7 @@ HDF5ImageLocator::Location HDF5ImageSource::Resolve(int64_t global) const {
StoredPixelFormat HDF5ImageSource::GetStoredPixelFormat() const {
auto loc = locator_.Resolve(0);
HDF5DataSet dataset(*loc.file, "/entry/data/data");
HDF5DataSet dataset(*loc.file, loc.dataset);
HDF5DataType datatype(dataset);
return {static_cast<int64_t>(datatype.GetElemSize()) * 8, datatype.IsSigned()};
}
@@ -85,12 +85,13 @@ std::vector<HDF5DataSourceMessage> HDF5ImageSource::GetSourceMapping(uint64_t fi
const HDF5ImageSource::OpenDataset &
HDF5ImageSource::GetDataset(const HDF5ImageLocator::Location &loc) const {
if (auto it = dataset_cache_.find(loc.file.get()); it != dataset_cache_.end())
auto key = std::make_pair(loc.file.get(), loc.dataset);
if (auto it = dataset_cache_.find(key); it != dataset_cache_.end())
return it->second;
OpenDataset entry;
entry.file = loc.file;
entry.dataset = std::make_unique<HDF5DataSet>(*loc.file, "/entry/data/data");
entry.dataset = std::make_unique<HDF5DataSet>(*loc.file, loc.dataset);
HDF5DataSpace dataspace(*entry.dataset);
HDF5DataType datatype(*entry.dataset);
@@ -98,7 +99,7 @@ HDF5ImageSource::GetDataset(const HDF5ImageLocator::Location &loc) const {
if (dataspace.GetNumOfDimensions() != 3)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
"/entry/data/data dataset must be 3D");
loc.dataset + " dataset must be 3D");
if (datatype.IsFloat())
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
@@ -115,20 +116,37 @@ HDF5ImageSource::GetDataset(const HDF5ImageLocator::Location &loc) const {
if (entry.direct_chunk)
entry.algorithm = dcpl.GetCompression();
if (entry.direct_chunk && !loc.path.empty()) {
entry.raw = std::make_shared<RawFile>(loc.path);
if (!entry.raw->IsOpen())
entry.raw.reset();
hid_t fcpl = H5Fget_create_plist(loc.file->GetID());
if (fcpl >= 0) {
hsize_t user_block = 0;
if (H5Pget_userblock(fcpl, &user_block) >= 0)
entry.user_block = user_block;
H5Pclose(fcpl);
if (entry.direct_chunk) {
// A positional read needs the file the PIXELS are in, and that is not always the file that
// was opened: a dataset reached through an external link lives elsewhere, and a chunk
// address is an offset into ITS file - as is the user block the address counts from. Ask
// HDF5 which file the dataset ended up in rather than assuming it is the one we asked.
std::string path = loc.path;
const hid_t owner = H5Iget_file_id(entry.dataset->GetID());
if (owner >= 0) {
const ssize_t name_size = H5Fget_name(owner, nullptr, 0);
if (name_size > 0) {
std::string name(static_cast<size_t>(name_size), '\0');
if (H5Fget_name(owner, name.data(), name.size() + 1) >= 0)
path = name;
}
const hid_t fcpl = H5Fget_create_plist(owner);
if (fcpl >= 0) {
hsize_t user_block = 0;
if (H5Pget_userblock(fcpl, &user_block) >= 0)
entry.user_block = user_block;
H5Pclose(fcpl);
}
H5Fclose(owner);
}
if (!path.empty()) {
entry.raw = std::make_shared<RawFile>(path);
if (!entry.raw->IsOpen())
entry.raw.reset();
}
}
return dataset_cache_.emplace(loc.file.get(), std::move(entry)).first->second;
return dataset_cache_.emplace(std::move(key), std::move(entry)).first->second;
}
std::optional<HDF5ImageSource::DirectChunk>