leonarski_fandClaude Opus 5
d01e69e407
_process.h5 is an NXmxIntegrated master that links to the ORIGINAL image files rather than writing images of its own (write_images = false). Its pixel metadata therefore has to describe those files - but it was filled from experiment_, which Rugnux pins to signed 32-bit because that is the container HDF5MetadataSource hands images out in. The virtual dataset was consequently typed int32 over unsigned 16- or 32-bit sources, so HDF5 converted every value on read: the 0xFFFFFFFF error marker of a uint32 source does not survive, and error_value and underload_value described a container the file does not contain. Our own reader never saw it, because it resolves the mapping and opens the source file itself. Only consumers that go through the virtual view - DIALS, XDS via Durin, plain h5py - read the converted values. Take the format from the reader instead. GetStoredPixelFormat() reports the bit depth and signedness of /entry/data/data as stored, which is deliberately not the same thing as the experiment's image format, and rugnux fills the start message from it. The writer is left alone on purpose: it must be able to produce a master before the mapped files exist, or when they are not readable, and the mapping strips the source directory, so it cannot open them to ask. Measured on a 20-image set with int16 sources: the _process.h5 virtual dataset goes from H5T_STD_I32LE to H5T_STD_I16LE, matching the source, and DIALS reports trusted_range (-32767, 32765) instead of (-2147483647, 32765). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 lines
2.8 KiB
C++
75 lines
2.8 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include "HDF5ImageSource.h"
|
|
#include "../common/JFJochException.h"
|
|
|
|
void HDF5ImageSource::Configure(HDF5ImageLocator::Layout layout) {
|
|
locator_.Configure(std::move(layout));
|
|
}
|
|
|
|
void HDF5ImageSource::Clear() {
|
|
locator_.Clear();
|
|
}
|
|
|
|
HDF5ImageLocator::Location HDF5ImageSource::Resolve(int64_t global) const {
|
|
return locator_.Resolve(global);
|
|
}
|
|
|
|
StoredPixelFormat HDF5ImageSource::GetStoredPixelFormat() const {
|
|
auto loc = locator_.Resolve(0);
|
|
HDF5DataSet dataset(*loc.file, "/entry/data/data");
|
|
HDF5DataType datatype(dataset);
|
|
return {static_cast<int64_t>(datatype.GetElemSize()) * 8, datatype.IsSigned()};
|
|
}
|
|
|
|
std::vector<HDF5DataSourceMessage> HDF5ImageSource::GetSourceMapping(uint64_t first_image,
|
|
std::optional<uint64_t> image_count,
|
|
uint64_t total_images,
|
|
uint64_t stride) const {
|
|
return locator_.GetSourceMapping(first_image, image_count, total_images, stride);
|
|
}
|
|
|
|
CompressedImage HDF5ImageSource::ReadImageAt(std::vector<uint8_t> &buffer,
|
|
const HDF5ImageLocator::Location &loc) const {
|
|
return LoadImageDataset(buffer, *loc.file, loc.local_index);
|
|
}
|
|
|
|
CompressedImage HDF5ImageSource::LoadImageDataset(std::vector<uint8_t> &tmp, HDF5Object &file, hsize_t number) {
|
|
std::vector<hsize_t> start = {static_cast<hsize_t>(number), 0, 0};
|
|
|
|
const std::string dataset_name = "/entry/data/data";
|
|
|
|
HDF5DataSet dataset(file, dataset_name);
|
|
HDF5DataSpace dataspace(dataset);
|
|
HDF5DataType datatype(dataset);
|
|
HDF5Dcpl dcpl(dataset);
|
|
|
|
if (dataspace.GetNumOfDimensions() != 3)
|
|
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
|
|
"/entry/data/data dataset must be 3D");
|
|
|
|
auto dim = dataspace.GetDimensions();
|
|
|
|
CompressionAlgorithm algorithm = CompressionAlgorithm::NO_COMPRESSION;
|
|
auto chunk_size = dcpl.GetChunking();
|
|
|
|
if ((chunk_size.size() == 3) && (chunk_size[0] == 1) && (chunk_size[1] == dim[1]) && (chunk_size[2] == dim[2])) {
|
|
dataset.ReadDirectChunk(tmp, start);
|
|
algorithm = dcpl.GetCompression();
|
|
} else {
|
|
dataset.ReadVectorToU8(tmp, start, {1, dim[1], dim[2]});
|
|
algorithm = CompressionAlgorithm::NO_COMPRESSION;
|
|
}
|
|
|
|
if (datatype.IsFloat())
|
|
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
|
|
"Float datasets not supported at this time");
|
|
|
|
return {
|
|
tmp, dim[2], dim[1],
|
|
CalcImageMode(datatype.GetElemSize(), datatype.IsFloat(), datatype.IsSigned()),
|
|
algorithm
|
|
};
|
|
}
|