leonarski_fandClaude Opus 5
d01e69e407
_process.h5 is an NXmxIntegrated master that links to the ORIGINAL image files rather than writing images of its own (write_images = false). Its pixel metadata therefore has to describe those files - but it was filled from experiment_, which Rugnux pins to signed 32-bit because that is the container HDF5MetadataSource hands images out in. The virtual dataset was consequently typed int32 over unsigned 16- or 32-bit sources, so HDF5 converted every value on read: the 0xFFFFFFFF error marker of a uint32 source does not survive, and error_value and underload_value described a container the file does not contain. Our own reader never saw it, because it resolves the mapping and opens the source file itself. Only consumers that go through the virtual view - DIALS, XDS via Durin, plain h5py - read the converted values. Take the format from the reader instead. GetStoredPixelFormat() reports the bit depth and signedness of /entry/data/data as stored, which is deliberately not the same thing as the experiment's image format, and rugnux fills the start message from it. The writer is left alone on purpose: it must be able to produce a master before the mapped files exist, or when they are not readable, and the mapping strips the source directory, so it cannot open them to ask. Measured on a 20-image set with int16 sources: the _process.h5 virtual dataset goes from H5T_STD_I32LE to H5T_STD_I16LE, matching the source, and DIALS reports trusted_range (-32767, 32765) instead of (-2147483647, 32765). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
49 lines
2.1 KiB
C++
49 lines
2.1 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <cstdint>
|
|
#include <optional>
|
|
#include <vector>
|
|
|
|
#include "HDF5ImageLocator.h"
|
|
#include "../common/CompressedImage.h"
|
|
|
|
// Raw-pixel side of the reader. Turns a global image number into a CompressedImage, using
|
|
// HDF5ImageLocator to find the file (with its open-file cache). This is the part whose "links
|
|
// to files stay" constant: switching which master the per-image metadata is read from never
|
|
// touches it. Caller must hold the global hdf5_mutex (HDF5 is not thread-safe).
|
|
// Bit depth and signedness of /entry/data/data as it is STORED. Deliberately not the same thing as
|
|
// the experiment's image format: the reader hands every image out in a signed 32-bit container
|
|
// whatever the file holds (see HDF5MetadataSource), so an output file that links to the original
|
|
// images rather than writing its own must describe them with this, not with the experiment.
|
|
struct StoredPixelFormat {
|
|
int64_t bit_depth = 0;
|
|
bool is_signed = false;
|
|
};
|
|
|
|
class HDF5ImageSource {
|
|
public:
|
|
void Configure(HDF5ImageLocator::Layout layout);
|
|
void Clear();
|
|
|
|
[[nodiscard]] StoredPixelFormat GetStoredPixelFormat() const;
|
|
|
|
// Where image `global` physically lives. Also used by the metadata source to find the data
|
|
// file that holds a legacy/VDS image's per-image metadata.
|
|
HDF5ImageLocator::Location Resolve(int64_t global) const;
|
|
|
|
// Read the pixels at a resolved location into a CompressedImage backed by `buffer`.
|
|
CompressedImage ReadImageAt(std::vector<uint8_t> &buffer, const HDF5ImageLocator::Location &loc) const;
|
|
|
|
std::vector<HDF5DataSourceMessage> GetSourceMapping(uint64_t first_image,
|
|
std::optional<uint64_t> image_count,
|
|
uint64_t total_images,
|
|
uint64_t stride = 1) const;
|
|
|
|
private:
|
|
HDF5ImageLocator locator_;
|
|
static CompressedImage LoadImageDataset(std::vector<uint8_t> &tmp, HDF5Object &file, hsize_t number);
|
|
};
|