leonarski_fandClaude Opus 5.5
aea13044b5
The broker rebuilds the outgoing StartMessage from DiffractionExperiment, and FillMessage hard-coded countrate_correction_enabled and flatfield_enabled to false. JFJochReceiverLite parsed the true values from the detector's stream2 start message and dropped them, so every DECTRIS file written through the broker said neither correction was applied - DECTRIS enables both by default. pixel_mask_applied had the same defect: it reported the local apply_mask setting (an FPGA feature), not what the detector did to the pixels, which ReceiverLite forwards byte for byte. These now come from the stream: DetectorSetup carries them, ReceiverLite copies them in Configure, FillMessage reads them. Two more fields the DECTRIS stream sends and NXmx defines are passed through to the master file: countrate_correction_lookup_table (uint32, possibly bslz4/bszstd compressed in the stream) and virtual_pixel_interpolation_applied. The flatfield is deliberately not written - it makes the master file too large. PSI EIGER is unchanged: jfjoch never enables rate correction and the detector server starts with it off. The reader also opens the DECTRIS "hdf5 nexus v2024.2 nxmx" layout, where /entry/data/data is 4D [image, channel, y, x] and the pixel mask is kept per channel. Only the first channel is read; this is compatibility, not full multichannel support. A _process.h5 made from such a file links its pictures to that channel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
83 lines
4.1 KiB
C++
83 lines
4.1 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <map>
|
|
#include <memory>
|
|
#include <optional>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "../writer/HDF5Objects.h" // HDF5ReadOnlyFile, HDF5VirtualDatasetMapping, HDF5DataSetLayout
|
|
#include "../common/JFJochMessages.h" // FileWriterFormat, HDF5DataSourceMessage
|
|
|
|
// Turns a global image number into the HDF5 file + local index that physically holds its pixels,
|
|
// for all three on-disk layouts (legacy linked data files, VDS, contiguous/integrated). This is
|
|
// the part of the reader whose "links to files stay" constant: it knows where the raw images
|
|
// live, independent of which master file the per-image metadata is read from.
|
|
//
|
|
// Open data-file handles are cached, so scanning many images (e.g. reprocessing) does not reopen
|
|
// the same file on every read. HDF5 is not thread-safe, so every call must be made with the
|
|
// global hdf5_mutex held by the caller; the locator does no locking of its own.
|
|
class HDF5ImageLocator {
|
|
public:
|
|
struct Location {
|
|
std::shared_ptr<HDF5ReadOnlyFile> file;
|
|
uint32_t local_index = 0;
|
|
// Path the file was opened from. Needed to open it a second time as a plain file, for the
|
|
// positional reads HDF5ImageSource does outside the mutex.
|
|
std::string path;
|
|
// Where the images sit INSIDE that file. /entry/data/data everywhere DECTRIS writes, but a
|
|
// VDS names its source dataset and is free to name another one, so take it at its word.
|
|
std::string dataset = "/entry/data/data";
|
|
// Channel to read when the dataset is 4D, [image, channel, y, x], as the DECTRIS "hdf5 nexus
|
|
// v2024.2 nxmx" format writes it (one channel per threshold). Only the first channel of the
|
|
// master is read.
|
|
hsize_t channel = 0;
|
|
};
|
|
|
|
// One data file of a legacy multi-file dataset, with the dataset the master's link names
|
|
// inside it - the same "take the link at its word" the VDS branch already does.
|
|
struct LegacyFile {
|
|
std::string path;
|
|
std::string dataset;
|
|
};
|
|
|
|
// Layout description, filled by the reader once the master file has been parsed. All paths
|
|
// are absolute: legacy data files and VDS mapping filenames are resolved relative to the
|
|
// master before being handed over, so the locator never deals with relative paths.
|
|
struct Layout {
|
|
FileWriterFormat format = FileWriterFormat::NoFile;
|
|
HDF5DataSetLayout data_layout = HDF5DataSetLayout::CONTIGUOUS;
|
|
std::shared_ptr<HDF5ReadOnlyFile> master_file;
|
|
std::string master_filename;
|
|
std::vector<LegacyFile> legacy_files;
|
|
size_t images_per_file = 1;
|
|
std::vector<HDF5VirtualDatasetMapping> vds_mappings;
|
|
};
|
|
|
|
void Configure(Layout layout);
|
|
void Clear();
|
|
|
|
// Resolve a global image number to {file, local index}. Throws if the image is not covered
|
|
// by the layout. Does not bounds-check against the total image count - the caller does that.
|
|
Location Resolve(int64_t global_image) const;
|
|
|
|
// Source mapping for re-writing a derived file (e.g. _process.h5) so it links back to the
|
|
// original pixel sources rather than to a master. total_images is supplied by the caller.
|
|
// stride is the step between consecutive images of the derived file in the SOURCE: image i of the
|
|
// output comes from source image first_image + i * stride. It has to match the stride the caller
|
|
// processed with, or the pictures and the per-image analysis in the derived file describe
|
|
// different frames.
|
|
std::vector<HDF5DataSourceMessage> GetSourceMapping(uint64_t first_image,
|
|
std::optional<uint64_t> image_count,
|
|
uint64_t total_images,
|
|
uint64_t stride = 1) const;
|
|
|
|
private:
|
|
Layout layout_;
|
|
mutable std::map<std::string, std::shared_ptr<HDF5ReadOnlyFile> > file_cache_;
|
|
std::shared_ptr<HDF5ReadOnlyFile> OpenCached(const std::string &path) const;
|
|
};
|