leonarski_fandClaude Opus 5
d01e69e407
_process.h5 is an NXmxIntegrated master that links to the ORIGINAL image files rather than writing images of its own (write_images = false). Its pixel metadata therefore has to describe those files - but it was filled from experiment_, which Rugnux pins to signed 32-bit because that is the container HDF5MetadataSource hands images out in. The virtual dataset was consequently typed int32 over unsigned 16- or 32-bit sources, so HDF5 converted every value on read: the 0xFFFFFFFF error marker of a uint32 source does not survive, and error_value and underload_value described a container the file does not contain. Our own reader never saw it, because it resolves the mapping and opens the source file itself. Only consumers that go through the virtual view - DIALS, XDS via Durin, plain h5py - read the converted values. Take the format from the reader instead. GetStoredPixelFormat() reports the bit depth and signedness of /entry/data/data as stored, which is deliberately not the same thing as the experiment's image format, and rugnux fills the start message from it. The writer is left alone on purpose: it must be able to produce a master before the mapped files exist, or when they are not readable, and the mapping strips the source directory, so it cannot open them to ask. Measured on a 20-image set with int16 sources: the _process.h5 virtual dataset goes from H5T_STD_I32LE to H5T_STD_I16LE, matching the source, and DIALS reports trusted_range (-32767, 32765) instead of (-2147483647, 32765). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
82 lines
3.6 KiB
C++
82 lines
3.6 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <map>
|
|
#include <memory>
|
|
#include <optional>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "JFJochReader.h"
|
|
#include "JFJochReaderSpots.h"
|
|
#include "HDF5ImageSource.h"
|
|
#include "HDF5MetadataSource.h"
|
|
#include "../writer/HDF5Objects.h"
|
|
#include "../image_analysis/IntegrationOutcome.h"
|
|
|
|
// Composes one shared raw-image source with one or more metadata sources ("snapshots") over the
|
|
// same images: the original _master.h5 plus optional reprocessing _process.h5 results. The active
|
|
// snapshot supplies the dataset and per-image metadata; the image source always supplies pixels,
|
|
// so switching snapshots never reloads image data.
|
|
class JFJochHDF5Reader : public JFJochReader {
|
|
HDF5ImageSource image_source_;
|
|
std::map<std::string, std::shared_ptr<HDF5MetadataSource> > snapshots_;
|
|
std::shared_ptr<HDF5MetadataSource> active_metadata_;
|
|
std::string active_snapshot_;
|
|
size_t number_of_images = 0;
|
|
|
|
bool LoadImage_i(std::shared_ptr<JFJochReaderDataset> &dataset,
|
|
DataMessage &message,
|
|
std::vector<uint8_t> &buffer,
|
|
int64_t image_number,
|
|
bool update_dataset) override;
|
|
|
|
HDF5ImageLocator::Location GetImageLocation(int64_t image_number) const;
|
|
|
|
public:
|
|
~JFJochHDF5Reader() override = default;
|
|
|
|
void ReadFile(const std::string &filename);
|
|
|
|
uint64_t GetNumberOfImages() const override;
|
|
|
|
void Close() override;
|
|
|
|
// stride is the step in the SOURCE between consecutive images of the derived file: image i of the
|
|
// output comes from source image first_image + i * stride. Pass the stride the caller processed
|
|
// with, or the linked pictures and the per-image analysis describe different frames.
|
|
std::vector<HDF5DataSourceMessage> GetHDF5DataSource(
|
|
uint64_t first_image = 0,
|
|
std::optional<uint64_t> image_count = {},
|
|
uint64_t stride = 1
|
|
) const;
|
|
|
|
// Pixel format of the stored images. Needed by anything that links to them instead of writing
|
|
// its own, since the experiment describes the reader's container rather than the file.
|
|
[[nodiscard]] StoredPixelFormat GetStoredPixelFormat() const;
|
|
|
|
std::vector<IntegrationOutcome> ReadReflections(size_t start_image = 0, std::optional<size_t> end_image = {}) const;
|
|
|
|
std::vector<SpotToSave> ReadSpots(int64_t image) const override;
|
|
// True if the loaded file stores spot-finding results; false for a plain DECTRIS dataset (no
|
|
// /entry/MX), so callers can find spots instead of reusing absent ones.
|
|
[[nodiscard]] bool HasStoredSpots() const;
|
|
|
|
CompressedImage ReadCalibration(std::vector<uint8_t> &tmp, const std::string &name) const;
|
|
|
|
std::shared_ptr<JFJochReaderRawImage> GetRawImage(int64_t image_number) override;
|
|
|
|
// Metadata snapshots over the same images. The original file is registered as "Original" on
|
|
// ReadFile and is the initial active snapshot; reprocessing results can be added later.
|
|
void RegisterSnapshot(const std::string &name, const std::string &master_path);
|
|
void RemoveSnapshot(const std::string &name); // "Original" cannot be removed
|
|
void SetActiveSnapshot(const std::string &name);
|
|
std::vector<std::string> SnapshotNames() const;
|
|
std::string ActiveSnapshot() const;
|
|
|
|
// All snapshot datasets (name -> dataset), for overlaying every run's plots at once.
|
|
std::vector<std::pair<std::string, std::shared_ptr<const JFJochReaderDataset>>> AllSnapshotDatasets() const;
|
|
};
|