leonarski_fandClaude Opus 5
259b43154e
Two ways the written files could misdescribe themselves without anyone noticing. The pixel format is stated twice and the two were never compared: each image carries its own type as a CBOR tag, which is what the data files are written with, while the master is typed from the start message. A stream whose header contradicts its images produced data files of one type under a master declaring another, and with NXmxVDS, HDF5 then converts silently on every read. HDF5DataFile::CreateFile now checks the two agree and refuses the run otherwise - the point where the values first meet, so it covers every path into the writer. Two test fixtures were relying on exactly that inconsistency. The HDF5 writer tests wrote uint16 buffers under a JUNGFRAU experiment, which converts to photon counts by default and so declares int16; they never read the pixels back, so it went unnoticed. The receiver-lite tests feed frames from compression_benchmark.h5, which really are signed int16, through a DECTRIS experiment, which declares unsigned by default - the same class of bug the pixel_signed propagation fixed on the live path. Both now declare what they send. Second: a virtual dataset whose source file is absent reads as the fill value, and HDF5 defaults that to zero, so a data file that was not copied alongside the master is indistinguishable from frames of genuine zero counts. Measured with DIALS on a four-file set with one file removed: 25 frames of pure zeros, no error and no warning. The image VDS is now filled with the error marker instead, which sits outside underload_value..saturation_value, so a reader masks those frames. Same measurement after the change: -32768 throughout, which DIALS excludes. Only the images ask for a fill value; the per-image metadata datasets keep the default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
74 lines
3.2 KiB
C++
74 lines
3.2 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include "../common/JFJochMessages.h"
|
|
|
|
#include "HDF5Objects.h"
|
|
|
|
namespace HDF5Metadata {
|
|
std::string MasterFileName(const StartMessage &msg);
|
|
std::string DataFileName(const StartMessage &msg, int64_t file_number);
|
|
}
|
|
|
|
class NXmx {
|
|
std::shared_ptr<HDF5File> hdf5_file;
|
|
const StartMessage start_message;
|
|
const std::string filename;
|
|
std::string tmp_filename;
|
|
bool overwrite = false;
|
|
bool calibration_group_created = false;
|
|
|
|
void LinkToData(const StartMessage &start, const EndMessage &end);
|
|
void LinkToData_VDS(const StartMessage &start, const EndMessage &end);
|
|
void LinkToData_ProcessingVDS(const StartMessage &start, const EndMessage &end);
|
|
void LinkToReflections_VDS(const StartMessage &start, const EndMessage &end);
|
|
|
|
// fill_value, when given, is what the dataset reads as where no source file is available - see
|
|
// the note in HDF5NXmx.cpp. Left unset for everything but the images.
|
|
std::unique_ptr<HDF5DataSet> VDS(const StartMessage &start,
|
|
const std::string& name,
|
|
const std::vector<hsize_t> &dim,
|
|
const HDF5DataType &data_type,
|
|
const std::optional<int64_t> &fill_value = {});
|
|
|
|
std::unique_ptr<HDF5DataSet> VDS(const StartMessage &start,
|
|
const std::string& name_src,
|
|
const std::string& name_dest,
|
|
const std::vector<hsize_t> &dim,
|
|
const HDF5DataType &data_type,
|
|
const std::optional<int64_t> &fill_value = {});
|
|
|
|
void Detector(const StartMessage &start);
|
|
void Detector(const StartMessage &start, const EndMessage &end);
|
|
void DetectorModule(const std::string &name, const std::vector<int32_t> &origin,
|
|
const std::vector<int32_t> &size, const std::vector<double> &fast_axis,
|
|
const std::vector<double> &slow_axis, const std::string &nx_axis, double pixel_size_mm);
|
|
|
|
void Facility(const StartMessage &start);
|
|
void Beam(const StartMessage &start);
|
|
void Metrology(const StartMessage &start, const EndMessage &end);
|
|
void Sample(const StartMessage &start, const EndMessage &end);
|
|
void Attenuator(const StartMessage &start);
|
|
|
|
void SaveCBORImage(const std::string& hdf5_path, const CompressedImage &image);
|
|
void AzimuthalIntegration(const StartMessage &start, const EndMessage &end);
|
|
void ADUHistogram(const EndMessage &end);
|
|
void EndResultVectors(const EndMessage &end);
|
|
void UserData(const StartMessage &start);
|
|
void MX(const StartMessage &start);
|
|
void ROI(const StartMessage &start);
|
|
void Fluorescence(const StartMessage &start);
|
|
|
|
public:
|
|
NXmx(const StartMessage& start);
|
|
~NXmx();
|
|
NXmx(const NXmx &other) = delete;
|
|
NXmx& operator=(const NXmx &other) = delete;
|
|
void Finalize(const EndMessage &end);
|
|
void WriteCalibration(const CompressedImage &image);
|
|
|
|
std::shared_ptr<HDF5File> GetFile();
|
|
};
|