Files
Jungfraujoch/reader/JFJochReaderDataset.h
leonarski_fandClaude Opus 5.5 637d2c09cc pixel_sum on the CPU and GPU path, under the device mask, and read back from HDF5
pixel_sum existed only on the FPGA path, so a DECTRIS master file carried an all-zero
/entry/image/pixel_sum. The CPU and GPU image preprocessors now compute it in the pass that takes the
minimum and maximum, as a signed 64-bit sum, and MXAnalysisWithoutFPGA reports it: Lite, live stills,
live rotation and rugnux. On the simulated card it equals the card's own sum, for 16-bit and for
summed 32-bit frames.

The sum excludes error and saturated pixels and the device mask (detector and calibration). The hot
pixels and the beam-stop shadow a run finds on its own frames do not change it, though they stay masked
in the image and in the other statistics, so the number does not depend on what a run decided. Those
two have bits of their own (PixelMask::RunMaskBits); the byte-per-pixel mask keeps the difference, 1 for
the device mask and 2 for the run's alone, and the CPU preprocessor gets a packed mask of the second.
Live stills reports this sum of the assembled image in place of the card's, and with the run's mask out
of it the two agree on the test frames.

Frames a live rotation run does not reach stay 0 in the master file, as before, and have no dataset in
the data files. The reader now reads pixel_sum into the dataset and the per-image message; a file
without it opens as before.

(ported from rc184-2 15af14357)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQsU5a8AJUdJPgWevAK
2026-11-08 20:08:56 +01:00

126 lines
5.7 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <string>
#include <array>
#include <optional>
#include <vector>
#include <map>
#include <memory>
#include "../common/DiffractionGeometry.h"
#include "../common/DiffractionExperiment.h"
#include "../common/StillsRunWidth.h"
#include "../common/PixelMask.h"
#include "../common/AzimuthalIntegrationMapping.h"
struct JFJochReaderDataset {
std::string arm_date;
DiffractionExperiment experiment;
// Shared, not copied, across dataset snapshots: the mask is constant for a run, so a per-frame
// dataset refresh / mutable copy must not clone the full-detector (~tens of MB) mask. Held as
// shared_ptr<const>; the rare edit (user mask) builds a fresh mask (copy-on-write). Never null.
std::shared_ptr<const PixelMask> pixel_mask = std::make_shared<const PixelMask>();
std::optional<int64_t> error_value;
// The master file's detect_ice_rings key, kept as written rather than collapsed to a bool, so a
// consumer can tell "the file asked for this" from "the file said nothing". Offline processing
// defaults ice handling by geometry when the file is silent (see rugnux_cli).
std::optional<bool> file_detect_ice_rings;
std::string jfjoch_release;
// Change of basis (3x3 integers, row major) from the setting the per-image reflections and
// lattices in this file were written in to the setting of experiment's unit cell and space group:
// hkl_cell = M . hkl_written. The per-image data are written as each image is processed, but the
// space group - and the conventional setting that comes with it - is only settled after the merge,
// so the two are not always the same setting. Empty means they are. Read straight from
// /entry/MX/reindexMatrix; the reader applies it, so everything this class hands out is already in
// the cell's setting.
std::optional<std::array<int32_t, 9>> reindex_matrix;
// The e.s.d.s of the experiment's unit cell, a b c alpha beta gamma (/entry/sample/unit_cell_esd); empty
// where the file has none.
std::optional<UnitCell> unit_cell_esd;
// Stills: the rocking curve the run's frames were predicted through (/entry/MX/stillsRockingWidth*).
std::optional<StillsRunWidth> stills_run_width;
// Stills: the cell the run was given, whose population of frames it merged (/entry/MX/stillsGivenUnitCell).
std::optional<UnitCell> stills_given_cell;
std::vector<float> az_int_bin_to_q;
std::vector<float> az_int_bin_to_phi;
size_t azimuthal_bins = 0;
size_t q_bins = 0;
std::vector<float> spot_count;
std::vector<float> spot_count_indexed;
std::vector<float> spot_count_low_res;
std::vector<float> spot_count_ice_rings;
std::vector<float> spot_count_ice_control;
std::vector<float> indexing_result;
std::vector<float> indexing_lattice_count;
std::vector<float> bkg_estimate;
// The XFEL pulse ID and event code of each image (/entry/xfel), where the file records them.
std::vector<uint64_t> xfel_pulse_id;
std::vector<uint64_t> xfel_event_code;
// NaN entries are frames with no value (CANNOT SAY), not zeros.
std::vector<float> spindle_blind_fraction;
std::vector<float> ice_ring_score;
std::vector<float> resolution_estimate;
std::vector<float> efficiency;
std::vector<float> profile_radius;
std::vector<float> mosaicity_deg;
std::vector<float> b_factor;
std::vector<float> integrated_reflections;
std::vector<float> image_scale_factor;
std::vector<float> image_scale_cc;
std::vector<int64_t> max_value;
std::vector<int64_t> pixel_sum;
// Per-image sweep-quality code from /entry/MX/sweepQuality: 0 = the image is in no flagged range,
// otherwise a 1-based index into sweep_quality_reasons (the vocabulary stored beside it, so the
// codes read back without this source). Both empty when the file carries no sweep quality, which
// means the run never looked - not that every image was clean.
std::vector<uint8_t> sweep_quality;
std::vector<std::string> sweep_quality_reasons;
// Per-image disposition from /entry/MX/frameDisposition: a 0-based index into
// frame_disposition_codes (merged, downgraded, rejected). What was DONE with the image, where
// sweep_quality above says what was SEEN.
std::vector<uint8_t> frame_disposition;
std::vector<std::string> frame_disposition_codes;
// Maps this dataset's image index -> the original/collected image number it came from.
// Empty means identity (image i == original image i). Lets a dataset be a subset (or strided
// selection) of the truly collected images: reprocessing snapshots over a sub-range, and (in
// future) a main dataset that was filtered on-the-fly during collection.
std::vector<int> source_image_number;
std::vector<std::string> roi;
std::vector<std::vector<int64_t>> roi_sum;
std::vector<std::vector<int64_t>> roi_sum_sq;
std::vector<std::vector<int64_t>> roi_max;
std::vector<std::vector<int64_t>> roi_npixel;
std::vector<std::vector<float>> roi_x;
std::vector<std::vector<float>> roi_y;
// ROI definitions stored in the master file. The logical definitions populate
// experiment.ROI() (they re-derive with the current geometry); roi_map is the
// per-pixel bitmask as written (constant footprint), with roi_bit_index mapping
// each ROI name to its bit.
std::vector<uint16_t> roi_map;
std::map<std::string, uint16_t> roi_bit_index;
std::vector<std::string> calibration_data;
JFJochReaderDataset() = default;
JFJochReaderDataset(const JFJochReaderDataset &other) = default;
};