Files
Jungfraujoch/reader/JFJochReaderDataset.h
T
leonarski_fandClaude Opus 5 beae384bb7
Build Packages / Create release (push) Successful in 32s
Build Packages / build:viewer-tgz:cpu (push) Successful in 7m40s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 6m54s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 7m7s
Build Packages / build:viewer-tgz:cuda (push) Successful in 8m29s
Build Packages / build:windows:nocuda (push) Successful in 16m47s
Build Packages / build:windows:cuda (push) Successful in 19m18s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 21m1s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m53s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m3s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m48s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m34s
Build Packages / build:rugnux:windows (push) Successful in 10m19s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 15m38s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 16m35s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / Build documentation (push) Successful in 1m19s
Build Packages / build:rpm (rocky8) (push) Successful in 15m59s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 15m34s
Build Packages / build:rpm (rocky9) (push) Successful in 16m26s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m48s
Build Packages / Unit tests (push) Successful in 1h34m17s
scale/merge: what each stretch of a sweep costs the merged data, measured
The sweep-quality diagnostic says a range is weak; it cannot say whether that
weakness costs anything. A 60 degree stretch of poor diffraction and a radiation
damage tail look alike in scale and correlation, and on one dataset they measure
-0.0758 +- 0.0005 and -0.0019: one is 150 sigma of real harm, the other is
nothing once the corrections have done their work.

So measure it, per batch of the sweep, as the change in CC1/2 when that batch is
left out - Assmann, Brehm & Diederichs (2016) Acta Cryst. D72, 1071-1081. In the
sigma-tau form, so no half-set is drawn and nothing depends on a seed; against the
observed scatter of each reflection's own observations rather than the error
model's sigmas, which would invert the sign and make a bad batch read as added
precision; and on the 10 degree batch the damage monitor already uses, because a
single frame is far below what the statistic can resolve - the precision goes as
one over the root of the reflection count, and 0.1 degrees of rotation does not
have the reflections.

Report only. Nothing is dropped, no observation is weighted differently, and every
existing key of every dataset tested is unchanged to the digit. The disposition
column says merged for all of them; acting on it needs a frame attribution this
code does not yet have, since a full is currently labelled with its peak frame
rather than with the frames its partials came from.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011GxZqDiFP3KqriBhNdcR56
2026-09-13 16:10:44 +02:00

114 lines
4.9 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <string>
#include <array>
#include <optional>
#include <vector>
#include <map>
#include <memory>
#include "../common/DiffractionGeometry.h"
#include "../common/DiffractionExperiment.h"
#include "../common/PixelMask.h"
#include "../common/AzimuthalIntegrationMapping.h"
struct JFJochReaderDataset {
std::string arm_date;
DiffractionExperiment experiment;
// Shared, not copied, across dataset snapshots: the mask is constant for a run, so a per-frame
// dataset refresh / mutable copy must not clone the full-detector (~tens of MB) mask. Held as
// shared_ptr<const>; the rare edit (user mask) builds a fresh mask (copy-on-write). Never null.
std::shared_ptr<const PixelMask> pixel_mask = std::make_shared<const PixelMask>();
std::optional<int64_t> error_value;
// The master file's detect_ice_rings key, kept as written rather than collapsed to a bool, so a
// consumer can tell "the file asked for this" from "the file said nothing". Offline processing
// defaults ice handling by geometry when the file is silent (see rugnux_cli).
std::optional<bool> file_detect_ice_rings;
std::string jfjoch_release;
// Change of basis (3x3 integers, row major) from the setting the per-image reflections and
// lattices in this file were written in to the setting of experiment's unit cell and space group:
// hkl_cell = M . hkl_written. The per-image data are written as each image is processed, but the
// space group - and the conventional setting that comes with it - is only settled after the merge,
// so the two are not always the same setting. Empty means they are. Read straight from
// /entry/MX/reindexMatrix; the reader applies it, so everything this class hands out is already in
// the cell's setting.
std::optional<std::array<int32_t, 9>> reindex_matrix;
std::vector<float> az_int_bin_to_q;
std::vector<float> az_int_bin_to_phi;
size_t azimuthal_bins = 0;
size_t q_bins = 0;
std::vector<float> spot_count;
std::vector<float> spot_count_indexed;
std::vector<float> spot_count_low_res;
std::vector<float> spot_count_ice_rings;
std::vector<float> spot_count_ice_control;
std::vector<float> indexing_result;
std::vector<float> indexing_lattice_count;
std::vector<float> bkg_estimate;
// NaN entries are frames with no value (CANNOT SAY), not zeros.
std::vector<float> spindle_blind_fraction;
std::vector<float> ice_ring_score;
std::vector<float> resolution_estimate;
std::vector<float> efficiency;
std::vector<float> profile_radius;
std::vector<float> mosaicity_deg;
std::vector<float> b_factor;
std::vector<float> integrated_reflections;
std::vector<float> image_scale_factor;
std::vector<float> image_scale_cc;
std::vector<int64_t> max_value;
// Per-image sweep-quality code from /entry/MX/sweepQuality: 0 = the image is in no flagged range,
// otherwise a 1-based index into sweep_quality_reasons (the vocabulary stored beside it, so the
// codes read back without this source). Both empty when the file carries no sweep quality, which
// means the run never looked - not that every image was clean.
std::vector<uint8_t> sweep_quality;
std::vector<std::string> sweep_quality_reasons;
// Per-image disposition from /entry/MX/frameDisposition: a 0-based index into
// frame_disposition_codes (merged, downgraded, rejected). What was DONE with the image, where
// sweep_quality above says what was SEEN.
std::vector<uint8_t> frame_disposition;
std::vector<std::string> frame_disposition_codes;
// Maps this dataset's image index -> the original/collected image number it came from.
// Empty means identity (image i == original image i). Lets a dataset be a subset (or strided
// selection) of the truly collected images: reprocessing snapshots over a sub-range, and (in
// future) a main dataset that was filtered on-the-fly during collection.
std::vector<int> source_image_number;
std::vector<std::string> roi;
std::vector<std::vector<int64_t>> roi_sum;
std::vector<std::vector<int64_t>> roi_sum_sq;
std::vector<std::vector<int64_t>> roi_max;
std::vector<std::vector<int64_t>> roi_npixel;
std::vector<std::vector<float>> roi_x;
std::vector<std::vector<float>> roi_y;
// ROI definitions stored in the master file. The logical definitions populate
// experiment.ROI() (they re-derive with the current geometry); roi_map is the
// per-pixel bitmask as written (constant footprint), with roi_bit_index mapping
// each ROI name to its bit.
std::vector<uint16_t> roi_map;
std::map<std::string, uint16_t> roi_bit_index;
std::vector<std::string> calibration_data;
JFJochReaderDataset() = default;
JFJochReaderDataset(const JFJochReaderDataset &other) = default;
};