leonarski_fandClaude Opus 5
f0cdb027e1
Three defaults, each settled by measurement rather than by argument. The arbiter throughout is structure-referenced - anomalous peak height where a crystal can carry it, and otherwise the agreement of the ice bands with a fixed external model against resolution-matched DECOY bands carrying no ice. The band-versus-decoy contrast is used because R-free here tracks completeness, and every one of these switches moves completeness. The damage is real and it localizes: over the rotation battery the ice bands' excess amplitude reaches +9.6% on a smooth-ice crystal and +35% on the worst, while a clean control sits at +0.6% (z +0.45). On the worst crystal, nine of the ten largest excess peaks in a q scan land on hexagonal ring positions. Turning ice handling off leaves the contrast unchanged and forcing it on a clean crystal does not create one, so it is the ice and not the machinery. MERGE-TIME RING MASK -> OFF. It deletes reflections, which no other program does by default - AIMLESS, DIALS, xia2, XDS and CrystFEL all keep ice-band reflections in the merge and exclude them only from the model fit; autoPROC is the sole exception. On the one battery crystal where the mask fires and an anomalous arbiter can score it, dropping the band moved the mean peak height at the known sites by -0.001 +- 0.018 sigma, 2% of the site height, while removing 1149 unique reflections whose mean I/sigma was 3.62 against the dataset's own 3.05 - better than average data - and costing 17 completeness points in that shell. It fires on 5 of 37 crystals, changes no space group, and those 5 disagree in sign: it clearly helps the two most heavily iced, is a wash on two and costs a third. So it stays as a switch, worth setting by hand on a badly iced crystal where it shows in the high shell, but it is not a default. RADIAL BACKGROUND -> AUTO, gated per image. The correction models the background as a function of radius alone, and that is exactly when it works. On a crystal with pure smooth powder ice it removes 43% of the bands' excess amplitude, with the improvement 7x larger inside the bands than outside; on a crystal whose ice is discrete crystallite spots - no smooth ring to model - the excess amplitude GREW by half; on clean data it is inert to four decimals. The two ice channels already separate those morphologies, so --background-radial takes on|off|auto and auto applies it to an image when that image's peak-excluded score reaches --ice-min-score. Auto never engages without such a score, because the plain profile carries the Bragg peaks and cannot support an absolute threshold. Per image rather than per run, and that was tested rather than assumed: the gate fires on 100% and 94% of frames on the two crystals that want it, and on 1.5% of frames - 32 blocks, 23 of them single frames - on the textured-ice crystal. A seam statistic against off + f*(on - off) is null on both mixed runs, every merge statistic is bracketed by the pure arms, and the textured crystal's auto arm lands on `off` rather than on `on`'s harm. A run-level gate would need the score before the pass that integrates, i.e. rotation-only plumbing, and buys nothing measurable. The kernel was already built unconditionally, so flipping the flag per image is free - except on the GPU, where the launches were gated on a construction-time n_rad. That is why the buffers are now allocated whenever the correction could run, and Run() decides per image. DETECTION -> the geometry's default when the file is silent: on for rotation, off for stills, with the command line and then the file taking precedence. A rotation sweep sits on the same rings for the whole run, so ice there is a coherent systematic and the presence gate keeps it inert on a clean crystal; a serial stills run has too few spots per image to spend any on flagging. The master file's key is kept as written rather than collapsed to a bool, so "the file said nothing" is distinguishable from "the file said no" - it used to fall silently to off, taking the exclusion from the scale fit with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
89 lines
3.3 KiB
C++
89 lines
3.3 KiB
C++
// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <string>
|
|
#include <optional>
|
|
#include <vector>
|
|
#include <map>
|
|
#include <memory>
|
|
|
|
#include "../common/DiffractionGeometry.h"
|
|
#include "../common/DiffractionExperiment.h"
|
|
#include "../common/PixelMask.h"
|
|
#include "../common/AzimuthalIntegrationMapping.h"
|
|
|
|
struct JFJochReaderDataset {
|
|
std::string arm_date;
|
|
|
|
DiffractionExperiment experiment;
|
|
// Shared, not copied, across dataset snapshots: the mask is constant for a run, so a per-frame
|
|
// dataset refresh / mutable copy must not clone the full-detector (~tens of MB) mask. Held as
|
|
// shared_ptr<const>; the rare edit (user mask) builds a fresh mask (copy-on-write). Never null.
|
|
std::shared_ptr<const PixelMask> pixel_mask = std::make_shared<const PixelMask>();
|
|
|
|
std::optional<int64_t> error_value;
|
|
|
|
// The master file's detect_ice_rings key, kept as written rather than collapsed to a bool, so a
|
|
// consumer can tell "the file asked for this" from "the file said nothing". Offline processing
|
|
// defaults ice handling by geometry when the file is silent (see rugnux_cli).
|
|
std::optional<bool> file_detect_ice_rings;
|
|
|
|
std::string jfjoch_release;
|
|
|
|
std::vector<float> az_int_bin_to_q;
|
|
std::vector<float> az_int_bin_to_phi;
|
|
size_t azimuthal_bins = 0;
|
|
size_t q_bins = 0;
|
|
|
|
std::vector<float> spot_count;
|
|
std::vector<float> spot_count_indexed;
|
|
std::vector<float> spot_count_low_res;
|
|
std::vector<float> spot_count_ice_rings;
|
|
std::vector<float> spot_count_ice_control;
|
|
|
|
std::vector<float> indexing_result;
|
|
std::vector<float> indexing_lattice_count;
|
|
std::vector<float> bkg_estimate;
|
|
std::vector<float> ice_ring_score;
|
|
std::vector<float> resolution_estimate;
|
|
std::vector<float> efficiency;
|
|
std::vector<float> profile_radius;
|
|
std::vector<float> mosaicity_deg;
|
|
std::vector<float> b_factor;
|
|
std::vector<float> integrated_reflections;
|
|
std::vector<float> image_scale_factor;
|
|
std::vector<float> image_scale_cc;
|
|
std::vector<int64_t> max_value;
|
|
|
|
// Maps this dataset's image index -> the original/collected image number it came from.
|
|
// Empty means identity (image i == original image i). Lets a dataset be a subset (or strided
|
|
// selection) of the truly collected images: reprocessing snapshots over a sub-range, and (in
|
|
// future) a main dataset that was filtered on-the-fly during collection.
|
|
std::vector<int> source_image_number;
|
|
|
|
std::vector<std::string> roi;
|
|
std::vector<std::vector<int64_t>> roi_sum;
|
|
std::vector<std::vector<int64_t>> roi_sum_sq;
|
|
std::vector<std::vector<int64_t>> roi_max;
|
|
std::vector<std::vector<int64_t>> roi_npixel;
|
|
std::vector<std::vector<float>> roi_x;
|
|
std::vector<std::vector<float>> roi_y;
|
|
|
|
// ROI definitions stored in the master file. The logical definitions populate
|
|
// experiment.ROI() (they re-derive with the current geometry); roi_map is the
|
|
// per-pixel bitmask as written (constant footprint), with roi_bit_index mapping
|
|
// each ROI name to its bit.
|
|
std::vector<uint16_t> roi_map;
|
|
std::map<std::string, uint16_t> roi_bit_index;
|
|
|
|
std::vector<std::string> calibration_data;
|
|
|
|
JFJochReaderDataset() = default;
|
|
JFJochReaderDataset(const JFJochReaderDataset &other) = default;
|
|
};
|
|
|
|
|
|
|