leonarski_fandClaude Opus 5
61a7c91b90
The per-image ice score was read off the PLAIN azimuthal profile. That profile is a per-ring mean, so a few strong Bragg reflections landing in a ring's q bin lift it exactly as ice would. Measured over 37 rotation crystals, that did not merely add noise - it INVERTED the metric: the two highest-scoring crystals had no ice at all (4.23 and 4.06), while a clean control read 1.57. A decoy null - the identical statistic evaluated at q positions where hexagonal ice cannot be - reaches 1.51 at its 99th percentile and 2.70 at its maximum, so that metric cannot support any absolute threshold whatsoever. The adaptive spot finder already computes the right input for its own threshold: a sigma-clipped per-resolution-ring background, in the same bins. A powder ring is azimuthally smooth and survives the clip; Bragg peaks do not. On the clipped profile the clean population tightens to 1.00-1.22 and the crystals with confirmed ice sit at 2.08-2.37, against a decoy null that never exceeds 1.29. That channel is blind to one thing: ice in large crystallites diffracts as DISCRETE spots and leaves the radial profile flat. So a second channel counts found spots on the rings against the same q width of ice-free flanks beside them. The two barely overlap - the smooth-ice crystals read 2.1-2.4 / ~1.0 and the textured ones ~1.1 / 3.8-17.6, while a clean crystal reads 1.04 on both. Both are then used as a GATE (--ice-min-score 1.5, --ice-min-spot-ratio 2.0, both calibrated on the battery, 0 disables): the eleven fixed hexagonal bands cover 16-26 % of the unique reflections at typical resolutions whether or not the crystal has ice, so flagging, the exclusion from the scale fit and the merge-time CC1/2 ring mask are now all skipped when neither channel sees any. The gate is applied in the full pipeline and in --scale, which reads the stored per-image values back out of the _process.h5. Also fixes the merge-time mask's control: the shoulder now excludes reflections that are themselves on an ice ring. The rings are not evenly spaced - 1.947/1.916/1.882 A sit 0.05-0.06 apart in q - so for those three the [w,3w) shoulder landed squarely on the neighbours and the test compared ice against ice. Measured, that is the only thing this changes: it removes firings on those three rings and leaves every other firing's CC pair identical to three decimals. And the online ice half-width, which was 0.02 in the API against 0.03 offline, so the same data got a narrower band online than the measured ~0.06 ring FWHM justifies. Battery (37 rotation crystals, against the previous behaviour): space groups 34/37 in both and NO crystal's space group changes; 6 crystals gain unique reflections, 1 loses. Best of them gains 7082 unique reflections with R_meas 16.0 -> 14.3, CC1/2 95.9 -> 97.3 and ISa 13.7 -> 19.0; another goes R_meas 54.9 -> 42.9, CC1/2 84.0 -> 90.4, ISa 3.9 -> 5.5; a third reaches CC1/2 99.4 from 95.7 at an unchanged reflection count. The one crystal that loses reflections improves on both R_meas and CC1/2. Not done here: the ScanResult/API/plot-type/frontend/viewer layers for the new spot_count_ice_control (they need the OpenAPI regeneration). Message, CBOR, HDF5 write/read and the receiver plots are. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
138 lines
4.7 KiB
C++
138 lines
4.7 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <map>
|
|
#include <shared_mutex>
|
|
#include <string>
|
|
|
|
#include "StatusVector.h"
|
|
#include "Histogram.h"
|
|
#include "ADUHistogram.h"
|
|
#include "AutoIncrVector.h"
|
|
#include "DiffractionExperiment.h"
|
|
#include "AzimuthalIntegrationProfile.h"
|
|
#include "JFJochMessages.h"
|
|
#include "Plot.h"
|
|
#include "ScanResult.h"
|
|
|
|
struct MeanProcessingTime {
|
|
float compression;
|
|
float preprocessing;
|
|
float azint;
|
|
float spot_finding;
|
|
float indexing;
|
|
float refinement;
|
|
float integration;
|
|
float bragg_prediction;
|
|
float processing;
|
|
float indexing_analysis;
|
|
float image_scale;
|
|
};
|
|
|
|
class JFJochReceiverPlots {
|
|
mutable std::mutex m; // protects xfel_pulse_id, xfel_event_code and az_int_profile
|
|
|
|
std::optional<GoniometerAxis> goniometer;
|
|
std::optional<GridScanSettings> grid_scan;
|
|
int64_t default_binning = 1;
|
|
std::unique_ptr<AzimuthalIntegrationProfile> az_int_profile;
|
|
AutoIncrVector<uint64_t> xfel_pulse_id;
|
|
AutoIncrVector<uint64_t> xfel_event_code;
|
|
|
|
StatusVector bkg_estimate;
|
|
StatusVector ice_ring_score;
|
|
StatusVector spot_count;
|
|
StatusVector spot_count_low_res;
|
|
StatusVector spot_count_indexed;
|
|
StatusVector spot_count_ice;
|
|
StatusVector spot_count_ice_control;
|
|
|
|
StatusVector indexing_solution;
|
|
StatusVector indexing_lattice_count;
|
|
StatusVector indexing_uc_a;
|
|
StatusVector indexing_uc_b;
|
|
StatusVector indexing_uc_c;
|
|
StatusVector indexing_uc_alpha;
|
|
StatusVector indexing_uc_beta;
|
|
StatusVector indexing_uc_gamma;
|
|
StatusVector error_pixels;
|
|
StatusVector saturated_pixels;
|
|
StatusVector strong_pixels;
|
|
StatusVector receiver_delay;
|
|
StatusVector receiver_buf_available;
|
|
StatusVector receiver_buf_in_sending;
|
|
StatusVector receiver_buf_in_preparation;
|
|
StatusVector image_collection_efficiency;
|
|
StatusVector packets_received;
|
|
StatusVector max_value;
|
|
StatusVector resolution_estimate;
|
|
StatusVector integrated_reflections;
|
|
StatusVector image_scale_factor;
|
|
StatusVector image_scale_cc;
|
|
StatusVector compression_ratio;
|
|
|
|
// StatusVector objects are fully thread-safe (protected by internal mutex)
|
|
// It is OK to have concurrent access to StatusVector
|
|
// roi_m lock is needed to make sure that std::map<std::string, ROIStatus> is not mutable within critical section
|
|
// so no new elements added outside of a unique lock, but it is OK to modify ROIStatus under shared lock
|
|
struct ROIStatus {
|
|
StatusVector sum;
|
|
StatusVector max_count;
|
|
StatusVector pixels;
|
|
StatusVector x;
|
|
StatusVector y;
|
|
StatusVector mean;
|
|
};
|
|
mutable std::shared_mutex roi_m;
|
|
std::map<std::string, ROIStatus> roi_status;
|
|
|
|
StatusVector profile_radius;
|
|
StatusVector mosaicity_deg;
|
|
StatusVector b_factor;
|
|
StatusVector pixel_sum;
|
|
|
|
StatusVector beam_center_x;
|
|
StatusVector beam_center_y;
|
|
|
|
StatusVector spot_finding_time;
|
|
StatusVector indexing_time;
|
|
StatusVector refinement_time;
|
|
StatusVector integration_time;
|
|
StatusVector bragg_prediction_time;
|
|
StatusVector total_processing_time;
|
|
StatusVector preprocessing_time;
|
|
StatusVector compression_time;
|
|
StatusVector azint_time;
|
|
StatusVector indexing_analysis_time;
|
|
StatusVector image_scale_time;
|
|
|
|
MultiLinePlot GetROIPlot(PlotType type, int64_t nbins, float start, float incr,
|
|
const std::optional<float> &fill_value) const;
|
|
public:
|
|
void Setup(const DiffractionExperiment& experiment, const AzimuthalIntegrationMapping& mapping);
|
|
|
|
void Add(const DataMessage& msg, const AzimuthalIntegrationProfile &profile);
|
|
void AddEmptyImage(const DataMessage& msg);
|
|
MultiLinePlot GetPlots(const PlotRequest& request);
|
|
|
|
void GetXFELPulseID(std::vector<uint64_t>& v) const;
|
|
void GetXFELEventCode(std::vector<uint64_t>& v) const;
|
|
|
|
std::optional<float> GetIndexingRate() const;
|
|
std::optional<float> GetBkgEstimate() const;
|
|
std::optional<float> GetIceRingScore() const;
|
|
// Pooled over the run: spots on the hexagonal rings over the same q width of ice-free control
|
|
// flanks. 1 = spots spread evenly, > 1 = they pile up on the rings (textured ice).
|
|
[[nodiscard]] std::optional<float> GetIceRingSpotRatio() const;
|
|
std::vector<float> GetIceRingScoreArray() const;
|
|
|
|
std::vector<float> GetAzIntProfile() const;
|
|
MultiLinePlot GetAzIntProfilePlot(bool force_1d = false, PlotAzintUnit azint_unit = PlotAzintUnit::Q_recipA) const;
|
|
|
|
MeanProcessingTime GetMeanProcessingTime() const;
|
|
|
|
void GetPlotRaw(std::vector<float> &v, PlotType type, const std::string &roi);
|
|
};
|