Files
Jungfraujoch/common/Reflection.h
T
leonarski_fandClaude Opus 5 3d7891c347 rugnux: weight every CC1/2 by the information a reflection carries
The resolution cutoff, the shell table and the overall CC1/2 counted every
unique reflection equally. The merge itself is inverse-variance weighted, so
an observation from a frame the crystal barely diffracted on enters it at
1/G^2 of a good one - honestly, with its sigma - but a reflection measured
only on such frames is scaled-up noise that then counts as much as a
well-measured pair in every Pearson CC1/2 read off the merge. On a sweep
where half the frames are weak the curve collapses at every resolution: the
cut lands at 3.7 A on a P1 crystal whose good frames reach 1.5 A, and the
UNUSABLE verdict fires (CC1/2 0.40 beside I/sigma 10).

Each merged reflection now carries cc_weight: the precision its half-sets
would have had with every observation at the run's typical frame scale, over
the precision they have. G_ref = sum G^3 / sum G^2 over the usable
observations is the precision-weighted typical scale, which the dead frames
cannot drag down however many there are; the factor per observation is
max(1, (G_ref/G)^2), with G the frame's total scale (partial scale, flux and
the fulls' own G) taken before the correction surfaces and before collapsed
frames are dropped, so nothing intensity- or resolution-dependent enters it.
On a sweep without a weak stretch every weight is 1 and the CC1/2 is the
plain Pearson it was. The cutoff fit, the shell table and the overall CC1/2
(and so the UNUSABLE verdict and the report's shell checks) all read the same
weighted statistic.

The two extra per-group sums are accumulated on both merge paths, the host
loop and MergeAccumKernel, from one per-frame factor array; a host recompute
of the device sums agrees to 1e-15 relative on every merge after the
corrected corr is uploaded, and a host-merge run gives the same cut, space
group, CC1/2 and ISa on five sets.

Weighting by the half-set error variance alone (1/(v0+v1)) is not this: v
grows with the intensity, so it weights the weak end of the intensity
distribution and biases homogeneous data coarser.

Measured (written resolution, together with the weighted outlier median):
a P1 sweep with a long weak stretch 3.73 -> 1.40 A against a 1.63 A XDS
reference, UNUSABLE withdrawn (overall CC1/2 0.40 -> 0.97); a second 3.61 ->
3.00 A; one with most of the sweep out of beam 7.35 -> 5.20 A. Lysozyme,
thaumatin and two insulin sets unchanged to 0.01 A (one lysozyme sweep with a
weak wedge 1.13 -> 1.16 A, from the median), same space groups throughout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 18:45:03 +02:00

95 lines
4.8 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <cstdint>
#include <optional>
#include <cmath>
#include "SpotToSave.h"
struct Reflection {
int32_t h;
int32_t k;
int32_t l;
float image_number; // Can be in-between for 3D integration
float delta_phi_deg; // phi angle from XDS - difference from middle of current frame (NOT an absolute angle)
float predicted_x;
float predicted_y;
float observed_x;
float observed_y;
float d;
float I;
float bkg;
float var_bkg; // non-signal (background) part of sigma^2, carried to the merge
float sigma;
float dist_ewald;
// The reciprocal Lorentz factor (rotation only - a still's Lorentz factor is one) times the
// reciprocal polarization factor, and nothing else. This is what LP means everywhere in the
// field, and it is what the CBOR key "rlp" and the HDF5 dataset "lp" store the reciprocal of.
// What it is not is a scale: the fitted per-image scale and the partiality stay out of it and
// are divided in separately below. (Named after DIALS's prescaling_correction.)
float prescaling_corr;
// The sensor's angle-dependent efficiency, QE(0)/QE(alpha): always <= 1, and exactly 1 where the
// sensor is opaque or its material and thickness are unknown. It is carried BESIDE
// prescaling_corr rather than inside it, because LP and detector response are two different
// things and every file the field reads keeps them apart. It is one of the three factors whose
// product is the total deterministic correction, with prescaling_corr above and flight_corr
// below. Defaulted to 1 so a reflection read from a file written before this existed is a no-op
// rather than a zero.
float qe_corr = 1.0f;
// The medium in the sample-to-pixel flight path, exp(D/L * (1/cos(alpha) - 1)) with D the
// normal-incidence distance, L the medium's attenuation length and alpha the angle of incidence
// on the detector: always >= 1, because an oblique reflection crossed more of the medium than
// one arriving head-on. Exactly 1 under --flight-path vacuum. Carried beside the two above for the same
// reason they are carried apart - it is neither beam geometry nor detector response but the
// medium in between, and unlike either of them it is set by the flight distance. The total
// deterministic correction on a reflection is prescaling_corr * qe_corr * flight_corr, and every
// site that corrects an intensity multiplies all three. Defaulted to 1 so a reflection read
// from a file written before this existed is a no-op rather than a zero.
float flight_corr = 1.0f;
float partiality; // fraction of the reflection recorded in the sampled (rocking) slice
float zeta;
float image_scale_corr; // I_true = image_scale_corr * I; = prescaling_corr * qe_corr * flight_corr / (partiality * image_scale)
bool observed = false;
bool on_ice_ring = false; // sits on a hexagonal-ice powder ring: excluded from scaling, kept for merging
};
struct MergedReflection {
int32_t h = 0;
int32_t k = 0;
int32_t l = 0;
float I = NAN;
float sigma = NAN;
float I_half[2] = {NAN, NAN};
float sigma_half[2] = {NAN, NAN};
// Weight of this reflection in a CC1/2: the precision its half-sets would have had with every
// observation at the run's typical frame scale, over the precision they have. 1 on a sweep
// without a weak stretch; small for a reflection measured only where the crystal barely
// diffracted, whose scaled-up noise would otherwise count as much as a well-measured pair.
float cc_weight = 1.0f;
float d = 0.0;
// Any observation of this reflection sat on an ice ring. The intensity is still merged and
// written - the ring contaminates it, it does not make it absent - and this only marks it so
// a consumer that must not read the ring as crystal signal can leave it out.
bool on_ice_ring = false;
bool rfree_flag = false;
float F = NAN; // French-Wilson amplitude |F| (filled by ApplyFrenchWilson at end of merge)
float sigmaF = NAN; // its sigma
// Anomalous (Bijvoet) split of this reflection's own observations, kept even when the merge is
// Friedel-averaged (I above is the Friedel mean). Lets I(+)/I(-) be written and CCano reported by
// default without scaling anomalously; NaN when a hand was not measured or for centrics.
float I_plus = NAN;
float sigma_plus = NAN;
float I_minus = NAN;
float sigma_minus = NAN;
// French-Wilson amplitudes of the two hands (filled by ApplyFrenchWilson from I_plus/I_minus).
float F_plus = NAN;
float sigmaF_plus = NAN;
float F_minus = NAN;
float sigmaF_minus = NAN;
};