leonarski_fandClaude Opus 5
0a342fcb13
The resolution cutoff, the shell table and the overall CC1/2 counted every unique reflection equally. The merge itself is inverse-variance weighted, so an observation from a frame the crystal barely diffracted on enters it at 1/G^2 of a good one - honestly, with its sigma - but a reflection measured only on such frames is scaled-up noise that then counts as much as a well-measured pair in every Pearson CC1/2 read off the merge. On a sweep where half the frames are weak the curve collapses at every resolution: the cut lands at 3.7 A on a P1 crystal whose good frames reach 1.5 A, and the UNUSABLE verdict fires (CC1/2 0.40 beside I/sigma 10). Each merged reflection now carries cc_weight: the precision its half-sets would have had with every observation at the run's typical frame scale, over the precision they have. G_ref = sum G^3 / sum G^2 over the usable observations is the precision-weighted typical scale, which the dead frames cannot drag down however many there are; the factor per observation is max(1, (G_ref/G)^2), with G the frame's total scale (partial scale, flux and the fulls' own G) taken before the correction surfaces and before collapsed frames are dropped, so nothing intensity- or resolution-dependent enters it. On a sweep without a weak stretch every weight is 1 and the CC1/2 is the plain Pearson it was. The cutoff fit, the shell table and the overall CC1/2 (and so the UNUSABLE verdict and the report's shell checks) all read the same weighted statistic. The two extra per-group sums are accumulated on both merge paths, the host loop and MergeAccumKernel, from one per-frame factor array; a host recompute of the device sums agrees to 1e-15 relative on every merge after the corrected corr is uploaded, and a host-merge run gives the same cut, space group, CC1/2 and ISa on five sets. Weighting by the half-set error variance alone (1/(v0+v1)) is not this: v grows with the intensity, so it weights the weak end of the intensity distribution and biases homogeneous data coarser. Measured (written resolution, together with the weighted outlier median): a P1 sweep with a long weak stretch 3.73 -> 1.40 A against a 1.63 A XDS reference, UNUSABLE withdrawn (overall CC1/2 0.40 -> 0.97); a second 3.61 -> 3.00 A; one with most of the sweep out of beam 7.35 -> 5.20 A. Lysozyme, thaumatin and two insulin sets unchanged to 0.01 A (one lysozyme sweep with a weak wedge 1.13 -> 1.16 A, from the median), same space groups throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
63 lines
4.2 KiB
C++
63 lines
4.2 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <optional>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "../../common/Logger.h"
|
|
#include "../../common/Reflection.h" // MergedReflection
|
|
#include "../../common/ScalingSettings.h" // ResolutionCutoffMethod
|
|
|
|
// Automatic high-resolution cutoff from the CC1/2 fall-off of the merged half-sets (DIALS-style).
|
|
// The merge itself, the error model and the per-image _process.h5 are left untouched - only the
|
|
// written reflections and the reported shell table should be trimmed to the returned d_cut.
|
|
//
|
|
// Method (see docs/rugnux_resolution_cutoff_design.md): bin CC1/2 against s = 1/d^2 in fine bins,
|
|
// fit a logistic CC1/2(s) = 1/(1+exp(k*(s-s0))) to the contiguous-from-low-res fall-off, take the s
|
|
// where the fit crosses cc_target, then extend by one mean (10-shell) shell width in s ("one shell
|
|
// too far", generous). Where the fitted crossing lands past the bins the fit was made over - a
|
|
// fall-off region too ragged for a logistic to follow, so the crossing is an extrapolation rather
|
|
// than something the bins show - the crossing is read off the bins themselves instead, and
|
|
// everything after it is unchanged. d_cut is nullopt when the fit is degenerate (too few bins, flat)
|
|
// or CC1/2 never falls below cc_target inside the measured range - the caller then keeps the full
|
|
// range. cc_target in (0,1); merged must carry finite I_half[0]/I_half[1] to contribute.
|
|
struct ResolutionCutoffResult {
|
|
std::optional<double> d_cut; // high-resolution limit (A); nullopt => keep the full range
|
|
// Where CC1/2 crosses cc_target, BEFORE the deliberate one-shell extension - i.e. the
|
|
// resolution the data are judged to reach, as opposed to the (coarser in s, finer in d) limit the
|
|
// reflections are actually written to. This is the number to quote. Set whenever the fit produced
|
|
// a crossing inside the measured range, even when no cut was applied.
|
|
std::optional<double> d_fit;
|
|
std::string note; // human-readable description of the decision (for logging)
|
|
};
|
|
|
|
// The per-frame factor of the CC1/2 weight (MergedReflection::cc_weight): max(1, (G_ref/G)^2), the
|
|
// variance an observation from a frame at scale G carries over the same observation at the typical
|
|
// scale G_ref = sum G^3 / sum G^2 over the observations (n_obs[f] of them on frame f). Precision-
|
|
// weighted, so frames the crystal barely diffracted on cannot drag the typical scale down however many
|
|
// of them there are. A frame without a finite positive scale counts as G = 1.
|
|
std::vector<double> CCHalfFrameFactors(const std::vector<double> &frame_scale,
|
|
const std::vector<int64_t> &n_obs);
|
|
|
|
ResolutionCutoffResult ComputeCCHalfLogisticCutoff(const std::vector<MergedReflection> &merged,
|
|
double cc_target, Logger &logger);
|
|
|
|
// Resolve the effective high-resolution limit and trim `merged` to it, in one place shared by the
|
|
// stills merge, the rotation merge and the offline --scale path. A manual limit (manual_limit) wins;
|
|
// otherwise, unless this is a P1 space-group search merge (for_search), the CCHalfLogistic method
|
|
// takes the auto CC1/2 cutoff. Reflections beyond the resolved limit are erased from `merged`.
|
|
// Returns the applied limit (nullopt => the full range was kept). The decision is logged as before.
|
|
// fit_limit_out, when given, receives ResolutionCutoffResult::d_fit - the CC1/2 crossing without the
|
|
// one-shell extension, for reporting. Left untouched when no automatic fit ran (a manual limit, a
|
|
// search merge, or the method turned off), so a caller can tell "not fitted" from "fitted".
|
|
std::optional<double> ApplyResolutionCutoff(std::vector<MergedReflection> &merged,
|
|
std::optional<double> manual_limit,
|
|
ResolutionCutoffMethod method,
|
|
double cc_target,
|
|
bool for_search,
|
|
Logger &logger,
|
|
std::optional<double> *fit_limit_out = nullptr);
|