leonarski_fandClaude Opus 5.5
87f0bdc870
FitErrorModelGPU puts the samples in the host's rank order with four stable radix passes, so the equal-count bins hold the same samples; computes every normalised deviation with the host's rounding, so every bin median is the host's to the bit; and hands the per-bin terms to the host's own update step, now ErrorModelUpdate (a pure extraction from FitErrorModel). The one thing that differs is the order each bin's two sums are added in: a fixed tree on the device, the order nth_element happened to leave the bin in on the host. a and b^2 can therefore differ from the host fit in their last bits - measured on two synthetic pools: relative difference 2e-16 and 3e-13 in a, 5e-16 in b^2. The device fit is deterministic (same pool, same bits). p.mtz md5 nevertheless unchanged on myob, cytc, 8a1a and 8qaw (GPU build): the difference does not survive the float rounding of the merged sigmas there. A knife-edge decision elsewhere could still see it, which is why this is kept as its own commit. The host fit took 0.4-0.6 s per fit on the large merges, about half of it the initial equal-count split, with only 16-way parallelism in the iterations. Device peak about 72 bytes per sample (the rank sort; each stage's buffers are freed when it is done); the free memory is checked first and a device without it stops with a message naming the CPU build. Kept last on the branch so that it can be dropped on its own. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
62 lines
3.4 KiB
C++
62 lines
3.4 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <cstddef>
|
|
#include <vector>
|
|
|
|
// The error model of the rotation merge: var = a * s2 + b^2 * <I>^2, fitted so that the scatter of
|
|
// symmetry equivalents matches it - in each bin of counting I/sigma, the median of dev2 / var is the
|
|
// median of a chi^2 with one degree of freedom.
|
|
//
|
|
// One sample per observation: s2 is its counting variance as the merge rebuilds it at the reflection's
|
|
// mean intensity, I2 that mean squared, dev2 its leverage-corrected squared deviation from the mean.
|
|
//
|
|
// Two choices, each against a measured failure of the fit this replaces, which took the medians of s2,
|
|
// I2 and dev2 separately over bins of I2:
|
|
// * Bins are equal counts in COUNTING I/sigma (I2 / s2), not in I2. b is identified by how far the
|
|
// bins reach into the regime where b*<I> dominates the counting term, and that is what this ranks
|
|
// on; ranking on I2 mixes weak high-resolution reflections into the strong bins.
|
|
// * The median is taken of the NORMALISED deviation, each sample divided by the variance the previous
|
|
// iteration gave it. A median of dev2 over observations whose variances differ is not 0.455 times
|
|
// their mean variance, and the ratio of three medians read a too low and b too high: on most of 28
|
|
// sets the core of the normalised deviations then scattered at up to 1.8 times its stated variance
|
|
// in the weak and middle bins and at 0.1-0.7 times it in the strongest.
|
|
// A median, not a mean: on heavy-tailed data (split and powder-contaminated crystals) a mean calibrates
|
|
// the sigmas on the tails, the merge's outlier test widens with them, and CC1/2 fell 0.80 -> 0.71 on one
|
|
// such set where this fit raises it to 0.83.
|
|
struct ErrorModelSample {
|
|
double s2, I2, dev2;
|
|
float d;
|
|
};
|
|
|
|
struct ErrorModelFit {
|
|
bool active = false; // the pool was large enough to fit
|
|
double a = 1.0, b2 = 0.0;
|
|
bool b_measured = true; // false: no bin reached where b could be seen; b is held at 0
|
|
bool b_resolved = true; // b^2 stood clear of two of its standard errors
|
|
};
|
|
|
|
// Scratch: the partitioned copy of the pool (reused between calls to save the allocation), with the rank
|
|
// key and the current normalised deviation.
|
|
struct ErrorModelBinned {
|
|
double snr2, s2, I2, dev2, z2;
|
|
};
|
|
|
|
// The fit's equal-count bins of counting I/sigma; the strongest bin's median (I/sigma)^2 must exceed
|
|
// ERROR_MODEL_B_LEVER_MIN for b to be fitted (see FitErrorModel); and the median of a chi^2 with one
|
|
// degree of freedom, which the median normalised deviation of a bin is compared with.
|
|
constexpr int ERROR_MODEL_BINS = 16;
|
|
constexpr double ERROR_MODEL_B_LEVER_MIN = 4.0;
|
|
constexpr double ERROR_MODEL_CHI2_1_MEDIAN = 0.454936;
|
|
|
|
ErrorModelFit FitErrorModel(const std::vector<ErrorModelSample> &pool, std::vector<ErrorModelBinned> &scratch,
|
|
size_t nthreads);
|
|
|
|
// One update of the fit from its per-bin terms - bs2 and bI2 the sums of s2 / var and I2 / var under
|
|
// the current (a, b^2), bd2 the bin's median normalised deviation expressed as the sum it stands for:
|
|
// the new (a, b^2) into a, b2 and fit. False when the fit stops, settled or no longer determined.
|
|
bool ErrorModelUpdate(const std::vector<double> &bs2, const std::vector<double> &bI2, const std::vector<double> &bd2,
|
|
double &a, double &b2, ErrorModelFit &fit);
|