Jungfraujoch/image_analysis/scale_merge/ErrorModel.h

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only

#pragma once

#include <cstddef>
#include <vector>

// The error model of the rotation merge: var = a * s2 + b^2 * <I>^2, fitted so that the scatter of
// symmetry equivalents matches it - in each bin of counting I/sigma, the median of dev2 / var is the
// median of a chi^2 with one degree of freedom.
//
// One sample per observation: s2 is its counting variance as the merge rebuilds it at the reflection's
// mean intensity, I2 that mean squared, dev2 its leverage-corrected squared deviation from the mean.
//
// Two choices, each against a measured failure of the fit this replaces, which took the medians of s2,
// I2 and dev2 separately over bins of I2:
//  * Bins are equal counts in COUNTING I/sigma (I2 / s2), not in I2. b is identified by how far the
//    bins reach into the regime where b*<I> dominates the counting term, and that is what this ranks
//    on; ranking on I2 mixes weak high-resolution reflections into the strong bins.
//  * The median is taken of the NORMALISED deviation, each sample divided by the variance the previous
//    iteration gave it. A median of dev2 over observations whose variances differ is not 0.455 times
//    their mean variance, and the ratio of three medians read a too low and b too high: on most of 28
//    sets the core of the normalised deviations then scattered at up to 1.8 times its stated variance
//    in the weak and middle bins and at 0.1-0.7 times it in the strongest.
// A median, not a mean: on heavy-tailed data (split and powder-contaminated crystals) a mean calibrates
// the sigmas on the tails, the merge's outlier test widens with them, and CC1/2 fell 0.80 -> 0.71 on one
// such set where this fit raises it to 0.83.
struct ErrorModelSample {
    double s2, I2, dev2;
    float d;
};

struct ErrorModelFit {
    bool active = false;       // the pool was large enough to fit
    double a = 1.0, b2 = 0.0;
    bool b_measured = true;    // false: no bin reached where b could be seen; b is held at 0
    bool b_resolved = true;    // b^2 stood clear of two of its standard errors
};

// Scratch: the partitioned copy of the pool (reused between calls to save the allocation), with the rank
// key and the current normalised deviation.
struct ErrorModelBinned {
    double snr2, s2, I2, dev2, z2;
};

ErrorModelFit FitErrorModel(const std::vector<ErrorModelSample> &pool, std::vector<ErrorModelBinned> &scratch,
                            size_t nthreads);