Error model: calibrate each bin on the median NORMALISED deviation, binned by counting I/sigma

The rotation merge fitted var = a*s2 + b^2*<I>^2 from three separate medians (s2, I2, dev2) per
bin of I2. A median of dev2 over observations whose variances differ is not 0.455 times their mean
variance, so the ratio of medians read a too low and b too high: on the scaled fulls of 28 sets
(in-house, open and private) the core of the normalised deviations scattered at up to 1.8x its
stated variance in the weak and middle bins and at 0.1-0.7x in the strongest. Reproduced on
synthetic samples with a known model (a 1.3 read as 1.12, ISa 33 read as 31).

Now (ErrorModel.h/.cpp, host-only, so the GPU and CPU paths share it):
- bins are equal counts in counting I/sigma (I2/s2), where b is identified;
- each bin is calibrated on the median of dev2/var with var from the previous iteration, iterated
  to a fixed point - heterogeneity inside a bin no longer biases it, and the median keeps it robust
  to tails;
- s2 is the counting variance the merge actually applies the model to (rebuilt at the reflection's
  mean), not the observation's own sigma^2.
The separate 6-sigma misfit refit is gone: the median does not need it.

A mean-based fit (misfits cut at z^2 > 2 ln N) was tried first: it calibrates the total variance
best (median rms log chi2 over the 28 sets 0.14 vs 0.19 here) but on heavy-tailed data it sizes
the sigmas on the tails, the merge's outlier test widens with them, and CC1/2 fell 0.80 -> 0.71 on
a powder-contaminated set (0.83 with this fit). Rejected for that.

Offline, 28 sets: rms log chi2 of the median normalised deviation over (counting I/sigma x
resolution) 0.208 -> 0.139 (better on 23), of the mean 0.239 -> 0.191 (better on 22).
Battery, 45 of 48 sets against the 9b6736 run (3 lost to CUDA OOM from GPU contention): space group
unchanged on all; d_min unchanged except two poor multi-lattice sets (1.69 -> 1.56, 1.96 -> 1.90);
ISa x1.13 (median), in-house ISa/XDS 0.73 -> 0.93; ISa*R_meas_lo/0.8 0.93 -> 1.05 (XDS ~1.2);
CC1/2 over the XDS range +0.002 (mean; up 0.014-0.031 on the three poorest sets, else +-0.0001);
CC_model +0.0011, R_model_shell_scaled -0.0007 (mean over 17 open sets); CC_anom +0.003 (mean).
Six private sets: space group, d_min and CC1/2 unchanged, ISa up by 14-67% towards XDS's.

Remaining misfit, not addressed: the excess variance grows slower than <I>^2 (the effective
fractional error falls 2-2.5x from counting I/sigma 5 to 200), so the strongest reflections still
scatter below their sigma on open sets. A third, linear term (as in Aimless) fits it better on most
sets but leaves b unidentified on some (b -> 0 on 4 of 28); not landed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C

tests/MergeScaleTest.cpp

@@ -4,6 +4,7 @@
 #include <catch2/catch_all.hpp>
 
 #include <random>
+#include "../image_analysis/scale_merge/ErrorModel.h"
 #include "../image_analysis/scale_merge/HKLKey.h"
 #include "../image_analysis/scale_merge/Merge.h"
 #include "../image_analysis/scale_merge/ResolutionCutoff.h"
@@ -406,3 +407,72 @@ TEST_CASE("ResolutionCutoff_RaggedFallOffIsReadOffTheBins") {
     REQUIRE(rc.d_cut);
     CHECK(*rc.d_cut > 2.30);                        // coarser than the band that correlates again
 }
+
+namespace {
+    // Samples of var = a*s2 + b^2*I2 over four decades of counting I/sigma, with counting variances
+    // spread over a factor of 100 at every intensity, and a fraction of gross outliers.
+    std::vector<ErrorModelSample> SyntheticErrorModelSamples(double a, double b, double max_snr,
+                                                             double outlier_fraction) {
+        std::mt19937 rng(7);
+        std::uniform_real_distribution<double> u(0.0, 1.0);
+        std::normal_distribution<double> z(0.0, 1.0);
+        std::vector<ErrorModelSample> out;
+        for (int i = 0; i < 200000; ++i) {
+            const double s2 = std::pow(10.0, 2.0 * u(rng));
+            const double snr = max_snr * std::pow(10.0, -4.0 * u(rng));
+            const double I2 = snr * snr * s2;
+            const double e = z(rng);
+            double dev2 = (a * s2 + b * b * I2) * e * e;
+            if (u(rng) < outlier_fraction) dev2 *= 400.0;
+            out.push_back({s2, I2, dev2, 2.0f});
+        }
+        return out;
+    }
+}
+
+// The fit recovers a and b from observations whose counting variances differ widely inside every
+// intensity bin (where a ratio of bin medians does not), with 0.3% gross outliers in the pool.
+TEST_CASE("ErrorModel_RecoversAAndBThroughOutliers") {
+    const auto pool = SyntheticErrorModelSamples(1.3, 0.03, 300.0, 0.003);
+    std::vector<ErrorModelBinned> scratch;
+    const auto fit = FitErrorModel(pool, scratch, 4);
+    REQUIRE(fit.active);
+    CHECK(fit.b_measured);
+    CHECK(fit.b_resolved);
+    CHECK(fit.a == Catch::Approx(1.3).epsilon(0.03));
+    CHECK(1.0 / std::sqrt(fit.b2) == Catch::Approx(1.0 / 0.03).epsilon(0.05));
+}
+
+// A few observations whose counting variance is hugely overstated - their scatter is ordinary - do not
+// set the fit: each sample counts by its deviation relative to its own variance.
+TEST_CASE("ErrorModel_OverstatedCountingVarianceDoesNotSetTheFit") {
+    auto pool = SyntheticErrorModelSamples(1.3, 0.03, 300.0, 0.0);
+    for (size_t i = 0; i < pool.size(); i += 200)
+        pool[i].s2 *= 1e4;
+    std::vector<ErrorModelBinned> scratch;
+    const auto fit = FitErrorModel(pool, scratch, 4);
+    REQUIRE(fit.active);
+    CHECK(fit.a == Catch::Approx(1.3).epsilon(0.03));
+    CHECK(1.0 / std::sqrt(fit.b2) == Catch::Approx(1.0 / 0.03).epsilon(0.05));
+}
+
+// The same pool on one thread and on many gives the same numbers.
+TEST_CASE("ErrorModel_SameOnAnyNumberOfThreads") {
+    const auto pool = SyntheticErrorModelSamples(1.1, 0.05, 100.0, 0.001);
+    std::vector<ErrorModelBinned> scratch;
+    const auto one = FitErrorModel(pool, scratch, 1);
+    const auto many = FitErrorModel(pool, scratch, 16);
+    CHECK(one.a == many.a);
+    CHECK(one.b2 == many.b2);
+}
+
+// Data that never reach where b could be seen: a is fitted alone and b held at 0.
+TEST_CASE("ErrorModel_BNotMeasuredOnWeakData") {
+    const auto pool = SyntheticErrorModelSamples(0.9, 0.05, 1.5, 0.0);
+    std::vector<ErrorModelBinned> scratch;
+    const auto fit = FitErrorModel(pool, scratch, 4);
+    REQUIRE(fit.active);
+    CHECK(!fit.b_measured);
+    CHECK(fit.b2 == 0.0);
+    CHECK(fit.a == Catch::Approx(0.9).epsilon(0.03));
+}