model: nine null replicates, now that they cost the slowest and not the sum

The verdict is (real - mean)/sd of the null sample, so its reliability is set by
how well the SPREAD is pinned, not the mean. The relative error on an sd from n
draws is 1/sqrt(2(n-1)), and a sample that happens to come out narrow is exactly
what turns a model that does not fit into one that appears to.

Measured on the case that sits nearest the gate - an unrelated protein, 1.7-1.8
sigma against a threshold of 3 - the chance of it reading over the gate on a
different seed is 31% at n=3, 17% at n=5 and 6% at n=9.

Nine is affordable only because the replicates now run concurrently: the null
costs the slowest of them rather than their sum, and the slowest of nine is
barely above the slowest of five. Measured end to end on the accepted case, 5.98 s
against 6.46 s for five - inside the run-to-run scatter, so the four extra
replicates are free.

Verdicts unchanged, on more evidence: correct model ACCEPTED at +17.69 sigma
(+14.96 at n=5), an unrelated protein REJECTED at +1.70, the same model rotated
90 degrees REJECTED at -0.37. Both rejected runs still write .mtz, .cif, .hkl and
_unmerged.mtz byte-identical to a run with no model at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

docs/CPU_DATA_ANALYSIS_DECISIONS.md

@@ -140,7 +140,7 @@ a cut calibrated on one pair misjudges the next. In one measured arm an unrelate
 
 The only null that fits both the model and the data is therefore **made out of them**: the same model
 is re-oriented at random about its own centroid and run through the identical path — the same scaling,
-the same rigid-body placement, the same R — five times, and the real fit is asked how far above the
+the same rigid-body placement, the same R — nine times, and the real fit is asked how far above the
 resulting distribution it sits (`MODEL_FIT_SIGMA`, accepted at 3σ). The replicates are placed as well
 as fitted, or the comparison would be between a placed model and unplaced nulls and the margin would
 be inflated by the placement rather than by the model. Measured on one rotation data set: the crystal's
@@ -166,12 +166,20 @@ one that has to deliver a map seconds after the last image — and it is why the
 indexing probe is a handful of scaling fits and runs first anyway, so whether the expensive part is
 worth paying for is known before it starts.
 
+The count is nine, and it is the *spread* that sets it rather than the mean: the verdict is
+(real − mean)/sd of the sample, the relative error on an sd from $n$ draws is $1/\sqrt{2(n-1)}$, and a
+sample that happens to come out narrow is what turns a model that does not fit into one that appears
+to. Measured on the case nearest the gate — an unrelated protein at 1.83σ against a threshold of 3 —
+the chance of reading over the gate on a different seed is 31 % at $n=3$, 17 % at $n=5$ and 6 % at
+$n=9$. Nine is affordable only because the replicates run at once: the null costs the slowest of them
+rather than their sum, and the slowest of nine is barely above the slowest of five.
+
 The null costs one full fit and one rigid-body placement per replicate. The replicates share nothing
 — each is the same model under a different rotation, scored the same way — so each takes a copy of
 the model and of its structure factors and they run **concurrently**, on the run's own thread budget
-(`-N`); the real model is not touched by any of them. Measured on an idle machine, the five cost
-2.5 s together where running them one after another costs 10 s, on a `--mode scale` run that merges
-in 2 s. What remains is the cost of the *slowest* replicate: how many placement evaluations an
+(`-N`); the real model is not touched by any of them. Measured on an idle machine, five replicates
+cost 2.5 s together where running them one after another costs 10 s, on a `--mode scale` run that
+merges in 2 s. What remains is the cost of the *slowest* replicate: how many placement evaluations an
 orientation needs varies by nearly half between them, so the concurrency saturates around 4×, and
 more threads than replicates buy nothing. The orientations are drawn up front, in order, from the
 fixed seed, so replicate $i$ gets the same orientation whatever order the threads run in — a σ that

rugnux/ModelValidation.cpp

@@ -69,10 +69,18 @@ constexpr double ANOMALOUS_INVERSION_SIGMA = 5.0;
 // signal is there and what carries it; a full site list is what the map file is for.
 constexpr size_t MAX_ANOMALOUS_SITES = 10;
 
-// How many random placements of the same model the real fit is compared against. Five is enough to
-// put a mean and a spread on a distribution whose arms are several sigma apart in the cases that
-// matter, and each replicate costs a full fit and a rigid-body placement.
-constexpr int NULL_REPLICATES = 5;
+// How many random placements of the same model the real fit is compared against. The verdict is
+// (real - mean)/sd of this sample, so what matters is not the mean but how well the SPREAD is
+// pinned: the relative error on an sd from n draws is 1/sqrt(2(n-1)), and a sample that happens to
+// come out narrow is what turns a model that does not fit into one that appears to. Measured on the
+// case that sits closest to the gate - a model of an unrelated protein, 1.83 sigma against a
+// threshold of 3 - the chance of it reading over the gate on a different seed is 31% at n=3, 17% at
+// n=5, 6% at n=9.
+//
+// Nine rather than five because the replicates run concurrently: the null costs the SLOWEST of them
+// rather than their sum, and the slowest of nine is barely above the slowest of five, so the extra
+// four are close to free in wall clock (measured 2.5 s against 2.6 s) up to the thread count.
+constexpr int NULL_REPLICATES = 9;
 
 // Fixed, so the same data and the same model give the same verdict on every run.
 constexpr unsigned NULL_SEED = 20260902;