Stills merge: outlier rejection on by default (10 sigma), the band linear in the error model's sigma
The stills merge had no working outlier rejection. --reject-outliers was 0 for stills, and when set its band took the systematic part as a factor exp(+-n s), s the spread of ln(I / median) - on stills a reflection's observations are partials and per-crystal scales, many near zero, so s is wide and the band let through observations hundreds of sigma out. On a high-multiplicity broad-bandpass run with a spotty powder contaminant, single grains in a weak reflection's box were that reflection's whole merged intensity (half-set means -2 vs 2284 on 141 observations); the shells holding the powder rings and the lowest shell merged at CC1/2 0.6-0.75 beside 0.99 elsewhere, at multiplicity ~400-1000. Stills now test the plain linear band, n sqrt(a sigma^2 + (b <I>)^2) about the median - the error model's b is the spread as it is - at n = 10 by default (rotation keeps 6 and its multiplicative band). The stills merge statistics now report the rejected count (OBSERVATIONS_REJECTED was always 0 there). Measured (REFMAC 0-cycle R-free of an isomorphous room-temperature model, depositor's free set, scale mode on stored integrations, vs the same code with --reject-outliers 0): - broad-bandpass in-house stills, 50k frames: 0.318 -> 0.265; 20k: 0.274 -> 0.244; R_split 0.29 -> 0.06, CC1/2 0.79 -> 0.98. 0.1% of the observations are rejected. - monochromatic in-house stills, 30k frames: 0.216 -> 0.212; R_split 0.126 -> 0.087. - n = 6 gains ~0.003 more there but costs the public XFEL sets 0.003-0.006 on three of four; 20 keeps most of the gain. Still battery, 12 non-float sets, against rc180: R_split better on all 12; R-free change mean -0.0005 (range -0.006..+0.003, plus pal86 +0.009 that is within that set's run-to-run spread of ~0.007 - a rerun of rc180's behaviour gave 0.192 against the baseline's 0.185); one draw becomes a win (lcls242_s), nothing else changes class. Rotation is untouched (the stills band and default are gated on the experiment not being a rotation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 3a7142801056f21c720956255f1207e0bf1a8f43)
This commit is contained in:
1 parent
47960ccaf7
commit
cf7676f5d5
7 files changed
+63
-7
No files matched your search
@@ -1113,6 +1113,45 @@ TEST_CASE("MergeOnTheFly_RejectsAnOutlierInTheBand") {
|
||||
CHECK(merge.RejectedCount() == 1);
|
||||
}
|
||||
|
||||
// Stills: the observations of a reflection spread from near zero to its full intensity (partials), so
|
||||
// the spread of ln(I / median) is wide. One powder grain in a weak reflection's box, at 200 times its
|
||||
// intensity, must still be rejected - the band is the error model's, n sigma about the median - while
|
||||
// none of the partials is.
|
||||
TEST_CASE("MergeOnTheFly_StillsRejectsAGrainAmongPartials") {
|
||||
DiffractionExperiment x;
|
||||
x.SetSpaceGroup(gemmi::get_spacegroup_by_name("P 1"));
|
||||
ScalingSettings s = x.GetScalingSettings();
|
||||
s.OutlierRejectNsigma(6.0);
|
||||
x.ImportScalingSettings(s);
|
||||
REQUIRE(!x.IsRotationIndexing());
|
||||
|
||||
std::mt19937 rng(5);
|
||||
std::normal_distribution<float> gauss(0.0f, 1.0f);
|
||||
std::uniform_real_distribution<float> partial(0.1f, 1.0f);
|
||||
std::vector<IntegrationOutcome> outcomes(40);
|
||||
for (size_t i = 0; i < outcomes.size(); ++i) {
|
||||
for (int h = 1; h <= 50; ++h) {
|
||||
Reflection r{};
|
||||
r.h = h; r.k = 2; r.l = 3;
|
||||
r.d = 2.0f;
|
||||
const float I_true = 100.0f * static_cast<float>(h);
|
||||
r.I = I_true * partial(rng) + 3.0f * gauss(rng);
|
||||
if (i == 7 && h == 1)
|
||||
r.I = 200.0f * I_true;
|
||||
r.sigma = 3.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
r.partiality = 1.0f;
|
||||
r.image_scale_corr = 1.0f;
|
||||
outcomes[i].reflections.push_back(r);
|
||||
}
|
||||
}
|
||||
MergeOnTheFly merge(x);
|
||||
merge.RefineErrorModel(outcomes);
|
||||
REQUIRE(merge.ErrorModelActive());
|
||||
merge.AddImages(outcomes);
|
||||
CHECK(merge.RejectedCount() == 1);
|
||||
}
|
||||
|
||||
// The capture term widens the sigma of a partial by (c (1 - p) <I>): a partial merges at a lower weight
|
||||
// than a full of the same intensity, and not at all differently with c = 0.
|
||||
TEST_CASE("MergeOnTheFly_CaptureTermDownweightsPartials") {
|
||||
|
||||
Reference in new issue
Block a user