resolution: a cut the shell table refutes is not quoted

One run was written to 1.089 A where I/sigma was zero and R_meas 1776 per cent,
and the summary said only "Merged to 1.09 A". Two things let that through.

The logistic's crossing was taken wherever the fitted curve met the target, even
when that point lay past every bin the curve was fitted through - an extrapolation
of a fall-off the data never showed, quoted as a measurement. The crossing now has
to lie inside the fitted bins, with the one-shell extension still there to spare;
outside them the number is read off the bins themselves instead.

And the shipped guard asked only for the finest shell whose CC1/2 still reached the
target, with no requirement that the curve get there monotonically. A noise shell
that climbs back over the bar therefore became the edge of the data. It is the
climbing back that disqualifies it, and the program already noticed - it printed
"CC1/2 is not monotone with resolution" and then cut there anyway, because the test
was developer-only and decided nothing. It decides now, in the report everyone
reads, and the duplicate is gone so there is one such test rather than two.

The dataset above is written to 1.527 A: CC1/2 0.209 to 0.688, I/sigma 0.40 to
1.26, R_meas 245 to 145 per cent. Forty-four of fifty-one reports are unchanged to
the character; of the seven that move, five are cut coarser and every headline
number of all five improves, one loses a fit that was an extrapolation without
changing anything written, and one gains a quotable fit and loses a warning.

This is a guard, not the cause. On four of those five the reflections doing the
damage are ice, which the resolution fit now leaves out for its own reasons; the
guard still earns its place, because on those four removing the ice alone does not
stop the cut being quoted past what the shells support.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011GxZqDiFP3KqriBhNdcR56
This commit is contained in:
leonarski_fandClaude Opus 5 committed 2026-09-14 14:17:39 +02:00
1 parent 5f1292ca3b
commit d00175c8b4
6 files changed
+145 -38

No files matched your search

@@ -107,9 +107,11 @@ ResolutionCutoffResult ComputeCCHalfLogisticCutoff(const std::vector<MergedRefle
result.note = "no low-resolution CC1/2 plateau";
return result;
}
size_t first_below = cc_bin.size();
size_t keep = cc_bin.size();
for (size_t i = 0; i < cc_bin.size(); ++i) {
if (cc_bin[i] < cc_target) {
first_below = i;
keep = std::min(cc_bin.size(), i + 1 + EXTEND_BINS_PAST_FALLOFF);
break;
}
@@ -160,7 +162,26 @@ ResolutionCutoffResult ComputeCCHalfLogisticCutoff(const std::vector<MergedRefle
// plateau -> the fall-off crossing), NOT the full measured range: when the detector reaches far
// past where the crystal diffracts (a high-res-configured detector on a low-res crystal), the
// full range is dominated by high-res noise, so s_range/10 would be a huge over-extension.
const double s_cross = s0 + std::log(1.0 / cc_target - 1.0) / k;
double s_cross = s0 + std::log(1.0 / cc_target - 1.0) / k;
// The crossing has to be one the bins show, not one the logistic extrapolates to. On a real
// fall-off it sits between the last bin at the target and the first bin below it, with the two
// extension bins to spare. A crossing past the last fitted bin means the logistic never followed
// these data down - it settled on a shallow slope through a fall-off region that is not one -
// and every resolution read off it is one the bins refute. The bins' own crossing is the honest
// answer then: where the measured CC1/2 passes the target, between the two bins that straddle it.
bool logistic = true;
if (s_cross > s_bin.back()) {
if (first_below >= cc_bin.size()) {
result.note = "CC1/2 does not fall off within the measured range";
return result;
}
logistic = false;
const double f = (cc_bin[first_below - 1] - cc_target)
/ (cc_bin[first_below - 1] - cc_bin[first_below]);
s_cross = s_bin[first_below - 1] + f * (s_bin[first_below] - s_bin[first_below - 1]);
}
const double delta_s = (s_cross - s_lo) / SHELLS_FOR_EXTENSION;
const double s_final = s_cross + delta_s;
@@ -194,7 +215,9 @@ ResolutionCutoffResult ComputeCCHalfLogisticCutoff(const std::vector<MergedRefle
}
result.d_cut = d_cut;
result.note = "CC1/2 logistic fall-off, +1 shell";
result.note = logistic ? "CC1/2 logistic fit"
: "the logistic fit extrapolates past the bins it was made over, so the "
"measured CC1/2";
return result;
}
@@ -212,9 +235,9 @@ std::optional<double> ApplyResolutionCutoff(std::vector<MergedReflection> &merge
if (rc.d_cut) {
effective_d_min = rc.d_cut;
if (rc.d_fit)
logger.Info("Auto resolution cutoff: {:.2f} A (CC1/2 fit crosses {:.2f} at {:.2f} A, "
logger.Info("Auto resolution cutoff: {:.2f} A ({} crosses {:.2f} at {:.2f} A, "
"written one shell past it; override with --scaling-high-resolution)",
*rc.d_cut, cc_target, *rc.d_fit);
*rc.d_cut, rc.note, cc_target, *rc.d_fit);
else
logger.Info("Auto resolution cutoff: {:.2f} A ({}; override with --scaling-high-resolution)",
*rc.d_cut, rc.note);