delta-CC1/2: a flagged stretch both CC1/2 call harmful is removed below the least-harm floor
A stretch the per-image channels flag is removed whole when keeping it lowers the plain CC1/2 by at least DELTA_CC_MIN_HARM. The floor is there for a stretch the merge still learns from - a weak tail on a crystal merging at 0.999 - but it also kept stretches that carry nothing: frames with the sample out of the beam, at a tenth of the run's scale, whose removal raises the plain CC1/2 by 0.001-0.002 at 6-20 standard errors and whose noise dominates the reported R_meas. Such a stretch is now removed when the information-weighted CC1/2 says, as significantly, that keeping it is harmful too, and its plain cost is beyond the null of the unflagged batches: then the merged intensities do not want it either. The decision no longer depends on whether the batch loop happened to slide its edges over the same frames, which flipped between versions on a strong data set with two out-of-beam arcs. The plain and weighted measurements of every flagged stretch are logged at debug level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UqXYdL4qBGYRiSXSGXfCLu
This commit is contained in:
1 parent
f0832ee39e
commit
d04f91fd74
1 file changed
+22
-5
@@ -3853,10 +3853,14 @@ void RotationScaleMerge::MeasureBatchDeltaCCHalf(int n_groups) {
|
||||
// measured the same way, DELTA_CC_SIGMA robust standard deviations below their median; with
|
||||
// fewer than DELTA_CC_MIN_HEALTHY_BATCHES of them there is no null and nothing is removed;
|
||||
// * and the cost has to be at least DELTA_CC_MIN_HARM in CC: on a crystal merging at 0.999 a tail
|
||||
// that costs 0.0002 is many standard errors, and harmless.
|
||||
// that costs 0.0002 is many standard errors, and harmless - unless the information-weighted CC1/2
|
||||
// says the same, as significantly: then the merged intensities do not want the stretch either, and
|
||||
// there is nothing its removal could cost. The floor protects a stretch the merge still learns
|
||||
// from, not one whose frames carry nothing but noise (a sample that has left the beam).
|
||||
// The worst stretch goes first and the rest are measured again against what is left, as above.
|
||||
unit_weight = true;
|
||||
double healthy_floor = -std::numeric_limits<double>::infinity();
|
||||
double healthy_null = -std::numeric_limits<double>::infinity();
|
||||
double healthy_floor = healthy_null;
|
||||
{
|
||||
std::vector<std::pair<int, int>> healthy;
|
||||
for (int b = 0; b < n_batch; ++b) {
|
||||
@@ -3871,7 +3875,8 @@ void RotationScaleMerge::MeasureBatchDeltaCCHalf(int n_groups) {
|
||||
if (static_cast<int>(v.size()) >= DELTA_CC_MIN_HEALTHY_BATCHES) {
|
||||
const double median = MedianOf(v);
|
||||
for (double &d : v) d = std::fabs(d - median);
|
||||
healthy_floor = std::min(median - DELTA_CC_SIGMA * 1.4826 * MedianOf(v), -DELTA_CC_MIN_HARM);
|
||||
healthy_null = median - DELTA_CC_SIGMA * 1.4826 * MedianOf(v);
|
||||
healthy_floor = std::min(healthy_null, -DELTA_CC_MIN_HARM);
|
||||
}
|
||||
}
|
||||
// Ledger ranges closer together than one rocking event are one stretch.
|
||||
@@ -3885,10 +3890,22 @@ void RotationScaleMerge::MeasureBatchDeltaCCHalf(int n_groups) {
|
||||
while (true) {
|
||||
// A stretch already wholly out touches no reflection and reads as not measured.
|
||||
const std::vector<RangeDelta> stretch_delta = measure_all(stretches);
|
||||
unit_weight = false;
|
||||
const std::vector<RangeDelta> weighted_delta = measure_all(stretches);
|
||||
unit_weight = true;
|
||||
for (size_t i = 0; i < stretches.size(); ++i)
|
||||
if (stretch_delta[i].ok)
|
||||
logger->Debug("delta-CC1/2 flagged stretch {}-{}: plain {:+.4f} +/- {:.4f} (z {:+.1f}), weighted {:+.4f} "
|
||||
"(z {:+.1f}), healthy null {:+.4f}", stretches[i].first, stretches[i].second,
|
||||
stretch_delta[i].delta_cc, stretch_delta[i].se, stretch_delta[i].delta_z / stretch_delta[i].se_z,
|
||||
weighted_delta[i].delta_cc, weighted_delta[i].ok ? weighted_delta[i].delta_z / weighted_delta[i].se_z : 0.0,
|
||||
healthy_null);
|
||||
int worst = -1;
|
||||
for (size_t i = 0; i < stretches.size(); ++i) {
|
||||
const RangeDelta &r = stretch_delta[i];
|
||||
if (r.ok && r.delta_z < -DELTA_CC_SIGMA * r.se_z && r.delta_cc < healthy_floor
|
||||
const RangeDelta &r = stretch_delta[i], &w = weighted_delta[i];
|
||||
const bool weighted_agrees = w.ok && w.delta_z < -DELTA_CC_SIGMA * w.se_z;
|
||||
if (r.ok && r.delta_z < -DELTA_CC_SIGMA * r.se_z
|
||||
&& (r.delta_cc < healthy_floor || (weighted_agrees && r.delta_cc < healthy_null))
|
||||
&& (worst < 0 || r.delta_z < stretch_delta[worst].delta_z))
|
||||
worst = static_cast<int>(i);
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user