Joint sweeps: a sweep on no axes of the reference whose cell is off the target's is processed again at the shared distance

A sweep whose cell is several percent short (header distance kept) can share too few reflections with the
reference to be put on its axes at all, and the distance re-run skipped every such sweep: the cell error
that kept it off the axes was never corrected. Every sweep with a merge whose cell differs from the target's
by more than SHARED_CELL_AGREEMENT is now processed again from the distance that makes the two cells agree,
unless the two volumes are a whole multiple apart (another lattice). For a sweep on no axes there is no
held-out residual to compare; its second run is taken where it has no fewer fulls, and the relation made
again decides its join on its intensities.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdJDrM65gxsCn2PrbfK
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-11-14 20:33:35 +01:00
1 parent 2489700184
commit d7df69838f
2 files changed
+49 -36

No files matched your search

+6 -4
View File
@@ -118,10 +118,12 @@ about 1 it joins, clearly below it (another crystal) it is left out, and in betw
joins unless it is shown to hurt: adding it raises the held-out residual of the others' merge by more than 1%,
two standard errors down (a sweep left out is named in the log). A sweep that found no lattice of the crystal on its own (it merged nothing, or correlates on no axes)
is processed again given the cell of the joined sweep reaching the finest resolution. The crystal has one cell:
any sweep on the reference's axes - the ones just rescued too - whose own cell is more than 0.3% off that sweep's
(where the cell is measured rather than the distance) is processed a second time, from the distance that would
make the two cells agree, and that run replaces the first only where its intensities agree better with the
joined sweeps' (a lower held-out residual by more than twice its error) and it has no fewer fulls. Their fulls are
any sweep with a merge - joined or not, the ones just rescued too - whose own cell is more than 0.3% off that
sweep's (where the cell is measured rather than the distance), but not a whole multiple of it in volume, is
processed a second time, from the distance that would make the two cells agree. That run replaces the first only
where it has no fewer fulls and, for a sweep on the reference's axes, its intensities agree better with the
joined sweeps' (a lower held-out residual by more than twice its error); a sweep on none is then judged on its
intensities like every other. Their fulls are
scaled and merged together: a per-frame scale within each sweep, the sweeps' dose order read from
their relative B factors (not from the input order or the header times), a decay along that order or
one B per sweep, whichever predicts held-out frames better, then the space-group search, the
+43 -32
View File
@@ -4209,20 +4209,22 @@ int RugnuxCli::RunJointSweeps(const RugnuxSweepPlan &plan) {
replaced = false;
}
// Every sweep on the reference's axes, joined or not - the ones just rescued too - on the target sweep's
// cell: a cell off by a few percent is itself a reason for intensities to disagree, so the join is decided
// after this, on the runs it keeps. A sweep whose cell is off the target's is processed again from the
// distance that would make the two agree, and that run replaces its first one only where it agrees better
// with the joined sweeps' intensities (HeldOutAgainst) by more than the two residuals' errors, and at no loss
// of observations: a distance is not changed on noise, and a run that integrates fewer reflections does not
// measure them better.
// Every sweep with a merge - joined or not, the ones just rescued too - on the target sweep's cell: a cell off
// by a few percent is itself a reason for intensities to disagree, or for a sweep not to be put on the
// reference's axes at all, so the join is decided after this, on the runs it keeps. A sweep whose cell is off
// the target's is processed again from the distance that would make the two agree - unless the two are
// different lattices, whose volumes are a whole multiple (2 or more) apart. On the reference's axes, the
// second run replaces the first only where it agrees better with the joined sweeps' intensities
// (HeldOutAgainst) by more than the two residuals' errors; on none, there is nothing to compare, and the
// relation made again below decides on its intensities. Never at a loss of observations: a distance is not
// changed on noise, and a run that integrates fewer reflections does not measure them better.
for (size_t k = 0; target >= 0 && k < members.size(); ++k) {
const std::string &decision = relation.sweeps[k].decision;
if (decision == "no merge" || decision == "no axes" || static_cast<int>(k) == target
|| !std::isfinite(members[k].distance_mm))
if (decision == "no merge" || static_cast<int>(k) == target || !std::isfinite(members[k].distance_mm))
continue;
const double scale = VolumeLength(members[target].cell) / VolumeLength(members[k].cell);
if (std::fabs(scale - 1.0) <= SHARED_CELL_AGREEMENT)
const double volume_ratio = scale * scale * scale;
if (std::fabs(scale - 1.0) <= SHARED_CELL_AGREEMENT || std::max(volume_ratio, 1.0 / volume_ratio) >= 1.5)
continue;
const auto distance = static_cast<float>(members[k].distance_mm * scale);
logger.Info("Sweep {}: its cell is {:+.2f}% off sweep {}'s; processed again from {:.2f} mm", k + 1,
@@ -4230,29 +4232,38 @@ int RugnuxCli::RunJointSweeps(const RugnuxSweepPlan &plan) {
JointMemberSweep again = run_member(k, distance, std::nullopt, "_cell");
if (again.merged.empty())
continue;
// Against the merge of the other joined sweeps, the target first (the scale).
std::vector<const JointMemberSweep *> base{&members[target]};
std::vector<gemmi::Op> base_cob{relation.sweeps[target].cob};
for (size_t j = 0; j < members.size(); ++j)
if (relation.sweeps[j].included && j != k && static_cast<int>(j) != target) {
base.push_back(&members[j]);
base_cob.push_back(relation.sweeps[j].cob);
}
const JointMemberSweep &reference = members[relation.reference];
const std::vector<HeldOutGain> gains = HeldOutAgainst(base, base_cob, reference, {&members[k], &again});
const HeldOutGain &before = gains[0], &after = gains[1];
const double gain = before.change - after.change;
const bool better = std::isfinite(gain) && gain > 2.0 * std::hypot(before.se, after.se);
const bool no_loss = again.fulls.size() >= members[k].fulls.size();
logger.Info("Sweep {}: against the joined sweeps (scale: sweep {}), held-out residual {:+.2f}% +- {:.2f}% at "
"its own cell ({:.3f} {:.3f} {:.3f}, {} fulls), {:+.2f}% +- {:.2f}% processed again ({:.3f} {:.3f} "
"{:.3f}, {} fulls): {}", k + 1, target + 1, 100.0 * before.change, 100.0 * before.se,
members[k].cell.a, members[k].cell.b, members[k].cell.c, members[k].fulls.size(),
100.0 * after.change, 100.0 * after.se, again.cell.a, again.cell.b, again.cell.c,
again.fulls.size(),
!better ? "the first run stands - the second agrees no better beyond the errors"
: !no_loss ? "the first run stands - the second has fewer fulls"
: "the second run is taken");
bool better = true;
if (decision == "no axes")
logger.Info("Sweep {}: on no axes of the reference at its own cell, so nothing to compare; {} fulls there "
"({:.3f} {:.3f} {:.3f}), {} processed again ({:.3f} {:.3f} {:.3f}): {}", k + 1,
members[k].fulls.size(), members[k].cell.a, members[k].cell.b, members[k].cell.c,
again.fulls.size(), again.cell.a, again.cell.b, again.cell.c,
no_loss ? "the second run is taken" : "the first run stands - the second has fewer fulls");
else {
// Against the merge of the other joined sweeps, the target first (the scale).
std::vector<const JointMemberSweep *> base{&members[target]};
std::vector<gemmi::Op> base_cob{relation.sweeps[target].cob};
for (size_t j = 0; j < members.size(); ++j)
if (relation.sweeps[j].included && j != k && static_cast<int>(j) != target) {
base.push_back(&members[j]);
base_cob.push_back(relation.sweeps[j].cob);
}
const JointMemberSweep &reference = members[relation.reference];
const std::vector<HeldOutGain> gains = HeldOutAgainst(base, base_cob, reference, {&members[k], &again});
const HeldOutGain &before = gains[0], &after = gains[1];
const double gain = before.change - after.change;
better = std::isfinite(gain) && gain > 2.0 * std::hypot(before.se, after.se);
logger.Info("Sweep {}: against the joined sweeps (scale: sweep {}), held-out residual {:+.2f}% +- {:.2f}% "
"at its own cell ({:.3f} {:.3f} {:.3f}, {} fulls), {:+.2f}% +- {:.2f}% processed again ({:.3f} "
"{:.3f} {:.3f}, {} fulls): {}", k + 1, target + 1, 100.0 * before.change, 100.0 * before.se,
members[k].cell.a, members[k].cell.b, members[k].cell.c, members[k].fulls.size(),
100.0 * after.change, 100.0 * after.se, again.cell.a, again.cell.b, again.cell.c,
again.fulls.size(),
!better ? "the first run stands - the second agrees no better beyond the errors"
: !no_loss ? "the first run stands - the second has fewer fulls"
: "the second run is taken");
}
if (better && no_loss) {
again.reprocessed_from_mm = members[k].distance_mm;
members[k] = std::move(again);