mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 73 Wiki Activity

rugnux: decide the space group after the second pass, not the first

Browse Source 
Pass 1 searched for the space group on the geometry the run started with, stashed the answer, and
reinstated it before pass 2's merge - so the decision was made on the worse of the two passes and
then fed forward as a constraint. Four guards existed to reconcile it with what pass 2 later found.

Now pass 1 does not search at all. It keeps its first merge, which is where mosaicity smoothing and
geometry post-refinement already happen, and skips the in-symmetry re-merge that existed only to feed
the search. prepass_merge_sg_, prepass_promoted_point_group_, the reinstatement, the
primitive-to-conventional reindex it needed, and lattice_conflicts_with_prepass_sg all go.

The obstacle was the pass-adoption guard, which compared completeness and CC1/2 across what could now
be two different space groups. It compares each pass's FIRST merge instead - P1 on both sides, full
range, before correction surfaces - which pass 2 produces anyway.

One guard had to be replaced rather than removed. The index-time veto is re-keyed from pass 1's GROUP
to its LATTICE and made one-directional: a centred pass-1 lattice against a primitive pass-2 one.
Without it a C2 crystal fell to P1, and the obvious narrower fix - exempting a pass-1 P1 - would not
have caught it, because that crystal's pass-2 lattice is monoclinic-P rather than triclinic. A
centred lattice and its primitive sub-cell share a primitive volume, so the supercell arm cannot see
that demotion.

Corpus of 93 datasets, both arms at -N 6, every log validated: 78 comparable, 65 with a byte-identical
p.hkl. Two crystals gain their reference point group, one gains its screw axis, one is lost. Per
shell, over the 72 unchanged crystals, the median move is +0.25 points of R_meas and zero in
<I/sigma>, CC1/2 and completeness. Load-matched compute is +0.4%, and the header-geometry fallback
fires three times in the old arm against once in the new.

What is lost is one crystal whose 222 the operator correlations still confirm: the merge chi-squared
ratio moves 3% across a fixed bound at the refined geometry. A second crystal in the XDS harness
crosses the H bound the same way. Both bounds are in SearchSpaceGroup and are recorded elsewhere as
miscalibrated; recalibrating them is deliberately left to its own change.

The comment at the long-axis rescue is corrected while here: the resolution setting only sizes the
FFT histogram, and coarsening moves the true axis DOWN the direction ranking - 1996 to 16361 of
16384 - rather than making it more robust as the comment claimed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
This commit is contained in:
leonarski_f
2026-08-29 19:58:17 +02:00
co-authored by Claude Opus 5
parent 8634e9a72b
commit 4ef6bd9952
2 changed files with 126 additions and 187 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rugnux/Rugnux.cpp
+109 -164
View File
@@ -1052,8 +1052,6 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
prepass_rotation_scale_.reset();
prepass_result_.reset();
force_rotation_result_.reset();
prepass_merge_sg_.reset();
prepass_promoted_point_group_ = false;
bragg_adaptive_.reset();
logger.Info("Rotation two-pass geometry post-refinement: first pass (header geometry) -> {}_01_*",
@@ -1106,24 +1104,6 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
bragg_adaptive_->GetR2(), bragg_adaptive_->GetR3());
}
// Space group: the second pass RE-INDEXES DE NOVO (clear the group here) so the indexer's pseudo-
// symmetry safeguards recover the true cell - reusing pass-1's group in the indexer forces build_sr
// onto a wrong / doubled cell (a huge oblique cell collapses; a pseudo-centred cell doubles). Pass-1's
// group is instead reinstated for the MERGE ONLY (RunPipeline, before RotationScaleMerge), so the
// symmetry stays fixed with no re-search / no flip.
//
// This holds for a group the centred-lattice test determined too, even though its conventional
// setting is not the one the de-novo primitive frame comes back in. That is precisely what the
// primitive->conventional reindex below does, and when the reindex declines - pass 2's metric is
// not the group's - the centring check right after it stops the pass being adopted at all. Letting
// such a group be re-searched instead put the second pass's determination back in play at a
// slightly different geometry, which on a lattice whose centring is pseudo-symmetric to a couple of
// tenths of a percent is a coin toss: measured on a C-centred monoclinic crystal, pass 1 confirmed
// the centring and pass 2 dropped it, and the run merged in P1.
experiment_.SpaceGroupNumber(std::nullopt);
prepass_merge_sg_ = pass1.space_group_number;
prepass_promoted_point_group_ = pass1.twinning.laue_class_was_chosen_by_promotion;
logger.Info("Rotation two-pass geometry post-refinement: second pass (refined geometry, canonical) -> {}_*",
base_prefix);
config_.output_prefix = base_prefix; // the refined pass is the canonical result (no _02 suffix)
@@ -1202,43 +1182,31 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
force_rotation_result_.reset();
}
}
// The space group was DECIDED in pass 1 and only reused by pass 2, which therefore has no search
// of its own to report. Carry pass 1's over, or the evidence behind the decision (operator CCs,
// absence counts, the refused higher symmetry) is never shown on the default rotation path.
if (!pass2.space_group_search.has_value())
pass2.space_group_search = pass1.space_group_search;
// Pass 2 is normally the better answer, which is why it is the canonical output - but it is not
// guaranteed to be, and until now it was adopted whatever it produced. Two ways it can be wrong:
// it merges more unique reflections than the cell it settled on can hold (completeness above
// 100% is arithmetically impossible and means the cell is wrong), or its CC1/2 collapses
// relative to pass 1. Measured on a large-cell crystal: pass 1 195538 unique at 92.6% and
// CC1/2 0.98, pass 2 134667 at "117%" and CC1/2 0.62. Both bounds are set where only a failure
// guaranteed to be. Two ways it can be wrong: it merges more unique reflections than the cell it
// settled on can hold (completeness above 100% is arithmetically impossible and means the cell is
// wrong), or its CC1/2 collapses relative to pass 1. Both bounds are set where only a failure
// reaches them, so a normal run - where pass 2 is a little better - keeps pass 2.
//
// The CC1/2 compared is the one measured BEFORE the scaling correction surfaces, on both sides.
// Pass 1 fits no surfaces (its intensities are discarded, so it has nowhere to put one) and its
// reported CC1/2 is therefore already the uncorrected number; setting pass 2's CORRECTED CC1/2
// against it would credit the refined geometry with whatever the surfaces did and blunt the
// comparison in the process.
// The comparison is made on each pass's SEARCH merge, not on the final one. Each pass
// determines its own space group, so the two final merges can be in different groups, and
// completeness and CC1/2 in different groups are not the same measurement. The search merge is
// P1 on both sides, over the full resolution range and before any correction surface is fitted
// - the one merge the two passes make in the same terms.
if (!cancelled_ && pass1.has_merge_statistics && pass2.has_merge_statistics) {
const auto completeness = [](const ProcessResult &r) {
const auto &o = r.merge_statistics.overall;
return o.possible_unique_reflections > 0
? 100.0 * o.unique_reflections / o.possible_unique_reflections : 0.0;
};
const double compl2 = completeness(pass2);
const double cc1 = pass1.cc_half_before_corrections;
const double cc2 = pass2.cc_half_before_corrections;
const double compl1 = pass1.search_merge_completeness;
const double compl2 = pass2.search_merge_completeness;
const double cc1 = pass1.search_merge_cc_half;
const double cc2 = pass2.search_merge_cc_half;
constexpr double MAX_CREDIBLE_COMPLETENESS = 100.5; // rounding headroom, nothing more
constexpr double MAX_CC_HALF_LOSS = 0.05;
if (pass2.lattice_conflicts_with_prepass_sg
if (pass2.lattice_conflicts_with_fixed_sg
|| compl2 > MAX_CREDIBLE_COMPLETENESS || cc2 < cc1 - MAX_CC_HALF_LOSS) {
logger.Warning("Two-pass: the refined pass is worse than the header-geometry pass "
"(completeness {:.1f}% vs {:.1f}%, CC1/2 before corrections {:.3f} vs "
"{:.3f}) - going back to the header geometry. The refined geometry did "
"not help this crystal.",
compl2, completeness(pass1), cc2, cc1);
compl2, compl1, cc2, cc1);
// The refined pass has already written the canonical files, so re-run at the header
// geometry to replace them - the same remedy the supercell collapse above uses, and it
// only costs a pass on a crystal that was going to be wrong otherwise.
@@ -1246,25 +1214,22 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
.DetectorDistance_mm(header_geometry[2]);
if (prepass_rotation_scale_ && gonio_snapshot)
experiment_.Goniometer(*gonio_snapshot); // and the header rotation angles with it
experiment_.SpaceGroupNumber(std::nullopt);
config_.output_prefix = base_prefix;
auto redo = RunPipeline(observer, /*write_output=*/true, /*geometry_prepass=*/false);
if (!redo.space_group_search.has_value())
redo.space_group_search = pass1.space_group_search;
redo.post_refine = pass1.post_refine;
redo.pass_number = pass2.pass_count + 1;
redo.pass_count = pass2.pass_count + 1;
redo.pass_decision = fmt::format(
"header geometry re-adopted: the post-refined pass was worse "
"(completeness {:.1f}% vs {:.1f}%, CC1/2 before corrections {:.3f} vs {:.3f})",
compl2, completeness(pass1), cc2, cc1);
compl2, compl1, cc2, cc1);
return redo;
}
if (pass2.pass_decision.empty())
pass2.pass_decision = fmt::format(
"post-refined geometry adopted (completeness {:.1f}% vs {:.1f}%, "
"CC1/2 before corrections {:.3f} vs {:.3f})",
compl2, completeness(pass1), cc2, cc1);
compl2, compl1, cc2, cc1);
}
if (pass2.pass_decision.empty())
pass2.pass_decision = (prepass_detector_geometry_ || prepass_rotation_scale_)
@@ -1333,6 +1298,12 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
Logger logger("Rugnux");
ProcessResult result;
// Each pass determines its own space group, so start every one of them from the group the run was
// ASKED for. A pass that determined one leaves it on experiment_, and the pass after it - the
// re-runs in Run(), or a second Run() on the same object - would otherwise inherit that answer and
// skip the search that is the whole point of running again.
experiment_.SpaceGroupNumber(user_fixed_sg_);
const auto dataset = reader_.GetDataset();
if (!dataset)
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid,
@@ -1965,9 +1936,9 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
// axis was likely lost: the unconstrained FFT either collapsed it to a short sub-multiple or let
// a denser supercell over-fit the accumulated cloud (a small global-orientation error throws the
// many high-order reflections off along the fine axis, so the true cell scores worst on the raw
// cloud). Recover the true metric with a COARSE-resolution pass - only low-order reflections,
// where the fine axis stays robust - then RE-INDEX at full resolution constrained by that cell as
// a reference (the -C path): the reference filter drops the collapsed/supercell candidates and
// cloud). Recover the true metric with a second search whose FFT unit-cell bound is widened past
// the default, then RE-INDEX at full resolution constrained by the cell that comes back, as a
// reference (the -C path): the reference filter drops the collapsed/supercell candidates and
// refines an accurate global lattice. Only runs after a poor standard pass, so well-indexing
// crystals never pay for it and their result is untouched.
auto max_axis = [](const RotationIndexerResult &r) {
@@ -1977,8 +1948,12 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
if (!cancelled_ && best.result.has_value()
&& best.score < 0.5 * static_cast<double>(validation.size())) {
auto coarse_settings = experiment_.GetIndexingSettings();
coarse_settings.FFT_HighResolution_A(3.5f); // low-order reflections only -> robust long axis
// Widen the search bound too, and note this costs nothing: the histogram length is
// The resolution setting only sizes the FFT histogram - it does not filter which spots go
// into it - so this is here to pay for the widened bound below, not to make the long axis
// easier to pick out. Measured, coarsening does the opposite: the acceptance angle widens
// (0.159 -> 0.579 deg) but the true axis falls from rank 1996 to 16361 of 16384 directions.
coarse_settings.FFT_HighResolution_A(3.5f);
// The widened bound is what the rescue turns on, and it costs nothing: the histogram length is
// 4*pi*max_length/high_resolution, so coarsening the resolution to 3.5 A has already
// SHRUNK the transform by 1.75x - spending that back on reach leaves the coarse pass at
// 1.14x the standard one. Without it the rescue cannot recover what it exists to recover:
@@ -2108,16 +2083,11 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
}
// Second pass: compare the de-novo lattice with pass 1's HERE, before integrating every image
// with it. A markedly larger cell is the bistable supercell collapse, and a centring the group
// carried over from pass 1 cannot describe is the same disagreement seen from the symmetry side.
// Both were already rejected - the first in Run(), the second before the merge - but only after
// the whole pass had run, so each cost an entire extra pass on a dataset that ended up on pass-1's
// lattice anyway. Take pass-1's result now instead: the same remedy, one pass earlier. A triclinic
// de-novo cell is left alone, since that is the demotion the merge reindexes into the group's
// conventional setting.
// with it. A markedly larger cell is the bistable supercell collapse, which was already
// rejected in Run() - but only after the whole pass had run, so it cost an entire extra pass
// on a dataset that ended up on pass-1's lattice anyway. Take pass-1's result now instead:
// the same remedy, one pass earlier.
if (!geometry_prepass && prepass_result_.has_value()) {
const auto *reuse_sg = prepass_merge_sg_.has_value()
? gemmi::find_spacegroup_by_number(static_cast<int>(*prepass_merge_sg_)) : nullptr;
// Primitive volumes, like the scheme comparison above: a centred conventional cell is an
// exact integer multiple of its primitive one, so two settings of the SAME lattice differ
// by that factor and comparing the conventional cells reads a mere change of setting as a
@@ -2127,12 +2097,17 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
const double v2 = std::abs(best.result->lattice
.ToPrimitive(best.result->search_result.centering).CalcVolume());
const bool supercell = v1 > 1.0 && v2 > 1.5 * v1;
const bool wrong_centering = reuse_sg != nullptr
&& best.result->search_result.system != gemmi::CrystalSystem::Triclinic
&& best.result->search_result.centering != reuse_sg->centring_type();
if (supercell || wrong_centering) {
logger.Info("Two-pass: that lattice ({}-centred, volume {:.0f} A^3) disagrees with pass 1 "
"({}-centred, {:.0f} A^3) - integrating with pass-1's lattice instead",
// Dropping to the primitive sub-cell of pass-1's centred lattice is the same
// disagreement seen from the other side, and the volumes above cannot see it: the two
// have the SAME primitive metric. Integrating there never predicts the reflections the
// centring implies, and the pass then merges in P1 (measured on a C-centred monoclinic
// crystal). Only this direction - a centring pass 2 finds where pass 1 had none is the
// refined geometry doing its job, and the intensity test in the search confirms it.
const bool lost_centring = prepass_result_->search_result.centering != 'P'
&& best.result->search_result.centering == 'P';
if (supercell || lost_centring) {
logger.Info("Two-pass: that lattice ({}-centred, volume {:.0f} A^3) disagrees with "
"pass 1 ({}-centred, {:.0f} A^3) - integrating with pass-1's lattice instead",
best.result->search_result.centering, v2,
prepass_result_->search_result.centering, v1);
indexer->ForceRotationIndexerResult(*prepass_result_);
@@ -2564,26 +2539,6 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
return true;
};
// Two-pass second pass: if the de-novo indexer DEMOTED to a triclinic (primitive) cell while the reused
// merge space group is higher-symmetry (a huge oblique cell whose constrained centred refine was
// ill-posed, so the pseudo-symmetry guard kept the primitive), the reflections are in the primitive
// frame but the merge expects the group's CONVENTIONAL setting - reindex them into it (the same
// primitive->conventional change of basis the space-group search applies to a demoted result), else the
// merge folds the wrong equivalents. No-op when the indexer already produced the group's setting.
if (prepass_merge_sg_ && end_msg.rotation_lattice.has_value() && end_msg.rotation_lattice_type.has_value()
&& end_msg.rotation_lattice_type->crystal_system == gemmi::CrystalSystem::Triclinic) {
const auto *msg_sg = gemmi::find_spacegroup_by_number(static_cast<int>(*prepass_merge_sg_));
if (msg_sg && msg_sg->crystal_system() != gemmi::CrystalSystem::Triclinic) {
const auto cand = LatticeSearch(*end_msg.rotation_lattice);
// Only when LatticeSearch recovers the reused group's own metric (system + centring): otherwise
// the reused group and the indexed metric disagree and reindexing would be wrong - leave it.
if (cand.system == msg_sg->crystal_system() && cand.centering == msg_sg->centring_type()
&& reindex_into(cand))
logger.Info("Two-pass: reindexed the de-novo primitive cell into the space-group {} "
"conventional setting for the merge", static_cast<int>(*prepass_merge_sg_));
}
}
// A space group the USER fixed carries its own Bravais lattice, and that is the setting its
// reflections have to be indexed in - but the indexer answers to the metric, not to the group, and
// LatticeSearch hands back the MOST symmetric setting the metric supports. A tetragonal-P lattice
@@ -2627,65 +2582,45 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
}
}
// Two-pass second pass only: the indexer above ran DE NOVO (Run() cleared the space group so its
// pseudo-symmetry safeguards find the true cell); now reinstate pass-1's space group for the merge,
// exactly as a user -S would, so the symmetry is fixed without a re-search. No-op on pass 1 / single pass.
// The lattice a pass indexed has to carry the Bravais lattice of the group it merges in. Only a
// group the USER fixed can fail that - one the pass determined for itself was determined FROM
// this lattice - and the re-seating above has already had its chance to find a setting that
// does. What comes out otherwise is not a merely suboptimal answer: the absence rule is applied
// across a frame the reflections are not in, and the statistics stop being arithmetic (measured:
// 173.5% complete on a triclinic-P lattice merged in C2, and an undefined R_meas on an F-centred
// cubic one merged in a trigonal-P group).
//
// Only when the lattice pass 2 actually found HAS that group's centring. Re-indexing de novo can
// land in a different setting from pass 1 - most often the primitive sub-cell of a centred
// lattice, which is what the reindex above exists to undo, and which it declines to do when the
// metric does not match. Stamping a centred group onto a primitive cell is not a small error:
// the centring absence rule then removes half the reflections that genuinely exist, so the merge
// reports more unique reflections than its own cell can hold (measured: a C2 group on a cell of
// exactly half the C-centred volume, reported as 117% complete with CC1/2 0.62 against the first
// pass's 92.6% and 0.98). When they disagree, let pass 2 search for itself rather than trust a
// group that belongs to a different lattice.
if (prepass_merge_sg_ && end_msg.rotation_lattice_type.has_value()) {
const auto *reuse_sg = gemmi::find_spacegroup_by_number(static_cast<int>(*prepass_merge_sg_));
if (reuse_sg && end_msg.rotation_lattice_type->centering != reuse_sg->centring_type()) {
logger.Warning("Two-pass: the second pass indexed a {}-centred lattice, but the first pass "
"determined {}, whose lattice is {}-centred - the group cannot describe this "
"lattice, so this pass will not be adopted",
end_msg.rotation_lattice_type->centering, reuse_sg->xhm(),
reuse_sg->centring_type());
result.lattice_conflicts_with_prepass_sg = true;
}
}
// The same conflict, but for a group the USER fixed, and on a pass that has nothing to fall back
// on - the first of the two, or a single pass. The re-seating above has had its chance and no
// setting of the lattice this crystal indexes as carries the group's Bravais lattice, so there is
// no frame in which the merge means anything. What comes out is not a merely suboptimal answer:
// the absence rule is applied across a frame the reflections are not in, and the statistics stop
// being arithmetic (measured: 173.5% complete on a triclinic-P lattice merged in C2, and an
// undefined R_meas on an F-centred cubic one merged in a trigonal-P group). Nor is there anything
// to fall back ON - the group is the user's assertion, and quietly determining a different one
// would answer a question that was not asked. Refuse, and name the cell that WAS indexed so the
// user can act on it.
//
// The SECOND pass keeps the flag-and-do-not-adopt handling just above instead: its lattice comes
// from a de-novo re-index at the post-refined geometry, and the first pass - whose lattice did
// carry the group - is still there to go back to. Refusing there would throw away a good answer.
if (user_fixed_sg_ && !prepass_merge_sg_
&& end_msg.rotation_lattice_type.has_value() && end_msg.unit_cell.has_value()) {
// The two-pass second pass is flagged and not adopted rather than refused: its lattice comes from
// a de-novo re-index at the post-refined geometry, and the first pass - whose lattice did carry
// the group - is still there to go back to. A pass with nothing to fall back on refuses instead;
// the group is the user's assertion, and quietly determining a different one would answer a
// question that was not asked. Name the cell that WAS indexed so the user can act on it.
if (user_fixed_sg_ && end_msg.rotation_lattice_type.has_value() && end_msg.unit_cell.has_value()) {
const auto *fixed_sg = gemmi::find_spacegroup_by_number(static_cast<int>(*user_fixed_sg_));
if (fixed_sg && end_msg.rotation_lattice_type->centering != fixed_sg->centring_type()) {
const auto &uc = *end_msg.unit_cell;
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, fmt::format(
"The space group {} was fixed for this run, and its lattice is {}-centred - but "
"this crystal indexes as a {}-centred {} lattice (a={:.3f} b={:.3f} c={:.3f} "
"alpha={:.2f} beta={:.2f} gamma={:.2f}), and no setting of that lattice carries the "
"group's. Merging in {} would apply its absence rule to reflections that are not in "
"its frame, so nothing it reported would describe this crystal. Re-run without a fixed "
"space group to have it determined from the data, or fix one whose lattice this "
"crystal has.",
fixed_sg->xhm(), fixed_sg->centring_type(), end_msg.rotation_lattice_type->centering,
gemmi::crystal_system_str(end_msg.rotation_lattice_type->crystal_system),
uc.a, uc.b, uc.c, uc.alpha, uc.beta, uc.gamma, fixed_sg->xhm()));
if (prepass_result_) {
logger.Warning("Two-pass: the second pass indexed a {}-centred lattice, but the fixed "
"space group {} has a {}-centred lattice - the group cannot describe "
"this lattice, so this pass will not be adopted",
end_msg.rotation_lattice_type->centering, fixed_sg->xhm(),
fixed_sg->centring_type());
result.lattice_conflicts_with_fixed_sg = true;
} else {
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, fmt::format(
"The space group {} was fixed for this run, and its lattice is {}-centred - but "
"this crystal indexes as a {}-centred {} lattice (a={:.3f} b={:.3f} c={:.3f} "
"alpha={:.2f} beta={:.2f} gamma={:.2f}), and no setting of that lattice carries the "
"group's. Merging in {} would apply its absence rule to reflections that are not in "
"its frame, so nothing it reported would describe this crystal. Re-run without a fixed "
"space group to have it determined from the data, or fix one whose lattice this "
"crystal has.",
fixed_sg->xhm(), fixed_sg->centring_type(), end_msg.rotation_lattice_type->centering,
gemmi::crystal_system_str(end_msg.rotation_lattice_type->crystal_system),
uc.a, uc.b, uc.c, uc.alpha, uc.beta, uc.gamma, fixed_sg->xhm()));
}
}
}
if (prepass_merge_sg_)
experiment_.SpaceGroupNumber(*prepass_merge_sg_);
const auto &rot_ss = experiment_.GetScalingSettings();
const bool is_rotation = experiment_.IsRotationIndexing(); // rotation indexing -> rotation scaling/merge
@@ -2809,12 +2744,22 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
const auto initial_sg = experiment_.GetGemmiSpaceGroup();
auto sm = scale_and_merge(initial_sg ? initial_sg->short_name() : "P1", !initial_sg.has_value());
// The two-pass quality guard judges the second pass against the first on THIS merge, which both
// passes make in the same terms - P1 (or the group the user fixed), full range, no correction
// surfaces - where their final merges need not even be in the same group. See RunAllPasses.
{
const auto &o = sm.statistics.overall;
result.search_merge_completeness = o.possible_unique_reflections > 0
? 100.0 * o.unique_reflections / o.possible_unique_reflections : 0.0;
result.search_merge_cc_half = sm.cc_half_before_corrections;
}
// Rotation two-pass geometry pre-pass: this first scale/merge has now fitted a frame-order-smoothed
// mosaicity and written it back onto the per-frame outcomes. Capture it for the second pass's Bragg
// prediction (so prediction uses the smoothed value, not a fresh per-image estimate) and post-refine +
// apply the detector geometry from this integration. The pre-pass then CONTINUES through the space-
// group search (below) so the second pass can reuse the same space group and lattice - re-searching at
// the slightly changed geometry could flip a borderline determination and mix the two passes.
// apply the detector geometry from this integration. That is all the pre-pass is for: everything
// the second pass consumes from it is fitted here, and the space group is left to the second pass
// (see the search gate below).
if (geometry_prepass && rsm && experiment_.IsRotationIndexing()) {
const auto &outcomes = indexer->GetIntegrationOutcome();
prepass_mosaicity_.assign(outcomes.size(), NAN);
@@ -2864,8 +2809,20 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
// weak-reflection profile-fit runaway) has an astronomical resolution-normalised E and wrecks the
// intensity-correlation / second-moment statistics. The final in-symmetry merge keeps the full
// range (only this P1 search pass is cut); a manual --scaling-high-resolution, if coarser, wins.
// The geometry pre-pass does not determine the space group. Its integration is at the header
// geometry, which is the thing the second pass exists to improve on, and the symmetry evidence
// moves with the geometry: measured on a rotation crystal, 0.33 mm of distance and 0.65 px of
// beam centre took the screw-axis evidence from -19 to +311 nats, where merging the same
// integration in symmetry instead of P1 moved it 2.5. Deciding on the worse integration also
// fails in one direction only - every statistic the search weighs is a ratio of the candidate
// merge's scatter to the subgroup's, so poorly-integrated data inflate them and REFUSE real
// symmetry (corpus-wide the misses run ten under-calls to one over-call). So the search runs on
// the canonical pass, at the refined geometry, and the pre-pass skips it along with the
// in-symmetry re-merge that only exists to serve it.
const bool search_space_group = !geometry_prepass && !initial_sg.has_value();
double d_min_search = 0.0;
if (!initial_sg.has_value()) {
if (search_space_group) {
std::vector<std::pair<float, float>> rs; // (d, I/sigma) over the full P1 merge
rs.reserve(sm.merged.size());
for (const auto &m : sm.merged)
@@ -2911,10 +2868,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
// True when the point group below was chosen BY THE SEARCH (rather than given by the user) and is
// above triclinic - i.e. a promotion was made, which is what makes a later "the Laue class is
// holohedral so there is no twin law" conclusion circular.
// Pass 2 reuses pass-1's group instead of searching, so the promotion it needs to report was
// made there; a first / single pass has no carried-over group and decides this below.
bool promoted_point_group = prepass_merge_sg_.has_value() && prepass_promoted_point_group_;
if (!experiment_.GetGemmiSpaceGroup().has_value()) {
bool promoted_point_group = false;
if (search_space_group) {
SearchSpaceGroupOptions sg_opts;
sg_opts.nthreads = static_cast<size_t>(std::max(1, config_.nthreads));
sg_opts.merge_friedel = experiment_.GetScalingSettings().GetMergeFriedel();
@@ -3179,22 +3134,12 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
} else {
// A space group was fixed by the user; surface it so the viewer/CLI can still show it.
result.space_group_number = experiment_.GetSpaceGroupNumber();
// ...and record it, exactly as the search arm above does with the group it determined.
// This arm also carries the two-pass second pass, which REUSES the first pass's group
// rather than re-searching, so without this the _process.h5 of every two-pass rotation run
// came out with a cell but no group - and re-merging it defaulted to P1.
// ...and record it, exactly as the search arm above does with the group it determined, or
// the _process.h5 comes out with a cell but no group and re-merging it defaults to P1.
if (const auto sg = experiment_.GetSpaceGroupNumber(); sg.has_value())
end_msg.space_group_number = static_cast<uint64_t>(*sg);
}
// Rotation two-pass geometry pre-pass: the determined space group is now fixed on experiment_, so the
// second pass REUSES it instead of re-searching at the slightly changed geometry (which could flip a
// borderline determination and mix the two passes). The lattice is deliberately NOT forced - the second
// pass re-indexes at the refined geometry so the CELL comes out self-consistent with it (forcing the
// pre-pass lattice would pin the nominal cell and undo the geometry refinement). Unlike a throwaway
// pre-pass, this one now CONTINUES to the write below so its own (header-geometry) result is saved
// as the first pass.
// Reference-based indexing-ambiguity resolution (rotation). When a reference MTZ is supplied and
// the crystal has an indexing ambiguity (merohedral: lattice symmetry higher than the Laue group),
// pick the reindexing that best correlates with the reference intensities and re-merge in it. The
rugnux/Rugnux.h
+17 -23
View File
@@ -195,17 +195,24 @@ struct ProcessResult {
// measurement on both sides of the comparison.
double cc_half_before_corrections = NAN;
// The same two numbers for the pass's FIRST merge - P1 when the pass searches for a space group,
// otherwise the group the user fixed - which is what the rotation two-pass quality guard compares.
// The final merges of the two passes can be in different groups, where completeness and CC1/2 are
// no longer the same measurement; this merge is in the same terms on both sides.
double search_merge_completeness = 0.0;
double search_merge_cc_half = NAN;
// Per-reflection (I, sigma) of the final merged reflections, for the ISa diagnostic: I/sigma
// plotted against I flattens off at the asymptote the error model reports as ISa, so the plot shows
// whether that number describes the data. Strided down to a few thousand points - this is a shape,
// not a reflection list, and the reflections themselves are in the .mtz/.cif. Empty when no merge ran.
std::vector<std::pair<float, float>> merged_i_sigma;
// Two-pass second pass: the lattice this pass indexed does not have the centring of the space
// group the first pass determined - most often because it landed on the primitive sub-cell of a
// centred lattice. The group cannot describe this lattice, so the pass is not trustworthy whatever
// its statistics say. Always false on a first / single pass.
bool lattice_conflicts_with_prepass_sg = false;
// Two-pass second pass: the lattice this pass indexed does not have the centring of the space group
// the USER fixed - most often because it landed on the primitive sub-cell of a centred lattice. The
// group cannot describe this lattice, so the pass is not trustworthy whatever its statistics say.
// Always false on a first / single pass (which refuses outright) and when no group was fixed.
bool lattice_conflicts_with_fixed_sg = false;
// Twinning analysis of the final merged intensities (l_test_pairs == 0 when not computed).
TwinningAnalysisResult twinning;
@@ -299,27 +306,14 @@ class Rugnux {
// (RunPipeline calls indexer->ForceRotationIndexerResult when this is set); reset right after that re-run.
std::optional<RotationIndexerResult> force_rotation_result_;
// Two-pass second pass: pass-1's space group, reinstated for the MERGE ONLY (RotationScaleMerge) after the
// indexer has run de novo. The indexer runs with no fixed group so its pseudo-symmetry safeguards recover
// the true cell (reusing the group in the indexer forces build_sr onto a wrong / doubled cell); the merge
// then folds equivalents in pass-1's group so the symmetry stays fixed without a re-search. Empty outside
// the second pass, and for a reindex-derived group (left to re-search).
std::optional<int64_t> prepass_merge_sg_;
// The space group the USER fixed (-S on the CLI, the settings panel in the viewer), captured before
// anything can clear it - the two-pass driver blanks the experiment's group between the passes so the
// second pass re-indexes de novo, so after that point the experiment can no longer say whether the
// group in force was asserted or determined. A group the user asserted is treated differently from
// one the run determined: it is re-seated onto its own Bravais lattice if the metric has that
// setting, and the run refuses rather than merging in it if the metric does not.
// anything can clear it - a pass that determines a group of its own writes it onto the experiment,
// so after that point the experiment can no longer say whether the group in force was asserted or
// determined. A group the user asserted is treated differently from one the run determined: it is
// re-seated onto its own Bravais lattice if the metric has that setting, and the run refuses rather
// than merging in it if the metric does not. Each pass is also restored to it before it starts.
const std::optional<int64_t> user_fixed_sg_;
// Whether the group in prepass_merge_sg_ was PROMOTED by pass-1's search rather than given. Pass 2
// reuses the group without searching, so it cannot work this out for itself - and without it the
// canonical output reports "the Laue class is holohedral, so no twin law exists" about a Laue class
// the search chose, which is the circular claim that reasoning exists to flag.
bool prepass_promoted_point_group_ = false;
// Whether the beam centre has already been placed by a measurement the pre-scan cannot better:
// a stills bundle adjustment, or an earlier pre-scan of this same run. The pre-scan estimate is