From 3b06bcee78c3541dd394016c79d5165a5921347b Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sun, 25 Oct 2026 23:57:56 +0100 Subject: [PATCH] Refinement: the trajectory ends at its best step, steps try the positional weight halved too, the final ladder must win Three changes to how the trajectory of commit 2bfb84bc4 moves and ends: - A step is taken while its trial lowers R-free or R-work by more than the tie and its R-free stays within the tie of the best so far; the trajectory then ends at its best step. On a deposited model, whose free set it was fitted to, R-work falls while R-free rises, and the trajectory now ends where it began instead of walking on while R-work falls. - Positional steps try the weight halved as well as doubled: at a fixed weight the geometry loosens as the model improves (the X-ray gradient grows with D) until the trials leave the bounds, which stopped 8dqb's trajectories at 2 A (angles > 2.0 deg at w0/128 by step 6). - The final ladder's choice replaces the trajectory's end only where it displaces it (geometry within bounds, R-free lower by more than the tie); before, its first rung was taken by default, two more cycles of movement that on deposited models cost R-free. Deposited models (14 parity sets), R-free after refinement vs refine-combined: mean +0.0058 -> +0.0016 (e.g. 7n2s +0.012 -> +0.003, 9rcs +0.013 -> -0.008, 7d98 +0.015 -> +0.003, 9yl4 +0.010 -> +0.003), the time halved against refine-combined. MD starts, median R-free (REFMAC 0-cycle scorer), previous commit -> this: 5f6m 0.167 -> 0.168, 8dqb 0.294 -> 0.297, 7arr 0.278 -> 0.290 (REFMAC 0.311), 11if 0.244 -> 0.245, 5reo 0.258 -> 0.259, 7d98 0.335 -> 0.335. Determinism: run A == run B == --trials-at-once 1 on 5reo, 9rcs and a 5f6m MD start. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit c2b55aab1aaff637d9484dec0d25003b5c4a3857) --- docs/RUGNUX_MODEL_AND_REFERENCE.md | 10 +++-- .../structure_refinement/Refinement.cpp | 44 +++++++++++++------ 2 files changed, 37 insertions(+), 17 deletions(-) diff --git a/docs/RUGNUX_MODEL_AND_REFERENCE.md b/docs/RUGNUX_MODEL_AND_REFERENCE.md index e45ceb222..6969e165c 100644 --- a/docs/RUGNUX_MODEL_AND_REFERENCE.md +++ b/docs/RUGNUX_MODEL_AND_REFERENCE.md @@ -246,9 +246,13 @@ lists them. The dictionary's hydrogens are added and ride on their parents. Meta donor atoms (O, N, S) the model's LINK / `_struct_conn` records bind them to: by the dictionary's own Zn and Fe links to Cys and His where they apply, otherwise at the library's typical distance for the metal, the donor element and the number of such records of the ion (Mg-O 2.07 A, Na-O 2.42 A, K-O 2.82 A at their -usual coordination). A contact without a record, or one to a symmetry mate, is not restrained as a bond. The X-ray weight is chosen from a -ladder of complete refinements by R-free, among those whose bond and angle rmsd stay within 0.02 A and 2 -degrees; B-factor refinement is then tried at a few weights and kept only where it lowers R-free beyond +usual coordination). A contact without a record, or one to a symmetry mate, is not restrained as a bond. The B factors of bonded atoms and of +atoms in contact (waters included) are restrained to each other in proportion to their size. The weights of +the X-ray term against the geometry and against that B restraint are chosen by R-free among short +refinements whose bond and angle rmsd stay within 0.02 A and 2 degrees: three weights from the placed +model, each with the B factors held and refined; then a trajectory of further short refinements from the +best, each trying the weights as they are and changed, taken while R-free or R-work falls beyond 0.001 and +R-free stays within 0.001 of its best; then a short final ladder, kept only where it lowers R-free beyond 0.001. The free set is the merge's own, so R-free after `--refine` judges the refinement against reflections it never fitted - but where the model was deposited, those reflections were most likely its depositor's working reflections, and a refinement against new data then moves R-free up from the unrefined value rather than diff --git a/image_analysis/structure_refinement/Refinement.cpp b/image_analysis/structure_refinement/Refinement.cpp index 153988e24..d89692ff7 100644 --- a/image_analysis/structure_refinement/Refinement.cpp +++ b/image_analysis/structure_refinement/Refinement.cpp @@ -1327,8 +1327,13 @@ RefinementResult RefineModel(const gemmi::Structure &placed, const std::vector &outcomes, const char *stage) { int best = -1; for (size_t k = 0; k < outcomes.size(); k++) { @@ -1337,8 +1342,7 @@ RefinementResult RefineModel(const gemmi::Structure &placed, const std::vectorwithin_bounds || t.r_free < b->r_free - R_FREE_TIE))) + if (best < 0 || displaces(o.trial, outcomes[best].trial)) best = static_cast(k); } return best; @@ -1359,29 +1363,41 @@ RefinementResult RefineModel(const gemmi::Structure &placed, const std::vector> trials = {{w, adp}}; if (step % 2 == 1 && adp > 0) { trials.emplace_back(w, adp / 4); trials.emplace_back(w, 4 * adp); - } else if (2 * w <= w_max) - trials.emplace_back(2 * w, adp); + } else { + trials.emplace_back(w / 2, adp); + if (2 * w <= w_max) + trials.emplace_back(2 * w, adp); + } outcomes = RunTrials(trial_slots, trials, refinable, state.pos, state.b, &state.fit, settings.macro_cycles, settings); k = choose(outcomes, fmt::format("step {}", step + 1).c_str()); if (k < 0) break; - const bool converged = state.trial.r_free - outcomes[k].trial.r_free < R_FREE_TIE && - state.trial.r_work - outcomes[k].trial.r_work < R_FREE_TIE; - state = std::move(outcomes[k]); - if (converged) + const RefinementWeightTrial &t = outcomes[k].trial; + const bool gains = displaces(t, state.trial) || (t.within_bounds && t.r_work < state.trial.r_work - R_FREE_TIE); + if (!gains || !(t.within_bounds && t.r_free < best.trial.r_free + R_FREE_TIE)) break; + state = std::move(outcomes[k]); + if (displaces(state.trial, best.trial)) + best = state; } + state = std::move(best); - // 3. The final weights: a short ladder from where the trajectory ended, tighter geometry first. + // 3. The final weights: a short ladder from where the trajectory ended, tighter geometry first, its choice + // kept where it displaces that. if (state.ran && !ctx.Cancelled()) { const double w = state.trial.weight, adp = state.trial.adp_weight; std::vector> trials; @@ -1391,7 +1407,7 @@ RefinementResult RefineModel(const gemmi::Structure &placed, const std::vector= 0) + if (k >= 0 && displaces(outcomes[k].trial, state.trial)) state = std::move(outcomes[k]); } for (size_t s = 1; s < trial_slots.size(); s++)