mx/Jungfraujoch
3
2 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 88 Wiki Activity

RESULT-CHANGING: Refinement: displacement trials on the positions held, the model chosen then a step with them

Browse Source 
Each displacement-model trial (TLS cuts, the model's own groups, U at two weights) was a whole refinement:
the positions (30 iterations a cycle), the displacement parameters, a macro-cycle longer than a step. The
positions were 43% of a TLS trial's evaluations and 60% of a U trial's, and the third cycle moved R-free by
~0 on the traced battery. Now the trials refine the displacement parameters alone, on the positions where the
weights ended, for as long as a step - so they differ in the displacement model only - and the model chosen
then takes one step with the positions refined, kept where it displaces its trial.

Held positions alone lost where a displacement model is a large change (a TLS model 0.044 below the state:
+0.009 R-free; a U model 0.037 below: +0.003); the step with the positions gives that back.

Battery subset against rc181 297ef9dc7 (28 sets, 50 refinements, refine arm and 4YBB included):
R-free mean -0.0011, 13 lower by more than 0.002, none higher by more than 0.0015 (worst +0.0015, +0.0007 on
one refine set); refine arm -0.0016 / 0 / +0.0007 / -0.0009 / -0.0002, 4YBB -0.0004 with the same model;
target evaluations -22%. 4YBB displacement phase on one RTX 5080: 1431 -> 509 evaluations for the trials, plus
one step of the model chosen. Trial placement and order are unchanged; the outcome is the same for any number
of slots and cards (Refinement_trials_run_ahead passes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-29 02:32:43 +01:00
1 parent 297ef9dc78
commit 365e496c0e
2 files changed
+34 -13
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/CPU_DATA_ANALYSIS_REFINEMENT.md
+3 -1
View File
@@ -308,7 +308,9 @@ are:
3. **The final ladder**: the trajectory's weights times $\{1, 2\}$, kept where one displaces it (the weight halved
was never kept on 55 refinements, and taking it away changed no choice).
4. **The displacement model**, from where the weights ended (the B factors as they are: isotropic, or the model's U
held), each trial a macro-cycle longer than a step: TLS groups with a residual B - the model's own, one per chain
held), each trial as long as a step on the positions where the weights ended, held - only the displacement
parameters refine, so the trials differ in the displacement model alone; the model chosen then takes a step with
the positions refined, kept where it displaces its trial: TLS groups with a residual B - the model's own, one per chain
of at least 50 refinable atoms, and the chains cut into 2, 4, 8 and 16 contiguous segments of at least 10 residues
as TLSMD cuts them (by dynamic programming over the least-squares fit of an isotropic TLS model, a quadratic B(r),
to the refined B factors), each cut where it gives more groups than the one before; and a U per atom, at the U
image_analysis/structure_refinement/Refinement.cpp
+31 -12
View File
@@ -1562,7 +1562,7 @@ struct TrialOutcome {
TrialOutcome RunTrial(TrialSlot slot, const std::vector<int> &refinable, const std::vector<gemmi::Position> &pos0,
const std::vector<double> &b0, const std::vector<AtomAniso> &aniso0, const Context::Fit *fit,
double w, double adp_w, int cycles, const RefinementSettings &settings, const NcsCopies *ncs,
double ncs_sigma, const AdpHypothesis *hyp = nullptr) {
double ncs_sigma, const AdpHypothesis *hyp = nullptr, bool hold_positions = false) {
Context &ctx = *slot.ctx;
RefinementTopology &topo = *slot.topo;
const RefinementStage stage(hyp != nullptr ? "rung.adp" : adp_w > 0 ? "rung.b" : "rung.xyz");
@@ -1614,17 +1614,17 @@ TrialOutcome RunTrial(TrialSlot slot, const std::vector<int> &refinable, const s
}
if (adp_w > 0)
ctx.MoveOverallB();
{
const RestraintSet set = RestraintsNow(ctx, topo, ncs, ncs_sigma);
const RefinementStage s1("set_restraints");
ctx.engine->SetRestraints(0, set);
}
{
if (!hold_positions) {
{
const RestraintSet set = RestraintsNow(ctx, topo, ncs, ncs_sigma);
const RefinementStage s1("set_restraints");
ctx.engine->SetRestraints(0, set);
}
const RefinementStage s2("minimise.xyz");
Minimise(ctx, xyz, settings.xyz_iterations, xyz_minimiser);
}
if (kind == AdpHypothesis::Kind::Tls) {
ctx.ApplyTls(); // the bodies' U at where the atoms moved
ctx.ApplyTls(); // the bodies' U at the atoms as they are
const RefinementStage s4("minimise.tls");
Minimise(ctx, tls_target, settings.b_iterations, tls_minimiser);
}
@@ -1721,7 +1721,8 @@ std::vector<TrialOutcome> RunTrials(const std::vector<TrialSlot> &trial_slots,
const std::vector<int> &refinable, const std::vector<gemmi::Position> &pos0,
const std::vector<double> &b0, const std::vector<AtomAniso> &aniso0,
const Context::Fit *fit, int cycles, const RefinementSettings &settings,
const NcsCopies *ncs, const std::vector<const AdpHypothesis *> &hyps = {}) {
const NcsCopies *ncs, const std::vector<const AdpHypothesis *> &hyps = {},
bool hold_positions = false) {
std::vector<TrialOutcome> out(trials.size());
std::vector<std::exception_ptr> errors(trial_slots.size());
// the threads shared among the slots that run (the outcome does not depend on how many each has)
@@ -1747,7 +1748,8 @@ std::vector<TrialOutcome> RunTrials(const std::vector<TrialSlot> &trial_slots,
for (size_t j = s; j < trials.size(); j = next++)
if (const size_t i = order[j]; !trial_slots[s].ctx->Cancelled())
out[i] = RunTrial(trial_slots[s], refinable, pos0, b0, aniso0, fit, trials[i].w, trials[i].adp,
cycles, settings, ncs, trials[i].ncs, hyps.empty() ? nullptr : hyps[i]);
cycles, settings, ncs, trials[i].ncs, hyps.empty() ? nullptr : hyps[i],
hold_positions);
} catch (...) {
errors[s] = std::current_exception();
}
@@ -2889,7 +2891,11 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
}
// 4. The displacement model, decided late, on the free set: from where the weights ended, TLS bodies (the
// model's own groups, one per chain, the chains in TLSMD segments) with a residual B, and a U per atom at
// its balanced weight and a quarter of it - each trial a macro-cycle longer than a step. One displaces the one before it (the first
// its balanced weight and a quarter of it - each trial as long as a step, on the positions where the weights
// ended, held, so that the trials differ in the displacement model alone; the model chosen then takes a
// step with the positions refined, kept where it displaces its trial (against trials a macro-cycle longer
// that refined the positions too, on 50 refinements of the battery R-free 0.001 lower on average and none
// higher by more than 0.0015, at 78% of the evaluations). One displaces the one before it (the first
// where the weights ended, the B factors as they are) only where its geometry is within bounds and both its
// R-free (beyond the tie) and the -ln L of the free reflections are lower: the free set is the test of the
// parameters it adds.
@@ -2972,7 +2978,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
if (t.empty())
return;
std::vector<TrialOutcome> ran = RunTrials(trial_slots, t, refinable, state.pos, state.b, state.aniso,
&state.fit, settings.macro_cycles + 1, settings, ncs_p, h);
&state.fit, settings.macro_cycles, settings, ncs_p, h, true);
for (size_t k = 0, m = 0; k < trials.size(); k++)
if (these[k])
outcomes[k] = std::move(ran[m++]);
@@ -3023,6 +3029,19 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
if (best >= 0) {
result.adp_model = outcomes[best].trial.adp;
state = std::move(outcomes[best]);
// the model chosen refined on with the positions, a step from where its trial ended, kept where it
// displaces that
AdpHypothesis chosen = *which[best];
chosen.bodies = state.tls;
chosen.tls_b = state.tls_b;
std::vector<TrialOutcome> on = RunTrials(trial_slots, {trials[best]}, refinable, state.pos, state.b,
state.aniso, &state.fit, settings.macro_cycles, settings,
ncs_p, {&chosen});
if (on[0].ran) {
logger.Info("Refinement, displacements, positions refined: {}", on[0].line);
if (displaces(on[0].trial, state.trial))
state = std::move(on[0]);
}
}
logger.Info("Refinement: displacement model {}", result.adp_model);
}