Merge branch 'rc188-f' into rc188-int

This commit is contained in:
leonarski_f committed 2026-11-18 16:10:22 +01:00
commit 96cd005b21
6 files changed
+371 -165

No files matched your search

+17 -1
View File
@@ -652,6 +652,16 @@ structure refinement with phenix.refine" (2012), Acta Cryst. D68, 352-367
A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal
structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0).
**What the free set decides** — the free set decides the weights and the displacement model alone, and only
where its two measures agree: a candidate is taken where its free $-\ln L$ is lower and its R-free not higher
than the incumbent's by more than the paired standard error of the difference; every other step of the
refinement is taken on the work-set target. The approach of deciding on more than one free measure, and of keeping the
free set for few choices, is PDB-REDO's: R. P. Joosten, K. Joosten, G. N. Murshudov and A. Perrakis,
"PDB_REDO: constructive validation, more than just looking for errors" (2012), Acta Cryst. D68, 484-496
[doi:10.1107/S0907444911054515](https://doi.org/10.1107/S0907444911054515), and R. P. Joosten, F. Long,
G. N. Murshudov and A. Perrakis, "The PDB_REDO server for macromolecular structure model optimization" (2014),
IUCrJ 1, 213-220 [doi:10.1107/S2052252514009324](https://doi.org/10.1107/S2052252514009324).
**Robust scaling of the model** — the overall scale and anisotropic B are fitted by iteratively
reweighted least squares with the pseudo-Huber weight. P. J. Huber, "Robust estimation of a location
parameter" (1964), Ann. Math. Statist. 35, 73-101
@@ -692,7 +702,13 @@ SHELXL" (2015), Acta Cryst. C71, 3-8
[doi:10.1107/S2053229614024218](https://doi.org/10.1107/S2053229614024218) documents them. Which
displacement model a refinement keeps is decided on the free set rather than by a rule of
resolution: E. A. Merritt, "To B or not to B: a question of resolution?" (2012), Acta Cryst. D68,
468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320).
468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320). Between a model
with more displacement parameters and one with fewer, the R-factor ratio test of W. C. Hamilton, "Significance
tests on the crystallographic R factor" (1965), Acta Cryst. 18, 502-510 (DOI not verified), decides first,
over a range of restraint weights as Merritt (2012) applies it, with PDB-REDO's (Joosten et al. 2012, above)
95%/30% rule and its bound of 18 reflections per atom for a U per atom; its F point is from E. Paulson,
"An approximate normalization of the analysis of variance distribution" (1942), Ann. Math. Statist. 13,
233-235 [doi:10.1214/aoms/1177731608](https://doi.org/10.1214/aoms/1177731608).
**Population and crossover** — the members' refinement on blurred data is a continuation (homotopy) on the
resolution, the deformation of the landscape of L. Piela, J. Kostrowicki and H. A. Scheraga, "On the
+1 -1
View File
@@ -18,7 +18,7 @@ This document describes the crystallographic algorithms implemented in Jungfrauj
12. auxiliary statistics (Wilson plot, ⟨I/σ(I)⟩, CC1/2, CCref),
13. amplitude estimation (French–Wilson) and R-free test-set flagging,
14. optional model-based validation: rigid-body placement of a supplied model (molecular replacement where the rigid body does not place it, or for a predicted model), R-free against it, sigma_A-weighted 2mFo−DFc / mFo−DFc electron-density maps, and an anomalous difference map with the strongest anomalous sites named.
15. restrained refinement of the placed model (wherever a model is placed): rigid bodies and group B, then positions and B against a maximum-likelihood amplitude target with the dictionary's geometry, the X-ray weight chosen by R-free within geometry bounds, and the unexplained blobs of the refined difference map.
15. restrained refinement of the placed model (wherever a model is placed): rigid bodies and group B, then positions and B against a maximum-likelihood amplitude target with the dictionary's geometry, the X-ray weight and the displacement model chosen on the free set (free -ln L and R-free agreeing) within geometry bounds, every other step taken on the work-set target, and the unexplained blobs of the refined difference map.
The reference is split into five parts, in pipeline order; the section numbers run continuously
+61 -33
View File
@@ -42,8 +42,8 @@ no twin law is tested, the data carrying the larger symmetry exactly. The refine
it was refined in. An atom's anisotropic U (ANISOU, `_atom_site_anisotrop`) is kept. Where the model has TLS
groups (`_pdbx_refine_tls*`, PDB `REMARK 3`) whose atoms bring no U of their own, their U is made from the
groups, and the B column is read either as the total B (as phenix.refine writes it) or as the residual to the
groups (as REFMAC writes it) - whichever of the two, and of the isotropic B as it stands, scores the lowest
R-free on the model as given (`REFINE_ADP_INPUT`). Occupancies are kept as they are, and
groups (as REFMAC writes it) - whichever of the two, and of the isotropic B as it stands, the free set prefers on
the model as given (§15.7; `REFINE_ADP_INPUT`). Occupancies are kept as they are, and
so are alternative conformations, each refined with its own restraints.
Two stages follow.
@@ -51,12 +51,12 @@ Two stages follow.
1. **Placement.** Each chain of at least 50 non-water atoms is moved as a rigid body - an angle-axis
rotation about its centroid, scaled by its rms radius so that a unit of rotation moves a typical atom by
about an angstrom, and a translation - against the X-ray target (§15.2), by Ceres' L-BFGS for 30
iterations. Then one B shift per residue. Each of the two is **kept only where it lowers R-free by more
than 0.001**: a deposited model was refined atom by atom already, and moving its chains or shifting its
B factors residue by residue then only makes it worse. The placement stage needs no dictionary.
iterations. Then one B shift per residue. Each of the two is **kept only where it lowers the $-\ln L$ of the
working reflections**, the target it minimised: the free set decides the weights and the displacement model
alone (§15.7). The placement stage needs no dictionary.
2. **Restrained refinement.** Every atom's position and isotropic B, against the X-ray target and the
geometry of the dictionary (§15.4-§15.5) - the built-in one with the user's restraint files over it
(`--restraints`) - in weight trials chosen by R-free (§15.7). Residues the
(`--restraints`) - in weight trials chosen on the free set (§15.7). Residues the
dictionary does not describe, and residues with an atom it does not know, are **held fixed**: they stay
in the structure factors, in the bulk-solvent mask and in the repulsions the refined atoms see, but
they do not move. So do atoms of zero occupancy.
@@ -344,29 +344,52 @@ $w_B\,T_\mathrm{X\text{-}ray} + E_B$ for the B factors. Rather than fixing $w$ a
runs **short refinements at several weights and keeps the one that does best on the free set** - a weight
chosen late, on the evidence, instead of early at a rule.
**What the free set decides.** The free set decides the weights - the X-ray weights of the positions and of the B
factors and the NCS restraints' $\sigma$ - the displacement model (§15.7, 6.), the reading of the model's TLS groups
(§15.1) and whether the model as given is kept over the refinement (7.), and nothing else. Each of these choices asks
both of its measures: a candidate is preferred to the incumbent only where its $-\ln L$ of the free reflections is
lower **and** its R-free is not higher by more than the standard error of the difference. The two are scored on the
same free reflections, so the difference is paired: R-free is a sum of one term per free reflection (per free twin
group under a twin law, whose members share their $F_\mathrm{calc}$), and the standard error is that of the sum of
the terms' differences - two trials from one start differ far less from reflection to reflection than either
scatters about the truth, so this is narrower than the error of one R-free, $R_\mathrm{free}/\sqrt{2N_\mathrm{free}}$.
One measure alone, or a fixed tie, picks the noise of a few hundred reflections as often as a better model (PDB-REDO
likewise takes a change only where its free measures agree, Joosten et al. 2014). $-\ln L$ free is that of the
refinement's own likelihood parameters - the variance fitted to the free reflections; with D and the variance fitted
to the working reflections instead, it kept out a per-atom U that overfits at 2.2 Å, but held back every
refinement (on 17 structures R-free 0.0016 and R-work 0.008 higher on average), and it is not used. Every other step - a step of the
trajectory, a round, the placement's rigid bodies and B per residue, the solvent build's rounds, the side-chain
field, the population's child - is taken on the **work-set target**: the target the trial minimises,
$w\,T_\mathrm{X\text{-}ray} + E_\mathrm{geometry}$ (with the NCS restraints at their $\sigma$), plus
$w_B\,T_\mathrm{X\text{-}ray} + E_B$ where the B factors are refined, $T_\mathrm{X\text{-}ray}$ the $-\ln L$ of the
working reflections - at the weights the two candidates share, and taken where it lowers that by more than
$10^{-4}$ of it, relative, and the geometry stays within bounds where it was. A step at other weights (X-ray weights
or NCS $\sigma$) is not compared on a target that changed with them - the positions and the B factors minimise
targets of their own, whose sum a change of weight can raise while it lowers each - and the free set decides it as
it decides every weight. The free set thus tests a
handful of choices, each between a few candidates, and the R-free reported is not the best of the many steps the
refinement took.
The starting weights balance the gradients: $w_0 = |\nabla E_\mathrm{geometry}| / |\nabla T_\mathrm{X\text{-}ray}|$,
the geometry's gradient taken on the model shaken by 0.1 Å (a refined model's geometry would give none), and
$w_{B,0}$ likewise from the B restraints on the B factors shaken by 20%. Each **trial** is two macro-cycles
(the scale, solvent and likelihood refitted, then the positions, then the B factors) from where it starts.
A trial is **within bounds** where its rmsd from the dictionary is at most 0.02 Å for bonds and 2° for
angles; one trial **displaces** another where it is within bounds and its R-free is lower by more than 0.001
(differences smaller than that do not decide, and the trial listed first - on a ladder, the tighter
geometry - is kept). The trials
are:
angles; one trial **displaces** another where it is within bounds and the free set prefers it as above (the trial
listed first - on a ladder, the tighter geometry - is kept otherwise). The trials are:
1. **The ladder from the placed model**: $w = w_0 \times \{1/1024, 1/128, 1/16\}$, each with the B factors
held and refined at $w_{B,0} \times \{1/16, 1/4, 1\}$ - twelve trials (the B factors are refined first at
the middle rung only, and at the others where that comes within two ties of holding them there). The lower
the middle rung only, and at the others unless holding them there displaces refining them). The lower
B weights are for a model whose B factors arrive far from where they refine - a prediction's pLDDT - whose
balanced weight is not the one its B wants: they took 0.002-0.010 off R-free on five such starts, and
nothing off eight deposited models.
2. **The trajectory**: up to eight steps, each a set of trials from where the last step ended - the weights
as they are, and in turn the positions' weight halved and doubled (halved because the geometry loosens at
one weight as the model improves, until it leaves the bounds) or the B factors' weight divided and
multiplied by four. A step is taken while it lowers R-free or R-work by more than 0.001 and its R-free
stays within 0.001 of the best so far, and the trajectory ends at its best step. On a deposited model,
whose free set the depositor's refinement has already fitted, R-work falls while R-free rises, and the
trajectory ends where it began.
multiplied by four. The step's weights are chosen among its trials on the free set; the step is taken while
that trial lowers the work-set target (or, at other weights, displaces where the last step ended), and the
trajectory ends where the last step taken ended.
3. **The population** (off by default, `RefinementSettings::population`: with the minimiser of §15.6 its members
took nothing off R-free over 19 refinements that the second round did not), from where the trajectory ended:
up to twelve members (at most as many as the trial slots, so that they run in one wave; none below two) refined
@@ -380,8 +403,8 @@ are:
members see those around it. Their **crossover** is made on one 2mFo-DFc map averaged over the members with
the free reflections left out: each residue is taken from the member whose atoms sit in the most density,
the junctions are mended by a macro-cycle at the ladder's tightest rung, and the child is refined by one
trial at the trajectory's weights. The child displaces the state by the rule every trial is held to; the
members themselves are never chosen among on the free set, whose best of twelve would be the best of twelve. On
trial at the trajectory's weights. The child is taken where it lowers the work-set target, as a step is; the
free set never sees the members. On
13 predictions placed by MR the child was kept on four (R-free 0.005-0.011 lower) and on one of 22 refined
models (0.009 lower, the rest unchanged); the gain, where there is one, comes mostly in the rounds that
follow from the child.
@@ -402,14 +425,15 @@ are:
sweep would otherwise take more than $4 \times 10^8$ evaluations (states times reflections, about 3 s on one
card). An FFT gradient cannot carry a side chain 1.5 Å at 1.7 Å
resolution; the field, a search, can, and it is scored by the likelihood rather than by a map, which on an
incomplete model rewards whatever fills unmodelled density. The free reflections are never seen by the field:
one step at the trajectory's weights is then taken from it and judged against the same step from the model as
it was, kept only where both its R-free and its $-\ln L$ of the free reflections are lower and the geometry
within bounds, undone otherwise (a model whose side chains the data already hold moves nothing and costs the
incomplete model rewards whatever fills unmodelled density. The free reflections are never seen by the field
nor by its test: one step at the trajectory's weights is then taken from it and judged against the same step
from the model as it was, kept only where it lowers the work-set target and the geometry stays within bounds,
undone otherwise (a model whose side chains the data already hold moves nothing and costs the
sweep alone). The log names each side chain moved with its chi angles before and after and the energy the move
took off (`REFINE_SIDE_CHAIN_FIELD_MOVED`, `REFINE_SIDE_CHAIN_FIELD_PRUNED`). Twinned data and a model with
anisotropic atoms are refined without the field.
5. **The final ladder**: the trajectory's weights times $\{1, 2\}$, kept where one displaces it (the weight halved
5. **The final ladder**: the trajectory's weights times $\{1, 2\}$, chosen on the free set, the choice taken where it
advances from where the trajectory ended as a step does (the weight halved
was never kept on 55 refinements, and taking it away changed no choice).
6. **The displacement model**, from where the weights ended (the B factors as they are: isotropic, or the model's U
held), each trial as long as a step on the positions where the weights ended, held - only the displacement
@@ -425,8 +449,7 @@ are:
reflections from where the weights ended; on 49 refinements of 33 structures neither the cuts' rule nor leaving
out the U weight $\times 4$ changed a choice taken. A
trial displaces the one before it (the first: where the weights ended) only where its geometry is within bounds
and both its R-free (beyond the tie) and the $-\ln L$ of the free reflections are lower: the free set tests the
parameters a model adds, so the choice is made on the evidence of each run, not at a resolution (Merritt 2012).
and the free set prefers it: the free set tests the parameters a model adds, so the choice is made on the evidence of each run, not at a resolution (Merritt 2012). Between a model with more displacement parameters (TLS over a B per atom, a U per atom over either) and one with fewer, Hamilton's R-ratio test decides first, as Merritt (2012) and PDB-REDO apply it: the larger passes where its R-work is below the smaller's by more than the ratio $\sqrt{b/(n-m)\,F_{b,n-m,0.05} + 1}$, $n$ the working reflections, $m$ its displacement parameters (the positions are held) and $b$ the extra ones (Hamilton 1965; the F point by Paulson's approximation). The restraints are credited as $wR$ observations, $w$ unknown: each model's is scanned from 0 to the bound at which its restraints take every parameter they restrain off its count (Merritt's $N_\mathrm{atoms}/R$ for a B per atom, $6N_\mathrm{atoms}/R$ for a U per atom), six values each, independently. The larger model is taken where it passes more than 95% of the 36 tests, the smaller where it passes fewer than 30%, and in between the free set decides as everywhere else (PDB-REDO falls back on Z(R-free) and the R-free - R-work gap at fixed cut-offs instead; the paired rule is the one every other free-set choice here uses). A U per atom is tried only above 18 working reflections per refinable atom, PDB-REDO's bound.
The refined U are written as ANISOU and the TLS groups as `_pdbx_refine_tls*` and `REMARK 3` (`REFINE_ADP_MODEL`,
`REFINE_ANISO_ATOMS`, `REFINE_TLS_GROUPS`). It is decided after the solvent build (§15.10), on the model with its
solvent: a water a round put in refines a U of its own in the per-atom trial as every refinable atom does, and no
@@ -437,14 +460,16 @@ are:
B factors as given and its displacement model after them; on a placed prediction, whose B factors are its pLDDT,
the B factors' weight balanced on them is a hundred times too stiff (the B similarity's gradient goes as $1/B$),
the middle rung wins where the tightest wins once the B factors are refined, and the TLS bodies fitted from the
refined B factors come 0.02 lower in R-free than from the B factors as given. A round is taken while it lowers
R-free beyond the tie and undone otherwise; where its ladder and trajectory leave the model where it was, the
refined B factors come 0.02 lower in R-free than from the B factors as given. A round is taken where it advances as a
step does - on the work-set target where its weights are those it started from, on the free set where it chose
others - and undone otherwise; where its ladder and trajectory leave the model where it was, the
hypotheses would repeat the same trials, and it ends there - from a model refined before, a round is one ladder
and one step. No round runs where the refinement does not displace the model as given.
7. **The model as given** is the first candidate in order of trust: where the refinement does not displace it by the
rule every trial is held to (its geometry within bounds and R-free lower beyond the tie), the model is written as
it came, scaled to the data, and the solvent build's edits are undone (`REFINE_AS_GIVEN`). What comes out is never
worse on the free set than what went in. This holds only where the free set is free to the model as given: a model
7. **The model as given** is kept over the refinement where it displaces the refinement on the free set (its geometry
within bounds and the free set preferring it) and the refinement is not better on both R-work and R-free (the
work-set target cannot choose here, the refinement having minimised it from the model as given): the model is
written as it came, scaled to the data, and the solvent build's edits are undone (`REFINE_AS_GIVEN`). What comes
out is never worse on both free measures than what went in. This holds only where the free set is free to the model as given: a model
placed as given on a free set drawn by the run (in `--mode refine`, a file without free flags) has
most likely been refined against those reflections, and its R-free as given does not compete.
@@ -453,7 +478,7 @@ without them, and then with them at $\sigma = 0.1$ Å at the B factors' weight t
X-ray weight a rung up as often as not, the B factors' choice it never changed on 39 refinements), and every step of
the trajectory also tries them changed - switched on where they are off, and in turn tightened ($\sigma$ halved) and
loosened ($\sigma$ doubled, off beyond 0.4 Å) where they are on. Listed after their twin without, the restraints are
kept only where they lower R-free by more than the tie. A model without copies runs exactly the trials it ran before.
kept only where the free set prefers them. A model without copies runs exactly the trials it ran before.
$w$ never exceeds $w_0/8$. The trials of one set run side by side - on one GPU, on several, or on the CPU's
threads - each from the same start and taking nothing from the one before it, so the result does not depend
@@ -576,8 +601,8 @@ and removed and waters and ions relabelled, as phenix.refine builds waters withi
trajectory (§15.7) ends, a round reads the maps and the audit of that model and changes its solvent; then the solvent
alone - its waters and single-atom ions, the rest held - is refined at the weights the trajectory ended at (its B
too, at its balanced weight where the trajectory held B, and 10 iterations a macro-cycle), and the round is kept
where that lowers R-free; where it does not but the round as built does (refining the solvent alone can lose what its
sites gained), it
where that lowers the work-set target at those weights (§15.7); where it does not but the round as built does
(refining the solvent alone can lose what its sites gained), it
is kept as built; otherwise it is undone. The rest of the model goes on with its B factors held or refined as the
trajectory had them. The rounds stop at one that changes nothing, at one undone, or after two (a third gains
nothing measurable); the final ladder of §15.7 then
@@ -786,6 +811,7 @@ What the refinement does not do yet - each a known gap, not a design choice:
- A. A. Vagin, R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long & G. N. Murshudov, "REFMAC5 dictionary: organization of prior chemical knowledge and guidelines for its use", *Acta Cryst.* **D60** (2004), 2184-2195 (the restraint dictionary, §15.4).
- M. Wojdyr, "GEMMI: A library for structural biology", *J. Open Source Softw.* **7** (2022), 4200 (the topology, the riding hydrogens and the blob search, §15.4, §15.9).
- V. Schomaker & K. N. Trueblood, *Acta Cryst.* **B24** (1968), 63-76, and M. D. Winn, M. N. Isupov & G. N. Murshudov, *Acta Cryst.* **D57** (2001), 122-133 (TLS, and the B column total or residual to it, §15.1, §15.5); J. Painter & E. A. Merritt, *Acta Cryst.* **D62** (2006), 439-450 (TLSMD's segments, §15.7).
- W. C. Hamilton, "Significance tests on the crystallographic R factor", *Acta Cryst.* **18** (1965), 502-510, as E. A. Merritt, "To B or not to B: a question of resolution?", *Acta Cryst.* **D68** (2012), 468-477 [doi:10.1107/S0907444911028320](https://doi.org/10.1107/S0907444911028320) applies it to the displacement model, and E. Paulson, "An approximate normalization of the analysis of variance distribution", *Ann. Math. Statist.* **13** (1942), 233-235 [doi:10.1214/aoms/1177731608](https://doi.org/10.1214/aoms/1177731608) (the F point) (§15.7).
- F. L. Hirshfeld, *Acta Cryst.* **A32** (1976), 239-244, A. Thorn, B. Dittrich & G. M. Sheldrick, *Acta Cryst.* **A68** (2012), 448-451, and G. M. Sheldrick, *Acta Cryst.* **C71** (2015), 3-8 (the rigid-bond, similarity and isotropy restraints of anisotropic atoms, §15.5); E. A. Merritt, *Acta Cryst.* **D68** (2012), 468-477 (the displacement model decided by test, §15.7).
- A. Blondel & M. Karplus, "New formulation for derivatives of torsion angles and improper torsion angles in molecular mechanics: elimination of singularities", *J. Comput. Chem.* **17** (1996), 1132-1141 (the torsion derivatives, §15.5).
- M. M. Harding, "Geometry of metal-ligand interactions in proteins", *Acta Cryst.* **D57** (2001), 401-411, and "Small revisions to predicted distances around metal sites in proteins", *Acta Cryst.* **D62** (2006), 678-682; F. Leonarski, L. D'Ascenzo & P. Auffinger, "Mg2+ ions: do they bind to nucleobase nitrogens?", *Nucleic Acids Res.* **45** (2017), 987-1004 (the metal-site distances, §15.5).
@@ -800,6 +826,8 @@ What the refinement does not do yet - each a known gap, not a design choice:
- A. T. Brünger, M. Karplus & G. A. Petsko, "Crystallographic refinement by simulated annealing: application to crambin", *Acta Cryst.* **A45** (1989), 50-61 (the starting weight from the ratio of the gradients, §15.7).
- P. V. Afonine et al., "Towards automated crystallographic structure refinement with phenix.refine", *Acta Cryst.* **D68** (2012), 352-367 (weights chosen among refinements by R-free within geometry bounds, and the overall B moved into the atoms, §15.3, §15.7).
- A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures", *Nature* **355** (1992), 472-475 (R-free).
- I. J. Tickle, R. A. Laskowski & D. S. Moss, "R<sub>free</sub> and the R<sub>free</sub> ratio. II. Calculation of the expected values and variances of cross-validation statistics in macromolecular least-squares refinement", *Acta Cryst.* **D56** (2000), 442-450 [doi:10.1107/S0907444999016868](https://doi.org/10.1107/S0907444999016868) (the standard error of one R-free, against which the paired error of §15.7 is set).
- R. P. Joosten, F. Long, G. N. Murshudov & A. Perrakis, "The PDB_REDO server for macromolecular structure model optimization", *IUCrJ* **1** (2014), 213-220 [doi:10.1107/S2052252514009324](https://doi.org/10.1107/S2052252514009324), and R. P. Joosten, K. Joosten, G. N. Murshudov & A. Perrakis, "PDB_REDO: constructive validation, more than just looking for errors", *Acta Cryst.* **D68** (2012), 484-496 [doi:10.1107/S0907444911054515](https://doi.org/10.1107/S0907444911054515) (choices taken only where more than one free measure agrees, §15.7).
- G. J. Kleywegt, "Use of non-crystallographic symmetry in protein structure refinement", *Acta Cryst.* **D52** (1996), 842-857; I. Usón, E. Pohl, T. R. Schneider, Z. Dauter, A. Schmidt, H.-J. Fritz & G. M. Sheldrick, "1.7 Å structure of the stabilized REIv mutant T39K. Application of local NCS restraints", *Acta Cryst.* **D55** (1999), 1158-1167; R. A. Nicholls, F. Long & G. N. Murshudov, "Low-resolution refinement tools in REFMAC5", *Acta Cryst.* **D68** (2012), 404-417; J. J. Headd et al., "Flexible torsion-angle noncrystallographic symmetry restraints for improved macromolecular structure refinement", *Acta Cryst.* **D70** (2014), 1346-1356 (local NCS restraints with a robust loss, decided by R-free, and Murshudov et al. 2011 above for the copies found by alignment, §15.5, §15.7).
- P. Emsley, B. Lohkamp, W. G. Scott & K. Cowtan, "Features and development of Coot", *Acta Cryst.* **D66** (2010), 486-501 (unmodelled blobs, §15.9).
- N. Naleem, A. Henning-Knechtel, S. Kirmizialtin & P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and annotating Mg2+ ions in RNA structures", *Nucleic Acids Res.* **54** (2026), gkag120; F. Leonarski, L. D'Ascenzo & P. Auffinger, *Nucleic Acids Res.* **45** (2017), 987-1004 and *RNA* **25** (2019), 173-192; F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar & P. Auffinger, *Nucleic Acids Res.* **53** (2025), gkae1148 (the coordination bands, distances and exclusion zone of the solvent audit, §15.10).
+7 -5
View File
@@ -287,12 +287,14 @@ element and the number of such records of the ion (Mg-O 2.07 A, Na-O 2.42 A, K-O
usual coordination), where the model's distance is within 0.33 A of that (Mg-O up to 2.4 A); a record at an
outer-sphere distance, a contact without a record, or one to a symmetry mate, is not restrained as a bond. The B factors of bonded atoms and of
atoms in contact (waters included) are restrained to each other in proportion to their size. The weights of
the X-ray term against the geometry and against that B restraint are chosen by R-free among short
refinements whose bond and angle rmsd stay within 0.02 A and 2 degrees: three weights from the placed
the X-ray term against the geometry and against that B restraint are chosen on the free set among short
refinements whose bond and angle rmsd stay within 0.02 A and 2 degrees - a candidate taken only where its free
-ln L is lower and its R-free not higher than the incumbent's by more than the standard error of their paired
difference over the free reflections: three weights from the placed
model, each with the B factors held and refined; then a trajectory of further short refinements from the
best, each trying the weights as they are and changed, taken while R-free or R-work falls beyond 0.001 and
R-free stays within 0.001 of its best; then a short final ladder, kept only where it lowers R-free beyond
0.001. The free set is the merge's own, so the refined R-free judges the refinement against reflections it
chosen one, each trying the weights as they are and changed, taken while it lowers the work-set target (the
refined target at the chosen weights); then a short final ladder. The free set decides the weights and the
displacement model only; every other step is taken on the work-set target. The free set is the merge's own, so the refined R-free judges the refinement against reflections it
never fitted - but where the model was deposited, those reflections were most likely its depositor's working
reflections, and a refinement against new data then moves R-free up from the unrefined value rather than
down. `R_WORK`/`R_FREE` of the model as given are never replaced. On a GPU a refinement takes seconds; the CPU
+279 -125
View File
@@ -91,8 +91,62 @@ constexpr double REPULSION_MARGIN = 0.5; // A: pairs this much beyond their co
constexpr double MAX_RMSD_BONDS = 0.02, MAX_RMSD_ANGLES = 2.0;
// The minimiser's iterations per macro-cycle where the solvent build refines the solvent alone.
constexpr int SOLVENT_ITERATIONS = 10;
// R-free differences smaller than this do not decide between two weights: the tighter geometry is taken.
constexpr double R_FREE_TIE = 0.001;
// The free set decides the weights and the displacement model alone, and only where both of its measures agree: a
// candidate is preferred to the incumbent where its -ln L of the free reflections is lower and its R-free not higher
// than the incumbent's by more than the standard error of the difference. The two are scored on the same free
// reflections, so the difference is paired: R-free is a sum of one term per free twin group (FreeRTerms), and the
// standard error is that of the sum of the groups' differences.
// As PDB-REDO decides on more than one free measure, Joosten, Long, Murshudov & Perrakis (2014) IUCrJ 1, 213-220
bool FreeSetPrefers(const std::vector<float> &r_terms, double nll_free, const std::vector<float> &r_terms_b,
double nll_free_b) {
const size_t n = r_terms.size();
double sum = 0, sum2 = 0;
for (size_t k = 0; k < n; k++) {
const double d = r_terms[k] - r_terms_b[k];
sum += d;
sum2 += d * d;
}
const double se = n > 1 ? std::sqrt(std::max(sum2 - sum * sum / n, 0.0) * n / (n - 1)) : 0.0;
return nll_free < nll_free_b && sum < se;
}
// The upper alpha point of the F distribution with d1 and d2 degrees of freedom, by Paulson's normal approximation of
// F^(1/3) (z the normal deviate of alpha) - accurate to a few percent at the counts a displacement model has.
// Following Paulson (1942) Ann. Math. Statist. 13, 233-235
double FQuantile(double d1, double d2, double z) {
const double u = 2.0 / (9.0 * d1), v = 2.0 / (9.0 * d2), a = 1.0 - v, c = 1.0 - u;
const double q = a * a - z * z * v;
const double x = (a * c + std::sqrt(std::max(a * a * c * c - q * (c * c - z * z * u), 0.0))) / q;
return x * x * x;
}
// Of two displacement models, `large` with more parameters than `small`: the fraction of Hamilton's R-ratio tests it
// passes over the range of weights its restraints may be credited at - R_small / R_large above
// sqrt(b / (n - m) F(b, n - m, 0.05) + 1), n the working reflections, m the larger model's displacement parameters
// (the trials hold the positions), b the extra ones. Each model's restraints are credited as w R observations, w from
// 0 up to the bound at which they take every parameter they restrain off its count - Merritt's N_atoms / R for a B
// per atom (as his 3 N_atoms / R for the positions), 6 N_atoms / R for a U per atom; the two weights are scanned
// independently, six values each.
// Following Hamilton (1965) Acta Cryst. 18, 502-510, as Merritt (2012) Acta Cryst. D68, 468-477 and PDB-REDO
// (Joosten et al. 2012 Acta Cryst. D68, 484-496) apply it to the displacement model
double HamiltonPassFraction(const RefinementWeightTrial &large, const RefinementWeightTrial &small, int n_work) {
int pass = 0, tests = 0;
for (int i = 0; i <= 5; i++)
for (int j = 0; j <= 5; j++, tests++) {
const double m_small = small.adp_parameters - small.adp_restrained * i / 5.0;
const double m_large = large.adp_parameters - large.adp_restrained * j / 5.0;
const double b = m_large - m_small, dof = n_work - m_large;
pass += b <= 0 || (dof > 0 && small.r_work / large.r_work > std::sqrt(b / dof * FQuantile(b, dof, 1.6449) + 1.0));
}
return static_cast<double>(pass) / tests;
}
// The displacement model: of the larger and the smaller, the larger where it passes more than this fraction of the
// tests (HamiltonPassFraction), the smaller where it passes fewer than the other, and the free set's choice between
// (as PDB-REDO; Joosten et al. 2012).
constexpr double HAMILTON_LARGER = 0.95, HAMILTON_SMALLER = 0.30;
// A U per atom is tried only above this many working reflections per refinable atom (as PDB-REDO; Joosten et al. 2012).
constexpr double U_MIN_REFLECTIONS_PER_ATOM = 18;
// Every other step is taken on the work-set target (WorkTarget) where it lowers it by more than this, relative.
constexpr double TARGET_TOLERANCE = 1e-4;
// The rounds of RefineOnce's weight and displacement-model choice (4b.): the first from the model as given, each
// after it from the model the one before ended with.
// The minimiser (RefinementMinimiser): a solve stops where an iteration lowers the target by less than this,
@@ -1564,9 +1618,10 @@ void RefinePlacement(Context &ctx, bool group_b, RefinementResult &result) {
if (body.size() >= 50)
bodies.push_back(body);
}
// Each step is kept only where it lowers R-free beyond the tie: a deposited model was refined atom by atom
// already, and moving its chains or shifting its B factors residue by residue then only makes it worse.
RFactors before = ctx.Refit();
// Each step is kept only where it lowers the -ln L of the working reflections, the target it minimised (the free
// set decides the weights and the displacement model alone).
ctx.Refit();
double before = ctx.Score().nll_work;
if (!bodies.empty()) {
const std::vector<gemmi::Position> pos0 = ctx.Positions();
std::vector<double> q(6 * bodies.size(), 0.0);
@@ -1577,13 +1632,15 @@ void RefinePlacement(Context &ctx, bool group_b, RefinementResult &result) {
const CancelCheck cancel_check(ctx);
ceres::Solve(cancel_check.Options(30), problem, q.data(), &summary);
rb->Place(q.data());
const RFactors after = ctx.Refit();
if (after.r_free < before.r_free - R_FREE_TIE) {
ctx.Refit();
const double after = ctx.Score().nll_work;
if (after < before) {
result.rigid_bodies = static_cast<int>(bodies.size());
before = after;
} else {
ctx.SetPositions(pos0);
before = ctx.Refit();
ctx.Refit();
before = ctx.Score().nll_work;
}
}
if (group_b) {
@@ -1596,7 +1653,8 @@ void RefinePlacement(Context &ctx, bool group_b, RefinementResult &result) {
const CancelCheck cancel_check(ctx);
ceres::Solve(cancel_check.Options(30), problem, q.data(), &summary);
gb->Place(q.data());
result.group_b = ctx.Refit().r_free < before.r_free - R_FREE_TIE;
ctx.Refit();
result.group_b = ctx.Score().nll_work < before;
if (!result.group_b)
ctx.SetB(b0);
}
@@ -1897,6 +1955,39 @@ std::string DomainText(const RefinementReflections &refl, const XrayRowModel &ro
// As REFMAC5 drops twin domains of small fraction, Murshudov et al. (2011) Acta Cryst. D67, 355-367
constexpr double TWIN_DOMAIN_MIN = 0.05;
// Each observed reflection's twin group, named by its lowest observed row (itself where there is no twin law).
std::vector<int> TwinGroups(const RefinementReflections &refl) {
const size_t n = refl.Size();
std::vector<int> group(n);
for (size_t m = 0; m < n; m++) {
group[m] = static_cast<int>(m);
for (size_t d = 0; d < refl.Laws(); d++)
if (refl.mates[d][m] < static_cast<int>(n))
group[m] = std::min(group[m], refl.mates[d][m]);
}
return group;
}
// R-free of the model as the last Fcalc left it, as one term per free twin group, in the order of the groups (the
// members of a group share their F_calc, so their terms are summed into one, as TwinNull sums them): the terms
// FreeSetPrefers pairs.
std::vector<float> FreeRTerms(const Context &ctx) {
const std::vector<std::complex<float>> fmodel = TwinnedModel(ctx.refl, ctx.rows, ctx.fcalc);
const std::vector<int> group = TwinGroups(ctx.refl);
double fo = 0;
std::vector<double> sum(ctx.refl.Size(), 0.0);
for (size_t m = 0; m < ctx.refl.Size(); m++)
if (ctx.refl.free[m]) {
fo += ctx.refl.fo[m];
sum[group[m]] += std::fabs(ctx.refl.fo[m] - std::abs(fmodel[m]));
}
std::vector<float> terms;
for (size_t m = 0; m < ctx.refl.Size(); m++)
if (ctx.refl.free[m] && group[m] == static_cast<int>(m))
terms.push_back(static_cast<float>(sum[m] / fo));
return terms;
}
// The null of a twinned refinement, for the model as the last Refit() left it: its R-free at the same fraction with
// every reflection's mate replaced by an unrelated observed one of the same resolution and the same set (free with
// free, work with work, as a law's twin pairs are). The reflections of each set are put in order of 1/d^2 with the two
@@ -1920,14 +2011,7 @@ constexpr double TWIN_STOP_Z = 0.5;
void TwinNull(const Context &ctx, RefinementResult &result, size_t first = 0) {
RefinementReflections null = ctx.refl;
const size_t n = null.Size();
// each observed reflection's twin group, named by its lowest observed row
std::vector<int> group(n);
for (size_t m = 0; m < n; m++) {
group[m] = static_cast<int>(m);
for (size_t d = 0; d < ctx.refl.Laws(); d++)
if (ctx.refl.mates[d][m] < static_cast<int>(n))
group[m] = std::min(group[m], ctx.refl.mates[d][m]);
}
const std::vector<int> group = TwinGroups(ctx.refl);
for (size_t d = first; d < null.Laws(); d++)
std::iota(null.mates[d].begin(), null.mates[d].end(), 0);
for (const bool free : {false, true}) {
@@ -2246,6 +2330,26 @@ struct TrialOutcome {
int evaluations = 0;
};
// The work-set target's terms of the model as it stands (RefinementWeightTrial), at the last Refit()'s fit: what every
// decision but the weights' and the displacement model's is made on (WorkTarget).
void MeasureWorkTarget(Context &ctx, const RefinementTopology &topo, const NcsCopies *ncs, double ncs_sigma,
RefinementWeightTrial &t) {
const RefinementScore sc = ctx.Score();
t.nll_work = sc.nll_work;
t.nll_free = sc.nll_free;
t.free_r_terms = FreeRTerms(ctx);
const std::vector<std::array<double, 4>> xyzb = ctx.XyzB();
t.e_geometry = EvaluateRestraints(RestraintsNow(ctx, topo, ncs, ncs_sigma), xyzb, nullptr);
t.e_b = EvaluateRestraints(BRestraintsNow(ctx, topo), xyzb, nullptr);
}
// The work-set target of a trial at the X-ray weights w (positions) and adp (B, 0: held), its NCS restraints at its own
// sigma: what a trial at those weights minimises, the X-ray term of the positions and the geometry, and that of the B
// factors and the B similarity where they are refined.
double WorkTarget(const RefinementWeightTrial &t, double w, double adp) {
return w * t.nll_work + t.e_geometry + (adp > 0 ? adp * t.nll_work + t.e_b : 0.0);
}
// The X-ray target's gradient on the positions, its squared norm over `refinable`, for the atoms as they are.
double XrayGradientNorm(Context &ctx, const std::vector<int> &refinable) {
std::vector<std::array<double, 4>> g;
@@ -2471,16 +2575,25 @@ TrialOutcome RunTrial(TrialSlot slot, XyzCalibrations &calibrations, const std::
t.ncs_rmsd = NcsRmsd(*ncs, ctx.XyzB());
if (hyp != nullptr)
t.adp = hyp->name;
t.nll_free = ctx.Score().nll_free;
out.line = fmt::format("{}X-ray weight {:.3g}, {} {}{}: R-work {:.4f}, R-free {:.4f}{}, rmsd bonds {:.4f} A, "
"angles {:.2f} deg{}{}",
// the displacement parameters: a B per refinable atom, the TLS bodies' 20 each beside it, or a U per atom
int atoms_b = 0;
for (int i : refinable)
atoms_b += !ctx.atoms[i]->is_hydrogen();
t.adp_parameters = kind == AdpHypothesis::Kind::Anisotropic ? 6 * static_cast<int>(u_atoms.size())
: kind == AdpHypothesis::Kind::Tls ? atoms_b + 20 * static_cast<int>(hyp->bodies.size())
: atoms_b;
// of them the restrained ones: the B or the U of every atom (not the TLS bodies')
t.adp_restrained = kind == AdpHypothesis::Kind::Anisotropic ? 6 * static_cast<int>(u_atoms.size()) : atoms_b;
MeasureWorkTarget(ctx, topo, ncs, ncs_sigma, t);
out.line = fmt::format("{}X-ray weight {:.3g}, {} {}{}: R-work {:.4f}, R-free {:.4f}{}, target {:.6g}, rmsd bonds "
"{:.4f} A, angles {:.2f} deg{}{}",
std::string(tw.blur.empty() ? "" : "blurred, ") + (hyp != nullptr ? hyp->name + ", " : ""), w,
kind == AdpHypothesis::Kind::Anisotropic ? "U" : kind == AdpHypothesis::Kind::Tls ? "residual B" : "B",
adp_w > 0 ? fmt::format("refined at weight {:.3g}", adp_w) : std::string("held"),
ncs == nullptr || ncs->Empty() ? std::string()
: ncs_sigma > 0 ? fmt::format(", NCS at {:.3g} A (copies {:.3f} A apart)", ncs_sigma, t.ncs_rmsd)
: fmt::format(", no NCS (copies {:.3f} A apart)", t.ncs_rmsd),
rf.r_work, rf.r_free, fmt::format(", -LL free {:.1f}", t.nll_free),
rf.r_work, rf.r_free, fmt::format(", -LL free {:.1f}", t.nll_free), WorkTarget(t, w, adp_w),
t.geometry.rmsd_bonds, t.geometry.rmsd_angles,
t.geometry.metal_bonds > 0 ? fmt::format(", metal bonds {:.3f} A", t.geometry.rmsd_metal)
: std::string(),
@@ -2859,8 +2972,8 @@ double BalancedUWeight(Context &ctx, const URestraints &restraints, const std::v
// The displacements of the model as given, read where it has TLS groups whose atoms bring no U of their own: their U
// from the groups, its B column taken as the total B (the U's equivalent isotropic B, as phenix.refine writes it) or as
// the residual to the groups (as REFMAC writes it), and the isotropic B as it stands - the one of the three whose
// R-free is lowest, beyond the tie in that order. `tls`: the groups, their atoms those of ctx.atoms but the hydrogens
// the residual to the groups (as REFMAC writes it), and the isotropic B as it stands - a displacement model, chosen on
// the free set (FreeSetPrefers), in that order. `tls`: the groups, their atoms those of ctx.atoms but the hydrogens
// (which ride on their parents' B). Returns the R-factors of the reading kept.
// Following Winn, Isupov & Murshudov (2001) Acta Cryst. D57, 122-133 (the B column total or residual to the TLS)
RFactors ReadInputDisplacements(Context &ctx, const std::vector<TlsBody> &tls, Logger &logger, RefinementResult &result) {
@@ -2906,6 +3019,8 @@ RFactors ReadInputDisplacements(Context &ctx, const std::vector<TlsBody> &tls, L
readings.push_back({result.adp_input == "ANISOU" ? "ANISOU" : "isotropic", b0, aniso0});
int best = -1;
std::vector<RFactors> rf;
std::vector<double> nll_free;
std::vector<std::vector<float>> r_terms;
for (size_t k = 0; k < readings.size(); k++) {
ctx.SetB(readings[k].b);
ctx.SetAniso(readings[k].aniso);
@@ -2914,9 +3029,11 @@ RFactors ReadInputDisplacements(Context &ctx, const std::vector<TlsBody> &tls, L
ctx.atoms[i]->b_iso = ctx.atoms[ctx.rider_parent[i]]->b_iso;
ctx.ForgetStarts();
rf.push_back(ctx.Refit());
logger.Info("Refinement: the model's {} TLS groups ({} atoms) read as {}: R-work {:.4f}, R-free {:.4f}", tls.size(),
tls_u.size(), readings[k].name, rf[k].r_work, rf[k].r_free);
if (best < 0 || rf[k].r_free < rf[best].r_free - R_FREE_TIE)
nll_free.push_back(ctx.Score().nll_free);
r_terms.push_back(FreeRTerms(ctx));
logger.Info("Refinement: the model's {} TLS groups ({} atoms) read as {}: R-work {:.4f}, R-free {:.4f}, -LL free "
"{:.1f}", tls.size(), tls_u.size(), readings[k].name, rf[k].r_work, rf[k].r_free, nll_free[k]);
if (best < 0 || FreeSetPrefers(r_terms[k], nll_free[k], r_terms[best], nll_free[best]))
best = static_cast<int>(k);
}
ctx.SetB(readings[best].b);
@@ -3161,6 +3278,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
result.r_work_start = r.r_work;
result.r_free_start = r.r_free;
result.score_start = ctx.Score();
const std::vector<float> start_r_terms = FreeRTerms(ctx);
logger.Info("Refinement{}: {} atoms ({} riding hydrogens), {} reflections to {:.2f} A ({} free), on the {}; as "
"given R-work {:.4f}, R-free {:.4f}; k_sol {:.3f}, b_sol {:.1f}, overall B {:.1f}, Wilson B {:.0f}, B "
"ceiling {:.0f}, mask grid {:.2f} A",
@@ -3178,6 +3296,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
given.trial.r_work = r.r_work;
given.trial.r_free = r.r_free;
given.trial.nll_free = result.score_start.nll_free;
given.trial.free_r_terms = start_r_terms;
given.pos = ctx.Positions();
given.b = ctx.B();
given.aniso = ctx.aniso;
@@ -3217,7 +3336,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
ctx.st.meta.refinement[0].tls_groups = TlsBodiesToGroups(ctx.Model(), ctx.tls);
for (size_t k = 1; k < ctx.st.meta.refinement.size(); k++)
ctx.st.meta.refinement[k].tls_groups.clear();
} else if (result.adp_model == "anisotropic")
} else if (result.adp_model.starts_with("anisotropic"))
for (gemmi::RefinementInfo &ref : ctx.st.meta.refinement)
ref.tls_groups.clear();
if (const std::vector<gemmi::TlsGroup> *groups = ctx.st.meta.get_tls_groups())
@@ -3426,10 +3545,24 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
o = TrialOutcome();
};
// A trial displaces another only where its geometry is within bounds and its R-free lower by more than the
// tie.
// A trial at other weights (or the model as given) displaces another only where its geometry is within
// bounds and the free set prefers it (FreeSetPrefers).
auto displaces = [](const RefinementWeightTrial &t, const RefinementWeightTrial &b) {
return t.within_bounds && (!b.within_bounds || t.r_free < b.r_free - R_FREE_TIE);
return t.within_bounds &&
(!b.within_bounds || FreeSetPrefers(t.free_r_terms, t.nll_free, b.free_r_terms, b.nll_free));
};
// A trial taken on from `s`: at the weights of s, where the work-set target (WorkTarget) is lower than that of s
// by more than TARGET_TOLERANCE, and the geometry within bounds where that of s is. A trial at other weights
// (X-ray weights or NCS restraints) is not compared on a target that changed with them - the positions and the
// B factors minimise targets of their own, whose sum a change of weight can raise as it lowers each: the
// weights are the free set's to decide (displaces).
auto advances = [&displaces](const RefinementWeightTrial &t, const RefinementWeightTrial &s) {
if (t.weight != s.weight || t.adp_weight != s.adp_weight || t.ncs_sigma != s.ncs_sigma)
return displaces(t, s);
if (t.within_bounds < s.within_bounds)
return false;
const double before = WorkTarget(s, s.weight, s.adp_weight);
return WorkTarget(t, s.weight, s.adp_weight) < before - TARGET_TOLERANCE * std::abs(before);
};
// Of a set of trials, the one kept: they come in order of trust, the first kept unless a later one displaces
// it. -1: none ran.
@@ -3459,9 +3592,8 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
// not change (on 39 refinements of 21 structures with copies it never did; the X-ray weight it does), and
// at the rung they chose and the next looser one (on 198 refinements of the battery with copies, NCS was
// never kept at any other).
// The B factors are refined first at the middle rung only, and at the others where that comes within two
// ties of holding them there: on 364 refinements of the battery B held was kept by 281, and the rule
// changed the rung kept on 4 (all within 0.006 in R-free). Their X-ray weight is tried at the balanced
// The B factors are refined first at the middle rung only, and at the others unless holding them there
// displaces refining them (on 364 refinements of the battery B held was kept by 281). Their X-ray weight is tried at the balanced
// one and at a quarter and a sixteenth of it: a model whose B factors arrive far from where they refine
// (a prediction's pLDDT) balances a weight that is not the one its B wants, and the lower rungs took
// 0.002-0.010 off R-free on five such starts, nothing off eight deposited models.
@@ -3500,8 +3632,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
if (ladder[i].w == mid && ladder[i].ncs == 0)
(ladder[i].adp == 0 ? held : refined) = &outcomes[i];
const bool refine_b = held == nullptr || refined == nullptr ||
(refined->ran && refined->trial.within_bounds &&
(!held->trial.within_bounds || refined->trial.r_free < held->trial.r_free + 2 * R_FREE_TIE));
(refined->ran && refined->trial.within_bounds && !displaces(held->trial, refined->trial));
auto rest_b = [&](const TrialWeights &l) { return l.ncs == 0 && !first(l); };
if (refine_b && !all_ahead)
run_ladder(rest_b);
@@ -3547,37 +3678,29 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
}
// 2. The trajectory: steps of a few macro-cycles from where the last one ended, each trying the weights as they
// are and, in turn, the positions' X-ray weight halved and doubled (halved only in the first step, from the
// ladder's coarse rung: on 364 refinements of the battery no later step kept it, of 91 that tried it) and
// are and, in turn, the positions' X-ray weight halved and doubled (the ladder's rungs are a factor of eight
// apart, and a trajectory that goes on from a looser weight than the data want can tighten it only so) and
// the B factors' divided and multiplied by four; where there are copies, also the NCS restraints changed:
// switched on where they are off, and in turn tightened and loosened (to off beyond NCS_SIGMA_MAX) where
// they are on.
// A step is taken while it lowers R-free or R-work by more than the tie and its R-free stays within the tie
// of the best so far; the trajectory ends at its best step (on a deposited model, whose free set it was
// fitted to, R-work falls while R-free rises, and it ends where it began).
// The trials of the step that went from the best so far: the final ladder goes from there too, and a trial
// of the same weights from the same model is the same computation.
// The weights of a step are chosen on the free set among its trials (displaces, the weights as they are
// first); the step is taken while that trial advances from where the last one ended (advances), and the
// trajectory ends where the last step taken ended.
// The trials of the step that went from where the trajectory ended: the final ladder goes from there too, and
// a trial of the same weights from the same model is the same computation.
std::vector<TrialOutcome> from_best;
// `fresh`: the first step fits the scale, the solvent and the likelihood anew (the model changed since).
// `label`: the steps' name in the log.
auto trajectory = [&](TrialOutcome state, bool fresh, std::vector<TrialOutcome> &from_best, const char *label) {
TrialOutcome best = state;
from_best.clear();
bool state_is_best = true;
// A step taken on R-work alone is followed by one more at most: on a model refined before, R-work falls
// step after step while R-free does not, and the trajectory ends at its best anyway - a model of 490k
// atoms spent six such steps (240 s) on a round it then gave back. (On 28 refinements of 22 deposited
// models and 6 predictions the rule changed nothing.)
int since_best = 0;
for (int step = 0; step < settings.max_steps && state.ran && !ctx.Cancelled() && since_best <= 1; step++) {
for (int step = 0; step < settings.max_steps && state.ran && !ctx.Cancelled(); step++) {
const double w = state.trial.weight, adp = state.trial.adp_weight, ncs_s = state.trial.ncs_sigma;
std::vector<TrialWeights> trials = {{w, adp, ncs_s}};
if (step % 2 == 1 && adp > 0) {
trials.push_back({w, adp / 4, ncs_s});
trials.push_back({w, 4 * adp, ncs_s});
} else {
if (step == 0)
trials.push_back({w / 2, adp, ncs_s});
trials.push_back({w / 2, adp, ncs_s});
if (2 * w <= w_max)
trials.push_back({2 * w, adp, ncs_s});
}
@@ -3588,27 +3711,18 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
std::vector<TrialOutcome> outcomes = RunTrials(trial_slots, trials, refinable, state.pos, state.b, state.aniso,
fresh && step == 0 ? nullptr : &state.fit,
settings.macro_cycles, settings, ncs_p);
if (state_is_best)
from_best = outcomes;
from_best = outcomes;
const int k = choose(outcomes, fmt::format("{} {}", label, step + 1).c_str());
if (k < 0)
break;
const RefinementWeightTrial &t = outcomes[k].trial;
// after a change of the model the first step is taken whatever it gains: it is where the change is
// judged from
const bool gains = (fresh && step == 0) || displaces(t, state.trial) ||
(t.within_bounds && t.r_work < state.trial.r_work - R_FREE_TIE);
if (!gains || !((fresh && step == 0) || (t.within_bounds && t.r_free < best.trial.r_free + R_FREE_TIE)))
if (!((fresh && step == 0) || advances(outcomes[k].trial, state.trial)))
break;
state = std::move(outcomes[k]);
state_is_best = (fresh && step == 0) || displaces(state.trial, best.trial);
since_best = state_is_best ? 0 : since_best + 1;
if (state_is_best) {
best = state;
from_best.clear();
}
from_best.clear();
}
return best;
return state;
};
if (!stopped)
state = trajectory(std::move(state), false, from_best, "step");
@@ -3620,9 +3734,8 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
// map averaged over the members with the free reflections left out (PopulationMap), each residue is taken
// from the member whose atoms sit in the most density (following Nwachukwu et al. (2013) Structure 21,
// 1923-1930); the child's junctions are mended by a macro-cycle
// at the ladder's tightest rung and the whole refined by one trial at the state's weights. The child displaces the state by the rule every trial is held to (geometry within
// bounds, R-free lower beyond the tie): the free set decides once, between two candidates - never among
// the members, whose best on the free set would be the best of N.
// at the ladder's tightest rung and the whole refined by one trial at the state's weights. The child is taken where it
// advances on the work-set target, as a step is (the free set does not see the members).
// The members are at most as many as the trial slots, one wave: on a large model, where the card holds
// few, more members would cost a wave each.
const size_t population = std::min<size_t>(std::max(settings.population, 0), trial_slots.size());
@@ -3651,7 +3764,8 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
blur += fmt::format("{}{:.0f}", blur.empty() ? "" : "/", b);
logger.Info("Refinement, population: blur {}, kick {:.1f} A: {}", blur.empty() ? "none" : blur,
members[k].kick, outcomes[k].line);
if (outcomes[k].trial.within_bounds && (best < 0 || outcomes[k].trial.nll_free < outcomes[best].trial.nll_free))
if (outcomes[k].trial.within_bounds &&
(best < 0 || WorkTarget(outcomes[k].trial, w, adp) < WorkTarget(outcomes[best].trial, w, adp)))
best = static_cast<int>(k);
}
if (best >= 0) {
@@ -3686,11 +3800,11 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
child = RunTrials(trial_slots, {{w, adp, ncs_s}}, refinable, child[0].pos, child[0].b, state.aniso,
&child[0].fit, settings.macro_cycles, settings, ncs_p);
const RefinementWeightTrial &c = child[0].trial;
const bool kept = child[0].ran && displaces(c, state.trial);
logger.Info("Refinement, population: {} members, the best on the free set R-free {:.4f}, -LL free {:.1f}; "
"crossover from {} members: {}; {} (state R-free {:.4f}, -LL free {:.1f})", outcomes.size(),
outcomes[best].trial.r_free, outcomes[best].trial.nll_free, used.size(), child[0].line,
kept ? "kept" : "not kept", state.trial.r_free, state.trial.nll_free);
const bool kept = child[0].ran && advances(c, state.trial);
logger.Info("Refinement, population: {} members, the best on the work-set target {:.6g}; crossover from "
"{} members: {}; {} (state target {:.6g})", outcomes.size(),
WorkTarget(outcomes[best].trial, w, adp), used.size(), child[0].line, kept ? "kept" : "not kept",
WorkTarget(state.trial, w, adp));
if (kept) {
state = std::move(child[0]);
from_best.clear();
@@ -3701,7 +3815,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
// 2b. The solvent build (SolventBuild.h), from where the trajectory ended: waters put in at the difference
// map's peaks, taken out where the maps no longer hold them, and waters and ions relabelled, on the maps
// and the solvent audit of that model; then the solvent alone refined, the rest held, at the weights the
// trajectory ended at. A round is kept where that lowers R-free, and undone otherwise (the build ends
// trajectory ended at. A round is kept where that lowers the work-set target at those weights, and undone otherwise (the build ends
// there); at most SOLVENT_BUILD_ROUNDS of them. What they kept is then refined with the rest by the final
// ladder (the trajectory run again from there gained nothing on twelve sets). Each round thus costs a map,
// an audit and a refinement of the solvent, where a refinement of its own cost the whole weight search
@@ -3881,6 +3995,16 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
ctx.Restore(state.fit);
const RFactors built = ctx.Refit();
const Context::Fit built_fit = ctx.Fitted();
RefinementWeightTrial built_trial = state.trial;
built_trial.r_work = built.r_work;
built_trial.r_free = built.r_free;
MeasureWorkTarget(ctx, topo, ncs_p, state.trial.ncs_sigma, built_trial);
// the round judged on the work-set target at the weights the trajectory ended at
const double target_before = WorkTarget(state.trial, state.trial.weight, state.trial.adp_weight);
auto lowers = [&](const RefinementWeightTrial &t) {
return WorkTarget(t, state.trial.weight, state.trial.adp_weight) <
target_before - TARGET_TOLERANCE * std::abs(target_before);
};
// (B refined, at its balanced weight where the trajectory held B: a new water's starting B is a
// guess; fewer iterations, the waters' few parameters converging fast)
RefinementSettings solvent_settings = settings;
@@ -3893,22 +4017,22 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
ncs_p);
logger.Info("Solvent build, round {}: the solvent refined alone: {}", round, local[0].line);
TrialOutcome after = std::move(local[0]);
if (after.ran && !(after.trial.r_free < state.trial.r_free) && built.r_free < state.trial.r_free) {
logger.Info("Solvent build, round {}: kept as built, R-work {:.4f}, R-free {:.4f}", round,
built.r_work, built.r_free);
if (after.ran && !lowers(after.trial) && lowers(built_trial)) {
logger.Info("Solvent build, round {}: kept as built, R-work {:.4f}, R-free {:.4f}, target {:.6g}",
round, built.r_work, built.r_free,
WorkTarget(built_trial, state.trial.weight, state.trial.adp_weight));
after.pos = changed_pos;
after.b = changed_b;
after.aniso = state.aniso;
after.fit = built_fit;
after.trial.r_work = built.r_work;
after.trial.r_free = built.r_free;
after.trial.geometry = state.trial.geometry;
after.trial.within_bounds = state.trial.within_bounds;
const double adp_weight = after.trial.adp_weight;
after.trial = built_trial;
after.trial.adp_weight = adp_weight;
}
if (!after.ran || !(after.trial.r_free < state.trial.r_free)) {
logger.Info("Solvent build, round {}: not kept (R-free {:.4f}, before {:.4f})", round,
after.trial.r_free, state.trial.r_free);
b.stop = fmt::format("round {} raised R-free", round);
if (!after.ran || !lowers(after.trial)) {
logger.Info("Solvent build, round {}: not kept (target {:.6g}, before {:.6g})", round,
WorkTarget(after.trial, state.trial.weight, state.trial.adp_weight), target_before);
b.stop = fmt::format("round {} did not lower the target", round);
// undone, in every slot
for (size_t sl = 0; sl < trial_slots.size(); sl++) {
for (auto u = undo[sl].rbegin(); u != undo[sl].rend(); ++u) {
@@ -3960,10 +4084,9 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
// side chain the field can search put in the rotamer of least energy - the -ln L of the working
// reflections at the fit as it stands, plus the refinement's own repulsion term at its X-ray weight -
// given the others, or pruned to CB where no rotamer fits; then one step at the trajectory's weights from
// the field, judged against the same step from the model as it was (the trajectory's end is the best of
// many trials, and a step from it alone regresses from it as often as not): kept where both its R-free
// and its -ln L of the free reflections are lower, the geometry within bounds, undone otherwise. The field
// never sees the free reflections. A model whose side chains the data already hold moves nothing and
// the field, judged against the same step from the model as it was: kept where it advances on the
// work-set target from that step (advances), undone otherwise. Neither the field nor its test sees the
// free reflections. A model whose side chains the data already hold moves nothing and
// costs the sweep alone. The atoms it prunes are left at occupancy 0, as the solvent build's placeholders
// are, and held.
if (settings.side_chain_field && !stopped && state.ran && !ctx.Cancelled() && !ctx.refl.Twinned() &&
@@ -4110,11 +4233,12 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
if (topo.refinable[i])
refinable.push_back(static_cast<int>(i));
TrialOutcome after = one_step(moved);
const bool kept = after.ran && after.trial.within_bounds && after.trial.r_free < unchanged.trial.r_free &&
after.trial.nll_free < unchanged.trial.nll_free;
logger.Info("Side-chain field: refined from the field R-free {:.4f}, -LL free {:.1f}; from the model as it "
"was R-free {:.4f}, -LL free {:.1f}: {}", after.trial.r_free, after.trial.nll_free,
unchanged.trial.r_free, unchanged.trial.nll_free, kept ? "kept" : "not kept");
const bool kept = after.ran && unchanged.ran && advances(after.trial, unchanged.trial);
logger.Info("Side-chain field: refined from the field R-free {:.4f}, -LL free {:.1f}, target {:.6g}; from "
"the model as it was R-free {:.4f}, -LL free {:.1f}, target {:.6g}: {}", after.trial.r_free,
after.trial.nll_free, WorkTarget(after.trial, after.trial.weight, after.trial.adp_weight),
unchanged.trial.r_free, unchanged.trial.nll_free,
WorkTarget(unchanged.trial, after.trial.weight, after.trial.adp_weight), kept ? "kept" : "not kept");
if (kept) {
result.side_chain_field_kept = true;
result.side_chain_field_moved = static_cast<int>(field.moves.size()) - pruned;
@@ -4132,8 +4256,8 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
}
}
// 3. The final weights: a short ladder from where the trajectory ended, tighter geometry first, its choice
// kept where it displaces that. A rung the step from there tried already is taken from that step.
// 3. The final weights: a short ladder from where the trajectory ended, the weight as it is first, its choice
// on the free set taken where it advances from there (advances). A rung the step from there tried already is taken from that step.
if (!stopped && state.ran && !ctx.Cancelled()) {
const double w = state.trial.weight, adp = state.trial.adp_weight, ncs_s = state.trial.ncs_sigma;
std::vector<TrialWeights> trials;
@@ -4156,21 +4280,21 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
if (!outcomes[i].ran)
outcomes[i] = std::move(ran[m++]);
const int k = choose(outcomes, "final");
if (k >= 0 && displaces(outcomes[k].trial, state.trial))
if (k >= 0 && advances(outcomes[k].trial, state.trial))
state = std::move(outcomes[k]);
}
// 4. The displacement model, decided late, on the free set: from where the weights ended, TLS bodies (the
// model's own groups, one per chain, the chains in TLSMD segments) with a residual B, and a U per atom at
// its balanced weight and a quarter of it - each trial as long as a step, on the positions where the weights
// ended, held, so that the trials differ in the displacement model alone; the model chosen then takes a
// step with the positions refined, kept where it displaces its trial (against trials a macro-cycle longer
// step with the positions refined, kept where it advances on the work-set target from its trial (against trials a macro-cycle longer
// that refined the positions too, on 50 refinements of the battery R-free 0.001 lower on average and none
// higher by more than 0.0015, at 78% of the evaluations). Where the geometry is out of bounds there, which
// held positions cannot mend, each trial refines them too, a macro-cycle longer than a step, and no step
// follows. One displaces the one before it (the first
// where the weights ended, the B factors as they are) only where its geometry is within bounds and both its
// R-free (beyond the tie) and the -ln L of the free reflections are lower: the free set is the test of the
// parameters it adds.
// where the weights ended, the B factors as they are) only where its geometry is within bounds and the free
// set prefers it (FreeSetPrefers: its -ln L of the free reflections lower, its R-free not higher beyond its
// standard error): the free set is the test of the parameters it adds.
// Following Merritt (2012) Acta Cryst. D68, 468-477 (the displacement model the data support, decided by test)
// The trials that would not change the choice are left out: the model's own TLS groups are tried only where
// one body per chain or the finest cut lowers R-free, the cuts between those only where also the finest
@@ -4190,9 +4314,14 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
state.trial.r_free, state.trial.nll_free);
std::vector<AdpHypothesis> hyps = AdpHypotheses(ctx, topo, own_tls);
const double w = state.trial.weight, adp = state.trial.adp_weight > 0 ? state.trial.adp_weight : adp_w0;
// a U per atom only where the data are many times its parameters
int atoms_b = 0;
for (int i : refinable)
atoms_b += !ctx.atoms[i]->is_hydrogen();
const bool u_tried = result.n_work > U_MIN_REFLECTIONS_PER_ATOM * atoms_b;
// the U weight balanced on the model with every refinable atom given a U
double u_w0 = adp;
{
if (u_tried) {
const std::vector<AtomAniso> keep = ctx.aniso;
std::vector<AtomAniso> all = keep.empty() ? std::vector<AtomAniso>(ctx.atoms.size()) : keep;
std::vector<char> refines_u(ctx.atoms.size(), 0);
@@ -4215,6 +4344,8 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
int own = -1, coarsest = -1, finest = -1, u_quarter = -1;
for (const AdpHypothesis &h : hyps) {
if (h.kind == AdpHypothesis::Kind::Anisotropic) {
if (!u_tried)
continue;
u_quarter = static_cast<int>(trials.size());
for (double f : {0.25, 1.0}) {
trials.push_back({w, u_w0 * f, ncs_s});
@@ -4298,20 +4429,34 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
continue;
logger.Info("Refinement, displacements: {}", outcomes[k].line);
result.adp_trials.push_back(t);
if (t.within_bounds && (!trial(best).within_bounds ||
(t.r_free < trial(best).r_free - R_FREE_TIE && t.nll_free < trial(best).nll_free)))
// the larger model where Hamilton's tests take it, the smaller where they refuse it, the free set's
// choice where they are inconclusive (or the two have as many parameters)
const RefinementWeightTrial &b = trial(best);
const bool larger = t.adp_parameters > b.adp_parameters;
const double pass = larger ? HamiltonPassFraction(t, b, result.n_work)
: HamiltonPassFraction(b, t, result.n_work);
bool takes = displaces(t, b);
if (t.adp_parameters != b.adp_parameters && (pass > HAMILTON_LARGER || pass < HAMILTON_SMALLER))
takes = t.within_bounds && (!b.within_bounds || (pass > HAMILTON_LARGER) == larger);
logger.Info("Refinement, displacements: {} against {}: the larger passes {:.0f}% of Hamilton's tests{}",
t.adp.empty() ? "the B factors as they are" : t.adp,
b.adp.empty() ? "the B factors as they are" : b.adp, 100 * pass, takes ? ", taken" : "");
if (takes)
best = static_cast<int>(k);
}
// (the model where the weights ended keeps the displacement model a round before chose for it)
if (result.adp_model.empty() || state.trial.adp.empty())
// (the model where the weights ended keeps the displacement model a round before chose for it, its U held by
// the weight trials since)
if (result.adp_model.empty())
result.adp_model = result.adp_input == "isotropic" ? "isotropic" : "the model's U held";
else if (state.trial.adp.empty() && result.adp_model != "isotropic" && !result.adp_model.ends_with("held"))
result.adp_model += ", U held";
if (best >= 0) {
result.adp_model = outcomes[best].trial.adp;
state = std::move(outcomes[best]);
}
if (best >= 0 && hold) {
// the model chosen refined on with the positions, a step from where its trial ended, kept where it
// displaces that
// advances on the work-set target from there
AdpHypothesis chosen = *which[best];
chosen.bodies = state.tls;
chosen.tls_b = state.tls_b;
@@ -4320,7 +4465,7 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
ncs_p, {&chosen});
if (on[0].ran) {
logger.Info("Refinement, displacements, positions refined: {}", on[0].line);
if (displaces(on[0].trial, state.trial))
if (advances(on[0].trial, state.trial))
state = std::move(on[0]);
}
}
@@ -4332,17 +4477,24 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
// weights on its B factors, the ladder (1.) from it with its displacements as they are (the U of TLS or
// of the anisotropic model held, B held or refined), the trajectory (2.) from the rung kept, and the
// displacement hypotheses (4.) from where that ends - the TLS bodies refined from the B factors the first
// choice left, which they could not be from the B factors as given. A round is kept where it lowers R-free
// beyond the tie and undone otherwise, RefinementSettings::rounds in all: from a placed prediction, whose B
// choice left, which they could not be from the B factors as given. A round is kept where it advances as a step
// does (advances) and undone otherwise, RefinementSettings::rounds in all: from a placed prediction, whose B
// factors are its pLDDT (6 A^2 under an overall B of 45), the first round balanced the B factors' weight
// a hundred times too stiff - the B similarity's gradient goes as 1 / B - and chose the middle rung with
// NCS; the second chose the tightest rung without, and from the B factors the first left, its TLS bodies
// 0.02 lower in R-free. Where the ladder and the trajectory leave the model where it was, the hypotheses
// would run the same trials from the same model again, so the round ends there: from a model refined
// before, a round is its ladder and one step. No round where the refinement does not displace the model
// as given (5.).
// before, a round is its ladder and one step. No round where the model as given is kept over the
// refinement (5.).
// The model as given kept over the refinement: where the free set prefers it, as one weight is preferred to
// another (displaces), and the refinement is not better on both R-work and R-free. The work-set target cannot
// choose here, the refinement having minimised it from the model as given.
auto given_kept = [&]() {
return displaces(given.trial, state.trial) &&
!(state.trial.r_free < given.trial.r_free && state.trial.r_work < given.trial.r_work);
};
for (int round = 2; round <= settings.rounds && settings.adp_hypotheses && !stopped && state.ran &&
!ctx.Cancelled() && (!settings.as_given_competes || displaces(state.trial, given.trial));
!ctx.Cancelled() && (!settings.as_given_competes || !given_kept());
round++) {
const TrialOutcome before = state;
const std::vector<TlsBody> own_tls = state.tls;
@@ -4402,23 +4554,25 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
}
const std::string adp_before = result.adp_model;
displacements(own_tls.empty() ? input_tls : own_tls);
const bool kept = state.trial.r_free < before.trial.r_free - R_FREE_TIE;
logger.Info("Refinement, round {}: R-free {:.4f} -> {:.4f}{}", round, before.trial.r_free, state.trial.r_free,
kept ? "" : ", not kept");
const bool kept = advances(state.trial, before.trial);
logger.Info("Refinement, round {}: R-free {:.4f} -> {:.4f}, target {:.6g} -> {:.6g}{}", round,
before.trial.r_free, state.trial.r_free,
WorkTarget(before.trial, state.trial.weight, state.trial.adp_weight),
WorkTarget(state.trial, state.trial.weight, state.trial.adp_weight), kept ? "" : ", not kept");
if (!kept) {
state = before;
result.adp_model = adp_before;
break;
}
}
// 5. The model as given, first in order of trust: the refinement is kept where it displaces it by the rule every
// trial is held to, else the model is given back as it came (the solvent build's edits undone), so that what
// comes out is never worse on the free set than what went in - where the free set is free to the model as
// given (RefinementSettings::as_given_competes).
if (settings.as_given_competes && state.ran && !ctx.Cancelled() && !displaces(state.trial, given.trial)) {
logger.Info("Refinement: the model as given kept (R-work {:.4f}, R-free {:.4f}), the refinement did not "
"displace it (R-work {:.4f}, R-free {:.4f})", given.trial.r_work, given.trial.r_free,
state.trial.r_work, state.trial.r_free);
// 5. The model as given: where it is kept over the refinement (given_kept), it is given back as it came (the
// solvent build's edits undone), so that what comes out is never worse on both free measures than what went
// in - where the free set is free to the model as given (RefinementSettings::as_given_competes).
if (settings.as_given_competes && state.ran && !ctx.Cancelled() && given_kept()) {
logger.Info("Refinement: the model as given kept (R-work {:.4f}, R-free {:.4f}, -LL free {:.1f}), the "
"refinement did not displace it (R-work {:.4f}, R-free {:.4f}, -LL free {:.1f})",
given.trial.r_work, given.trial.r_free, given.trial.nll_free, state.trial.r_work,
state.trial.r_free, state.trial.nll_free);
size_t a = 0, k = 0;
for (gemmi::Chain &ch : ctx.Model().chains)
for (gemmi::Residue &res : ch.residues) {
@@ -160,6 +160,12 @@ struct RefinementWeightTrial {
double ncs_rmsd = NAN; // the copies' C-alpha (P) rmsd after superposition, where there are copies
double r_work = 1, r_free = 1;
double nll_free = NAN; // -ln L of the free reflections
// The work-set target's terms where the trial ended (RefineOnce's WorkTarget): -ln L of the working reflections,
// the restraints' sum of the geometry (with the NCS restraints at ncs_sigma) and that of the B similarity.
double nll_work = NAN, e_geometry = NAN, e_b = NAN;
int adp_parameters = 0; // the displacement model's parameters (HamiltonPassFraction)
int adp_restrained = 0; // of them those the restraints act on, which HamiltonPassFraction credits
std::vector<float> free_r_terms; // R-free as one term per free twin group (FreeRTerms), for paired comparisons
GeometryStatistics geometry;
bool within_bounds = false; // the geometry bounds the choice is made inside
std::string adp; // the displacement model: "isotropic", "TLS ..." or "anisotropic"