Restrained refinement on the GPU: ML amplitude target, FFT gradient, dictionary restraints, R1 + R2 driver, blobs
RefineModel (Refinement.h) refines a placed model against the merge, reading and writing nothing: - placement stage (R1): every chain of 50+ atoms as a rigid body at once (L-BFGS on six parameters per chain, their gradient from the per-atom one through Ceres jets), then one B shift per residue; - restrained stage (R2): positions and isotropic B of every atom of a residue the embedded dictionary describes, the rest held fixed but kept in Fcalc and the mask; riding hydrogens added by gemmi's topology, moved with their parents, re-placed every macro-cycle. Macro-cycles refit the mask, the scale and bulk solvent (FitModelScale / ModelScaleGPUPool) and the likelihood's D and beta (ML on the free set in resolution bins), rebuild the repulsion pairs (symmetry mates included), then L-BFGS on positions and, where chosen, on B. - weights: a ladder of full refinements at multiples of the gradient-balanced X-ray weight, chosen by R-free within bond/angle bounds (ties to the tighter geometry), then the B refinement tried at its own weights and kept only where it beats B held by more than the tie. The target and its gradient (RefinementXray, RefinementGPU): the Rice/centric amplitude likelihood (RefinementMath.h, shared host/device code), the gradient by Agarwal's FFT method - the adjoint of the reciprocal-space symmetry composition into a half-l (CPU) or half-u (cuFFT) spectrum, one inverse real transform, then each atom's density derivatives (xyz and B) summed over its box. On the GPU everything per evaluation stays on the device: ModelDensityGPU's gather, r2c, a fused compose + likelihood kernel, a CSR gather for the adjoint (no atomics), c2r, one warp per atom for the gradient; the restraints are one kernel per restraint plus a per-atom gather of gradient slots; cub reductions. Deterministic. Restraints (RefinementRestraints): bonds, angles, torsions (with esd, no phi/psi, no alternative sugar puckers), chiral volumes, least-squares planes from gemmi's Topo; repulsions beyond 1-4 (vdW radii from the dictionary's energy library, 2.6 A for H-bond pairs, 1.9 A metal-N/O/S); B similarity of bonded and angle-related atoms. DensityBlobs (D1): gemmi's flood-fill blobs of mFo-DFc above 3 sigma peaking above 4 sigma, placed beside the model and labelled WATER / ION / LIGAND / MODEL_ERROR by distance, contacts, volume and the anomalous map. tools/rugnux_refine_test refines a model against a finished run's p.mtz (developer utility, not installed). Tests: likelihood derivative, the embedded library, restraint values on ideal residues and every restraint kind's gradient against finite differences (CPU, and the GPU against the CPU), the X-ray gradient against finite differences in P1, P212121, C2 and P6122 (CPU and GPU), and a synthetic refinement that recovers a shaken model. Credits in docs/ACKNOWLEDGEMENT.md and at the algorithms. Measured on the open arm's battery outputs against REFMAC 5.8, 10 cycles, same p.mtz, free set and start model (R-free): 5reo 0.2228 vs 0.2236, 8dyz 0.1733 vs 0.1651, 9chw 0.1995 vs 0.1969 (CPU engine). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 8efe6f2322b3bfdae580bb34f256e67730dfc3c2)
This commit is contained in:
1 parent
e58453a693
commit
b4b68d368f
19 files changed
+4220
No files matched your search
@@ -508,6 +508,51 @@ molecular replacement; the error itself is AlphaFold's: J. Jumper, R. Evans, A.
|
||||
accurate protein structure prediction with AlphaFold" (2021), Nature 596, 583-589
|
||||
[doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2).
|
||||
|
||||
**Structure refinement** (`--refine`) — the placed model is refined against the maximum-likelihood
|
||||
amplitude target, the Rice distribution of an observed amplitude given the model's (its centric
|
||||
counterpart for a centric reflection): N. S. Pannu and R. J. Read, "Improved structure refinement
|
||||
through maximum likelihood" (1996), Acta Cryst. A52, 659-668
|
||||
[doi:10.1107/S0108767396004370](https://doi.org/10.1107/S0108767396004370); G. N. Murshudov,
|
||||
A. A. Vagin and E. J. Dodson, "Refinement of macromolecular structures by the maximum-likelihood method"
|
||||
(1997), Acta Cryst. D53, 240-255 [doi:10.1107/S0907444996012255](https://doi.org/10.1107/S0907444996012255).
|
||||
Its two parameters, D and the variance, are fitted by maximum likelihood on the free reflections in
|
||||
resolution bins: V. Y. Lunin and T. P. Skovoroda, "R-free likelihood-based estimates of errors for phases
|
||||
calculated from atomic models" (1995), Acta Cryst. A51, 880-887
|
||||
[doi:10.1107/S010876739500688X](https://doi.org/10.1107/S010876739500688X); V. Y. Lunin, P. V. Afonine and
|
||||
A. G. Urzhumtsev, "Likelihood-based refinement. I. Irremovable model errors" (2002), Acta Cryst. A58,
|
||||
270-282 [doi:10.1107/S0108767302001046](https://doi.org/10.1107/S0108767302001046). The gradient with
|
||||
respect to every atom is taken by the FFT method - the derivatives of the target with respect to the
|
||||
structure factors transformed into a map and summed against each atom's density derivatives:
|
||||
R. C. Agarwal, "A new least-squares refinement technique based on the fast Fourier transform algorithm"
|
||||
(1978), Acta Cryst. A34, 791-809 [doi:10.1107/S0567739478001618](https://doi.org/10.1107/S0567739478001618).
|
||||
The positions and B factors are minimised by L-BFGS (Ceres): D. C. Liu and J. Nocedal, "On the limited
|
||||
memory BFGS method for large scale optimization" (1989), Math. Program. 45, 503-528
|
||||
[doi:10.1007/BF01589116](https://doi.org/10.1007/BF01589116). The geometry restraints are those of the CCP4
|
||||
Monomer Library, a subset of which rugnux carries (see THIRD_PARTY_NOTICES.md): A. A. Vagin,
|
||||
R. A. Steiner, A. A. Lebedev, L. Potterton, S. McNicholas, F. Long and G. N. Murshudov, "REFMAC5
|
||||
dictionary: organization of prior chemical knowledge and guidelines for its use" (2004), Acta Cryst. D60,
|
||||
2184-2195 [doi:10.1107/S0907444904023510](https://doi.org/10.1107/S0907444904023510); applied to the model
|
||||
through GEMMI's topology (links, modifications, planes, chiral volumes, torsions): M. Wojdyr, "GEMMI: A
|
||||
library for structural biology" (2022), J. Open Source Softw. 7, 4200
|
||||
[doi:10.21105/joss.04200](https://doi.org/10.21105/joss.04200). The torsion derivatives follow
|
||||
A. Blondel and M. Karplus, "New formulation for derivatives of torsion angles and improper torsion angles
|
||||
in molecular mechanics: elimination of singularities" (1996), J. Comput. Chem. 17, 1132-1141
|
||||
[doi:10.1002/(SICI)1096-987X(19960715)17:9<1132::AID-JCC5>3.0.CO;2-T](https://doi.org/10.1002/(SICI)1096-987X(19960715)17:9%3C1132::AID-JCC5%3E3.0.CO;2-T).
|
||||
The B factors of bonded atoms are restrained to each other, as REFMAC5 restrains them (Murshudov et al.
|
||||
1997, above). The weight of the X-ray term against the geometry starts from the ratio of the two
|
||||
gradients and is chosen among several full refinements by R-free with the geometry held within bounds,
|
||||
as phenix.refine chooses it: P. V. Afonine, R. W. Grosse-Kunstleve, N. Echols, J. J. Headd,
|
||||
N. W. Moriarty, M. Mustyakimov, T. C. Terwilliger, A. Urzhumtsev, P. H. Zwart and P. D. Adams,
|
||||
"Towards automated crystallographic structure refinement with phenix.refine" (2012), Acta Cryst. D68,
|
||||
352-367 [doi:10.1107/S0907444912001308](https://doi.org/10.1107/S0907444912001308). R-free itself:
|
||||
A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal
|
||||
structures" (1992), Nature 355, 472-475 [doi:10.1038/355472a0](https://doi.org/10.1038/355472a0).
|
||||
|
||||
**Unmodelled density** — the blobs `--refine` lists are the connected regions of the difference map above
|
||||
a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI's implementation
|
||||
(`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development
|
||||
of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493).
|
||||
|
||||
## Software and computing methods
|
||||
|
||||
Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading
|
||||
|
||||
Reference in new issue
Block a user