Metal sites were held only by the 1.9 A metal contact of the repulsions; REFMAC restrains the metal LINKs of
the coordinate file (its documentation), which was the one geometry term the 0-cycle comparison did not
share (one 2.8 A set: bond rmsd 0.018 against REFMAC's 0.042).
- monomers/metals.cif: the Monomer Library's own statistics of metal sites (metals.json, LGPL data as the
rest of monomers/), cut by update_monomers.py to the metal ions of its compound list and O, N, S donors:
mean distance, spread and number of sites per metal, donor element and coordination number.
- AddMetalLinks (before prepare_topology): every LINK / struct_conn record between a metal ion and an O, N
or S that no link of the dictionary describes (its ZN-CYS, ZN-HIS, FE-CYS, FE-HIS are taken as they are)
gets a link of one bond at EmbeddedMetalDistance for the two elements and the ion's coordination - the
number of its donor records, symmetry mates included - from the rows resting on at least 20 sites (the
coordination's own row, else the neighbours interpolated), sigma their spread (>= 0.02 A); a His or Cys
donor loses the proton the bond replaces (DEL-HD1 / DEL-HE2 / CYSmod1, as the library's links do).
Records only, no distance search; records to a symmetry mate count for the coordination but are not
restrained (bonds to symmetry mates are not, anywhere).
- The metal bonds are a restraint set of their own (RefinementTopology::metal_bonds), evaluated with the
geometry and reported apart (GeometryStatistics::rmsd_metal, REFINE_RMSD_METAL_BONDS,
REFINE_SCORE_RMSD_METAL_BONDS / _BONDS_ALL): their sigma is the spread of real sites, so they sit
further from ideal than covalent bonds, and the weight ladder's 0.02 A bound stays a bound on the
covalent ones. refine_parity.py compares REFMAC's bond rmsd (links included) with the combined one, and
takes --gpu.
Score-only geometry against REFMAC 5.8 0-cycle, all bonds (count): 2.8 A Zn/Fe set 0.0402 (4934) vs 0.042
(4935), was 0.0178; a Zn-Cys set 0.0186 (7102) vs 0.019 (7102); a Ca set 0.0297 vs 0.026 (REFMAC also
refines the sugars and uses one Ca-O distance); a Mg set 0.0114 vs 0.017 (REFMAC restrains 8 symmetry
links and puts Mg-O at 1.99 A, the library's statistics at 2.07). R of the scored model unchanged (+0.0001
on the first from the removed His/Cys protons). Refined R-free with the links vs without: +0.0016, -0.0002,
+0.0005, +0.0004.
Test: RefinementRestraints_metal_links (Zn-Cys4 by the dictionary's link, Mg(H2O)6 by the new ones, both
back at their distances).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 0fcffc5c71e03e184cff18f07524473462a57b68)
- RefinementSettings::max_trials_at_once (rugnux_refine_test --trials-at-once): 0 as many as the ladders and
the cards take, 1 one after another - the check that side by side changes nothing. Verified: refined
model and report identical, side by side vs one after another, GPU (5reo, 9yl4, 6o2h, 9rcs) and CPU
engine (6o2h, 9rcs, 5reo), and run to run.
- RefinementSettings::cancel, an atomic flag for the broker's live deadline: checked at every iteration of
the L-BFGS (positions, B, rigid bodies, group B: the minimiser stops where it stands), at every
macro-cycle and before every weight trial. The trials that ran are chosen among as usual; cut before the
first, the model is the placement's. RefinementResult::cancelled says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit f708d3d227bb42215ffe3d9a15c766dbd73b783c)
The score draws the mask on a grid of its own (min(d_min/3, 0.4 A), BulkSolventMask) on the host; on a GPU
engine that was a CPU mask and transform every cycle, per trial slot on one thread. Now the engine masks
on that grid itself:
- RefinementGPUSetup::mask_grid (BulkSolventMask::MaskGrid, the grid the host mask uses); where it is not
the structure factors' grid the engine reserves a grid, a spectrum and an R2C plan for it (counted in
DeviceBytes and the FFT work area), and Fmask masks (ModelMaskGPU), transforms and reads the rows there.
Refit takes the engine's Fmask on a GPU, BulkSolventMask on the CPU.
- ModelMaskGPU's shrink: each edge point marks its stencil as solvent instead of every macromolecule point
searching its stencil for an edge - the same idempotent stores to the same points, so the same mask to
the bit (ModelMaskGPU_* tests unchanged), and a thin shell of points visited instead of half the grid.
- BulkSolventMask finds its symmetry orbits at its first mask: a GPU refinement never draws one on the host
(1.1 s of trial-slot setup on the 3.6 A set).
The card's mask is gemmi's but at points exactly on an atom's radius (ModelMaskGPU.h): R-work/R-free of the
14 parity sets on the GPU are the CPU engine's to 4 digits. Refinement deterministic run to run on the GPU
(7 timing sets, refined model and report identical over 5 runs of successive builds that did not change
results).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 89b5d00907c9ad99e8b389f6272a3724f981cd10)
The score's robust fine pass (pseudo-Huber IRLS of ModelScaleBox::robust) on the device, so that refinement on
a GPU no longer fits its scale on the host every cycle - with the trials side by side that was a robust grid
search on one thread per slot (13-133 s per refinement on the seven timing sets, against 1.7-8 s before the
score commit).
- ModelScaleGPU: the weights multiply the WSSR and J^T J / J^T r in scale_sums (a weight row per fit,
reserved at the first robust fit); robust_residuals / robust_weights kernels; the median of each fit's
first residuals taken on the host as FitModelScale takes it (nth_element at n/2, on the fits' threads);
LevMarBatch split out of FitBatch so that the rounds restart from where a fit stands. FitSolvent(...,
robust) refits the coarse winner in the fine pass, as FitModelScale's robust fit does; the plain fit is
unchanged.
- ModelScaleGPUPool::Fit(..., robust); its streams at the highest priority: a fit is a chain of small
launches the host waits on, and on a card shared with the trials' long kernels each wait queued behind
them (refit.scale 58-87 ms -> 14-45 ms per cycle, the result the same to the bit).
- Refinement: the main context and every trial slot fit on their pool (robust); the CPU engine keeps
FitModelScale's robust fit.
The card's fit is FitModelScale's to rounding (float residuals and weights, the steps' anisotropic factor in
float), deterministic run to run and whichever engine a fit takes: R-work/R-free of the 14 parity sets on
the GPU equal the CPU engine's to 4 digits (largest difference 1.4e-6 on the 3.6 A set).
Test: ModelScaleGPUPool_RobustMatchesFitModelScale (one reflection in fifty a 4x outlier, five cells).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit d15f048cbd7ed2d6b430f9f9cb5018ca284da8f0)
Three terms of the score of a model - the same F_model at every stage, the as-given start and the
refined end alike - and a breakdown of it:
- Riding hydrogens from the start: the dictionary's hydrogens are added before the model is first
scored (REFMAC's 0-cycle restores them; they move R by 0.010-0.013 at 1.1 A). The engine is made
once, with them.
- The bulk-solvent mask on a grid of its own, min(d_min/3, 0.4 A), at most 2^26 points. On the
density grid (1.1 A at 3.3 A) gemmi's 0.8 A shrink found no grid offset and was skipped, which put
R 0.015-0.03 high at >= 2.5 A. 0.3 and 0.4 A agree to 0.001.
- The overall scale and anisotropic B by robust least squares (pseudo-Huber, iteratively
reweighted, c = median |residual|) in the fine pass of the k_sol/b_sol grid (ModelScaleBox::robust,
refinement only; MR and validation keep the plain fit). Plain least squares on |F| is the exact LS
optimum but lets a few badly-fitted strong reflections set the scale (1.1 A set: R 0.1706 against
REFMAC's 0.1600). The scale fit of refinement now runs on the CPU (no ModelScaleGPUPool).
RefinementSettings::score_only scores the model as given and stops; RefinementResult::score_start
carries every term (k, B, anisotropic B, k_sol, b_sol, mask spacing, R in 20 shells, -lnL work/free,
the likelihood FOM, hydrogens; per-reflection terms with score_only). GeometryStatistics gains rmsZ.
rugnux_refine_test: --score-only, --dump.
Measured against REFMAC 5.8.0431 ncyc 0 (refi bref iso) on the same model, data and flags, 14 open-arm
sets 1.1-3.6 A: R-work/R-free within 0.002 on 12 (was +0.003..+0.03); 7n2s and 9yl4 score 0.003/0.005
below REFMAC, which a 0.6-0.8 A mask grid reproduces. REFMAC with ANISOU stripped scores identically,
so isotropic B is not a parity term. Open: the likelihood's D/FOM (ours 0.01-0.11 higher at high
resolution); geometry differs only by REFMAC's metal LINK restraints and residues outside the
embedded dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 4f94b21611874ad9dcf89e6771fe311a0e10bb06)
(cherry picked from commit 00db6a6f539ac2da515c0a649474335f4c4b07b8)
Every rung of the weight ladder starts from the same model and depends on no other, so the six position
trials, and then the three B trials at the chosen weight, run at the same time: on the main model and engine
and on up to five slots beside it, each a copy of the model before the topology was made (the topology is
built again on it, hydrogens included) with its own engine. The slots go on the GPUs in turn from the main
engine's, as many as each card's budget takes, so a machine with several cards spreads them; a slot whose
engine cannot be made is left out, and with none the trials run one after another on the main engine as
before. The threads are shared out among the slots. The log lines and the choice are made afterwards in
ladder order, so the log, the trial table and the refined model are the same, bit for bit (GPU and CPU
engines checked).
5reo 3.9 -> 2.5 s, 9yl4 13.4 -> 7.8 s, 6o2h 3.1 -> 1.9 s, 8dyz 4.6 -> 2.7 s, 9rcs 2.9 -> 2.2 s (one RTX 5080, -N 8).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 0ea6a6db8aba640afc5249fe748922ddfaa2537a)
(cherry picked from commit 0c0176c8bf571ed26f8a68a9de40222487c6200e)
RefinementEngine::EvaluateWithRestraints evaluates both targets at the model's atoms. On the GPU the X-ray
chain is queued first, the restraints' coordinates are written into a pinned buffer while the device works
on it, the restraint kernels follow on the same stream, and the host waits once; the sums and gradients
come back through pinned buffers. Same kernels on the same inputs, same bits. The CPU engine keeps calling
the two in turn.
9yl4: evaluation 5.1 -> 4.7 ms, refinement 14.2 -> 13.4 s; 5reo 4.0 -> 3.9 s (RTX 5080, -N 8).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 51434b98b62a9e6322ffe811bf272a45ff6ce003)
(cherry picked from commit 0bcfa3f8041cec4bb4bc376b8559f30c36d3ac80)
BuildRepulsions walked the neighbours of every atom and only then dropped the pairs of a riding hydrogen;
those atoms are now skipped before the walk. The atoms are searched in 64 blocks under ParallelFor and the
blocks' pairs joined in atom order, so the restraint list is the same as before for any thread count. The
model is no longer copied for the NeighborSearch. Same bits.
9yl4: repulsions 3.37 -> 1.09 s (91 calls), refinement 17.7 -> 14.2 s; 5reo 0.38 -> 0.12 s
(RTX 5080, -N 8, loaded box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit ded11388a18f5a4138a73150c4bb073570152727)
(cherry picked from commit 9c96836a90c9f170008d343af2cd885a4e59b321)
Every evaluation uploaded both copies of the atoms' densities (the gather's and the gradient's) and their
boxes, although in a positions-only minimisation no B changes between two Refits. The engine now keeps one
copy (the gradient kernel reads ModelDensityGPU's atoms and boxes), uploads it only where some atom's
density was made again, and sends the positions from a pinned buffer without waiting. The densities that
are made again (every atom in a B minimisation) and the grid positions are worked out on the refinement's
threads. Same bits.
9yl4: upload 3.29 -> 0.56 s, host side 1.29 -> 0.61 s, refinement 22.3 -> 17.7 s; 5reo 4.8 -> 4.4 s
(RTX 5080, -N 8, loaded box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 46c3b67c91aee0aec3473b0fb1a4945c00bda070)
(cherry picked from commit 0aebc98110c8155932af647d6204891aec662294)
EstimateLikelihoodParameters fits each resolution bin on its own and writes only that bin's values, so
the bins run under ParallelFor; the result is the same, bit for bit. It was the largest single stage of
a refinement (107 calls, one core): 5reo 2.42 -> 0.56 s, 8dyz 2.46 -> 0.57 s, 9yl4 3.54 -> 0.88 s;
whole refinement 5reo 6.3 -> 4.8 s, 8dyz 6.9 -> 5.3 s (RTX 5080, -N 8, loaded box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 85aa34d3031c8e36be4462fca544bc801af45d5a)
(cherry picked from commit d84ba55252adf455d5b4ba62052dfbafa6290a64)
RefinementStage sums the time of each stage over a refinement (Refit and its parts, the evaluations and
their X-ray, restraint, host and upload shares, the repulsion build, the riding hydrogens, the rungs) and
logs it at debug level at the end; rugnux_refine_test -v shows it. No change to any result.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 9491f54e0beccf36c65ebcec9afff90d08485f86)
(cherry picked from commit 0ce206a88df960754476ce2f75ffaed90fbf3123)
On deposited models the per-residue B shift raised R-free at every resolution tried (2.56 A: 0.320 ->
0.327) and the rigid bodies rarely helped; each is now kept only where it lowers R-free by more than the
0.001 tie, otherwise the model goes on as it was. The positions' weight ladder runs from 1/1024 to 1/32 of
the gradient-balanced weight: the R-free optimum sat near 1/500 at 2.6 A and near 1/50 at 1.1-1.7 A.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit d444fb03f610db70903de04f53f0489a71540b37)
RefineModel (Refinement.h) refines a placed model against the merge, reading and writing nothing:
- placement stage (R1): every chain of 50+ atoms as a rigid body at once (L-BFGS on six parameters per
chain, their gradient from the per-atom one through Ceres jets), then one B shift per residue;
- restrained stage (R2): positions and isotropic B of every atom of a residue the embedded dictionary
describes, the rest held fixed but kept in Fcalc and the mask; riding hydrogens added by gemmi's
topology, moved with their parents, re-placed every macro-cycle. Macro-cycles refit the mask, the
scale and bulk solvent (FitModelScale / ModelScaleGPUPool) and the likelihood's D and beta (ML on
the free set in resolution bins), rebuild the repulsion pairs (symmetry mates included), then L-BFGS
on positions and, where chosen, on B.
- weights: a ladder of full refinements at multiples of the gradient-balanced X-ray weight, chosen by
R-free within bond/angle bounds (ties to the tighter geometry), then the B refinement tried at its own
weights and kept only where it beats B held by more than the tie.
The target and its gradient (RefinementXray, RefinementGPU): the Rice/centric amplitude likelihood
(RefinementMath.h, shared host/device code), the gradient by Agarwal's FFT method - the adjoint of the
reciprocal-space symmetry composition into a half-l (CPU) or half-u (cuFFT) spectrum, one inverse real
transform, then each atom's density derivatives (xyz and B) summed over its box. On the GPU everything
per evaluation stays on the device: ModelDensityGPU's gather, r2c, a fused compose + likelihood kernel,
a CSR gather for the adjoint (no atomics), c2r, one warp per atom for the gradient; the restraints are
one kernel per restraint plus a per-atom gather of gradient slots; cub reductions. Deterministic.
Restraints (RefinementRestraints): bonds, angles, torsions (with esd, no phi/psi, no alternative sugar
puckers), chiral volumes, least-squares planes from gemmi's Topo; repulsions beyond 1-4 (vdW radii from
the dictionary's energy library, 2.6 A for H-bond pairs, 1.9 A metal-N/O/S); B similarity of bonded and
angle-related atoms.
DensityBlobs (D1): gemmi's flood-fill blobs of mFo-DFc above 3 sigma peaking above 4 sigma, placed beside
the model and labelled WATER / ION / LIGAND / MODEL_ERROR by distance, contacts, volume and the anomalous
map.
tools/rugnux_refine_test refines a model against a finished run's p.mtz (developer utility, not
installed). Tests: likelihood derivative, the embedded library, restraint values on ideal residues and
every restraint kind's gradient against finite differences (CPU, and the GPU against the CPU), the X-ray
gradient against finite differences in P1, P212121, C2 and P6122 (CPU and GPU), and a synthetic
refinement that recovers a shaken model. Credits in docs/ACKNOWLEDGEMENT.md and at the algorithms.
Measured on the open arm's battery outputs against REFMAC 5.8, 10 cycles, same p.mtz, free set and start
model (R-free): 5reo 0.2228 vs 0.2236, 8dyz 0.1733 vs 0.1651, 9chw 0.1995 vs 0.1969 (CPU engine).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8efe6f2322b3bfdae580bb34f256e67730dfc3c2)
leonarski_f