Every rung of the old ladders (6 positional, then 3 B) was a full 10-cycle refinement from the placed model, so
~90% of the evaluations were thrown away and the kept trajectory had 100 iterations. The run is now:
1. a ladder from the placed model, 2 macro-cycles per trial: three positional weights (w0/1024, /128, /16), each
with the B factors held and refined (adp = 4 w), chosen by the old rule (tighter first, a later trial wins only
within the geometry bounds and by more than the R-free tie);
2. a trajectory of steps from where the last one ended, each two trials side by side - the weights as they are and
the X-ray term raised, in turn the positions' (x2, at most w0/8) and the B factors' (x4) - until a step lowers
neither R-work nor R-free by more than the tie (at most 8 steps);
3. a final ladder from there at x1/2, x1, x2.
A macro-cycle is 30 positional and 20 B iterations (the minimiser's memory kept within a trial). A trial starts
from the scale, solvent and likelihood fit its predecessor ended with (the same computation, so result-neutral),
and the slots that run share the threads.
On 5 battery MD starts (start 1 of 5f6m 1.09 A, 8dqb 2.04, 7arr 0.92, 11if 1.37, 7d98 3.32; R-free by rugnux's
scorer, REFMAC 0-cycle within 0.002), previous commit -> this, against REFMAC run to convergence (40-100 cycles):
5f6m 0.215 -> 0.167 (REFMAC 0.168), 8dqb 0.341 -> 0.290 (0.284), 7arr 0.356 -> 0.289 (0.288), 11if 0.295 ->
0.236 (0.244), 7d98 0.393 -> 0.337 (0.360). Time 5.4 / 8.6 / 7.3 / 3.9 / 10.0 s on the RTX 5080 (base 6-7 s).
Halving a weight during the trajectory never won in the logs, so steps only try raising it; the final ladder
can lower it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 2bfb84bc4f3c72e8da37e54b728839b65a636519)
- RefinementSettings::max_trials_at_once (rugnux_refine_test --trials-at-once): 0 as many as the ladders and
the cards take, 1 one after another - the check that side by side changes nothing. Verified: refined
model and report identical, side by side vs one after another, GPU (5reo, 9yl4, 6o2h, 9rcs) and CPU
engine (6o2h, 9rcs, 5reo), and run to run.
- RefinementSettings::cancel, an atomic flag for the broker's live deadline: checked at every iteration of
the L-BFGS (positions, B, rigid bodies, group B: the minimiser stops where it stands), at every
macro-cycle and before every weight trial. The trials that ran are chosen among as usual; cut before the
first, the model is the placement's. RefinementResult::cancelled says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit f708d3d227bb42215ffe3d9a15c766dbd73b783c)
Three terms of the score of a model - the same F_model at every stage, the as-given start and the
refined end alike - and a breakdown of it:
- Riding hydrogens from the start: the dictionary's hydrogens are added before the model is first
scored (REFMAC's 0-cycle restores them; they move R by 0.010-0.013 at 1.1 A). The engine is made
once, with them.
- The bulk-solvent mask on a grid of its own, min(d_min/3, 0.4 A), at most 2^26 points. On the
density grid (1.1 A at 3.3 A) gemmi's 0.8 A shrink found no grid offset and was skipped, which put
R 0.015-0.03 high at >= 2.5 A. 0.3 and 0.4 A agree to 0.001.
- The overall scale and anisotropic B by robust least squares (pseudo-Huber, iteratively
reweighted, c = median |residual|) in the fine pass of the k_sol/b_sol grid (ModelScaleBox::robust,
refinement only; MR and validation keep the plain fit). Plain least squares on |F| is the exact LS
optimum but lets a few badly-fitted strong reflections set the scale (1.1 A set: R 0.1706 against
REFMAC's 0.1600). The scale fit of refinement now runs on the CPU (no ModelScaleGPUPool).
RefinementSettings::score_only scores the model as given and stops; RefinementResult::score_start
carries every term (k, B, anisotropic B, k_sol, b_sol, mask spacing, R in 20 shells, -lnL work/free,
the likelihood FOM, hydrogens; per-reflection terms with score_only). GeometryStatistics gains rmsZ.
rugnux_refine_test: --score-only, --dump.
Measured against REFMAC 5.8.0431 ncyc 0 (refi bref iso) on the same model, data and flags, 14 open-arm
sets 1.1-3.6 A: R-work/R-free within 0.002 on 12 (was +0.003..+0.03); 7n2s and 9yl4 score 0.003/0.005
below REFMAC, which a 0.6-0.8 A mask grid reproduces. REFMAC with ANISOU stripped scores identically,
so isotropic B is not a parity term. Open: the likelihood's D/FOM (ours 0.01-0.11 higher at high
resolution); geometry differs only by REFMAC's metal LINK restraints and residues outside the
embedded dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 4f94b21611874ad9dcf89e6771fe311a0e10bb06)
(cherry picked from commit 00db6a6f539ac2da515c0a649474335f4c4b07b8)
RefinementStage sums the time of each stage over a refinement (Refit and its parts, the evaluations and
their X-ray, restraint, host and upload shares, the repulsion build, the riding hydrogens, the rungs) and
logs it at debug level at the end; rugnux_refine_test -v shows it. No change to any result.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 9491f54e0beccf36c65ebcec9afff90d08485f86)
(cherry picked from commit 0ce206a88df960754476ce2f75ffaed90fbf3123)
On deposited models the per-residue B shift raised R-free at every resolution tried (2.56 A: 0.320 ->
0.327) and the rigid bodies rarely helped; each is now kept only where it lowers R-free by more than the
0.001 tie, otherwise the model goes on as it was. The positions' weight ladder runs from 1/1024 to 1/32 of
the gradient-balanced weight: the R-free optimum sat near 1/500 at 2.6 A and near 1/50 at 1.1-1.7 A.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit d444fb03f610db70903de04f53f0489a71540b37)
RefineModel (Refinement.h) refines a placed model against the merge, reading and writing nothing:
- placement stage (R1): every chain of 50+ atoms as a rigid body at once (L-BFGS on six parameters per
chain, their gradient from the per-atom one through Ceres jets), then one B shift per residue;
- restrained stage (R2): positions and isotropic B of every atom of a residue the embedded dictionary
describes, the rest held fixed but kept in Fcalc and the mask; riding hydrogens added by gemmi's
topology, moved with their parents, re-placed every macro-cycle. Macro-cycles refit the mask, the
scale and bulk solvent (FitModelScale / ModelScaleGPUPool) and the likelihood's D and beta (ML on
the free set in resolution bins), rebuild the repulsion pairs (symmetry mates included), then L-BFGS
on positions and, where chosen, on B.
- weights: a ladder of full refinements at multiples of the gradient-balanced X-ray weight, chosen by
R-free within bond/angle bounds (ties to the tighter geometry), then the B refinement tried at its own
weights and kept only where it beats B held by more than the tie.
The target and its gradient (RefinementXray, RefinementGPU): the Rice/centric amplitude likelihood
(RefinementMath.h, shared host/device code), the gradient by Agarwal's FFT method - the adjoint of the
reciprocal-space symmetry composition into a half-l (CPU) or half-u (cuFFT) spectrum, one inverse real
transform, then each atom's density derivatives (xyz and B) summed over its box. On the GPU everything
per evaluation stays on the device: ModelDensityGPU's gather, r2c, a fused compose + likelihood kernel,
a CSR gather for the adjoint (no atomics), c2r, one warp per atom for the gradient; the restraints are
one kernel per restraint plus a per-atom gather of gradient slots; cub reductions. Deterministic.
Restraints (RefinementRestraints): bonds, angles, torsions (with esd, no phi/psi, no alternative sugar
puckers), chiral volumes, least-squares planes from gemmi's Topo; repulsions beyond 1-4 (vdW radii from
the dictionary's energy library, 2.6 A for H-bond pairs, 1.9 A metal-N/O/S); B similarity of bonded and
angle-related atoms.
DensityBlobs (D1): gemmi's flood-fill blobs of mFo-DFc above 3 sigma peaking above 4 sigma, placed beside
the model and labelled WATER / ION / LIGAND / MODEL_ERROR by distance, contacts, volume and the anomalous
map.
tools/rugnux_refine_test refines a model against a finished run's p.mtz (developer utility, not
installed). Tests: likelihood derivative, the embedded library, restraint values on ideal residues and
every restraint kind's gradient against finite differences (CPU, and the GPU against the CPU), the X-ray
gradient against finite differences in P1, P212121, C2 and P6122 (CPU and GPU), and a synthetic
refinement that recovers a shaken model. Credits in docs/ACKNOWLEDGEMENT.md and at the algorithms.
Measured on the open arm's battery outputs against REFMAC 5.8, 10 cycles, same p.mtz, free set and start
model (R-free): 5reo 0.2228 vs 0.2236, 8dyz 0.1733 vs 0.1651, 9chw 0.1995 vs 0.1969 (CPU engine).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8efe6f2322b3bfdae580bb34f256e67730dfc3c2)
leonarski_f