Deterministic, rounding changes. The per-bin ML fit took 4 rounds x (40 + 40) golden-section steps on the
free set and a 50-step golden section for D on the working set - each step a Bessel I0/I1 per row, ~4.6M per
Refit at 92k reflections, 27-37% of a trial's wall on the high-resolution sets. Each one-dimensional search is
now a safeguarded Newton iteration on the analytic first and second derivatives (bracketed on the sign of the
derivative, bisection where a step leaves the bracket or the curvature is not positive), within the same
brackets; the free-set alternation keeps its 4 rounds from the moments, so beta is the one it was (a converged
joint fit was tried: the 4 rounds are not converged in the low-resolution bins, and converging them moves
6iu9's FOM by -0.003, so it was not kept). D on the working set starts from the bin's D of the context's
previous Refit (XrayRowModel::bin_d), restored with a trial's fit and cleared at a trial's first Refit, so a
trial depends on nothing its slot ran before (side by side = one at a time, md5).
14 parity sets, score only: R-work/R-free identical to 4 digits, mean |FOM - REFMAC| 0.01186 -> 0.01186,
D within 1e-5 and var within 1e-4 (relative) per reflection. refit.ml (TIMING sum, -N 8): 5reo 1.47 -> 0.11 s,
5f6m 3.88 -> 0.26, 9yl4 1.37 -> 0.11, 5f6m MD start 4.97 -> 0.46. Wall: 5reo 2.03 -> 1.75, 5f6m 3.45 -> 2.60,
9yl4 7.99 -> 7.36, 7d98 4.04 -> 3.78, 5f6m MD start 6.10 -> 4.82 s. 5f6m MD starts 01-04 median R-free
0.1693 -> 0.1692.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit a91511945e0ccd7145a93877c0c231e87f86fb44)
BulkSolventMask::MaskGrid, documented as "sizes and spacing set, values not allocated", allocated and zeroed the
whole fine mask grid (up to 2^26 floats) - in BulkSolventMask's constructor and in every GPU engine's, i.e. once
more for each weight-trial slot, only for the grid's size. It now takes good_grid_size() as set_size_from_spacing()
does, and the CPU engine's Fmask() allocates its copy. rugnux_refine_test, RTX 5080, -N 8: 9yl4 8.6 -> 7.8 s,
7d98 4.85 -> 4.0 s (engine 0.66 -> 0.16 s, slots 0.60 -> 0.23 s on 9yl4); the high-resolution sets, whose mask
grid is small, are unchanged.
RefinementMinimiser keeps s.y of every pair it stores and Direction() takes rho and gamma from it instead of two
more dot products per pair per call (the same Dot() of the same vectors): Direction's CPU time -22% on 9yl4; no
wall change with the trials side by side, where other slots fill the card meanwhile.
Bitwise: refined CIF identical on 5reo, 5f6m, 9yl4, 7d98, a 5f6m MD start (GPU) and 6o2h (CPU engine).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 6f5caf5e1b1183a752acc5cca78bfb7ef002b12a)
EstimateLikelihoodParameters' golden-section searches over D at a fixed beta - four per bin on the free
reflections, and the one on the working reflections the D/FOM change added, 50 evaluations over 95 % of
the data - evaluated -ln L reading every row through the index and taking log(var) at every evaluation.
The rows are now gathered per search with their variance and its logarithm; the sum is the same
expressions in the same order, so D, beta and everything after are the same to the bit (refined model and
report identical on the seven timing sets). refit.ml 200 -> 140 ms per cycle on the 1.1 A set, one thread per
trial slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 23ad3bfc254d10af310f17dbb8f85616f666a783)
The likelihood's variance per bin (beta) is fitted by maximum likelihood on the free reflections as the whole
spread of Fo about D Fc, and D is then fitted on the working reflections with the variance eps beta + 2 sigma^2
(acentric; + sigma^2 centric), which is also the target's variance. Murshudov et al. (2011) Acta D67 355:
D from the working set, the variance from the free set. The measurement error is thereby counted twice, so D
and the figure of merit fall where sigma is a large part of the spread - at high resolution.
Measured on REFMAC 5.8.0431 0-cycle output (black box) of the 14 parity sets: its FOM inverted per reflection
gives var = eps a + 2 sigma^2 (acentric, fitted coefficient 2.00) and eps a + sigma^2 (centric); its D
(FC_ALL/FC_ALL_LS = the log's SigmaA_Fc1) and a are reproduced by this estimate (on REFMAC's own Fc, one set
to rms 0.009 in D over all 20 bins). The previous per-bin free-set ML put D at 0.39-0.48 where REFMAC has
0.06-0.13 in the outermost bins.
rugnux score vs REFMAC 0-cycle, 14 sets: mean |FOM difference| 0.056 -> 0.012 (max 0.112 -> 0.042), per-bin
D rms 0.107 -> 0.068; R-work/R-free unchanged (D does not enter them). The score's FOM is now the mean over
the working reflections, as REFMAC reports it, and the refined model's maps are weighted by the refinement's
own m and D rather than a separate sigma_A estimate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit d00d635e2874bff6cd216b7f24367362d0cc2e56)
(cherry picked from commit b93e0ea2fc1ffcc6bb40dae9b77816a321f4d6ec)
The score draws the mask on a grid of its own (min(d_min/3, 0.4 A), BulkSolventMask) on the host; on a GPU
engine that was a CPU mask and transform every cycle, per trial slot on one thread. Now the engine masks
on that grid itself:
- RefinementGPUSetup::mask_grid (BulkSolventMask::MaskGrid, the grid the host mask uses); where it is not
the structure factors' grid the engine reserves a grid, a spectrum and an R2C plan for it (counted in
DeviceBytes and the FFT work area), and Fmask masks (ModelMaskGPU), transforms and reads the rows there.
Refit takes the engine's Fmask on a GPU, BulkSolventMask on the CPU.
- ModelMaskGPU's shrink: each edge point marks its stencil as solvent instead of every macromolecule point
searching its stencil for an edge - the same idempotent stores to the same points, so the same mask to
the bit (ModelMaskGPU_* tests unchanged), and a thin shell of points visited instead of half the grid.
- BulkSolventMask finds its symmetry orbits at its first mask: a GPU refinement never draws one on the host
(1.1 s of trial-slot setup on the 3.6 A set).
The card's mask is gemmi's but at points exactly on an atom's radius (ModelMaskGPU.h): R-work/R-free of the
14 parity sets on the GPU are the CPU engine's to 4 digits. Refinement deterministic run to run on the GPU
(7 timing sets, refined model and report identical over 5 runs of successive builds that did not change
results).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 89b5d00907c9ad99e8b389f6272a3724f981cd10)
Three terms of the score of a model - the same F_model at every stage, the as-given start and the
refined end alike - and a breakdown of it:
- Riding hydrogens from the start: the dictionary's hydrogens are added before the model is first
scored (REFMAC's 0-cycle restores them; they move R by 0.010-0.013 at 1.1 A). The engine is made
once, with them.
- The bulk-solvent mask on a grid of its own, min(d_min/3, 0.4 A), at most 2^26 points. On the
density grid (1.1 A at 3.3 A) gemmi's 0.8 A shrink found no grid offset and was skipped, which put
R 0.015-0.03 high at >= 2.5 A. 0.3 and 0.4 A agree to 0.001.
- The overall scale and anisotropic B by robust least squares (pseudo-Huber, iteratively
reweighted, c = median |residual|) in the fine pass of the k_sol/b_sol grid (ModelScaleBox::robust,
refinement only; MR and validation keep the plain fit). Plain least squares on |F| is the exact LS
optimum but lets a few badly-fitted strong reflections set the scale (1.1 A set: R 0.1706 against
REFMAC's 0.1600). The scale fit of refinement now runs on the CPU (no ModelScaleGPUPool).
RefinementSettings::score_only scores the model as given and stops; RefinementResult::score_start
carries every term (k, B, anisotropic B, k_sol, b_sol, mask spacing, R in 20 shells, -lnL work/free,
the likelihood FOM, hydrogens; per-reflection terms with score_only). GeometryStatistics gains rmsZ.
rugnux_refine_test: --score-only, --dump.
Measured against REFMAC 5.8.0431 ncyc 0 (refi bref iso) on the same model, data and flags, 14 open-arm
sets 1.1-3.6 A: R-work/R-free within 0.002 on 12 (was +0.003..+0.03); 7n2s and 9yl4 score 0.003/0.005
below REFMAC, which a 0.6-0.8 A mask grid reproduces. REFMAC with ANISOU stripped scores identically,
so isotropic B is not a parity term. Open: the likelihood's D/FOM (ours 0.01-0.11 higher at high
resolution); geometry differs only by REFMAC's metal LINK restraints and residues outside the
embedded dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 4f94b21611874ad9dcf89e6771fe311a0e10bb06)
(cherry picked from commit 00db6a6f539ac2da515c0a649474335f4c4b07b8)
RefinementEngine::EvaluateWithRestraints evaluates both targets at the model's atoms. On the GPU the X-ray
chain is queued first, the restraints' coordinates are written into a pinned buffer while the device works
on it, the restraint kernels follow on the same stream, and the host waits once; the sums and gradients
come back through pinned buffers. Same kernels on the same inputs, same bits. The CPU engine keeps calling
the two in turn.
9yl4: evaluation 5.1 -> 4.7 ms, refinement 14.2 -> 13.4 s; 5reo 4.0 -> 3.9 s (RTX 5080, -N 8).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 51434b98b62a9e6322ffe811bf272a45ff6ce003)
(cherry picked from commit 0bcfa3f8041cec4bb4bc376b8559f30c36d3ac80)
EstimateLikelihoodParameters fits each resolution bin on its own and writes only that bin's values, so
the bins run under ParallelFor; the result is the same, bit for bit. It was the largest single stage of
a refinement (107 calls, one core): 5reo 2.42 -> 0.56 s, 8dyz 2.46 -> 0.57 s, 9yl4 3.54 -> 0.88 s;
whole refinement 5reo 6.3 -> 4.8 s, 8dyz 6.9 -> 5.3 s (RTX 5080, -N 8, loaded box).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 85aa34d3031c8e36be4462fca544bc801af45d5a)
(cherry picked from commit d84ba55252adf455d5b4ba62052dfbafa6290a64)
RefinementStage sums the time of each stage over a refinement (Refit and its parts, the evaluations and
their X-ray, restraint, host and upload shares, the repulsion build, the riding hydrogens, the rungs) and
logs it at debug level at the end; rugnux_refine_test -v shows it. No change to any result.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 9491f54e0beccf36c65ebcec9afff90d08485f86)
(cherry picked from commit 0ce206a88df960754476ce2f75ffaed90fbf3123)
RefineModel (Refinement.h) refines a placed model against the merge, reading and writing nothing:
- placement stage (R1): every chain of 50+ atoms as a rigid body at once (L-BFGS on six parameters per
chain, their gradient from the per-atom one through Ceres jets), then one B shift per residue;
- restrained stage (R2): positions and isotropic B of every atom of a residue the embedded dictionary
describes, the rest held fixed but kept in Fcalc and the mask; riding hydrogens added by gemmi's
topology, moved with their parents, re-placed every macro-cycle. Macro-cycles refit the mask, the
scale and bulk solvent (FitModelScale / ModelScaleGPUPool) and the likelihood's D and beta (ML on
the free set in resolution bins), rebuild the repulsion pairs (symmetry mates included), then L-BFGS
on positions and, where chosen, on B.
- weights: a ladder of full refinements at multiples of the gradient-balanced X-ray weight, chosen by
R-free within bond/angle bounds (ties to the tighter geometry), then the B refinement tried at its own
weights and kept only where it beats B held by more than the tie.
The target and its gradient (RefinementXray, RefinementGPU): the Rice/centric amplitude likelihood
(RefinementMath.h, shared host/device code), the gradient by Agarwal's FFT method - the adjoint of the
reciprocal-space symmetry composition into a half-l (CPU) or half-u (cuFFT) spectrum, one inverse real
transform, then each atom's density derivatives (xyz and B) summed over its box. On the GPU everything
per evaluation stays on the device: ModelDensityGPU's gather, r2c, a fused compose + likelihood kernel,
a CSR gather for the adjoint (no atomics), c2r, one warp per atom for the gradient; the restraints are
one kernel per restraint plus a per-atom gather of gradient slots; cub reductions. Deterministic.
Restraints (RefinementRestraints): bonds, angles, torsions (with esd, no phi/psi, no alternative sugar
puckers), chiral volumes, least-squares planes from gemmi's Topo; repulsions beyond 1-4 (vdW radii from
the dictionary's energy library, 2.6 A for H-bond pairs, 1.9 A metal-N/O/S); B similarity of bonded and
angle-related atoms.
DensityBlobs (D1): gemmi's flood-fill blobs of mFo-DFc above 3 sigma peaking above 4 sigma, placed beside
the model and labelled WATER / ION / LIGAND / MODEL_ERROR by distance, contacts, volume and the anomalous
map.
tools/rugnux_refine_test refines a model against a finished run's p.mtz (developer utility, not
installed). Tests: likelihood derivative, the embedded library, restraint values on ideal residues and
every restraint kind's gradient against finite differences (CPU, and the GPU against the CPU), the X-ray
gradient against finite differences in P1, P212121, C2 and P6122 (CPU and GPU), and a synthetic
refinement that recovers a shaken model. Credits in docs/ACKNOWLEDGEMENT.md and at the algorithms.
Measured on the open arm's battery outputs against REFMAC 5.8, 10 cycles, same p.mtz, free set and start
model (R-free): 5reo 0.2228 vs 0.2236, 8dyz 0.1733 vs 0.1651, 9chw 0.1995 vs 0.1969 (CPU engine).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8efe6f2322b3bfdae580bb34f256e67730dfc3c2)
leonarski_f