In broker live mode the refined model, its map coefficients and the difference-map blobs stayed in
the broker for the viewer routes and were lost at the next run. They now go to the writer in the END
message as the placed model and maps already do - EndMessage::refined_model (PlacedModelMessage),
refined_map_coefficients (MtzTable) and blobs (DensityBlobsMessage, one vector per field) - and the
writer writes <prefix>_refined_model.cif (+ .pdb), <prefix>_refined_maps.mtz and <prefix>_blobs.json,
the names the CLI gives them. The blobs' JSON is made by one function (DensityBlobsJson) that the CLI
now writes through as well, so the two files are byte-identical. Optional CBOR fields, no version bump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit c65220d96a96d2b1897f3c6632c94f9379b670a3)
- New docs/CPU_DATA_ANALYSIS_REFINEMENT.md (§15): the ML amplitude target and its FFT gradient,
scale/robust fit/fine solvent mask, D and the variance (work/free), the embedded Monomer Library
subset and topology, geometry/metal/KL-B restraints, the minimiser, the weight trials, refined maps,
blobs, GPU/CPU, and a "Not yet" list; with its own References. Linked from TECHNICAL.md and
CPU_DATA_ANALYSIS.md; §14 no longer says the structure is never refined.
- RUGNUX_REPORT.md: a Refinement (REFINE_*) section - every REFINE_ and REFINE_SCORE_ key, the
--mode refine input keys and the _blobs.json fields.
- RUGNUX_MODEL_AND_REFERENCE.md: "What it does not do yet"; RUGNUX_ADVANCED.md: --model mentions the
refinement, --no-refine as in the usage, a --mode refine options table; GPU_DATA_ANALYSIS.md: the
refinement's CUDA engines and the missing Metal path.
- ACKNOWLEDGEMENT.md: Brunger, Karplus & Petsko 1989 (gradient-balanced starting weight), Nocedal 1980
(two-loop recursion), Charbonnier et al. 1997 (pseudo-Huber), Jiang & Brunger 1994 (flat solvent
mask), Kullback & Leibler 1951; all DOIs checked against Crossref.
- In-source credit lines: the weight balancing (replacing a "Brunger (1992)-style" line that named the
R-free paper), Nocedal 1980, Charbonnier 1997, Jiang & Brunger 1994, Emsley et al. 2010 at
FindDensityBlobs, Read 1986 at the refined maps. Comments only.
- docs/THIRD_PARTY_NOTICES.md regenerated from the root file with update_version.sh's own step (it
lacked the CCP4 Monomer Library row).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 09375dba351bae0cc73e9a6f0f79455dc92b3a10)
Measured by stripping the deposited ligands and ions of 12 open-arm models (placement-refined, then the
blobs of mFo-DFc): ligands with a blob 34/39, called LIGAND 26/39, LIGAND blobs on a removed ligand or ion
24/33; ions with a blob 23/40. The ligand-size threshold was four resolution spheres, which called no glycan
at 3.3 A and every ion at 1.1 A; it is now one and a half, and never under 8 A^3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8e66d58ba87c35119df3ba8779ca193053f5cf00)
RefineModel (Refinement.h) refines a placed model against the merge, reading and writing nothing:
- placement stage (R1): every chain of 50+ atoms as a rigid body at once (L-BFGS on six parameters per
chain, their gradient from the per-atom one through Ceres jets), then one B shift per residue;
- restrained stage (R2): positions and isotropic B of every atom of a residue the embedded dictionary
describes, the rest held fixed but kept in Fcalc and the mask; riding hydrogens added by gemmi's
topology, moved with their parents, re-placed every macro-cycle. Macro-cycles refit the mask, the
scale and bulk solvent (FitModelScale / ModelScaleGPUPool) and the likelihood's D and beta (ML on
the free set in resolution bins), rebuild the repulsion pairs (symmetry mates included), then L-BFGS
on positions and, where chosen, on B.
- weights: a ladder of full refinements at multiples of the gradient-balanced X-ray weight, chosen by
R-free within bond/angle bounds (ties to the tighter geometry), then the B refinement tried at its own
weights and kept only where it beats B held by more than the tie.
The target and its gradient (RefinementXray, RefinementGPU): the Rice/centric amplitude likelihood
(RefinementMath.h, shared host/device code), the gradient by Agarwal's FFT method - the adjoint of the
reciprocal-space symmetry composition into a half-l (CPU) or half-u (cuFFT) spectrum, one inverse real
transform, then each atom's density derivatives (xyz and B) summed over its box. On the GPU everything
per evaluation stays on the device: ModelDensityGPU's gather, r2c, a fused compose + likelihood kernel,
a CSR gather for the adjoint (no atomics), c2r, one warp per atom for the gradient; the restraints are
one kernel per restraint plus a per-atom gather of gradient slots; cub reductions. Deterministic.
Restraints (RefinementRestraints): bonds, angles, torsions (with esd, no phi/psi, no alternative sugar
puckers), chiral volumes, least-squares planes from gemmi's Topo; repulsions beyond 1-4 (vdW radii from
the dictionary's energy library, 2.6 A for H-bond pairs, 1.9 A metal-N/O/S); B similarity of bonded and
angle-related atoms.
DensityBlobs (D1): gemmi's flood-fill blobs of mFo-DFc above 3 sigma peaking above 4 sigma, placed beside
the model and labelled WATER / ION / LIGAND / MODEL_ERROR by distance, contacts, volume and the anomalous
map.
tools/rugnux_refine_test refines a model against a finished run's p.mtz (developer utility, not
installed). Tests: likelihood derivative, the embedded library, restraint values on ideal residues and
every restraint kind's gradient against finite differences (CPU, and the GPU against the CPU), the X-ray
gradient against finite differences in P1, P212121, C2 and P6122 (CPU and GPU), and a synthetic
refinement that recovers a shaken model. Credits in docs/ACKNOWLEDGEMENT.md and at the algorithms.
Measured on the open arm's battery outputs against REFMAC 5.8, 10 cycles, same p.mtz, free set and start
model (R-free): 5reo 0.2228 vs 0.2236, 8dyz 0.1733 vs 0.1651, 9chw 0.1995 vs 0.1969 (CPU engine).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8efe6f2322b3bfdae580bb34f256e67730dfc3c2)
leonarski_f