exp(2 pi i h.t) depends only on the integer h.t (in units of 1/den), so the engine tabulates it once over the
range the rows and operators make, with the same op_phase() at the same double argument - the same bits. The
compose kernel reads the table at h.t; the scatter terms carry their table entry, and their row directly, so
neither hot kernel divides: the rotations go to the device in whole units (gemmi's are den times an integer
matrix), the term's row no longer has to be split from its operator. The table is a few thousand entries, the
scatter terms one int wider.
Outputs are byte-identical; the X-ray evaluation counter drops 7-10% on the demo-sized sets.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
Before the merge, the integrated reflections were walked twice - once per frame for what the
blank sweep ends are decided on, once more for the overload counts and the ice flags - and
each walk reads the whole reflection set from memory. The frame's sweep-end tally is now a
function of its own (TallySweepEnd), and the merge step makes it in the same per-frame walk
that counts the overloads and flags the ice, then adds the counts up over the frames still in,
so the trimmed and left-out frames count nothing, as before. Per frame the sums are the same
sums in the same order; the totals are integers. TrimBlankSweepEnds keeps its one-argument
form for the callers that walk the frames for nothing else.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
The first arm's space-group search now runs beside the wait for the all-observation merge and
the search started on it, instead of after both. Classifying a reflection for a candidate group
asks gemmi twice which operators fix it (once for the absence, once for the epsilon factor);
one pass over the operators answers both, with the absence read off a fixing operator's own
translation, which is what gemmi's test reduces to once the centring has been checked. The
delta-CC1/2 frame disposition builds its per-reflection totals on all threads, each taking a
range of reflections and walking every observation so the sums keep their order, and measures
up to sixteen candidate chains side by side with a worker's scratch made on first use. Same
numbers, same bits; only the waiting changed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
The device ingest staged every reflection to the card twice - once for what the sort and the
cut read, once again for the observations the cut kept - and the second staging was a full
copy of the reflections into pinned memory and over the bus. Where the whole set of reflections
and the ingest's own peak beside it fit a quarter of the card (decided on the card's total
memory, so the answer depends on the card and not on the moment), the first upload now goes
into one resident copy, and the observations are taken from it in a single extract; the second
staging stays as the fallback where the copy does not fit, and where the card reads host
memory in place. The extract kernel also hands over the predicted positions, which the host
had gathered itself from a downloaded index array, and the two pinned staging buffers are kept
across the ingest instead of being allocated per staging. Same observations, same order, same
bits.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
The trajectory ends where a step gains nothing, and that step ran from the best state at the weights the final
ladder then tries from the same state: on a positional step every rung, on a B step the middle one. A trial of
the same weights from the same model is the same computation (RunTrials: the outcome is the same on any slot),
so the final takes those outcomes from the step and runs only the rungs it has not got. The maps' Refit() then
starts from the fit the main slot would hold had it run its rung (Context::Restore of that rung's fit) - the
solvent grid and the likelihood's D it starts from otherwise depend on which trial the main slot ran last, and
that showed as last-bit differences in the map coefficients.
RunTrials gives the trials that refine B, the longest, to the slots first: on a machine whose cards hold a slot
each the ladder's two B rungs no longer share one card while a positional rung has one of its own.
Byte-identical (p.mtz, p.hkl, p.cif, p_model, p_refined_model, p_refined_maps.mtz, p_blobs.json, report minus
timing) on three rotation sets run live-path, and on two of them in --mode refine. Refinement wall on one
RTX 5080: 1.86 -> 1.71 s, 2.42 -> 2.17 s, 50.0 -> 40.8 s (one stage of the trajectory each).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
AnisotropyPriorHKL is the same tensor, bit for bit, as AnisotropyTensorHKL(AnalyzeAnisotropy(...)),
so the amplitudes no longer wait for the unmerged-observation part of the analysis (ScaledObservations):
the model work starts at once and the analysis, its text and the merge statistics are taken before
the rigid-body validation or after molecular replacement. The analysis takes its own copy of the
space group, which the enantiomorph relabelling changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
AnalyzeCentreOfSymmetry ran on one thread between the adopted group's merge and the analyses:
its orbit fold and L-test partner lookup were std::map (now hashed - the orbits are sorted by a
total order right after, and the lookups only test membership), and its 20 simulated
populations took their log/cos per reflection serially. The generator's draws are now taken
in the same order on one thread and turned into intensities on all of them (the Box-Muller
operands in the order GCC evaluated them; bit-exact against the old function on five groups at
1/4/32 threads).
AnalyzeAnisotropy's three BuildCells and the observation cells of the systematic floor find each
copy's nearest of 60 directions on all threads and fill the cells in the old order; the
observation sort is a ParallelSort (index-unique pairs, a total order). AnisotropyRunInfo
carries the worker count.
Live path, workstation, 3 A/B pairs, outputs identical (mtz/hkl/cif/model/maps/report):
thau centre stats 0.18 -> 0.06 s, analyses 0.57 -> 0.43 s; cytc 0.04 -> 0.016 s,
0.37 -> 0.32 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
The cell e.s.d.s and the unmerged MTZ read only the integrated observations and the group, both final once
the model has been read, and the refinement reads neither: both now start before the refinement instead of
after it. In MakeMaps the difference map is made on its own thread while the anomalous map is made.
Outputs identical (MTZ data, CIF/HKL/INS, refined model, maps, blobs, report minus provenance) on the
thaumatin and cytochrome C live runs. Live path without the unmerged MTZ (as the broker runs it), step
Refinement: thaumatin 2.76 -> 2.40 s, cytochrome C 1.99 -> 1.93 s. With the unmerged MTZ (CLI), Refinement +
Writing files: 3.21 -> 2.67 s and 3.33 -> 2.80 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
The Rugnux card says in passing that it takes a synchrotron rotation sweep or XFEL serial
stills, and ends with one sentence pointing at the two pages that list the public datasets it
is validated on (External test data: over 200 rotation sets, 207 in the open battery manifest;
External test data: serial crystallography: eleven CXIDB entries). The "two mountains, two
railways" line at the foot of the page is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 03e4431836a2f4ba816e6d3cdd57cc6ada87b95d)
The front page's hero image is now the cytochrome C collection at SLS X10SA as the broker's Rugnux
live page ran it: a detector frame on the page's indigo beside the AlphaFold model placed in its
electron density, over the two-lane timeline of the live analysis (the 18 s collection, the image
processing alongside it, scaling, merging and molecular replacement 4.9 s after the last image).
One WebP per theme (hero_live_light.webp, hero_live_dark.webp), shown by the same rules that pick
the theme's colour tokens, so the toggle and the system preference both switch it; the hero's art
column widens to 560 px on desktop (360 px on a phone). The lysozyme frame and its generator go;
the Sphinx extension copies the two new files into the site. Made by docs/landing/hero_live/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 5f2592390d9ec39428198df8cd8e18ad60167a79)
In broker live mode the refined model, its map coefficients and the difference-map blobs stayed in
the broker for the viewer routes and were lost at the next run. They now go to the writer in the END
message as the placed model and maps already do - EndMessage::refined_model (PlacedModelMessage),
refined_map_coefficients (MtzTable) and blobs (DensityBlobsMessage, one vector per field) - and the
writer writes <prefix>_refined_model.cif (+ .pdb), <prefix>_refined_maps.mtz and <prefix>_blobs.json,
the names the CLI gives them. The blobs' JSON is made by one function (DensityBlobsJson) that the CLI
now writes through as well, so the two files are byte-identical. Optional CBOR fields, no version bump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit c65220d96a96d2b1897f3c6632c94f9379b670a3)
Rugnux measured the cell's e.s.d.s (CellEsd) but they only reached the mmCIF's _cell.*_esd, the .ins's
ZERR and the report. They now travel in the END message (EndMessage::unit_cell_esd, CBOR key
unit_cell_esd, same layout as unit_cell), are written beside the cell in NXsample as unit_cell_esd
(float[6]: a b c alpha beta gamma, A and degrees) and are read back into
JFJochReaderDataset::unit_cell_esd. rugnux's _process.h5 and, through AddToEndMessage, the broker's
live master file get them. Rotation only; absent on stills and where nothing was measured.
Optional CBOR field, no version bump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 6902b5637aa92c27565f22bcf3fe11b5d2536038)
The held test read the X-ray gradient alone. At the floor, an atom whose X-ray term pushed
outward but whose restraints pulled it back in more strongly was held fixed against the total
target, and one whose restraints pushed it out was handed a gradient along the direction Set()
clamps away - a flat direction for L-BFGS, failed line searches and s.y <= 0 pairs.
On the 14 parity sets (--mode refine, GPU) the refined models are md5-identical; on the four
5f6m MD starts R-free 0.1707/0.1676/0.1720/0.1669 -> 0.1707/0.1690/0.1719/0.1668 (median
0.1692 -> 0.1699).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 2f4b0b5ea54fdac54de7ee6c5a1476d2fc019fd8)
Contacts() dropped every pair found in image 0 with a nonzero lattice shift ("the identity term
covers the contact"), but that pair is a contact with a translated copy of j, not with j itself:
in P1 no intermolecular repulsion or B-contact restraint was made at all, and in every group the
contacts along a, b and c were missing (MeasureGeometry's clash count with them). Such a pair is
now kept like a symmetry mate's, from the lower atom of the two.
An atom against its own image x -> T x is also met through T^-1 at the same distance (another
operator, or the same one at another lattice shift for a screw), and both were kept: atoms near
a 3-, 4- or 6-fold or a screw axis were repelled twice. For i == j the image first in fractional
coordinates of T x and T^-1 x is kept; an involution is kept once as before.
Test: in P1 two waters 2.4 A apart across a cell face get one repulsion and one B contact (none
before); a water 1.4 A from a 3-fold axis gets one repulsion with its images (two before).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 0d6e4d0491e655a49d59b709e1398843506d162a)
The environment variable overrode the current device that ScopedGpuPin sets, so a stray value
moved an indexing thread to another card. Recorded in the file header as a local modification,
together with the deterministic tie-break of c982ba4f6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 89acdeef939ee82cf15ee9c5fb52a016fb1441f8)
GPU: a slot was packed onto a card by its engine's bytes alone. The per-slot estimate now adds
the slot's ModelScaleGPUPool with the robust fit's buffers (ModelScaleGPUPool::DeviceBytes(n, 1,
true)), and RefinementGPUEngine::DeviceBytes counts the two restraint sets SetRestraints() holds,
not one. The 80%-of-card budget is unchanged, so where these do not fit fewer slots are packed
instead of an allocation failing inside a running trial.
CPU: OwnSlotDevices made a slot per wanted trial whatever the thread count, so -N 1 ran six trial
threads with six models, topologies and engines in memory. The main slot and those beside it are
now at most -N. Side by side equals sequential, so no result changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 3c04c01158266c267f8e5ffaaaa98d6d8fd2a747)
tf_z, rf_z, mr_d_min, the placed model's R-factors, CC, map and anomalous sigmas, the rigid
body's move, the refinement's R-factors and rmsds, and the headline d_min are set only where
finite (llg_z and null_sigma already were). nlohmann writes NaN/inf as null, which the page's
"!== undefined" checks let through to .toFixed(), and which the zod schema rejects for the
required blob distance - a blob with no atom within 12 A has an infinite one. That distance
is now optional (jfjoch_api.yaml; C++ model and frontend client regenerated), absent then.
The model cards test for != null, so a null that arrives anyway does not take the page down.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 62ff026aa507e5bda1daa5aa079f32fb5c21f16e)
A slot builder thread that throws (gemmi's topology or hydrogen placement, bad_alloc) would
call std::terminate - rugnux or the broker gone. The slot is now left out as one the card
cannot take is, with a warning, and MakeOwnSlot's GPU engine catches std::exception, not only
JFJochException. Both callers of RefineModel (after the merge, and --mode refine) catch
std::exception: the refinement is reported NOT_PERFORMED with "refinement failed: <what>" and
the run writes everything else (cell e.s.d.s, unmerged MTZ, report), as the refinement is
processing-inert.
Test: --mode refine on a cell too large for the bulk-solvent grid comes back with
REFINE= NOT_PERFORMED and the reason, not an exception.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit dc4e99fe4b3da2a9a3fa4290b5f70bfcc8c7bdb8)
The monoclinic cell of the .ins/mmCIF and fit tests is now 10/7/12, beta 100, and the
orthorhombic, hexagonal and tetragonal cells of the sampling tests are rounded synthetic ones.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 6f964bc0de0ca842f374e6aed83c8848e410173b)
M_PI is not defined by MSVC without _USE_MATH_DEFINES, and CellEsd.cpp is reachable from the
viewer build. The citation, so far only in CellEsd.h, now also sits at the algorithm it credits.
The other files new in rc180 under common/, image_analysis/, reader/, writer/, rugnux/ and
viewer/ use no M_* constants or POSIX-only calls (grep of the rc180 diff).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit a019bc5fd2084907af9379068af3265ffe6c9ef5)
Landing page (title, hero, lede, Rugnux card), the documentation home's Rugnux card and the opening
of RUGNUX.md now say what the program does: from diffraction images to merged data and, with a
model, on to a refined structure and maps.
RUGNUX.md Quick start is rebuilt around how a run is started: rotation (master file, or the first
frame of a CBF/marCCD/SMV sweep), serial stills (several masters or all of a dataset's files by
glob), serial stills with -C/-S and --crystfel-geometry (wavelength/distance flags only where the
files state neither), and with a model (positional or --model): placement by rigid body or MR,
refinement by default (--no-refine), maps and blobs, and that the model changes no processing
decision except the indexing choice on serial stills with -C/-S. It says plainly that the defaults
are meant to be the answer, and lists the output files with and without a model. The tutorial's
stills section drops -X ffbidx, -z and the resolution limit from its first example and shows the
same three stills commands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit bfca846332037688364b9d2d1c4250315f09410d)
Live refinement also runs on a broker without a GPU, on the CPU and slower, and the run's deadline
or cancel can cut it. Description only; regenerated broker/gen (Rugnux_live_refinement.h), the
frontend client (types.gen.ts, zod.gen.ts), broker/redoc-static.html and the python client doc
page - each differs from before only in this description.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 53d06c6e221fa1cf776822965560fb990cc69c73)
INTEGRATION: the stills prediction window and partiality from the run's
measured rocking curve (a d* table, kept in _process.h5 for --mode scale),
the model-error term in the per-crystal fits, --min-partiality defaults,
the adaptive radius on stills (background-ring reach).
IMAGE: reflections reaching into the beam-stop shadow are not integrated;
no-data pixels masked as dead, not as shadow.
INDEXING: the rotation-scale walk's refusal and refit; the stills geometry
refinement on the given cell's crystal form only.
DECISIONS: the model is refined after validation by default.
CPU_DATA_ANALYSIS: titles in the references that lacked them.
GPU: the still prediction kernel follows the rocking-curve window.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 5ae198b566b17de259fd0f0d6c8a0f8ea18eaa05)
RUGNUX_REPORT: a table of the headline keys (checked against
ResultReport.cpp); section 5 appears for --model, --mr or a model among
the inputs and holds the MR and REFINE_ keys; a --mode refine report has
one section; stills ambiguity resolved by the crystals' own cells first;
the developer-only CELL_ESD_OBSERVATIONS / CELL_DSPACING_FIT; the
"nothing here is refined" paragraph points at REFINE_*.
BENCHMARKING: the still battery, the refined R-free beside the placement
one, the Refinement and Doubled-cell steps in the timing table.
EXTERNAL_TEST_DATA_STILLS: the gain is measured, not passed; how the
battery runs these sets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit e9f5e3b63fb6208d6bd5984c1ecd0eb0da7c29ab)
RUGNUX: a model run also refines the model it placed; the model can be
given as an input; only the unmerged MTZ is written by default; the
_process.h5 is not a default output; a short "when a run fails" note.
TUTORIAL: the .ins, the refined model/maps/blobs, the log and _modules.h5
in the output list; the report is not written with -o ""; stills: -X Auto
picks ffbidx with a cell, a run over several files.
FORMATS: the XFEL stills formats and model/reference inputs told by content.
CALIBRATION: -C and -S as the calibrant. INSTALL: the CPU path runs every
indexer and the refinement. MODEL_AND_REFERENCE: the refined model is
written as .pdb too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 0062d72b95e996438ddc65dc343b526c84daa7fb)
RUGNUX_ADVANCED: --simulate-live/--fast-live rows, -e exclusive, stills
behaviour of --beam-center-check, -S, --min-partiality (0.5),
--capture-uncertainty, --adaptive-integration-radius (on for stills too),
--detector-distance/--rot1/--rot2/--no-refine-tilt; --background-radial
auto as it decides now; --refine-modules syntax; _cell.*_esd.
MODEL_AND_REFERENCE: the default refinement of a placed model in the
summary, the decision table and the step list; --mode scale does not refine.
INTEGRATION: the .ins written beside every .hkl (ZERR on rotation).
VIEWER: rugnux writes _process.h5 only with --no-merge/--write-process-h5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 51d2564356ba46bda8730c855ff8d3ee7c23d605)
Rotation runs now estimate the standard uncertainties of the cell. Every strong integrated
observation (I > 3 sigma, unclipped, unsaturated, off ice rings) gives |q| = 1/d at its observed
centroid, which depends on the detector geometry only; q^2 = h^T G* h is linear in the reciprocal
metric, so the components of G* the point group leaves free (projected from the group's rotation
parts, any setting) are fitted by weighted linear least squares with 4-sigma rejection, and the
GoF-scaled covariance is carried analytically onto a, b, c, alpha, beta, gamma (ties give equal
e.s.d.s, fixed angles zero). Least-squares cell e.s.d.s are known to be too small (Herbstein 2000),
so the same fit is made on 8 blocks of consecutive frames and their scatter / sqrt(8) is the second
estimate; the written e.s.d. is the larger of the two per parameter. Stills get none.
Outputs: <prefix>.ins next to every .hkl (TITL, CELL at the wavelength, ZERR with Z = order of the
group, LATT/SYMM from the space group - positive LATT only with -1 at the origin - a placeholder
organic SFAC/UNIT at 18 A^3 per non-H atom, HKLF 4, END); the mmCIF gains _cell.*_esd ("." where
fixed) and _cell_measurement.reflns_used/theta_min/theta_max, and writes the cell to 4/3 decimals;
report keys CELL_ESD, CELL_ESD_LSQ, CELL_ESD_BLOCKS, CELL_ESD_BLOCK_COUNT, CELL_ESD_REFLECTIONS,
CELL_ESD_THETA_RANGE_DEG (+ dev CELL_ESD_OBSERVATIONS, CELL_DSPACING_FIT).
On 13 small-molecule sets the block scatter is 10-40x the least-squares e.s.d. and is what gets
written; it lands within ~0.3-5x of XDS's CORRECT.LP e.s.d.s. SHELXT solves from the generated
.ins and SHELXL carries ZERR into its CIF su. Costs 12 ms on a small molecule, 0.16 s on 690k
lysozyme observations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit e74a3be76261fe1b6c0ff7b80c21afc4768c46cd)
--mode scale re-measured the rocking width on the stored reflections, so on any file whose frames were
predicted through the run's curve it merged with a different curve than the mx run did. The curve the
post-refinement used (formula, table, radial scale) is now carried in the EndMessage (StillsRunWidth) and
written to the master: /entry/MX/stillsRockingWidthFormula [gamma0 1/A, gamma_e rad, bandwidth sigma],
/entry/MX/stillsRadialScale, /entry/MX/stillsRockingWidthTableDstar and /entry/MX/stillsRockingWidthTable
(1/A). The reader returns it, and --mode scale passes it to the post-refinement. A file without it is
re-measured as before.
MeasureRunWidth grouped each frame's reflections by the frame's window, taken as the largest excitation
error it predicted. With a window that follows sigma(d*) that is the high-resolution window, applied to the
frame's low-resolution reflections too. Given the curve the frames were predicted through, the window is now
that curve at the reflection's own resolution; without one, the frame's largest excitation error as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 992343d0206489991880bb927efc79131134a1d9)
The embedded CCP4 Monomer Library subset grows from 62 to 84 compounds:
- the modified tRNA/rRNA nucleotides PSU, 5MC, 1MA, OMG, OMC, OMU, H2U, M2G, 7MG, 2MG, 5MU (RNA group,
joined by the library's backbone link "p");
- the cofactors NAD, NAI, NAP, NDP, FAD, FMN, HEM, HEC, SAM, SAH, and dioxygen (OXY). CMO is left out:
its dictionary has C-O at 1.25 A, where bound CO is near 1.15 A.
links_and_mods.cif gains the links these compounds use: HEC-CYS1/HEC-CYS2 (heme C thioethers),
CYS-HEMB/CYS-HEMC, CYS-FAD and HIS-FAD1/2, with their modifications.
AddMetalLinks now takes a metal atom of any residue, not only a one-atom ion, so a heme iron's LINK records
to His NE2 / Met SD / water become bonds at the library's metal-site distances; the site's coordination counts
the metal's bonds in its own dictionary (a heme's four pyrrole N) as well as the records. A metal within a
ligand is also a metal for the repulsions (1.9 A to N, O, S) and its dictionary bonds are kept with the metal
bonds. Note HEC.cif's own Fe-N(pyrrole) ideal is 1.90 A (HEM: 2.04 A), so a cytochrome c reports a metal-bond
rmsd near 0.1 A.
Test RefinementRestraints_deposited_ligands builds the topology of eleven deposited models cut down to the
ligand and its surroundings (tests/test_data/ligands, 66 KB, from PDB 1EHZ, 8BKH, 6K9I, 2W3V, 5VML, 6F3P,
8CON, 3TK0, 7TCD, 6W4H, 5EPE): every ligand restrained, bond rmsd < 0.03 A, the RNA backbone bonded to each
modified nucleotide, the heme Fe-His bonds, and the HEC thioethers. rugnux --mode refine on the full entries
against their deposited structure factors: R-free within +-0.003 of the build where these were held fixed,
ligand bond rmsd vs the dictionary from 0.004-0.058 A down to 0.002-0.042 A. rugnux grows by 425 KiB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit f4d407c8de6cb462320edb21a70665350e436d28)
Two mechanisms, as the code has them: with -S and no reference or model, the crystals made
consistent from the data (Brehm & Diederichs objective, cell-metric anchors); with -z or --model
(-C/-S), per image against the reference, crystals their own cell decides keeping that indexing,
the data-only result as a check. Removes the statements that the ambiguity needs a reference or
model (tutorial, quick start, overview, §10.9), adds the anchoring to the reference/model rows,
the "no -S" case (mixed merge, can pass for higher symmetry, warning), and the report keys in the
reference case (AMBIGUITY_DATA_DISAGREES in place of AMBIGUITY_REINDEXED, AMBIGUITY_TIME_S,
none in --mode scale).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8db1d3ecb97dba90471c4db238aff632d11bc3e8)
GPU_DATA_ANALYSIS.md: the refinement section at the level of the other engines - density gather,
cuFFT r2c/c2r, compose + likelihood target, CSR adjoint, warp-per-atom gradient, restraints and
slot gather, fine mask and robust scale fit (on-device median), reductions and determinism,
stream priority and blocking-sync waits, trial slots over cards, DeviceBytes budget, CPU fallback,
no Metal; indicative RTX 5080 timings.
Pipeline figures: a dashed Refinement box after Structure & map on both, the model branch labelled
as deciding nothing; the stills row gains the indexing-ambiguity step before the first merge.
RUGNUX_OVERVIEW.md: captions, a refinement paragraph, refined outputs under Write.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 8645fb2aca1abbbddec29cdfa99056471f8f38a5)
- New docs/CPU_DATA_ANALYSIS_REFINEMENT.md (§15): the ML amplitude target and its FFT gradient,
scale/robust fit/fine solvent mask, D and the variance (work/free), the embedded Monomer Library
subset and topology, geometry/metal/KL-B restraints, the minimiser, the weight trials, refined maps,
blobs, GPU/CPU, and a "Not yet" list; with its own References. Linked from TECHNICAL.md and
CPU_DATA_ANALYSIS.md; §14 no longer says the structure is never refined.
- RUGNUX_REPORT.md: a Refinement (REFINE_*) section - every REFINE_ and REFINE_SCORE_ key, the
--mode refine input keys and the _blobs.json fields.
- RUGNUX_MODEL_AND_REFERENCE.md: "What it does not do yet"; RUGNUX_ADVANCED.md: --model mentions the
refinement, --no-refine as in the usage, a --mode refine options table; GPU_DATA_ANALYSIS.md: the
refinement's CUDA engines and the missing Metal path.
- ACKNOWLEDGEMENT.md: Brunger, Karplus & Petsko 1989 (gradient-balanced starting weight), Nocedal 1980
(two-loop recursion), Charbonnier et al. 1997 (pseudo-Huber), Jiang & Brunger 1994 (flat solvent
mask), Kullback & Leibler 1951; all DOIs checked against Crossref.
- In-source credit lines: the weight balancing (replacing a "Brunger (1992)-style" line that named the
R-free paper), Nocedal 1980, Charbonnier 1997, Jiang & Brunger 1994, Emsley et al. 2010 at
FindDensityBlobs, Read 1986 at the refined maps. Comments only.
- docs/THIRD_PARTY_NOTICES.md regenerated from the root file with update_version.sh's own step (it
lacked the CCP4 Monomer Library row).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 09375dba351bae0cc73e9a6f0f79455dc92b3a10)
The generated spot-and-ring drawing is replaced by one 0.2 deg rotation frame of the in-house
lysozyme standard (EIGER2 16M, SLS X10SA): Bragg spots on their lunes, the beam stop and its arm,
the module gaps. docs/landing/hero_frame.py makes the 920 px WebP (67 KB) from the master
file - crop about the beam centre, log tone map from the frame's own background, block mean+max
downsampling - and is run by hand; the WebP is committed and copied by the landing-page step.
The dark theme shows it inverted through a CSS filter.
The hero's command line drops its explanatory sentence and gains the second form,
rugnux <diffraction image file> <CIF/PDB model file>.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit ff26000bfe7446cb4fda739c3b72596f8a8b7047)
MeasureRunWidth measured the width in eight shells of equal count in d*^2 and fitted the three-term
formula gamma0^2 + (gamma_e d*)^2 + (bw |q_z|)^2 to them. On runs predicting ~1000 reflections a frame
the equal-count shells all sat at 3.8-1.8 A, so the low-resolution width was the formula's
extrapolation, and the fit's form flipped with the number of frames (to a pure bandwidth at 50k frames).
The same window-group moment is now also measured in eight shells of equal width in d*, made
non-decreasing (pool adjacent violators, weighted by each shell's summed signal) and kept in RunWidth as
a table that the partiality interpolates linearly in d*. The formula is still fitted on the equal-count
shells and still sets the prediction window. With fewer than three measurable shells the formula is
used as before.
Still battery, 12 non-float sets, together with the sigma-following prediction window (the previous
commit; run 20261026-1021_1b28d6_still-part-comb against rc180 20261026-0857_3882b0_still-part-base1,
0-cycle REFMAC R-free vs deposited, both --no-refine): 6 better, 3 unchanged, 3 worse (by 0.002), sum of
changes -0.027 (the window alone: -0.007); largest gains pal90 -0.014, alvra103_int16 -0.011, lcls222
-0.005; largest losses euxfel152 +0.005, alvra213_int16 +0.005 (+0.010 with the window alone), euxfel83
+0.003. Win/draw/loss against the depositions unchanged at 9/2/1. Rotation: p.hkl byte-identical on
5reo, lyso_x06da_ref and thau_x10sa_16keV.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit de779bb10b1813eeae625e3994b1ca73d6079d7a)
Each still predicted through |dist_ewald| <= 2x its own profile radius (the RMS excitation error of its
indexed spots): one width at every resolution, differing eightfold between frames. Against the run's
measured rocking width that was 5-8 sigma at low resolution on one SACLA run (59 % of the 40-8 A predictions
empty) and below one sigma at high resolution on a crowded LCLS run (half of the curve never integrated).
Now the stills image pass first integrates a sample (a tenth of the run, at least 2000 frames) the old way,
scales it against its own merge and measures the run's rocking curve on it - centre (MeasureRadialScale) and
width (MeasureRunWidth), as the merge measures them - then predicts every frame through 2 sigma(d) of that
curve, centred on the reciprocal lattice scaled by the radial scale (BraggPredictionSettings::window_*, CPU
and CUDA). The post-refinement uses the same curve instead of measuring it again: re-measured through
windows that follow it, the constant term falls to ~0 and the result swung by up to +-0.015 per run.
Still battery, 12 non-float sets, against the st-partial baseline (0-cycle REFMAC R-free vs deposited):
9 better, 3 worse, sum of changes -0.013; the off-centre run -0.012, worst +0.008 on one JUNGFRAU run (its
sample measures a different width form than the full run). Integrated reflections 0.8-1.7x, battery wall
+13 % on a loaded box. Rotation untouched (window fields zero unless set); p.hkl byte-identical on three
rotation sets.
Tests: BraggPrediction_WindowFollowsResolution, BraggPrediction_CPU_GPU_consistency_window.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 1b28d671d780bc88525ceac5120017963d937bbc)
Bitwise. One warp per atom, each lane taking the box's points t = lane, lane + 32, ...: every point cost three
integer divisions/modulos to turn t into (u, v, w) and three more (imod) to wrap them into the grid. The lane
now steps (iu, iv, iw) on by 32 points with adds and carries, and wraps from the box's first point, wrapped once
per atom (a box is narrower than the cell, so one conditional subtraction per axis). The same points in the same
order per lane, the same arithmetic.
atom_gradient_kernel (nsys, side by side): 5reo 426 -> 237 us, 5f6m 887 -> 523, 9yl4 1436 -> 802, 7d98 692 ->
388 us per launch. Refined CIF md5 unchanged on the 5 GPU cases; side by side = one at a time. Wall: 5reo
1.43 -> 1.35, 5f6m 2.20 -> 2.00, 9yl4 6.88 -> 6.82, 7d98 3.58 -> 3.38, 5f6m MD start 4.16 -> 3.71 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 6554e9dcbe4b448eda582eb76d6a24c59fe6ab0c)
Bitwise. cudaStreamSynchronize spins a core for the whole wait; the weight trials' engines wait on a shared card
for most of a run, so about half of the user cycles of a refinement were the spin (PROFILE.md: 38-50%). A
StreamWaiter (CUDAMemHelpers.h: an event made with cudaEventBlockingSync, recorded and waited for) now takes the
refinement engine's waits - evaluation, Fcalc, Fmask, the uploads of positions, row model, blur and restraints.
The short waits of the host-driven loops stay spinning: the scale fit's Levenberg-Marquardt (one launch of
66-120 us per step) and the gather's pair count; making those block too cost 2-3% of wall (wake-up latency),
here the wall is unchanged. No process-wide cudaDeviceScheduleBlockingSync: that would change every GPU wait
of the broker.
CPU use (-N 8, time -v): 5reo 301-315% -> 190%, 5f6m 326% -> 181%, 9yl4 333% -> 222-228%, 7d98 307% -> 181%,
5f6m MD start 309-320% -> 151%. Wall, two interleaved runs each: 5reo 1.43-1.52 -> 1.43, 5f6m 2.19-2.20 ->
2.19-2.20, 9yl4 6.83-6.84 -> 6.88-6.89, 7d98 3.50-3.51 -> 3.58, 5f6m MD start 4.16-4.34 -> 4.16 s. Refined
CIF md5 unchanged on the 5 GPU cases; side by side = one at a time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit d23d2cf7bb99b30617a3f57c531b03db8bc03e05)
Bitwise. MakeOwnSlot copies the model as it was before the topology and reads only what the placement does not
change (cell, reflections, the engine's kind, the B ceiling), so the slots are now built on their threads right
after the start score, beside the rigid-body and group-B placement, and joined where they were built before
(also where the placement throws). The "slots" stage goes from 0.23 s to 0 on 9yl4; the placement is not
slowed (0.59 -> 0.60 s). Refined CIF md5 unchanged on 5reo, 5f6m, 9yl4, 7d98, 5f6m MD start (GPU) and 6o2h
(CPU); side by side = one at a time. Wall: 5reo 1.46-1.53 -> 1.54, 5f6m 2.26 -> 2.21, 9yl4 7.03 -> 6.83, 7d98
3.65 -> 3.56, 5f6m MD start 4.20 -> 4.15 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 0752a921a4293d4213894e3242c4ef36d8ce2a05)
Logic change (deterministic), kept on evidence. Every Refit searched the whole k_sol/b_sol box - an 88-point
coarse pass of plain least squares, then the robust fine pass around its winner. ModelScaleBox (and
ModelScaleGPUPool::Fit / ModelScaleGPU::FitSolvent) take an optional start; with one, the coarse pass is
skipped and the fine pass is laid around it. Refinement passes the grid winner of the context's previous
Refit (before MoveOverallB moves the overall B into b_sol); it travels with a trial's fit, and a trial's first
Refit without one (Context::ForgetStarts, which also clears the likelihood's D start) searches the whole box,
so a trial still depends on nothing its slot ran before (side by side = one at a time, md5).
Evidence (GPU, -N 8): full refinement of the 14 parity sets from their placed models - 12 refined CIFs
identical (md5), 9ig7 and 6iu9 differ, final R-work/R-free equal to 4 digits but 6iu9 R-work 0.2911 -> 0.2910;
of the 5 timing cases 4 identical, 7d98 (cif start) R-free 0.3096 -> 0.3095; 5f6m MD starts 01-04 median R-free
0.1692 -> 0.1692; 6o2h on the CPU engine identical (md5). The score-only parity table is not touched (one Refit,
no start). refit.scale (TIMING sum): 5f6m 1.52 -> 1.10 s. Wall: 5reo 1.68 -> 1.53, 5f6m 2.43 -> 2.24, 9yl4 7.19 ->
7.02, 7d98 3.70 -> 3.65, 5f6m MD start 4.57 -> 4.22 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit ff285fb525ecf4b480698d4aafe212a34301a8fe)
Bitwise. The first robust round copied every fine-pass fit's residuals to the host (up to 25 x n floats,
9 MB at 92k reflections) and took std::nth_element of each there. The median is now a radix selection on the
card (one block per fit, four byte-wide passes of integer counts over the float bits - the residuals are not
negative, so the bits order as the values): the element nth_element puts at n / 2, the same value; only the
fits' c come back. FitSolvent and RobustBatch lose the host threads they took for it.
Refined CIF md5 unchanged on 5reo, 5f6m, 9yl4, 7d98, 5f6m MD start. Wall (-N 8, quiet): 5reo 1.72 -> 1.68,
5f6m 2.57 -> 2.43, 9yl4 7.28 -> 7.19, 7d98 3.78 -> 3.70, 5f6m MD start 4.76 -> 4.57 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit dc53896c335fdabdf4d417323038e05b8b63535d)
leonarski_fandClaude Fable 5.1