6 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Fable 5.1	546566cff4	Refinement: the gather's pairs built in fixed-shape buffers, and each evaluation replayed as a CUDA graph ... ModelDensityGPU::Compute() no longer reads the pair count back to the host between its launches: the atoms are counted per brick with integer atomics, the counts scanned into the bricks' segments, the segments filled through atomics and then each brick put into model order by rank (one block per brick), which leaves the pairs exactly as the stable radix sort of the atom-major fill left them - the gather sums in the same order, bit for bit. The pair bound of the atoms set is checked on the host at SetAtoms() instead of on the device at every evaluation, and the buffers shrink from four to three ints per pair. With nothing coming back mid-evaluation, RefinementGPUEngine records the X-ray evaluation and the restraint evaluation each as a CUDA graph on its stream (cudaStreamBeginCapture, thread-local mode) and replays it, re-recording only where the shape changes (the atom count, a gradient asked or not, a restraint set re-uploaded). Two graphs, not one, so that the host still fills the restraints' positions while the X-ray half runs, as before. Where a stream cannot be captured the launches are queued as they were. Outputs byte-identical to 037c32d9f on the in-house cytochrome C live run (MTZ arrays, HKL, models, blobs, report). Workstation (RTX 5080): refinement 1.64 -> 1.50 s on that run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-27 05:57:57 +01:00
leonarski_fandClaude Fable 5.1	b16873036c	Refinement: density records sent from pinned memory, and the L-BFGS direction in fused sweeps ... Two byte-identical cuts to the per-evaluation host chain, which on a large model is twice the GPU's work per evaluation (34 ms against 16 ms of kernels per weight-trial slot, measured in nsys) and so bounds the refinement on one card with three slots, and the whole of it on a card with one slot. - The gather's per-atom density records (38 MB on a 242k-atom model, sent on every B-factor evaluation) live in a PinnedVector and go up by DMA; the boxes go through a pinned staging buffer of the density object. ModelDensityGPU::SetAtoms and RefinementGPUEngine::SetDensities take a pointer and a count. - RefinementMinimiser::Direction takes each pair's dot product in the same sweep that applies the previous pair's update to q: the dot products are serial chains of n additions, and the update's memory traffic now hides under them (20 -> 13 ms per call at 365k parameters). The same operations per element in the same order; the loop-2 coefficient stays written as (a[i] - b) with b on its own, because a[i] - rho * dot contracts into a fused multiply-add and changes the bits (checked bitwise in a standalone harness). - A debug line with the GPU engine's bytes and the budget. 4nwv (open arm, 242k atoms), live flags, one lock hold, loaded box: refinement 42.35 -> 38.40 s; refine.gpu.upload 8.14 -> 3.94 s. All outputs identical but provenance lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 22:48:28 +01:00
leonarski_fandClaude Fable 5.1	3189f71e8f	Refinement: the engine polls for its evaluations and the pair count comes back pinned; the slots beyond one per card go to the cards the ladder's long trials leave free ... The GPU engine's waits on its stream (every evaluation, the refits' transforms) and the model density's wait for the (brick, atom) pair count were blocking event and stream synchronisations; where a weight trial has a card to itself, as on a box with a card per trial, the wake-up of the sleeping thread is on the critical path of every evaluation. Both now poll, and the pair count and overflow flag come back into pinned memory instead of a pageable int (a pageable copy back is staged and blocks the host for the whole stream). OwnSlotDevices places the slots beyond one per card on the cards the ladder's long trials (those refining B) leave free, so that on four cards the three B rungs have a card each and the three B-held rungs share the fourth, instead of two cards each carrying a B rung beside a B-held one. The outcome does not depend on the placement. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 22:41:43 +01:00
leonarski_fandClaude Fable 5.1	604b270114	ModelDensityGPU: the gather stages each atom's box as the brick sees it, and only what the brick needs of the atom ... The staged record of an atom is now its isotropic Gaussians, occupancy and radius (an anisotropic atom's matrices are read from global memory on that rarer path) and its box as bit masks over the brick's eight points along each axis, with the base and wrap that turn a point into gemmi's unwrapped box index. The inner loop tests a point with three shifts instead of three box offsets and six shared reads, and the shared record is a quarter of its size, so a fourth block fits on a multiprocessor. The density arithmetic, the order over the atoms and the summation are unchanged, so the grid is bit for bit the CPU walk's as before. Outputs are byte-identical; refinement and molecular replacement each gain a few percent on the demo-sized sets. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 20:58:10 +01:00
leonarski_f	8261b7354e	v1.0.0-rc.176 (#86) ... * jfjoch_broker: Optional live Rugnux analysis of DECTRIS rotation collections inside the broker, with merged data, maps and the placed model written beside the master file, the rotation lattice in the scan result, and a "Rugnux live" tab and dashboard row in the web frontend. * Rugnux: Faster processing, with molecular replacement and model validation spread over several GPUs and run on the Apple GPU on macOS. * Rugnux: The polarization stated in miniCBF and d*TREK headers is used, twinned crystals' space groups are decided more reliably, and a run that finds no lattice still reports every image. Reviewed-on: #86 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-19 07:03:28 +02:00
leonarski_fandFilip Leonarski	19b135186d	v1.0.0-rc.175 (#85) ... * Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-16 15:34:43 +02:00