6 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Fable 5.1	546566cff4	Refinement: the gather's pairs built in fixed-shape buffers, and each evaluation replayed as a CUDA graph ... ModelDensityGPU::Compute() no longer reads the pair count back to the host between its launches: the atoms are counted per brick with integer atomics, the counts scanned into the bricks' segments, the segments filled through atomics and then each brick put into model order by rank (one block per brick), which leaves the pairs exactly as the stable radix sort of the atom-major fill left them - the gather sums in the same order, bit for bit. The pair bound of the atoms set is checked on the host at SetAtoms() instead of on the device at every evaluation, and the buffers shrink from four to three ints per pair. With nothing coming back mid-evaluation, RefinementGPUEngine records the X-ray evaluation and the restraint evaluation each as a CUDA graph on its stream (cudaStreamBeginCapture, thread-local mode) and replays it, re-recording only where the shape changes (the atom count, a gradient asked or not, a restraint set re-uploaded). Two graphs, not one, so that the host still fills the restraints' positions while the X-ray half runs, as before. Where a stream cannot be captured the launches are queued as they were. Outputs byte-identical to 037c32d9f on the in-house cytochrome C live run (MTZ arrays, HKL, models, blobs, report). Workstation (RTX 5080): refinement 1.64 -> 1.50 s on that run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-27 05:57:57 +01:00
leonarski_fandClaude Fable 5.1	b16873036c	Refinement: density records sent from pinned memory, and the L-BFGS direction in fused sweeps ... Two byte-identical cuts to the per-evaluation host chain, which on a large model is twice the GPU's work per evaluation (34 ms against 16 ms of kernels per weight-trial slot, measured in nsys) and so bounds the refinement on one card with three slots, and the whole of it on a card with one slot. - The gather's per-atom density records (38 MB on a 242k-atom model, sent on every B-factor evaluation) live in a PinnedVector and go up by DMA; the boxes go through a pinned staging buffer of the density object. ModelDensityGPU::SetAtoms and RefinementGPUEngine::SetDensities take a pointer and a count. - RefinementMinimiser::Direction takes each pair's dot product in the same sweep that applies the previous pair's update to q: the dot products are serial chains of n additions, and the update's memory traffic now hides under them (20 -> 13 ms per call at 365k parameters). The same operations per element in the same order; the loop-2 coefficient stays written as (a[i] - b) with b on its own, because a[i] - rho * dot contracts into a fused multiply-add and changes the bits (checked bitwise in a standalone harness). - A debug line with the GPU engine's bytes and the budget. 4nwv (open arm, 242k atoms), live flags, one lock hold, loaded box: refinement 42.35 -> 38.40 s; refine.gpu.upload 8.14 -> 3.94 s. All outputs identical but provenance lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 22:48:28 +01:00
leonarski_fandClaude Fable 5.1	3189f71e8f	Refinement: the engine polls for its evaluations and the pair count comes back pinned; the slots beyond one per card go to the cards the ladder's long trials leave free ... The GPU engine's waits on its stream (every evaluation, the refits' transforms) and the model density's wait for the (brick, atom) pair count were blocking event and stream synchronisations; where a weight trial has a card to itself, as on a box with a card per trial, the wake-up of the sleeping thread is on the critical path of every evaluation. Both now poll, and the pair count and overflow flag come back into pinned memory instead of a pageable int (a pageable copy back is staged and blocks the host for the whole stream). OwnSlotDevices places the slots beyond one per card on the cards the ladder's long trials (those refining B) leave free, so that on four cards the three B rungs have a card each and the three B-held rungs share the fourth, instead of two cards each carrying a B rung beside a B-held one. The outcome does not depend on the placement. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu	2026-10-26 22:41:43 +01:00
leonarski_f	e3782cabe4	Refinement GPU: atom densities uploaded only when some B moved, one device copy, positions through a pinned buffer ... Every evaluation uploaded both copies of the atoms' densities (the gather's and the gradient's) and their boxes, although in a positions-only minimisation no B changes between two Refits. The engine now keeps one copy (the gradient kernel reads ModelDensityGPU's atoms and boxes), uploads it only where some atom's density was made again, and sends the positions from a pinned buffer without waiting. The densities that are made again (every atom in a B minimisation) and the grid positions are worked out on the refinement's threads. Same bits. 9yl4: upload 3.29 -> 0.56 s, host side 1.29 -> 0.61 s, refinement 22.3 -> 17.7 s; 5reo 4.8 -> 4.4 s (RTX 5080, -N 8, loaded box). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit 46c3b67c91aee0aec3473b0fb1a4945c00bda070) (cherry picked from commit 0aebc98110c8155932af647d6204891aec662294)	2026-10-26 01:38:22 +01:00
leonarski_f	8261b7354e	v1.0.0-rc.176 (#86) ... * jfjoch_broker: Optional live Rugnux analysis of DECTRIS rotation collections inside the broker, with merged data, maps and the placed model written beside the master file, the rotation lattice in the scan result, and a "Rugnux live" tab and dashboard row in the web frontend. * Rugnux: Faster processing, with molecular replacement and model validation spread over several GPUs and run on the Apple GPU on macOS. * Rugnux: The polarization stated in miniCBF and d*TREK headers is used, twinned crystals' space groups are decided more reliably, and a run that finds no lattice still reports every image. Reviewed-on: #86 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-19 07:03:28 +02:00
leonarski_fandFilip Leonarski	19b135186d	v1.0.0-rc.175 (#85) ... * Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-16 15:34:43 +02:00