6 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	6f5c235fd7	Live molecular replacement: warmed up at model upload, searched beside the second pass and carried to the answer's merge ... - WarmUpModelValidation did nothing for a prediction (no crystal cell: "no atoms or no unit cell"), so a broker's first molecular replacement paid every first use on the GPU. A prediction is now warmed up by a search of its own (WarmUpMolecularReplacement): its search model in a P 1 21 1 cell four radii on each edge, that crystal's intensities to 2.5 A, the search, and the validation of what it placed. RunMolecularReplacement waits for a warm-up still running, as ValidateAgainstModel does. --simulate-live warms the model up as the broker does at upload. Result-identical. - fast_live with a prediction (or --mr): the search starts on the geometry pre-pass's merge, averaged into a group from that merge's own space-group search (to 3 A, standard setting), beside the second pass, on card 1 % cards, a quarter of the threads, under GPUWorkBeside. At the answer's merge CarryMolecularReplacement takes it over where it holds - same group and a cell within 1%, every copy refined by the rigid body beside the others against the answer's merge and its R-free within R_FREE_MARGIN of the one it was found at - and the merge is searched itself otherwise (and always where the twin hypothesis' merge is searched too). RESULT-CHANGING only in the refinement end point of the placement. Sim-live (10 ms/frame, RTX 5080): molecular-replacement step thaumatin 0.96 -> 0.31 s, cytochrome C 1.19 -> 0.25 s; merged reflections identical; thaumatin placed model identical (0.000 A), R-free 0.3765 both; cytochrome C a symmetry-equivalent placement, CA rmsd 0.03-0.05 A per copy, R-free 0.4443 -> 0.4431. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-20 07:51:07 +02:00
leonarski_fandClaude Opus 5.5	13dd2a8710	MolecularReplacement: the choice's null and comparisons start beside the work that decides where it falls ... The placed candidate of lowest R-free is known once the groups' searches are done. Its null and its comparisons with the other candidates (the copies carried into another group beside tNCS, the paired R) used to start only after the tNCS searches again and the null, one after the other. They now start as soon as the groups are done, beside the searches again, and the choice takes each where it comes to the same placements (matched by the placement itself); anything the choice does not reach is waited for and dropped. A null and a comparison are the same number whenever they are computed, so nothing changes but when they run. All of them are waited for before a merge's rigid-body engines are given back. The rigid-body pool gets engines for those fits at once (a null's replicates, a carry and a group searched again: at least 13); the transform's layout is still decided on the search's own fits. Measured on 4x Tesla T4 (24 threads), MR step, two runs each, byte-identical p.hkl, p_model.pdb, p_maps.mtz and MR log lines against the base on 1 and 4 cards: - cytochrome C (AF model, 3 copies, tNCS, enantiomorph decision): 4 cards 2.18 -> 1.74 s, 1 card 2.91 -> 2.71 s; - thaumatin (AF model, one copy, no tNCS): unchanged (1.62 / 2.49 s), it has no comparisons to overlap; - 9s02 (AF model, 4 cards): 8.53 -> 8.39 s, identical. [MolecularReplacement] and RigidBody* pass on 4 cards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-20 07:51:07 +02:00
leonarski_fandClaude Opus 5.5	97ef0936b2	CudaFFTPlan: one cuFFT plan per card and shape, shared by its users; rigid body transforms one grid at a time ... Every cuFFT plan has the driver compile cuFFT's PTX kernels: cufftMakePlanMany -> cuModuleLoadData -> the PTX JIT, under cuFFT's process-wide lock. A second plan of the same shape pays again, even while the first is alive. The CUDA JIT cache makes that ~1 ms. Without one (CUDA_CACHE_DISABLE, no writable $HOME) a plan costs 20-250 ms. That holds for every size (64^3 and 1D 256 included), every transform type and batch, on sm_75 and sm_120, with cuFFT 11.2, 12.0 and 12.4. No CUFFT_* setting changes it, so a choice of grid sizes cannot avoid it; only the number of plans made can be cut. Before this change, a plan was lent to one user at a time, so each concurrent user of a shape on a card made its own: 9-24 rigid-body engines, the translation slices and the score engines. Now a plan is made once per card and shape and shared. Each user keeps its own stream and work area, and CudaFFTPlan::R2C/C2R set both and launch under the plan's mutex. Shared plans give the same bits as private ones: 8 threads on one plan with their own work areas, 9 shapes x 160 transforms, 0 mismatches on RTX 5080 and T4. A shared work area, the negative control, gives 2-13 mismatches per shape where the plan has a work area. The rigid-body engine also dropped its batch-3 plan: the Jacobian transforms its three grids one at a time on the batch-1 plan. That gives the same bits (10 zone-grid shapes, sm_75 and sm_120) and one plan less per zone and card. The cytochrome C and thaumatin MR test sets, AF models (MR step, s): 4x T4 cytc cache on 2.22 -> 2.22 cache off 8.41 -> 4.00 plans 86 -> 24 4x T4 thau cache on 1.58 -> 1.61 cache off 11.13 -> 3.08 plans 131 -> 23 1x T4 cytc cache on 2.87 -> 2.81 cache off 8.82 -> 3.84 plans 80 -> 12 1x T4 thau cache on 2.41 -> 2.39 cache off 10.42 -> 3.17 plans 104 -> 11 RTX5080 cytc cache on 1.41 -> 1.12 cache off 6.5-10.7 -> 2.02 RTX5080 thau cache on 0.90 -> 0.85 cache off 8.03 -> 1.60 Outputs are bit-identical before and after, cache on or off: model, merged hkl, maps, and every MR and validation log line. The mtz/cif differ only in the command line they record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-20 01:51:28 +02:00
leonarski_f	7ed32b899f	v1.0.0-rc.177 (#87) ... * jfjoch_broker: Faster live Rugnux analysis, with the GPU indexers prepared at the start of a collection and kept between collections; Rugnux no longer writes `.ccp4` maps (the map coefficients are in `<prefix>_maps.mtz`). Reviewed-on: #87 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-19 16:07:31 +02:00
leonarski_f	8261b7354e	v1.0.0-rc.176 (#86) ... * jfjoch_broker: Optional live Rugnux analysis of DECTRIS rotation collections inside the broker, with merged data, maps and the placed model written beside the master file, the rotation lattice in the scan result, and a "Rugnux live" tab and dashboard row in the web frontend. * Rugnux: Faster processing, with molecular replacement and model validation spread over several GPUs and run on the Apple GPU on macOS. * Rugnux: The polarization stated in miniCBF and d*TREK headers is used, twinned crystals' space groups are decided more reliably, and a run that finds no lattice still reports every image. Reviewed-on: #86 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-19 07:03:28 +02:00
leonarski_fandFilip Leonarski	19b135186d	v1.0.0-rc.175 (#85) ... * Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-16 15:34:43 +02:00