7 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	52f94d2953	tests: FFT-indexer admission test fills every card, right before each Run ... IndexerThreadPool_FFTIndexerWaitsForGPUWorkBeside filled only the test thread's card, but an indexer worker takes its card round-robin (pin_gpu: a process-wide counter modulo the device count). On the CI runner, a container given all of the host's GPUs, the probe's worker could land on a card nobody had filled, build its indexer and fail "CHECK(probe.Run(x, recip).error != nullptr)"; the two checks that expect a wait passed there for the same wrong reason. On a one-card workstation the worker is always on device 0, so it never showed. Every card is now filled, each under its own device, and released the same way, and the fill is topped up right before each Run so memory given back in between by anything else does not let the indexer fit. The pools are trimmed on every card at the end. What the test checks is unchanged. Run 20x alone and with the [CUDAMemHelpers],[AdaptiveSpotFinderGPU] group, also with another process holding 4 GB of the card: all pass. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-11 16:39:43 +02:00
leonarski_fandClaude Opus 5.5	ee8c95de1c	CUDAMemHelpers test: the FFT indexer wait case trims the allocation pool after its pools ... The pools' indexers return their memory to the allocation pool when destroyed; without a trim the next GPU test in the same process (AdaptiveSpotFinderGPU_RunToRunReproducible) found the card taken and failed with out of memory. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-11 14:25:56 +02:00
leonarski_fandClaude Opus 5.5	8dbaa9f8d7	Merge branch 'fft-admit' into rc175: FFT indexer construction waits for GPU work beside it before failing ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-11 14:06:12 +02:00
leonarski_fandClaude Opus 5.5	3befddd649	rugnux: an FFT indexer that does not fit waits for the GPU work beside it, then is built again ... Several pipeline copies of one run build FFT indexer pools on the card at once - the rotation-scale walk's indexing probes start as the output pass begins its image loop, and the speculative geometry probe runs beside the merge - and each indexer is a cuFFT plan plus 0.6-0.9 GB of buffers. Construction had no admission control: a cufftPlanMany or device allocation that found the card full failed straight through, and since IndexerResult carries the exception that ends the run. With 7.2 GB of the card held by another process, a small-molecule rotation set failed 4 of 4 runs this way although each pass fits on its own. Construction now does what the merge's allocations do: hand back the pool's unused memory, wait for the work beside it to end, and build again. Who waits for whom is fixed so that nothing waits in a circle: - the main line waits for the probes (GPUWorkBeside, as the merge already does); - a probe waits for the main line's image pass (new GPUImagePass), never for a probe, itself included. The image pass waits for no probe, so it ends. The walk's indexing probes now hold a GPUWorkBeside too (they are exactly that work), and a run carries gpu_work_beside_ so its pools know which side they are on. Only when an indexer is built changes, never what it computes; with nothing to wait for it still fails at once. Same run with the 7.2 GB hog: 5/5 complete (also at 7.6 and 8.0 GB), p.hkl identical to the unhogged run; a card that cannot hold the run still fails in 0.5 s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-11 11:11:42 +02:00
leonarski_fandClaude Opus 5.5	09c41d067d	Pinned host memory from a process-wide pool; the compressed chunk uploaded from pinned staging ... The GPU engines are built per worker per step, and each took its page-locked host buffers with cudaHostAlloc / cudaHostRegister and gave them back with cudaFreeHost / cudaHostUnregister. Those calls map and lock pages under the process's memory-map lock, so all workers queue behind each other: on the 5080 workstation a myoglobin run made ~3400 of them (6-22 thread-s, depending on the load), 9gdj ~10400 (124 thread-s); on a four-socket host they cost 20-30 ms each. - CudaHostPtr takes its memory from a pool: a freed buffer is kept in a power-of-two size class for the next engine, up to 512 MiB in all; a buffer above 64 MiB is never kept. Rugnux::Run hands the pool back at its end. - PinnedVector (std::vector with the pool's allocator) replaces the registered vectors of AdaptiveSpotFinderGPU and ImagePreprocessorGPU; the Bragg predictors download into a pooled pinned buffer and copy into their output, instead of registering it. - BSLZ4DecoderGPU copies the compressed chunk into a pooled pinned buffer before the upload, so it is one DMA instead of the driver's pageable staging (9gdj: 163 GB pageable H2D, 52 s of copy time, -> 147 GB pinned, 13 s). Measured with nsys on the workstation: myob 3398 -> 1396 calls (674 of them the release at the end), cytc 2986 -> 1270, 9gdj 10428 -> 1748 (124 -> 3.7 thread-s). Output identical (p.hkl md5 on myob, cytc, 9gdj; CI tier battery). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3	2026-10-11 10:37:23 +02:00
leonarski_f	84228bf8be	v1.0.0-rc.173 (#83) ... * jfjoch_broker: Optional per-dataset authentication - statistics, images and plots can require a bearer token, which jfjoch_viewer supports. * jfjoch_viewer: Dark mode and a theme-matched colour scheme, a magnifier panel, and simpler contrast and background controls. * Rugnux: Multiple performance improvements on GPU and CPU (CPU-only processing up to 40% faster, faster image decoding on ARM), with unchanged results. * Rugnux: `--model` rigid-body refinement runs on the GPU, and the model-validation check is faster and more reliable. * Rugnux: Improved scaling and merging - error model, outlier rejection, absorption correction and French-Wilson amplitudes now agree more closely with XDS and ctruncate. * Rugnux: Improved integration - radial background on powder and ice rings, crowded rotation data keep their reflections, and CPU-only builds integrate large unit cells as GPU builds do. * Rugnux: More robust detector geometry - measured beam centre, X-ray bandwidth and goniometer rate, and geometry refinement accepted only on significant evidence. * Rugnux: Merged files are written in the standard setting, or in the setting of a reference MTZ, structure-factor mmCIF or model, with its free-R flags. * Rugnux: Richer report - ice and powder rings, further lattices, superstructure candidates and mosaicity, with warnings worded as prompts to check. * Rugnux: Clear error messages when a data set needs more GPU or host memory than is available. Reviewed-on: #83 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-29 15:57:32 +02:00
leonarski_f	6dfe065365	v1.0.0-rc.172 (#82) ... * Fixed `jfjoch_broker` cancelling every data collection with a CUDA "out of memory" error after long operation: GPU memory no longer leaks with each collection. * Rugnux scales a rotation sweep until the per-frame scales settle instead of for a fixed three rounds, and says so when they did not - merged intensities, and the space group, resolution cut and frame rejection read off them, change accordingly; `--scaling-iterations` is now the cap on that loop (default 100). * Rugnux places every frame of a marCCD, SMV or miniCBF series at the spindle angle its own header states, so a series with missing frames, or with angles written modulo 360, is no longer read at the wrong geometry or refused. * Every rotation run writes two diagnostic files beside its reflections: `<prefix>_detector.jpg`, the detector projection with the pixel mask and the detected beam-stop shadow drawn on it, and `<prefix>_plot.txt`, one row per image. Reviewed-on: #82 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-22 06:48:37 +02:00