20 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	04655bd8bf	MolecularReplacement: the rotation function on the Apple GPU (Metal), equal to rounding ... On macOS, where a Metal device may be used (not under --no-gpu), each merge's rotation stage - the sphere and the peaks' refinement - runs on the GPU: the same trilinear interpolation of the same transform in float, one threadgroup per orientation, every bin's sums of |F|^2, |F|^4 and |F|^2 (E^2 - 1) in a fixed order, and the correlation made of them in double on the host. The GPU reads its own copy of the transform in 2 x 2 x 2 bricks (1.3x faster than the grid's rows, as the eight corners then lie on fewer cache lines), made for the rotation stage and given back before the translation function; the seeds' orientations later on stay on the CPU. Where the GPU fails, the stage goes on on the CPU with a warning. The kernel is MolecularReplacementMetal.metal, carried in the binary as the Bragg integration's are. Elsewhere nothing changes. Measured on an Apple M6 (16 GB), alternating with rc176 + the CPU prefetch, two runs each: - 4nwv (the timing benchmark's moved model): rotation 78.9 -> 30.7 s, wall 344.5 -> 298.8 s; peak footprint 16.1-16.8 -> 18.5 GB. - 9s02: rotation 2.99 -> 0.28 s, wall 79.2 -> 76.4 s. p.hkl, the placed model and the maps are identical to the CPU's on both, and so is every line of the molecular-replacement log (RF Z, TF Z, R values). The RF itself agrees with the CPU's to 1e-4, with the same best orientation (MolecularReplacementMetal_MatchesCPU, [portable]). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018My7tqt8a4LyKKjP3FhScT	2026-10-18 12:51:49 +02:00
leonarski_fandClaude Opus 5.5	168a4e64a1	MolecularReplacement: prefetch the transform in the CPU rotation function, output unchanged ... The CPU rotation function interpolates the molecular transform at points scattered over its grid, four cache lines per point. Once the grid is far larger than the caches (835 MB for 4nwv: 16.9e9 interpolations over 17255 orientations) the loop waits on memory, at ~92 ns per interpolation per core on an Apple M6. MolecularTransform::Prefetch() asks for the rows a point's corners lie on, 4 reflections ahead of the one being computed, through one portable helper (__builtin_prefetch under GCC/Clang, nothing elsewhere). Same values, same order, so the output is bit-identical. Only for a transform of 256 MB or more: a smaller one sits mostly in the caches, where the extra address arithmetic only costs. In a standalone copy of the loop on the M6: 30% slower at 13-90 MB, even at ~130-175 MB, 1.5x faster at 400 MB, 1.75-1.8x at 835 MB (prefetch distance 4 to 8 equally good). M6 Mac mini, 4nwv, alternating runs on a quiet machine, against the previous commit: - rotation function 139.1/138.8 s -> 79.1/79.3/78.9 s - wall 400.7/415.7 s -> 338.0/343.1/345.6 s - --no-gpu: 450.4 s -> 385.4 s (rotation 140.3 -> 84.1 s) - 9s02 (14k reflections, 13 MB transform, below the threshold): rotation 2.83 s -> 2.98 s, wall 79.4 -> 80.4 s p.hkl, p_model.cif/pdb, the maps and the MTZ/CIF reflection data are byte-identical in every run (the files differ only in the git tag and command line in their headers). [MolecularReplacement] tests pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018My7tqt8a4LyKKjP3FhScT	2026-10-18 12:51:48 +02:00
leonarski_fandClaude Opus 5.5	6f361ccde5	HostWorkerPool: half the budget, MR placements reserve 16 B per grid point, output unchanged ... Without a GPU (macOS), rc176's worker pool - 1/3 of the 12.9 GB budget on a 16 GB Mac - let only two molecular-replacement placements run at once, as each reserved 32 B per grid point (2.1 GB on 4nwv) against a measured ~0.95 GB. MR rescoring went from 38.6 s to 69 s and 4nwv's wall time from 402 s (budget off) to 444 s. On the M6 Mac mini (16 GB), 4nwv, two runs each in alternating order on a quiet machine: - old build 234774ff0: 418.4 s, 20.0-21.3 GiB peak footprint, 576-682k swapouts - rc176 as shipped: 444.2 s, 14.5-15.2 GiB, 39-66k swapouts - this change: 402.4 s, 15.5-15.8 GiB, 66-93k swapouts - budget switched off: 401.7 s, 21.8-23.3 GiB, 268-433k swapouts 9zlo/9s02/9gjx: 71.9/79.0/82.8 s (old build 73.6/82.2/83.6), no swap. p.hkl byte-identical in every run: the pool decides only what runs at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018My7tqt8a4LyKKjP3FhScT	2026-10-18 08:42:02 +02:00
leonarski_fandClaude Opus 5.5	78c2a69267	Merge broker-rugnux-live into rc176: rugnux in jfjoch_broker as a live thread during a DECTRIS rotation collection (off by default), the rugnux library without file I/O (results in ProcessResult::outputs, the CLI writes them through JFJochRugnuxOutput), END carrying the merged table, map coefficients and placed model ... Branch reviewed at e16db07ff (review: owner's decisions accepted - _maps.mtz instead of .ccp4 in the viewer, -o "" computes everything incl. MR/validation, Ctrl+C/error paths may leave fewer files, the writer links gemmi/zlib) and merged at 55184eb09 with its review fixes (END size guard, frame drop as in Lite, outputs released before re-runs, pool priority, docs, frontend gating). Conflicts: rugnux/Rugnux.h (kept rc176's host_memory_budget_bytes, output_prefix removed as on the branch), rugnux/RugnuxPipeline.h (MtzTable future + rc176's GPUMerge pointer), MolecularReplacement.h/.cpp (rc176's GPU cost model, packed transform and split functions with the branch's RugnuxModel signature; search_model with rc176's other_budgets, result.model_name). tests/RugnuxLargeTest.cpp: rc176's host-budget test writes its files through WriteRugnuxOutputs now that the library writes none. Generated code (broker/gen, frontend client, Python client docs, Redoc) regenerated from the merged jfjoch_api.yaml with update_version.sh (version unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 21:01:38 +02:00
leonarski_fandClaude Opus 5.5	9c9b987928	Merge host-mem-2 into rc176: a host-memory budget from the machine's total RAM decides what runs side by side, the CPU ingest reads the reflections in place (24 B per partial), merge scratch is really freed, and CPU MR/validation workers are bounded by their bytes; output unchanged ... Peak live heap, CPU path -N10, before -> after: 4nwv scaling 14.8 -> 11.9 GiB, analyses 13.5 -> 7.3; 8qaw scaling 34.0 -> 27.1; 9v16 17.2 -> 13.5; 9zlo/9gjx/9s02 7.7/7.4/8.9 -> 6.3/6.3/7.5. With the budget of a 16 GB machine, 4nwv validation 18.4 -> 10.8-12.3 and MR ~17 -> 12.5. GPU 4nwv 16.2 -> 14.8. Byte-identical to its base on GPU and --no-gpu (timing standard + inhouse, 4nwv/8qaw/9v16/ lyso_x06da_ref/cytc_x10sa), at the machine's budget and with a 16 GB one. Conflict in RugnuxScaleMerge.cpp: perf-8gb-cpu's card-byte declaration of the merge and this branch's host-budget decisions on the post-refinement and the all-observation arm, kept both, in that order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 17:59:30 +02:00
leonarski_fandClaude Opus 5.5	5f731db81d	rugnux: the library returns its outputs in memory; the callers write the files ... The rugnux library wrote its results itself: _process.h5 through a FileWriter, the merged/twin/_mr reflection files, the unmerged MTZs, the maps and their MTZ, the placed models, _detector.jpg and _plot.txt, and on a doubled cell it renamed files on disk. It now writes none of them. - MtzTable (common/MtzTable.h): an MTZ file's header, columns and batch headers with no file behind it. It replaces MergedReflectionsMessage and MapCoefficientsMessage in the END message, with one CBOR encoding. WriteMtz (writer/WriteRugnuxFiles.h) is the one serialiser, used by rugnux and jfjoch_writer alike; it refuses a table whose columns differ in length, name a missing dataset, carry a malformed batch header or name a space group gemmi does not know. MergedMtzTable and UnmergedMtzTable build the tables the old writers built. - ProcessResult::outputs (RugnuxOutputs) holds, for the output pass only: the answer's merge (MTZ table plus what the .cif/.hkl are written from), the twin hypothesis' and the model's choice's merges, the unmerged and unmerged-partials tables, the maps (coefficients always; the grids only with ProcessConfig::keep_map_grids), the placed models, the detector JPEG and the per-image plot table, and the start message of the no-lattice _process.h5. Map RMS is taken from the grid in memory, as the CCP4 header stated it, whether or not the map is kept. - The per-image analysis goes to a caller-supplied RugnuxImageSink; rugnux and the viewer give a FileWriter-based ProcessFileSink. The geometry pre-pass no longer makes a _01_process.h5. - The doubled cell keeps both output sets: the answer's in outputs, the other's in DoubledCell::other_outputs with its suffix; the report names files by suffix after the caller's prefix. - The output prefix, write_process_h5 and the observation dump path are gone from ProcessConfig; --dump-observations is a callback. - What used to be gated on having files to write (model validation and MR, the twin-hypothesis merges, the finalist ledger, the scaled fulls, the unmerged tables) is gated on the output pass instead, so a caller with no prefix gets the same results. - RugnuxOutputFiles (new library JFJochRugnuxOutput, linked by rugnux, the viewer and the tests) writes every file from the result with the names and formats rugnux has always used. The viewer writes its job's files the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-17 16:51:20 +02:00
leonarski_fandClaude Opus 5.5	4bf41983ed	rugnux: the model is parsed by the caller and handed in as RugnuxModel ... The analysis read the --model file itself, four to five times per run (the model kind, every validation, molecular replacement, the stills reference intensities) and located the PAE sidecar from the file name. The library is to do no file I/O, so the model now arrives in memory: - RugnuxModel (image_analysis/structure_refinement/RugnuxModel.h): the gemmi structure as read, whether it is ModelCIF, the PAE matrix where a prediction came with one, and the name the report gives it. - ParseRugnuxModel / ReadRugnuxModelFile (writer/RugnuxModelFile.h, in JFJochRugnuxFiles): bytes in, gzip by magic, PDB/mmCIF by content, _ma_qa_metric detection as before; the file reader takes the PAE JSON from the same place PaePathFor named. - ProcessConfig::model replaces model_path; PrepareSearchModel, RunMolecularReplacement, ValidateAgainstModel and ModelReferenceIntensities take the model; result fields name it by model_name. - rugnux reads the model before the run starts and stops with an error where the file holds no model (it used to run and report the model as not validated). The viewer reads it when the job is set up. - The broker parses an uploaded model at once and answers 400 where it is not one; it keeps the parsed model in memory, with no temporary directory or file. It takes no PAE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-17 16:11:58 +02:00
leonarski_fandClaude Opus 5.5	af1bad8f47	MR on the CPU: placement refinements reserve their grids against the host worker pool ... The first copies' placements are refined side by side, one thread each, and without a GPU each makes its structure factors and rigid body on the CPU - about a gigabyte of grids on a large cell, five at once at the 4nwv molecular-replacement peak (14.9 GiB live with a 16 GB machine's budget). Each now reserves 32 B per point of its grid before it starts, as a null replicate does. What each computes, and where its score goes, does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 14:40:41 +02:00
leonarski_fandClaude Opus 5.5	2f3885259f	Merge mr-gpu-r2 into rc176: MR and model validation spread their rigid-body engines over the cards, and MR packs the transform where its cost model expects it faster; output unchanged ... Validated on the 4x T4 server (round 2): p.hkl, p_model.pdb, MR_* and the candidate table identical on 9zlo/9s02/9gjx/4nwv against round 1 on 1 and 4 cards; 4nwv on 4 cards 140.8 -> 127.9 s wall (MR 34.0 -> 21.2 s; rotation 18.4 -> 4.8 s), 9zlo/9s02/9gjx 1-3% faster. Conflicts were the crash fix both branches carry: the branch's side taken (the per-card Reserve of the device test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 12:35:42 +02:00
leonarski_fandClaude Opus 5.5	bc5577d32c	Merge mr-multigpu-crash-fix into rc176: rigid-body engine calls restore the caller's card and the MR GPU functions run on their own card, fixing an illegal memory access in MR with engines on other cards; output unchanged on one card ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 11:00:42 +02:00
leonarski_fandClaude Opus 5.5	210664f454	Model scale fit: no log ratio at |Fcalc| = 0, and no LM step once the model has underflowed ... Two ways the scale fit's normal matrix went singular and gemmi's jordan_solve threw "Trying to reverse singular matrix. Column 0 is zeroed.", ending the run: 1. The isotropic start (fit_isotropic_b_approximately) takes log(Fobs/|Fcalc|) over the strong reflections. At a phase-restricted reflection whose Fcalc and Fmask share a phase, some solvent pair of the k_sol/b_sol grid can cancel the model exactly in float, |Fcalc + k_sol exp(-b_sol s^2) Fmask| = 0. log(Fobs/0) = inf made that grid point's k_overall and B NaN, every sum of the Levenberg-Marquardt NaN, and the solve failed. This is what stopped three open-arm sets in the output pass after MR on the GPU (the scale fit after model validation's rigid body): one strong reflection, one grid point. The CPU path has the same exposure; which grid point hits an exact zero depends on the last bits of Fcalc, so the --no-gpu run of the same set went through. Such a reflection has no log ratio and is now left out of the start - in the GPU kernel, in FixedSolventFit::Start and, as a marked local change, in gemmi's own fit_isotropic_b_approximately, so the three stay equal and the held-solvent fits are covered too. 2. In FitModelScale's CPU Levenberg-Marquardt (ModelValidation_CCModelUnrelatedModelConfirmsNothing in a CUDA=OFF build): on 66 reflections of an unrelated model the fit walked to k_overall -424 and a B where k exp(-B s^2) |F| underflows at every reflection; the squares in J^T J were all 0 while J^T r kept a residue near 1e-210, which jordan_solve rejects. The fit now ends there, at its best so far, as it would at the lambda limit; that grid point then loses on R. The GPU fit takes the derivatives in float and their squares in double, so it cannot reach this state. Both changes only touch fits that threw before. Byte identity checked against an rc176 build on the standard and inhouse timing benchmarks and on 4nwv, 8tyy, 6oel (p.hkl, p_model.pdb, MTZ data, MR_* report keys). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 10:48:36 +02:00
leonarski_fandClaude Opus 5.5	80f634b641	MR GPU: engines spread over the cards; the transform packed where it is expected faster ... Engine placement: the rigid-body engines past this card's first go one card after the other in turn, from the next card and this one last, each within half of its budget. A small cell on four cards (24 engines of 0.09 GB) had put all of them on this card, which has room for them, where rc176 spread them; now 6/6/6/6. A large cell on four 15.6 GB cards keeps T+1/T+2/T+2/T+2. Layout: packing was refused wherever it cost any card an engine, which kept 4nwv plain on four T4s although the packed rotation function on ONE card (15.1 s) already beat plain over four (18.5 s). Now the two layouts' plans are compared on a small cost model, decided once before the search from sizes and counts alone: seconds = rotation interpolations (orientations x reflections x operators) x s/interpolation / cards with a copy + rigid-body fits at once x engine GB x s/(fit GB) / engines Constants measured on 4nwv on Tesla T4 (17255 x 244183 x 4 interpolations): 75.8 s plain and 15.1 s packed on one card; rescoring + null 28.5 s with one engine and 11.6 s with seven, 9 fits of 2.10 GB, so 19.7 s shared among the engines. Packing must still keep the search, the scores and (where plain has it) the rigid body on the GPU, and the budget must hold both grids while the packed one is made. Expected on four T4s: packed T+1/T+1/T+1/T+1, 8.7 s against 21.8 s plain. One card is unchanged (the engines are the same either way, so packed). The layout line now logs both expected times and the search's size. Layout and engine count change where the work runs, never a number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 10:34:09 +02:00
leonarski_fandClaude Opus 5.5	3fa17b9385	Model validation: the rigid-body engines on every card in turn, even where this card has room for all ... The engines past this card's first now go one card after the other, from the next card and this one last, each within half of its own budget, as MR's do. Before, this card took every extra engine it had room for and the other cards only the rest, so a small cell on four cards ran all its fits on one. Where this card has no room for a second engine (a large cell on 15.6 GB cards) the plan is unchanged: 1/3/3/3. This card's scale fits take the room its engines leave. Engines are interchangeable and deterministic, so only where fits run changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 10:34:05 +02:00
leonarski_fandClaude Opus 5.5	43e836573d	MR and model validation on the CPU: workers bounded by their bytes, not the thread count ... - The CPU translation function runs as many rotations at once as the host worker pool holds (8 B per grid point each, 550 MiB on a 400 A cell; 33 at once at 32 threads was 18 GiB), at most nthreads. Counted, not reserved: the rotations run on ParallelFor's pool, whose workers must not wait on memory. - Null replicates (model validation's and MR's) whose structure factors are made on the CPU reserve 32 B per point of their grid (ModelGridPoints) against the pool before they start, so the nine replicates, and nulls run side by side, hold at most the pool between them. Every replicate and rotation computes what it did and is stored by index: no result depends on how many run at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 09:31:05 +02:00
leonarski_fandClaude Opus 5.5	fbdf036e6f	Rigid-body engines leave the calling thread on its own card; the MR search runs on its transform's card ... Every RigidBodyGPUEngine call did cudaSetDevice(engine's card) and never set the caller back. Molecular replacement's group threads refine copies on the pool's engines - on any card where every engine on this one is taken - and then run the next translation function on the same thread. MolecularReplacementGPU ran its calls on whatever card the thread was on, so that function's buffers and cuFFT plan were made on the engine's card, beside the transform and the stream on the search's card: an illegal memory access, 9s02 with four cards visible, every run (the further-copies loop: refine a trial on another card's engine, then translation_function on the group thread). - Each engine call (and the constructor) now holds a DeviceGuard: on the engine's card for the call, back on the caller's after it. - MolecularReplacementGPU remembers the card its transform was uploaded to and runs every call (and its destructor) there, whichever card the calling thread is on. - RigidBodyGPU_CallsLeaveTheCallerOnItsCard: one engine per card, all taken from one thread; the thread's card is unchanged after each call (meaningful with two or more cards). No number changes: where every call runs is the same card as before for each object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 09:24:25 +02:00
leonarski_fandClaude Opus 5.5	a3f30d03c1	Rigid-body engines leave the calling thread on its own card; the MR search runs on its transform's card ... Every RigidBodyGPUEngine call did cudaSetDevice(engine's card) and never set the caller back. Molecular replacement's group threads refine copies on the pool's engines - on any card where every engine on this one is taken - and then run the next translation function on the same thread. MolecularReplacementGPU ran its calls on whatever card the thread was on, so that function's buffers and cuFFT plan were made on the engine's card, beside the transform and the stream on the search's card: an illegal memory access, 9s02 with four cards visible, every run (the further-copies loop: refine a trial on another card's engine, then translation_function on the group thread). - Each engine call (and the constructor) now holds a DeviceGuard: on the engine's card for the call, back on the caller's after it. - MolecularReplacementGPU remembers the card its transform was uploaded to and runs every call (and its destructor) there, whichever card the calling thread is on. - RigidBodyGPU_CallsLeaveTheCallerOnItsCard: one engine per card, all taken from one thread; the thread's card is unchanged after each call (meaningful with two or more cards). No number changes: where every call runs is the same card as before for each object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-17 09:24:16 +02:00
leonarski_fandClaude Opus 5.5	0f461a65a2	Merge mr-transform-layout: the corner-packed molecular transform beside MR on every card ... Combines mr-multigpu (per-card engine budgets, a transform copy on each other card, the rotation/translation functions split over the cards) with mr-transform-layout (the GPU transform optionally held corner-packed, same bits, 2.2x faster rotation function). Conflict (MolecularReplacement.cpp): the two new functions side by side (PackMolecularTransformGPU next to SplitRotationFunction/SplitTranslationFunction). On top of the textual resolution: - One layout decision for all cards, not one per card: the functions are split into equal slices, so a mix of packed and plain cards would run at the speed of the plain ones; either layout gives the same bits. - PackMolecularTransformGPU now plans with the other cards' budgets too and packs only where every card keeps what the plain plan gives it: this card's transform, engines and score engine, and on every other card its copy and its engine count. A packed copy is made after the card's engines are reserved; copy and engines within half of that card's budget leave room for the plain grid it is made from. - The copies on the other cards are made in the chosen layout, and the per-card plan (copies, engines, room) is made with the bytes actually placed. - Test: four 15.6 GB cards stay plain (packed, each other card would fall from T+2 to T+1); with 24 GB cards beside, packed. So on a 4x T4 4nwv runs split and plain, on one card packed; whether giving up 3 of 7 engines for the packed rotation function would pay there is left to a server measurement. Workstation (1x RTX 5080, --gpulock), against rc176: battery timing --benchmark standard (9zlo, 9s02, 9gjx, 4nwv) and --benchmark inhouse (myob, cytc, 4nwv) on GPU, inhouse also --no-gpu: p.hkl, p_model.pdb and the MR_ report block byte-identical on every set. 4nwv GPU: rotation 5.53 -> 2.50 s, MR step 20.1 -> 16.5 s (packed, 2.28 GB in place of 0.84 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-16 18:46:58 +02:00
leonarski_fandClaude Opus 5.5	3aa67a6059	MR GPU: corner-packed molecular transform for the rotation function ... Each sample of the molecular transform interpolates over 8 grid corners that lie in four u-rows of the half-l grid, 4-5 32-byte sectors per sample. On the GPU the transform can now be held packed: every point (i, j, l) stores its four (v, w) corners, so the two neighbouring records hold all eight, read from two places. The packed grid is cut to |u| <= reach as the plain one already is in v, and has the conjugate plane l = -1 in front; below l = -1 the eight corners are read mirrored about -i0 - 1 and conjugated. Same floats, same accumulation order: rotation and translation functions give the same bits (checked in MolecularReplacement_GPUMatchesCPU). The layout is chosen up front from the budget (PackMolecularTransformGPU): packed where the answer merge's plan holds every part beside it that it holds beside the plain transform, and the budget holds both grids while the packed one is made from the plain one on the device; logged either way. The upload no longer stages a host copy of the grid. The CPU At() is unchanged: a microbenchmark gained 1.3x single-threaded and ~1.0-1.1x at 16-32 threads, not worth 2.7x the host memory. 4nwv (moved model, timing benchmark), RTX 5080: rotation 5.56 -> 2.48 s, MR step 20.9 -> 16.7 s; transform on the card 0.84 -> 2.28 GB. p.hkl, p_model.pdb and the MR_ report block byte-identical on the timing sets (GPU and --no-gpu) and on 4nwv, 8tyy, 6oel, 3inp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-16 17:55:30 +02:00
leonarski_fandClaude Opus 5.5	bfc6ca8487	MR and model validation: use every card's memory, split MR's functions over the cards ... On a multi-GPU host the structure stage ran essentially on one card: the rotation function's launches all went to the search's card, and the rigid-body engines of model validation and MR were admitted only within half of that one card's budget, although RigidBodyGPUPool already placed engines on other cards. On a 4x T4 server 4nwv's MR rotation function took 75.8 s on one card, validation queued its 10 fits on one engine and MR's rescoring and null got one engine. - PlanModelValidationGPU / PlanMolecularReplacementGPU: this card is planned exactly as on a machine with it alone (GPU-vs-CPU placement unchanged: 80% of the total, less what is resident). The rigid-body engines it has no room for go on the other cards in turn, each against its own budget (80% of its total, less the resident bytes, counted on every card) within half of it. MR also puts a copy of the molecular transform on each other card within half of its budget. - RigidBodyGPUPool::Reserve takes per-card engine counts. - Validation forwards the most it holds on any card as gpu_bytes to a validation beside, and may run one beside only where twice that fits every card's budget. - MR splits every rotation and translation function into contiguous slices of the orientation list over the cards holding a transform copy (and room for the call's buffers), one thread per card, gathered back in order. Each orientation is scored by itself (one block / one map), nothing is summed across orientations, so N cards give the bytes of one. - ScoreEngines keeps a room per card; score engines always go on the search's own card; function buffers are held on the card they run on. Byte-identical at N=1 (workstation, one RTX 5080): timing benchmark (myob, cytc, 4nwv) and 4nwv, 8tyy, 6oel, 3inp with --model give the same p.hkl, MTZ data, p_model.pdb, maps and report against rc176. New tests: per-card plans on four 15.6 GB cards (validation 1/3/3/3 engines, MR T+1/T+2/T+2/T+2), and the split driven over 1-4 virtual cards on one device, bitwise equal to one card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4	2026-10-16 17:16:28 +02:00
leonarski_fandFilip Leonarski	19b135186d	v1.0.0-rc.175 (#85) ... * Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`). * Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection. * Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS). * Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead. * Rugnux: On macOS, Bragg integration runs on the Apple GPU. * Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand. * Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`. * Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit. * Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed. * Rugnux: The beam-stop shadow is found correctly on tilted detectors. * Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement. * jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps. * jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection. * jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images. * Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges. --------- Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home> Reviewed-on: #85 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-10-16 15:34:43 +02:00