A transform launched on a non-blocking stream straight after cufftPlanMany could read the
plan before its setup, still queued on the NULL stream, had finished: Xid 31, a read at a
fixed near-null address, "an illegal memory access" in the indexer threads. It showed at
transform length 3072, in 3 of 4 runs of one rotation set where a pool built its FFT
indexers at the start of integration (6 of 28 ci-tier sets, 9 of 25 null-model runs).
It went away under CUDA_LAUNCH_BLOCKING=1 and with a cudaStreamSynchronize(NULL) after
the plan (0 of 11 runs). The 3142 length the indexer used so far never hit it, but
nothing ordered it either.
Plans are built once per indexer, so the wait costs nothing measurable. The other cuFFT
users (beam-centre FFT, model structure factors, rigid body) create raw plans and are not
covered by this wrapper.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This reverts commit 64a8d8234. Measured on non-lysozyme sets (one search per frame scheme, 5000
frames each, with -C -S): on a body-centred orthorhombic crystal with a 235 A axis the single call
indexed 14.4% of frames against 15.9% with the 30/80/all escalation (79.7k -> 63.1k observations);
the escalation restored the frame count exactly (15.9%) on that set, on a P212121 set (73.6%) and on
lysozyme (55.0%). The lean seed matters for ffbidx on a long cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With a known cell the min-pix 1 list is searched again whatever the min-pix 2 list holds. On sparse
serial data the frames with almost no spot of two pixels are real hits: a weakly diffracting
hexagonal run (20000 frames, -C -S) lost half its indexed frames and observations to the gate
(6.5 -> 3.4%, 273k -> 120k observations, CC1/2 .83 -> .74); with the gate off for a known cell it
matches the original scheme (6.5%, 274k, .83). De novo the gate stays: there the cells a free search
fits to single pixels were unrelated to the crystal's.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The carried indexer answer now brings its spindle severity along, so with a goniometer (--force-still)
no severity-only pass runs per carried list. The min-pix 1 search gate counted ice-ring spots the
indexer leaves out; it now counts the spots indexing would use.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The peak search gave each of the 16384 directions to one thread, a chain of dependent
FP64 additions over 1572 bins - latency-bound on a card that holds far more threads.
Each direction's spectrum is now split into four consecutive runs of bins, one thread
each, and the runs' first-best peaks are combined in order with the same strict
comparison, so the first bin of highest prominence is still the one returned.
A run after the first sums its starting window afresh rather than carrying the running
sum in. The double sums of float magnitudes are exact unless a magnitude is below
~1e-9 of its window sum, so the background is the same double: 0 of 491520
directions differ in a synthetic still/rotation harness, and the hkl of 3000 stills
frames (-X fft) is byte-identical.
Kernel (isolated, RTX 5080): 0.87 -> 0.65 ms per call. The stills de-novo loop gains
~4% (128 -> 133 frames/s, ABAB, load ~10): the FFT chain as a whole is bound by GPU
memory bandwidth under the indexer pool's concurrency, so even a background in float
(6x faster kernel, not adopted) gives only ~9%. Rotation first pass: FFT kernels are
~3% of the indexing step.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A still indexed on few spots often gets a lattice that explains them as well as the true one: an axis
multiple, whose reciprocal lattice holds every true node plus the nodes between, or the true lattice in
another setting. Its cell is then not the run's, the merge leaves it out, and its reflections at the true
nodes - as good as any - are lost. On two 25k-frame XFEL runs of a tetragonal crystal these were 26-30%
of the indexed frames; for those holding the true cell as a sublattice, the reflections at true nodes had
the I/sigma of accepted frames and those between them none.
RelabelOnSublattice (new, scale_merge/SublatticeRelabel) compares lattices, not cell parameters: it
looks for integer combinations of the frame's reciprocal axes whose full metric is the run's cell within
the merge's own tolerances, volume ratio 1-4 (another setting, or a 2x-4x supercell). Where several such
bases exist - a nearly symmetric supercell fits along more than one axis - the one with the strongest
mean I/sigma at its nodes is taken, since only the true lattice has signal there. The frame's indices
are re-derived on that basis (h = h' M^-1), reflections that are not nodes are dropped, as are those
finer than any frame already on the run's cell reaches (the wrong lattice was also predicted to a wrong
resolution), and the frame gets the run's basis in its own orientation. It runs before the first stills
merge, and in --mode scale before the resolutions are recomputed from the cell.
--mode scale, both runs (indicative; the mx pairs are queued): 1060 and 920 crystals put back,
observations +8%, I/sigma 10.3 -> 10.6 and 10.7 -> 11.0, CC1/2 0.966 -> 0.969 and 0.970 -> 0.966.
Tests: SublatticeRelabel_DoubledAxisGoesBackToTheTrueCell, _OtherSettingAndUnrelatedLattice,
_SignalPicksTheAxisThatWasDoubled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The unmerged MTZ took every integrated still. A still whose cell is not close to the run's - the same
test, with the same tolerances, the merge applies - was indexed in another lattice (on few spots,
typically an axis multiple), so its H K L do not name the reflections a reading program takes them
for: aimless or careless would scale them in as wrong reflections. On two 25k-frame XFEL runs such
frames carried 60% of the integrated reflections, and the 824 MB file was the largest single cost
after the merge (about 7 s in --mode scale). Rotation runs are unchanged (one lattice).
The orientation matrix in the batch headers is likewise taken from the first crystal on the run's cell.
Test: Unmerged MTZ: on stills, only the crystals on the run's cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ExportReflections ran French-Wilson single-threaded: about 1 s on the P1 search merge of a 25k-frame
stills run (180k reflections). It now uses the merge's thread count, as AssignRfreeFlags does; both
give the same result at any worker count by construction. mtz/unmerged-mtz data, hkl, cif and report
identical (--mode scale, 25k-frame XFEL run).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The device scaling's peak, measured with nsys on four large sets (33-123 M observations, 4-16 M
distinct hkl), is 81 bytes an observation plus 291 bytes a distinct hkl to within 5%. Once the
ingest has counted both, an engine whose estimate is over 80% of the card's total memory scales on
the CPU instead, with one warning line saying why; a resident ingest is made again on the host,
which the CPU path needs. Decided once, before anything is scaled, on the card's total memory: the
answer depends on the card, not on the moment.
On a simulated 8 GB card (total clamped to 8.2e9, 7.75 GB free) a 33 M-observation, 15 M-hkl set
whose final merge alone needs ~7.5 GB on the GPU ran out of memory there; it now completes, and its
merged MTZ matches the 16 GB GPU run to float rounding (max relative difference 9e-7 on any
intensity column, same reflections, cell, space group and resolution cut). The largest estimate on
the battery is 12.45 GB, under 80% of a 16 GB card, so nothing changes there.
The all-observations arm is copied from the engine's card arrays only when that engine is on the
GPU.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each crystal's (psi_x, psi_y) tilt was refined by a Ceres problem rebuilt for every inner
iteration: one auto-differentiated residual block and one Cauchy loss per reflection. With the
scaling reference cleaned up it was 70% of the stills scale/merge. The same objective - Cauchy
on the sigma-weighted intensity residuals plus the Gaussian prior on the tilt, the tilt bounded -
is now minimised by a small Levenberg-Marquardt on the rho'-reweighted 2x2 normal equations,
the tilt applied to all reflections through one rotation matrix and the Jacobian taken by
central differences.
It reaches the same answer to optimizer precision: on the new synthetic test
(StillsPartialityRefine_RecoversTheTilt) partiality error 0.0559 -> 0.03019 with Ceres and
0.03016 with this, mean tilt 0.02065 vs 0.02067 deg; on two 25k-frame XFEL runs (--mode scale)
CC1/2, I/sigma, R_pim and the CC against a deposited lysozyme model agree to 0.0005, mean
|dpsi| 0.023 deg on both. Scale+merge 6.9 -> 3.0 s (--mode scale, loaded box).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The final stills merge drops every crystal whose cell is not close to the consensus cell
(MergeOnTheFly::Mask), but the scaling reference (MergeAll) and the partiality post-refinement
were made over all of them. On two 25k-frame XFEL runs of a tetragonal crystal those were 26-30%
of the indexed frames and 60% of the integrated reflections: weak frames (median 8 indexed
spots) indexed in a 2x/3x/4x supercell or an unrelated lattice, whose supercell-only reflections
are pure noise and whose indices do not name the true reflections. They went into the
reference every merged crystal was scaled against, and through the tilt refinement.
MergeAll and StillsPartialityRefine::Run now take the reference cell and use only the crystals
it accepts - the same test, with the same tolerances, as the merge.
Measured (both runs, mx path): CC1/2 0.9599 -> 0.9598 and 0.9645 -> 0.9644, I/sigma +0.05 and
+0.02, R_pim unchanged, space group and resolution unchanged; --mode scale against a deposited
tetragonal lysozyme model CC 0.9452 -> 0.9477 and 0.9461 -> 0.9473. Scale+merge 2.1x faster on
--mode scale (14.6 -> 6.9 s, loaded box).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Changes results - not for merging without the owner's approval.
The 30/80/all seed escalation exists for the de-novo search, which a flooded frame's full spot list
derails. ffbidx fits a known cell with a trimmed score that already discounts off-lattice spots, so
for it the escalation only multiplied calls: up to three per searched list, ~2.2 per indexed frame
with a fixed min-pix 2. The ffbidx path is GPU-bound (nsys: card 98.5% busy, ffbidx kernels 86%), so
calls are the loop rate. Stills only: rotation never resolves to ffbidx.
Measured on top of the one-search min-pix scheme (parent), two full 24960-frame serial runs, P41212
merge (box loaded by other agents, load 13-17):
run A: 1130 -> 1532 frames/s; obs 5.30M -> 5.47M; I/sig 13.02 -> 13.12; R_pim 6.46 -> 6.43%;
CC1/2 .973 -> .976; indexing rate 56.9 -> 57.3% (original scheme: 785 frames/s, 5.37M,
13.09, 6.48%, .970)
run B: 1017 -> 1488 frames/s; obs 4.89M -> 4.99M; I/sig 11.95 -> 11.94; R_pim 6.47 -> 6.42%;
CC1/2 .970 -> .972; indexing rate 52.4 -> 52.9% (original: 750, 4.91M, 12.00, 6.50%, .971)
Warm 6000-frame ABAB in one lock hold: 1453/1419 -> 1734/1810 frames/s, same indexing rate.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Changes results - not for merging without the owner's approval.
The per-image min-pix scheme indexed every frame at min-pix 3, 2 and 1, each with the 30/80/all seed
escalation, then indexed the winner again: 3.9 indexer calls per frame. Measured per frame on a
serial lysozyme run (2000 frames, de novo and against the known cell):
- where several lists index, the searches return the same lattice (97-98% of pairs);
- the gain over a fixed min-pix 2 has two sources: (A) frames min-pix 2 indexes but whose min-pix
1/3 list indexes ~26 more spots (a better per-frame refinement), and (B) frames with almost no
spot of two pixels (min-pix 3/2 lists of 1-3 spots, ~50 single pixels at min-pix 1) on which the
min-pix 1 search fits 8-9 spots. De novo, 189 of 194 such B lattices had cells unrelated to the
crystal's; against the known cell they index, but leaving them out raises I/sigma and lowers R_pim.
Now: the lattice is searched for on the min-pix 2 list (min-pix 3 where that does not index); the
first list that indexes hands its indexer answer to the others, which are only refined and scored
(FrameIndexerCalls::lattice_found). The min-pix 1 list is searched on its own only where the min-pix
2 list holds at least ViableCellMinSpots spots. Ties keep the larger min-pix, as before.
Measured on two full 24960-frame serial runs (RTX 5080, box loaded by other agents, load 9-20; loop
rate is frames/s of the image loop; I/sig, R_pim, observations of the P41212 merge):
known cell (ffbidx, -C): base 785 / 750 -> 1130 / 1017 frames/s (warm 6000-frame ABAB 860 -> 1453)
run A: obs 5.37M -> 5.30M, I/sig 13.09 -> 13.02, R_pim 6.48 -> 6.46%, CC1/2 .970 -> .973
run B: obs 4.91M -> 4.89M, I/sig 12.00 -> 11.95, R_pim 6.50 -> 6.47%, CC1/2 .971 -> .970
de novo (FFT): base 89 / 91 -> 174 / 195 frames/s
run A: obs 3.96M -> 3.91M, I/sig 10.18 -> 10.12, R_pim 6.94 -> 6.98%, CC1/2 .960 -> .967
run B: obs 3.91M -> 3.89M, I/sig 10.53 -> 10.52, R_pim 7.01 -> 7.03%, CC1/2 .965 -> .961
fixed min-pix 2, for scale: ffbidx 1496 / 1316 frames/s, obs 4.94M / 4.54M, I/sig 12.62 / 11.77.
The indexing rate drops (ffbidx 59.5 -> 56.9%, 55.2 -> 52.4%; FFT 60.5 -> 52.1%, 60.2 -> 47.9%):
the frames that go are the spurious B lattices; observations stay within 1.3%. Indexer calls per
frame 3.89 -> 1.67 (de novo, 2000 frames); the min-pix 1 search is skipped on 58% of frames.
De novo, 23 of 691 frames whose lattice was the crystal's now take a superlattice (13, same spots
indexed) or a cell fitted to 6-12 spots of a weak frame (10) from the min-pix 2 search.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
calculate_fft_result (the per-direction prominence peak search) was 61-65% of the GPU kernel time of
a de-novo stills run, and the card is what limits that loop. It is one dependent chain per direction,
latency-bound. Two exact changes, taken where the spectrum holds a whole background window (any
FFT max cell above ~30 A; the general loop stays for the rest):
- the sliding window written out as its three phases (clamped at the start, centred, clamped at the
end): the same additions and subtractions in the same order, so the running double sum is bit for
bit the same, but the loads no longer sit behind data-dependent while loops and are issued ahead;
- the window mean divides by a constant width, so the double division (a long FP64 iteration on a
consumer card) is replaced by the reciprocal multiply plus one correction by the exact remainder,
which returns the correctly rounded quotient (Markstein). Checked on the CPU against the division
for 4e8 random dividends over all odd widths 3-401: 0 differences.
Measured (RTX 5080, box loaded by other builds, load 12-25): kernel minimum 1.50 -> 0.83 ms, mean
under the 8-stream pool 4.7 -> 2.8 ms; de-novo stills 2000 frames ABAB 113/120 -> 127/138 frames/s.
p.hkl byte-identical (2000 and 3000 frames, base and new min-pix schemes).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
hide_gpus() in CUDAWrapper makes get_gpu_count() answer 0 without calling
cudaGetDeviceCount, so every GPU decision in the process (indexer Auto ->
FFTW, preprocessing/spot finding/decode, pre-scan, scaling engine, model
validation, MR) takes the path it takes on a machine without a GPU, and no
CUDA context is created. rugnux reads --no-gpu in main, ahead of the parser,
because set_gpu_blocking_sync() already asks CUDA for the devices. The
banner says 'CPU only (--no-gpu)'. The 'run it on the CPU' advice in the
out-of-GPU-memory messages and in the install docs now names --no-gpu
instead of CUDA_VISIBLE_DEVICES=.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stills scale/merge after the image loop was almost entirely serial:
- every scaling-reference merge (MergeAll: the per-image scaling reference and the two
partiality post-refinement references) ran French-Wilson single-threaded on its export,
about 3 s each, and assigned free-R flags - neither is read by a reference, which uses I
only. MergeAll now exports intensities alone (MergeOnTheFly::ExportIntensities).
- the merge accumulated into a std::map; it is now an unordered_map exported in key order,
which is the std::map order.
- AddImage, the error-model collection and the R_meas pass of MergeStats did the
per-observation work (systematic-absence test, ASU key, filters, error-model sigma,
outlier test, merged-I lookup) in one serial loop. They now reduce a block of images in
parallel and consume it in image order (ReduceImagesInOrder), so every floating-point
sum, half-set choice and hash-map insertion is made in the serial order.
Measured on a 25k-frame XFEL stills run (box loaded, load 9-14): merge P1 19.0 -> 7.1 s,
final-group merge 27.7 -> 13.2 s; on a second run 23.4 -> 7.7 s and 29.7 -> 13.6 s.
mtz/unmerged mtz data, hkl, cif and report identical on both runs, mx and --mode scale.
New test MergeOnTheFly_AddImagesIsTheSerialMerge checks the parallel merge against the
serial loop across a block boundary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stills per-image loop spent 99% of its time indexing (28.4 of 28.7 ms per frame on a 16 Mpx
serial set; read, decode, spot finding and integration together 0.33 ms), and only four threads
were ever busy:
- The 16 image workers each block on IndexerThreadPool, which had the default 4 threads, while the
FFT candidate post-processing is CPU work. Stills now size the pool to min(-N, 8); 8 gets 94% of
what 16 does (83 / 125 / 133 frames/s at 4 / 8 / 16 with the changes below) at half the card
memory - each GPU FFT indexer holds ~0.5 GB, 16 of them took the run to 12.7 GB.
- FFTIndexer::ReduceResults (widened path, taken on ~2/3 of calls) deduplicated ~2900 candidate
cells per call by a linear is_close scan: 46% of all CPU cycles. The accepted cells are now kept
ordered by a and only the 2%-window of a is asked; the answer (any close cell) is the same.
- FilterFFTResults built a 16k-entry multimap per call (11% of cycles); now a partial sort of
indices in the multimap's reverse order (magnitude descending, later direction first on ties).
- calculate_fft_result reads magnitudes from a bin-major transposed copy (in the histogram buffer,
no new memory): neighbouring lanes no longer read a spectrum apart. Bit-identical results
(checked against the old kernel on 4000+ calls, 0 mismatches); not faster on an idle card, but
67 -> 98 frames/s with the pool's streams sharing it.
- The stills min-pix choice indexes a frame at 3/2/1 and then the winner again; the seed of
strongest spots is often the same list each time (31% of indexer calls repeated an earlier call
of the same frame). IndexAndRefine answers a repeated list from a per-frame FrameIndexerCalls
memo kept by MXAnalysisWithoutFPGA for the duration of one frame.
Measured (RTX 5080, 32 threads, data in page cache), first 2000 frames, ABAB: 35/35 -> 127/127
frames/s, p.hkl md5 identical. Full 24960-frame run: loop 1081 s -> 277 s, wall 1132 s -> 328 s,
p.hkl md5 identical (P41212, 33234 unique); with a pool of 16 the loop is 222 s (the full run has
more spots per frame than its first 2000 frames, so the CPU side weighs more there). With a known cell (ffbidx path) 715 -> 920 frames/s on
2000 frames, p.hkl identical. Tests: jfjoch_test [Indexing] pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- MolecularReplacement: an engine whose making throws anything (std::bad_alloc, a gemmi error), not
only a JFJochException, is refused and its reservation given back, so a thread waiting for the
same engine is woken instead of waiting forever.
- RotationScaleMergeGPU::Alloc waits for the GPU work beside itself (GPU_BUSY_TIMEOUT), so its
buffers are allocated plainly Synchronous; SynchronousAfterWorkBeside added a second 10-minute wait
in front of it.
- CPU_DATA_ANALYSIS_INDEXING: an offline run decides its near ties on the whole sweep from the start;
only a live run (broker URL or --simulate-live) decides early and re-checks.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Last-bit change, not bit-exact: every partial whose frame does not contain its exact diffracting
condition gets a partiality - and so a slice-centroid walk along its Debye ring - that differs in
the last bits.
The partiality was (erf(b) - erf(a)) / 2 in float. Where the frame's slice lies in one tail of the
rocking curve both erf are near +-1, and their difference keeps only a few ulps of 1: glibc erf and
CUDA erff, one ulp apart, gave partialities ~1e-3 apart (relative) at p ~ 3e-5. The slice centroid
(RockingSliceCentroid_rad) divides by that partiality, and at zeta ~ 0.05 the walk it scales is
hundreds of pixels, so CPU and GPU placed such a reflection up to 0.1 px apart -
BraggPrediction_CPU_GPU_consistency_large_cell failed on one of them once --fmad=false (7ed970503)
moved the ulps. A slice in one tail now takes the same difference from erfcf, which keeps its relative
precision there; one helper (RockingSlicePartiality) serves both predictors.
Reflections placed more than 0.01 px apart by CPU and GPU in that test: 65 -> 0. BraggPrediction*,
BraggPredictionRot*, BraggPredictionRotGPU*, RockingSlice* pass. 7qij image passes 5.68/7.90 s ->
5.26/7.50 s (one lock hold; no cost).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Molecular replacement's structure-factor score engines were made one per group, copy count and
resolution, each admitted against a quarter of the card's total, and all kept to the end of the
search. On a large cell they summed to well over the card: traced with nsys on the rotated 4nwv
model (which failed on every run, base included), seven engines of 1.36 GB were held beside the
two rigid-body pools (5.9 GB) when a 288 MB buffer did not fit.
Each engine is still admitted, or the search scored on the CPU, exactly as before, on the card's
total memory. Now the engines kept on a card also stay within that quarter together: an engine
that does not fit beside them takes the place of the least recently used one no score is using,
and waits for a score to finish where all are in use; one taken out is made again when next asked
for. A score holds its engine for as long as it uses it (shared_ptr lease). Engines compute the
same numbers whenever they are made, so only when the card holds them changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The intermittent "out of GPU memory" on the 3600-frame cbf.gz set (about one run in three on a 16 GB
card, base included) was caught with allocation backtraces: FitErrorModelGPU's 42.8 MB in
MergeAndStats failed while the speculative geometry probe (GPUWorkBeside) held its FFT indexers on
the same card. The engine's own arrays (RotationScaleMergeGPU Impl::Alloc) wait for such work to end
and ask again; the merge's helpers - ErrorModelGPU, WilsonOutliersGPU, PairSortGPU and the cub
scratch of the engine itself - allocated directly and failed at once.
CudaAlloc::SynchronousAfterWorkBeside: Synchronous, and where the card is full after the pool is
trimmed, it waits for the GPU work beside the main line (wait_for_gpu_work_beside, 10 min as for
the engine) and asks once more. The four merge files use it. Only the probe passes register
GPUWorkBeside and they index only, so none of them merges and nothing waits for itself. When a
buffer is allocated is all that changes; what is computed is not.
Catch2 CudaDevicePtr_AfterWorkBesideWaitsForItsMemory: with the card filled by work beside, a
Synchronous buffer fails and a SynchronousAfterWorkBeside one gets its memory once the work ends.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A screw candidate whose absent class no reflection contradicts, refused only because its
evidence could not reach min_screw_absence_evidence even with every predicted-absent axial
reflection at the floor (one such reflection reaches 6.9 nats; a row without a control class
reaches nothing), left the symmorphic group adopted and the report saying the group follows from
the absences. Such a row is now SPACE_GROUP_SCREW_UNDETERMINED, with the screw group in
SPACE_GROUP_ALTERNATIVES. Report only: the result is kept apart (unreached_screws) and folded in
after every decision, so the written group and files do not change.
Measured on a monoclinic set whose merge holds five 0k0 reflections at noise level, one of them
predicted absent: P2 was reported as decided by the absences against a deposited P21; it is now
P2 with the b screw open and P21 offered. Over the full battery's candidate tables the condition
holds on that set and one private set (the same P2/P21 case) only. ci tier: no change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- All 84 DOIs checked (82 on Crossref, 2 DataCite): title, journal, volume, pages and first author
match; Matthews 1968 and Yeates 1997 links percent-encode their parentheses; the MOSFLM link
points at a page that answers.
- autoPROC: an explicit acknowledgement of Clemens Vonrhein (Global Phasing) for the principle that a
pipeline should verify the metadata it is given - beam centre, geometry, goniometer - and correct
it where it is wrong; Vonrhein et al. (2011) Acta Cryst. D67, 293-302 (DOI checked).
- Zhang, Sauter et al. (2006) DISTL, already credited in SpotFindingSettings.h, gets its entry.
- Molecular replacement: tNCS looked for in the native Patterson as Read, Adams & McCoy do; the same
credit at DetectTranslationalNcs (comment only).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- CPU_DATA_ANALYSIS_IMAGE §0: CBF byte-offset and marCCD widening are decoded on the card; the host
decoders are the CPU build's path.
- CPU_DATA_ANALYSIS_DECISIONS §14: the null runs for every model the rigid body is tried on; the
device choice is made once on the card's total (structure factors and rigid body a quarter, scale
fits an eighth) and a CUDA failure ends the validation, deciding nothing; a subgroup the model fits
is written as <prefix>_mr; new §14.10 on molecular replacement (model kind, rotation and
translation functions, likelihood, Matthews, tNCS and P1 rules, null, what it may change).
- CPU_DATA_ANALYSIS: scope and references gain molecular replacement.
- CPU_DATA_ANALYSIS_INDEXING: the near tie decided early and re-checked once every frame is in.
- RUGNUX: merging is the default and _process.h5 optional; live following of a collection with a
broker URL; --model's MR fallback.
- RUGNUX_INSTALL, RUGNUX_OVERVIEW, RUGNUX_TUTORIAL, RUGNUX_REPORT: GPU work list, <prefix>_mr outputs,
MR in --mode mx for rotation and stills, rigid body only in --mode scale.
- RUGNUX_ADVANCED: -N default as the usage states it.
- battery README: the ci tier has 28 sets.
- ModelValidation.cpp: a comment still said a GPU failure reran the validation on the CPU (comment only).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The ingest term was 90 bytes per reflection and 64 per run. Measured with cudaMemGetInfo at each
step of IngestUpload/IngestSort (relative to the free memory before the observation arrays):
- the second radix sort is the peak where runs are few: the 66 B of observation arrays, by_image,
perm, key, key_sorted and cub's alternate key/value buffers of a plain SortPairs - 102.3-103.0 B
per reflection on 7qij (51.1 M, 52.2 M, 78.1 M reflections) and 4nwv (32.7 M);
- the cut is the peak where runs are many: 98 B per reflection plus 56 per run (4nwv, 15.4 M runs:
125.8 B per reflection, 4106 MB against 3923 MB from the old formula);
- the upload holds the 66 B beside the two 64 MB staging buffers (~150 MB measured).
Now max(103 B per reflection + 58 B per run, 66 B per reflection + 160 MB): 2-4% above every
measured peak, where the old formula was 10% under on 7qij (7170 MB against 7986 MB for the main
engine) and 4.5% under on 4nwv. Its only consumer is whether the all-observation arm goes beside
the main engine, which changes when that merge runs, not what it gives.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A UINT16_WIDENED frame (marCCD) was page-locked where it lay with PinInputRegion, which keeps the
registration until the engine moves on to another frame. The pre-scan hands its kept frames to the
spot measurement and to the defective-pixel search at once, each with its own engines, so one
engine could unregister a frame while another engine's copy from it was still in flight. That
takes the context down ("unspecified launch failure"), seen as a rare crash at the end of the
pre-scan on a 3072x3072 marCCD sweep (1 run in 30 reproduced it).
The stored bytes now go through the engine's own page-locked staging buffer, as the CBF and BSLZ4
decoders already do. The pixels decoded are unchanged.
Test: ImagePreprocessorGPU_SharedStoredFramesManyEngines - four engines upload four shared marCCD
frames concurrently; it aborted with the launch failure before the change and passes after it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The first device decoder walked 256-byte tiles, one thread each, straight from global memory: on a
PILATUS 6M frame (~6 MB of stream) that is ~24k threads walking alone, and the two walks took
~0.5 ms of the card per frame - with the CBF decode gone from the workers the card is what bounds
the 7qij loops (kernels 67-78 % busy). Now each thread walks a 68-byte part that its block has read
into shared memory together (68, not 64, so the threads of a warp do not read the same banks), the
scan runs over tiles of 8 parts whose maps are composed first, and the write kernel stages a block's
pixels in shared memory and writes them out coalesced.
Measured on 36 real frames of 9 CBF sets (nsys, average per frame): 0.41 ms against ~0.6 ms before.
Output unchanged: the same tests, and the same 36 frames identical to the host decoder.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Every merge downloaded the fulls' frame and group keys and built both CSRs by serial counting
sorts on the host - four passes over the fulls, two of them scattering into an array as long
as the distinct hkl (15 M on 4nwv) - and uploaded them again. They are now stable radix sorts
of the resident fulls on the card, with the counts summed there; the host gets the counts per
frame and the group CSR it walks itself. The same permutations (stable sorts list each frame's
and each group's fulls in index order, as the counting sorts did): p.hkl identical on cytc
and 4nwv.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
On a set too large for the quality guard's engine to sit beside the main one (8qaw: 123 M
observations; 8a1a), the main engine was ingested, found not to leave room, dropped and
ingested again after the guard - several seconds of ingest for nothing. The pre-pass's
engine merged the guard's very frames earlier in the run, so its card memory per reflection
is now kept (prepass_engine_bytes_ / prepass_engine_reflections_) and both engines are
reckoned from it before either ingests: the guard at its frames' reflections, the main
engine at all of them (observations add in proportion; rocking events and distinct hkl at
most in proportion). Beside where the two fit the card less the 3 GB reserve, otherwise
the guard first. The re-ingest path is gone.
7qij, 9v16, cytc: beside; 8qaw, 8a1a: guard first. Same merges, same numbers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The rest of the resident ingest moves to the card: the delta_phi re-solve against each
frame's smoothed lattice, the one exact-Bragg angle per rocking event, the partiality
recomputed from the smoothed mosaicity (with corr brought to it) and the histogram of the
frames an event spans. The host keeps the per-frame work (the smoothed lattices, the
mosaicity window) and reads back only corr; nothing per observation crosses back, and the
host arrays the smoothing worked on are no longer built on this path.
The integer work is the host's to the bit (the event-span histogram). The floating-point
work differs in the last bits: every rounding on the card is explicit (no contraction, so
the same bits on any GPU), the host's compiler fuses some of the same products into sums,
and erf/atan2 come from the CUDA library. This is the one place in the lever where results
change; see the hand-back report for the sets and metrics it moves.
Measured on 7qij (main engine, 78 M reflections): the whole ingest 6.5 s host + 1.3 s device
before this lever -> 2.9 s, of which 2.4 s is the upload (memory-bound, beside the
post-refinement gather) and 0.25 s this smoothing. Same space group, error model,
resolution and reflection count; IMEAN differs at the 1e-7 level except on near-zero
intensities.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
With the GPU resident, the host ingest swept the source reflections four times (background
and h histogram, sort keys, the in-range build, the device staging), bucket-sorted the keys
and cut the raw-hkl runs: 5-6 s of host time per engine on a 78 M-reflection sweep, run
twice beside the quality guard. Now the frames' reflections are copied once, through two
pinned buffers, to the card, where the per-frame background is summed, the reflections
are ordered by two stable radix passes ((image, index), then raw hkl - the one order the
host's total comparator gives), cut into runs by run-length encoding, filtered by the
resolution range and compacted. The observations, the runs and their order are the host
ingest's to the bit; a new GPU test checks the layout against the host's sort and cut.
The geometry smoothing still runs on the host in this step, on arrays read back.
The all-observation arm's engine is cloned after the main engine's device ingest (its
DeviceBytes are known only then) and copies the runs on the card as well. DeviceBytes now
also covers the ingest's own peak (90 bytes per reflection, 64 per run before the cut).
Measured: p.hkl and MTZ data identical on cytc and 7qij.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The marCCD reader read each frame's 16-bit scanlines into scratch and widened them to int32 on the
worker thread (marccd::ReadInto, ~20 % of the image-loop worker CPU on 4nwv), and the 4-byte image was
uploaded. ReadRawImage now reads the scanlines straight into the image's buffer and hands them over as
a new read-only CompressionAlgorithm::UINT16_WIDENED (mode Int32): two bytes a pixel cross PCIe, from
page-locked memory, and the preprocessing kernel widens them itself.
It is not CompressedImageMode::Uint16, which calls 65535 an error pixel; a marCCD's 65535 is its
saturation marker. So the kernel compares each pixel with int32's error marker and the int32 saturation
cap - the comparisons are made on the pixel's value, in int64, rather than in the storage type - and
decides every pixel exactly as for the int32 image the host would have widened it to. The host widening
(JFJochDecompressPtr) is the CPU path.
Test: ImagePreprocessorGPU_StoredEncodingsMatchCPU, with a frame carrying 65535 at a saturation of
65535, against the CPU preprocessor on the host-widened image.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The CBF reader decoded every frame's x-CBF_BYTE_OFFSET stream on the worker thread - 19-30 % of the
image-loop worker CPU on CBF sets (7qij, 4mtk) - and the 4-byte image was then uploaded. ReadRawImage
now hands over the stream as the file stores it, in the image's own buffer, as a new read-only
CompressionAlgorithm::CBF_BYTE_OFFSET (mode Int32, what it decodes to), and ImagePreprocessorGPU
decodes it on the card into the same int32 image and runs the same preprocessing kernel on it.
The device decode (CBFDecoderGPU) cuts the stream into 256-byte tiles. A token is 1, 3, 7 or 15 bytes,
so a tile can be entered at 15 offsets; for each one thread works out where the decode leaves the
tile, how many pixels it emitted and their sum modulo 2^32 (the walks from different entries meet
within a few bytes, after which they are one walk). An exclusive scan composing these maps gives each
tile its true entry, first pixel and running value, and a second walk writes the pixels. A stream
that ends before the image is refused as on the host.
The host decoder moves to compression/ (DecodeCBFByteOffset) and is reached through
JFJochDecompressPtr, so every host consumer - the CPU build, the azimuthal-integration worker, the
pre-scan's shadow and gain passes - decodes exactly as before, now from the stream. It is the CPU
path, not a fallback: a frame the device refuses is not retried on the host. A process file written
from CBF input stores the decoded int32 image, as it always did. .cbf.gz is still inflated on the host.
Tests: CBFDecoderGPU_MatchesHost (every token size, 0x80 inside payloads, escape runs across tiles,
trailing CIF text), CBFDecoderGPU_ShortStreamIsRefused, ImagePreprocessorGPU_StoredEncodingsMatchCPU.
Checked locally against the host decoder on 36 real frames of 9 open-arm CBF sets (signed 32-bit,
signed and unsigned 16-bit, plain and gzip): identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The rotation predictor used to download every predicted reflection into pinned memory, copy it into
the host vector, build sort keys, std::sort them, gather the 88-byte structs into scratch and copy
them back (OrderOutput), and - on a frame over output_limit - partial_sort the structs themselves
(TruncateToOutput). On a large cell that is a quarter to a third of the image-loop worker CPU
(profiles of 4nwv and 7qij: TruncateToOutput 12-16 %, OrderOutput 9-13 %, plus the memmoves).
Now the same order is built on the device with stable cub radix sorts of an index: delta_phi, then
the packed hkl (OrderOutput's h, k, l, delta_phi), then - over the cap only - partiality descending
and excitation error ascending (TruncateToOutput, whose hkl tie-break the stable sort keeps). Only
the kept rows are gathered and downloaded. Floats are keyed by an order-preserving integer map with
-0 made +0, so the key order is the comparators' order; the only possible difference from the host
is between two solutions of the SAME hkl with bit-identical partiality and excitation error, whose
relative order partial_sort leaves to its heap and the stable sort to delta_phi.
The device buffers grow with half again of slack, so a frame predicting a few more than the last
record (the supercell probe's frames above all) no longer reallocates and predicts again; the host
side is sized for what is returned, at most output_limit.
Test: BraggPredictionRotGPU_DeviceOrderMatchesHost holds the device output against OrderOutput's
order and against TruncateToOutput applied on the host, at four caps.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
One --model now does the right thing by itself:
- A prediction (AlphaFold/ModelCIF, read from the file) goes straight to molecular replacement.
- An experimental structure is placed by the rigid body as before. The null now runs on every
--model run (no NOT_TESTED): "placed" = the fit beats its null (4.51 sd) in the answer's merge
or in the twin hypothesis' merge. Placed: behaviour unchanged (hand and reindex may go into
p.mtz); a subgroup the model fits and prefers is written as <prefix>_mr.
- Not placed (or no crystal cell): molecular replacement with the experimental-model path. It may
relabel the hand into p.mtz past the anomalous veto, never reindexes; another group or the twin
subgroup goes only to <prefix>_mr. Maps, R-free and CC come from validating the MR placement with
every decision switched off; MODEL_FIT stays the rigid body's verdict.
- The twin hypothesis' merge is made in memory with a model where a metric re-ask promoted the
group or the model's group has fewer rotations than the adopted one; written and reported only
where the model chooses it.
- --mr is kept as "--model, searched for by MR straight away".
- Report: MODEL_PLACEMENT= RIGID_BODY|MR|NOT_PLACED, MODEL_DECISIONS_TAKEN gains MR_ENANTIOMORPH,
MODEL_CHOICE_FILE_PREFIX/SPACE_GROUP; the MR keys move into 5. MODEL VALIDATION; MODEL_FIT absent
where no rigid body ran. MR NOT_PLACED is never a warning.
- Battery wrong-model step: check (c) also fails a wrong model placed by MR.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The scale fit of every model-validation fit (fit_model: the real model, the indexing probe's relabellings
and every null replicate) was FitModelScale on the host - a long-double Levenberg-Marquardt per solvent grid
point, ~80% of the validation's CPU on a 400 A cell. It now runs on the GPU through ModelScaleGPU::FitSolvent,
the engine the rigid body and molecular replacement already use, from a small pool (ModelScaleGPUPool: up to
four engines with their own streams, sized on the card's total memory, at most an eighth of it). The grid,
the fits and the winner rule are FitModelScale's; the sums are taken in double on the device, so R-factors
agree with the host's to rounding. The CPU build, and a GPU build without a card, keep FitModelScale.
The change-of-basis probe gridded each candidate frame with gemmi's serial put_model_density_on_grid, once
for the data as merged and once more for each alternative indexing. It now grids each frame once, with
ModelGrid's parallel PutModelDensityOnGrid (the same grid, bit for bit), from threads outside the pool so the
gridding spreads over it, and scores every labelling on that F_calc. Exact.
4nwv (122k atoms, I222, 400 A cell): model validation 33.2 s -> 14.0 s; every logged R, sigma and decision
the same.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- The option is --mr <file>. What kind of model it is is read from the file: a prediction where it
is ModelCIF (_ma_qa_metric) or has no cell and one B value per residue in 0-100 (AlphaFold PDB/mmCIF);
an experimental structure otherwise. A prediction is handled as before (pLDDT >= 70 trim, B from
pLDDT, PAE domains where the AlphaFold DB's PAE file sits beside it). An experimental structure is
taken as its polymer - no waters, ligands, hydrogens or alternative conformers, occupancy 1, isotropic
B as refined - with a coordinate error of 0.67 A (a prediction's at pLDDT 90) for sigma_A; a B column
above 100 no longer stops the search.
- The report section is MOLECULAR REPLACEMENT (EXPERIMENTAL), its keys MR_* (MR, MR_MODEL_FILE,
MR_MODEL_KIND, MR_DECISION, MR_SPACE_GROUP, MR_COPIES, MR_HAND_CHECK, MR_SECONDS, ...); the <prefix>_mr
outputs are unchanged.
- The twin hypothesis' merge that is made only for the model to test (a metric re-ask promoted the
indexed lattice's group) is written and reported only where the model chooses the subgroup.
Raw deposited coordinates of nine control sets, unconverted: every one placed and confirmed against the
deposition (all chains as one body), the groups the same as with the prediction or the prepared model.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
