The broker cancelled the whole data collection on the first indexed image when its algorithm was FFT and no
GPU was visible (the indexer pool refuses FFT there, and the receiver cancels on the exception). FFTW is too
slow for live stills, so it is not switched to; with None the pool reports the frame as not indexed and the
collection runs on. rugnux resolves -X FFT itself (cuFFT or FFTW) and never reaches this case.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each crystal's (psi_x, psi_y) tilt was refined by a Ceres problem rebuilt for every inner
iteration: one auto-differentiated residual block and one Cauchy loss per reflection. With the
scaling reference cleaned up it was 70% of the stills scale/merge. The same objective - Cauchy
on the sigma-weighted intensity residuals plus the Gaussian prior on the tilt, the tilt bounded -
is now minimised by a small Levenberg-Marquardt on the rho'-reweighted 2x2 normal equations,
the tilt applied to all reflections through one rotation matrix and the Jacobian taken by
central differences.
It reaches the same answer to optimizer precision: on the new synthetic test
(StillsPartialityRefine_RecoversTheTilt) partiality error 0.0559 -> 0.03019 with Ceres and
0.03016 with this, mean tilt 0.02065 vs 0.02067 deg; on two 25k-frame XFEL runs (--mode scale)
CC1/2, I/sigma, R_pim and the CC against a deposited lysozyme model agree to 0.0005, mean
|dpsi| 0.023 deg on both. Scale+merge 6.9 -> 3.0 s (--mode scale, loaded box).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The final stills merge drops every crystal whose cell is not close to the consensus cell
(MergeOnTheFly::Mask), but the scaling reference (MergeAll) and the partiality post-refinement
were made over all of them. On two 25k-frame XFEL runs of a tetragonal crystal those were 26-30%
of the indexed frames and 60% of the integrated reflections: weak frames (median 8 indexed
spots) indexed in a 2x/3x/4x supercell or an unrelated lattice, whose supercell-only reflections
are pure noise and whose indices do not name the true reflections. They went into the
reference every merged crystal was scaled against, and through the tilt refinement.
MergeAll and StillsPartialityRefine::Run now take the reference cell and use only the crystals
it accepts - the same test, with the same tolerances, as the merge.
Measured (both runs, mx path): CC1/2 0.9599 -> 0.9598 and 0.9645 -> 0.9644, I/sigma +0.05 and
+0.02, R_pim unchanged, space group and resolution unchanged; --mode scale against a deposited
tetragonal lysozyme model CC 0.9452 -> 0.9477 and 0.9461 -> 0.9473. Scale+merge 2.1x faster on
--mode scale (14.6 -> 6.9 s, loaded box).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In rugnux people choose the algorithm, not where it runs. -X fft (and the
old -F) now resolves at parse time to the GPU FFT indexer where a GPU is
visible and --no-gpu is not given, and to FFTW otherwise. -X cufft is the
GPU implementation alone, as -X fft was. -X cufft and -X ffbidx without a
GPU now stop at once with a clear message instead of failing later in the
indexer pool. Auto is unchanged and not redundant: on stills with a known
cell and a GPU it takes FFBIDX.
Only rugnux's CLI spelling changes. The internal IndexingAlgorithmEnum, the
broker and its OpenAPI enum keep FFT = the GPU implementation; the CLI
maps onto it, and the command line the viewer writes for a run (RugnuxCommandLine)
gives the enum's FFT back as -X cufft, which re-parses to the same thing.
A run without -X, and -X fft on a machine with a GPU, process exactly as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
hide_gpus() in CUDAWrapper makes get_gpu_count() answer 0 without calling
cudaGetDeviceCount, so every GPU decision in the process (indexer Auto ->
FFTW, preprocessing/spot finding/decode, pre-scan, scaling engine, model
validation, MR) takes the path it takes on a machine without a GPU, and no
CUDA context is created. rugnux reads --no-gpu in main, ahead of the parser,
because set_gpu_blocking_sync() already asks CUDA for the devices. The
banner says 'CPU only (--no-gpu)'. The 'run it on the CPU' advice in the
out-of-GPU-memory messages and in the install docs now names --no-gpu
instead of CUDA_VISIBLE_DEVICES=.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stills scale/merge after the image loop was almost entirely serial:
- every scaling-reference merge (MergeAll: the per-image scaling reference and the two
partiality post-refinement references) ran French-Wilson single-threaded on its export,
about 3 s each, and assigned free-R flags - neither is read by a reference, which uses I
only. MergeAll now exports intensities alone (MergeOnTheFly::ExportIntensities).
- the merge accumulated into a std::map; it is now an unordered_map exported in key order,
which is the std::map order.
- AddImage, the error-model collection and the R_meas pass of MergeStats did the
per-observation work (systematic-absence test, ASU key, filters, error-model sigma,
outlier test, merged-I lookup) in one serial loop. They now reduce a block of images in
parallel and consume it in image order (ReduceImagesInOrder), so every floating-point
sum, half-set choice and hash-map insertion is made in the serial order.
Measured on a 25k-frame XFEL stills run (box loaded, load 9-14): merge P1 19.0 -> 7.1 s,
final-group merge 27.7 -> 13.2 s; on a second run 23.4 -> 7.7 s and 29.7 -> 13.6 s.
mtz/unmerged mtz data, hkl, cif and report identical on both runs, mx and --mode scale.
New test MergeOnTheFly_AddImagesIsTheSerialMerge checks the parallel merge against the
serial loop across a block boundary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stills per-image loop spent 99% of its time indexing (28.4 of 28.7 ms per frame on a 16 Mpx
serial set; read, decode, spot finding and integration together 0.33 ms), and only four threads
were ever busy:
- The 16 image workers each block on IndexerThreadPool, which had the default 4 threads, while the
FFT candidate post-processing is CPU work. Stills now size the pool to min(-N, 8); 8 gets 94% of
what 16 does (83 / 125 / 133 frames/s at 4 / 8 / 16 with the changes below) at half the card
memory - each GPU FFT indexer holds ~0.5 GB, 16 of them took the run to 12.7 GB.
- FFTIndexer::ReduceResults (widened path, taken on ~2/3 of calls) deduplicated ~2900 candidate
cells per call by a linear is_close scan: 46% of all CPU cycles. The accepted cells are now kept
ordered by a and only the 2%-window of a is asked; the answer (any close cell) is the same.
- FilterFFTResults built a 16k-entry multimap per call (11% of cycles); now a partial sort of
indices in the multimap's reverse order (magnitude descending, later direction first on ties).
- calculate_fft_result reads magnitudes from a bin-major transposed copy (in the histogram buffer,
no new memory): neighbouring lanes no longer read a spectrum apart. Bit-identical results
(checked against the old kernel on 4000+ calls, 0 mismatches); not faster on an idle card, but
67 -> 98 frames/s with the pool's streams sharing it.
- The stills min-pix choice indexes a frame at 3/2/1 and then the winner again; the seed of
strongest spots is often the same list each time (31% of indexer calls repeated an earlier call
of the same frame). IndexAndRefine answers a repeated list from a per-frame FrameIndexerCalls
memo kept by MXAnalysisWithoutFPGA for the duration of one frame.
Measured (RTX 5080, 32 threads, data in page cache), first 2000 frames, ABAB: 35/35 -> 127/127
frames/s, p.hkl md5 identical. Full 24960-frame run: loop 1081 s -> 277 s, wall 1132 s -> 328 s,
p.hkl md5 identical (P41212, 33234 unique); with a pool of 16 the loop is 222 s (the full run has
more spots per frame than its first 2000 frames, so the CPU side weighs more there). With a known cell (ffbidx path) 715 -> 920 frames/s on
2000 frames, p.hkl identical. Tests: jfjoch_test [Indexing] pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- MolecularReplacement: an engine whose making throws anything (std::bad_alloc, a gemmi error), not
only a JFJochException, is refused and its reservation given back, so a thread waiting for the
same engine is woken instead of waiting forever.
- RotationScaleMergeGPU::Alloc waits for the GPU work beside itself (GPU_BUSY_TIMEOUT), so its
buffers are allocated plainly Synchronous; SynchronousAfterWorkBeside added a second 10-minute wait
in front of it.
- CPU_DATA_ANALYSIS_INDEXING: an offline run decides its near ties on the whole sweep from the start;
only a live run (broker URL or --simulate-live) decides early and re-checks.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replaces the kept f853596 run with 20261013-1944_a45135_rc175-a451359-full (open + inhouse; the
private arm is never kept). Against f853596: no verdict changes beyond 5ky6 fail -> unscored (screw
undeterminable), sum 4145 -> 4088 s; the other moves are the last bits of device code without FMA
contraction (8xtf ISa 7.7 -> 7.9) and anomalous sigma.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
After the END message the receiver waits for every image buffer the pusher still holds (for
ZeroMQ: zero-copy messages freed once libzmq has transmitted them) before calling Finalize();
on expiry the collection fails with "Send commands not finalized". The limit was a 10 s literal
repeated in the FPGA and Lite receivers; it is now one constant, SendFinalizeTimeout, in the
JFJochReceiver base class.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The measured-centre arm is a whole geometry pre-pass. It now integrates first a third of the pre-pass
frames, in 3k + 1 blocks of about 10 deg over the whole pre-pass arc with every third kept, and the
file's centre merges the same frames (its outcomes for the other frames are moved out while that
engine ingests). Where the measured centre loses on that subset on every line MeasuredCentreWins draws
by more than NEAR_TIE_SIGMA times the line's noise (Poisson for the strong counts, the Fisher standard
error for a CC1/2), it has lost and no full arm is run. Anything else, a near tie or a win, puts the
run back as the file's centre left it and runs the arm over all the pre-pass frames, deciding as before.
The run made again (decide_all_late_) runs the arm on all frames.
The same subset was tried for the metric-symmetry demoted arm and the walk refits and is not used
there: the held-out residual's standard error does not grow on a third of the frames, so a subset
margin overstates the evidence (one set's metric call flipped at 2.07 sigma), and a subset refit moved
one walk to a worse fixed point.
Against rc175 edde4439c, ci + 49 extra + 9 private sets: p.hkl and MTZ data identical on all 86. The
arm lost on the subset on 22 sets (no full arm), was a tie/win on 9 (full arm as before, +0.3-1.7 s).
Alternated runs: one set 62.1 -> 55.2 s, another 46.2 -> 42.4 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The walk's indexing probes start as the canonical pass's post-refinement fit ends and run beside its
merge and space-group search. The pass asked whether the walk takes a round only before the search
and before its analyses; it now also asks after the search's two arms, before the space-group
decisions and the merge in the adopted group, without waiting for anything. A pass the walk is going
to replace stops there instead of finishing the search. (Waiting for the probes before the search was
measured and dropped: it cost 0.2-3.6 s on sets where the walk took no round.)
Exact by construction (only a pass that is thrown away is cut short); p.hkl and MTZ identical on the
ci tier and 49 extra sets. Fired on one geometry-walk set, alternated runs 33.8 -> 31.6 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Each refit reads a share of the error left, so the walk's steps contract by about the ratio of the
last two. Where the step that ratio predicts moves the end of the sweep by less than half a frame,
the refit (an integration of the pre-pass frames and a post-refinement) and the indexing probe after
it are not made; the scale the validation frames last preferred is adopted, as before.
Measured on the walk sets of the battery against rc175 edde4439c: p.hkl and MTZ data identical; it
fired on one open set (2 refits instead of 3, same adopted scale), about -7 s there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The unit-test job had joined the battery workflow's concurrency group to keep the two off the GPU at
the same time. A group keeps one pending run or job and cancels it for a newer one, cancel-in-progress
or not, so the battery waiting for its runner was cancelled by the same push's unit-test job, on
every push since 140739a69.
The battery is now a job in build_and_test.yml that needs the unit tests and runs whatever their
outcome, so within one run the two take turns; no concurrency group, so a merge to main always gets
both, and a manual release (create_release) runs neither. A manual subset or the full battery is a
Run workflow with battery_subset (empty = full). Two pushes close together can still overlap on the
GPU; a runner with capacity 1 for the GPU jobs is the later fix.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Last-bit change, not bit-exact: every partial whose frame does not contain its exact diffracting
condition gets a partiality - and so a slice-centroid walk along its Debye ring - that differs in
the last bits.
The partiality was (erf(b) - erf(a)) / 2 in float. Where the frame's slice lies in one tail of the
rocking curve both erf are near +-1, and their difference keeps only a few ulps of 1: glibc erf and
CUDA erff, one ulp apart, gave partialities ~1e-3 apart (relative) at p ~ 3e-5. The slice centroid
(RockingSliceCentroid_rad) divides by that partiality, and at zeta ~ 0.05 the walk it scales is
hundreds of pixels, so CPU and GPU placed such a reflection up to 0.1 px apart -
BraggPrediction_CPU_GPU_consistency_large_cell failed on one of them once --fmad=false (7ed970503)
moved the ulps. A slice in one tail now takes the same difference from erfcf, which keeps its relative
precision there; one helper (RockingSlicePartiality) serves both predictors.
Reflections placed more than 0.01 px apart by CPU and GPU in that test: 65 -> 0. BraggPrediction*,
BraggPredictionRot*, BraggPredictionRotGPU*, RockingSlice* pass. 7qij image passes 5.68/7.90 s ->
5.26/7.50 s (one lock hold; no cost).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The rotation merge engine is released before model validation decides whether its GPU path fits,
but a predicted model (or --mr) skips that validation and went into molecular replacement with the
engine still on the card - about 10 GB on the AF 9v16 run, which then failed with 0.0 GB free on
some runs and on the others got fewer rigid-body engines. It is now released before the search
too, on the same condition (no finalist-ledger merge still running on it). Card peak of that run
15.75 -> 14.05 GB; MR keys identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Molecular replacement's structure-factor score engines were made one per group, copy count and
resolution, each admitted against a quarter of the card's total, and all kept to the end of the
search. On a large cell they summed to well over the card: traced with nsys on the rotated 4nwv
model (which failed on every run, base included), seven engines of 1.36 GB were held beside the
two rigid-body pools (5.9 GB) when a 288 MB buffer did not fit.
Each engine is still admitted, or the search scored on the CPU, exactly as before, on the card's
total memory. Now the engines kept on a card also stay within that quarter together: an engine
that does not fit beside them takes the place of the least recently used one no score is using,
and waits for a score to finish where all are in use; one taken out is made again when next asked
for. A score holds its engine for as long as it uses it (shared_ptr lease). Engines compute the
same numbers whenever they are made, so only when the card holds them changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The intermittent "out of GPU memory" on the 3600-frame cbf.gz set (about one run in three on a 16 GB
card, base included) was caught with allocation backtraces: FitErrorModelGPU's 42.8 MB in
MergeAndStats failed while the speculative geometry probe (GPUWorkBeside) held its FFT indexers on
the same card. The engine's own arrays (RotationScaleMergeGPU Impl::Alloc) wait for such work to end
and ask again; the merge's helpers - ErrorModelGPU, WilsonOutliersGPU, PairSortGPU and the cub
scratch of the engine itself - allocated directly and failed at once.
CudaAlloc::SynchronousAfterWorkBeside: Synchronous, and where the card is full after the pool is
trimmed, it waits for the GPU work beside the main line (wait_for_gpu_work_beside, 10 min as for
the engine) and asks once more. The four merge files use it. Only the probe passes register
GPUWorkBeside and they index only, so none of them merges and nothing waits for itself. When a
buffer is allocated is all that changes; what is computed is not.
Catch2 CudaDevicePtr_AfterWorkBesideWaitsForItsMemory: with the card filled by work beside, a
Synchronous buffer fails and a SynchronousAfterWorkBeside one gets its memory once the work ends.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A new last column, "disposition", gives each frame's merge disposition as
a number to plot directly: 1 merged, 0.5 downgraded, 0 rejected; nan for
a frame that never got a scale (never reached the merge) and on a run
that merged nothing, as the file writes every value nothing measured.
Existing columns keep their indices.
0.5 is a marker, not a weight. "Downgraded" is a label applied after the
merge (RotationScaleMerge::CountSweepDisposition marks the merged frames
inside a flagged sweep-quality range); the merge applies no factor of its
own to them - they are carried at whatever weight their own scale and
sigmas give - so there is no fixed or per-frame reduction factor that
could honestly be written instead.
Writer-only change: on three open-arm sets with downgraded and rejected
frames p.hkl is identical to the same commit without it, the first ten
columns are unchanged, and the column's 1/0.5/0 counts equal the report's
merged/downgraded/rejected counts.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Replaces the kept 57fef1c run with 20261013-0557_f85359_rc175-f853596-full (open + inhouse; the
private arm is never kept).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A screw candidate whose absent class no reflection contradicts, refused only because its
evidence could not reach min_screw_absence_evidence even with every predicted-absent axial
reflection at the floor (one such reflection reaches 6.9 nats; a row without a control class
reaches nothing), left the symmorphic group adopted and the report saying the group follows from
the absences. Such a row is now SPACE_GROUP_SCREW_UNDETERMINED, with the screw group in
SPACE_GROUP_ALTERNATIVES. Report only: the result is kept apart (unreached_screws) and folded in
after every decision, so the written group and files do not change.
Measured on a monoclinic set whose merge holds five 0k0 reflections at noise level, one of them
predicted absent: P2 was reported as decided by the absences against a deposited P21; it is now
P2 with the b screw open and P21 offered. Over the full battery's candidate tables the condition
holds on that set and one private set (the same P2/P21 case) only. ci tier: no change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Where the model's group, carried onto the data's axes (CellMappingOperators), is an index-2
subgroup of the adopted group, the merge the model is asked about the twin hypothesis on is the
one in that subgroup: a merge in another subgroup averages reflections the model's symmetry does
not relate. Where the data's own twin hypothesis (p_twin_*) is another subgroup, the model's merge
is made beside it, for the model only, and written - as <prefix>_mr - only where the model is
placed there and fits it better (the existing rule); p_twin_* and every data file are untouched.
A p_mr made from a subgroup merge carries the model's own group where it has the subgroup's
rotations (its screws and hand, unless the anomalous map at its atoms is inverted), without the
reflections that group extinguishes.
Measured: a hexagonal set whose deposited model is P3121 adopted P622 on twin-like data; its
data hypothesis is P6, and the model was asked about P6 and chose it (p_mr P6). Now it is asked
about P321 (twin R-free 0.296 at a fraction 0.45, answer's merge 0.398) and p_mr is P3121, the
deposited group. ci tier 28/28 and 14 named twin/screw sets: p.hkl, p.mtz data and p_twin data
identical; no other report key changed. AF models via --model (8 sets): decisions identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
determine_cutoff_radius() starts at the IT92 approximation of the radius. When the density there is
exactly the float cutoff (1e-5f) - which happens for a few B per element, e.g. a nitrogen at
B + blur = 277.64563 - neither loop runs, x1 == x2, and the interpolation returns 0/0 = NaN. Upstream
gemmi master has the same code.
A NaN radius cast to int gave MakeAtomBox() INT_MIN half-widths, so UseAtomBoxesByPlane() sized its
point buffer from a wrapped product (one point) and then wrote past it: the heap overflow (SIGSEGV, or
a crash later in FFTW) that killed molecular replacement after the rigid body of a large deposited
model, turned to imitate another crystal form, had failed (2 of 73476 atoms, in the CPU rigid body of
an MR placement). The GPU density path took the same NaN radius into its box sizes.
The outward search now steps at least once, so that case returns the point at the cutoff. Every other
B gives the same radius bit for bit (checked over ten elements, every 8th float B in 2-600 A^2: 3 NaN
before, none after, no finite radius changed).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Bounded memory and efficient closing of old data files on long runs, a failure ending the run in
an error state the broker hears about, and the pusher's log line when a stalled writer makes it
drop images; the master-file ordering moves here from the Fixes line. Owner-requested edit.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
ZMQMessage::data() returns const uint8_t *, and GCC refuses the implicit conversion to the
region's uint8_t *. In this mode the region is only read (deserialised and republished), never
written, so the cast is safe.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A send that blocked for over a second drops that image and turns the socket non-blocking for
the rest of the run, so every later image that finds the queue full is dropped too - without a
word in the log. Log the moment it happens, once per run, with the image number.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The slots of closed files stay in `files` (empty) for the whole run, and CloseOldFiles visited
all of them on every image: a thousand slots per image for a million-image run. Keep the open
file numbers in an ordered set and walk that; being ordered, the walk ends at the first file
that is not old enough to close.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every stage between the socket and the HDF5 write was bounded by a message count and never by
bytes: ZeroMQ receive watermark 100, cbor_fifo 200, outside_fifo 1000, repub_fifo 200 - up to
some 1500 images in flight per writer process, each a whole compressed image, so 6-12 GB for a
JUNGFRAU 4M and 30-60 GB for a 9M, and the writer's memory was a quantity nobody had chosen.
ReceiveRingBuffer is a fixed ring of bytes owned by the ImagePuller base class, so the ZeroMQ
and TCP pullers share it. A message is received straight into it (TCP reads the payload into
the ring; ZeroMQ copies the message in) and travels as a shared_ptr region in ImagePullerOutput;
the deserialised image points into that region, republish sends it as is, and the bytes come
back when the last holder lets go. Space is reclaimed in arrival order: a region released out
of turn - republish finishes with messages in its own order - is marked and waits for the ones
in front of it. Because of that, what republish may hold is capped at a quarter of the ring,
and an image over the cap is not republished, exactly as one that finds the republish queue
full; otherwise a stalled republish consumer would pin the ring and slow the writer to its
rate. A message larger than the whole ring (a START or END of a very long run) goes on the heap,
so it cannot wedge the receiver.
A full ring stalls the receive thread, and what that does depends on the stream. On TCP the
pusher waits behind the writer's BUSY heartbeats and gives up only after a minute in which no
image at all was written. On ZeroMQ the pusher's send blocks for one second, then drops the
image and sends without waiting for the rest of the run - so on ZeroMQ a ring that is smaller
than the writer's stalls costs images, where the old 1500-message queues absorbed them. That
is the trade: the budget is now a number, and it has to be chosen. jfjoch_writer sets it with
-m/--rcv_ring_mb, default 2048 MB per writer process; the ZeroMQ receive watermark default goes
from 100 to 10 messages, since it now only counts what sits beyond the ring. The ring's peak
use is in the end-of-run log line, and docs/JFJOCH_WRITER.md has a Memory section with the
sizing rule.
The puller's own default is no ring at all, which keeps every other user as it was: the
broker's DECTRIS stream2 puller (JFJochServices, JFJochReceiverService) stays zero-copy with
the ZeroMQ message as the buffer, and the broker's direct HDF5 writer (HDF5FilePusher ->
FileWriter) never goes through a puller. The puller threads now also catch std::exception, as
the writer does since the previous commit: deserialising the END message of a million-image
run allocates a vector per per-image array, and a bad_alloc leaving the thread ended the process.
Tests: ReceiveRingBufferTest (reuse, wrap-around, full ring, out-of-order release, heap
fallback, zero size, stop, a region outliving its ring); ZMQImageCommTest_SmallRingBuffer (64
one-megabyte images through a three-image ring, nothing lost, peak within the bound); one of
the two writers in TCPImageCommTest_2Writers_WithAck runs on a three-image ring.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ProcessDataImage, ProcessEndMessage and NotifyReceiverOnFinalizedWrite caught only
JFJochException, while ProcessStartMessage and FinalizeDataCollection already caught
std::exception. Anything else - std::bad_alloc from a per-image array, filesystem_error from
a path check, the JSON library refusing a string that is not UTF-8 while publishing a
file-done notification - left CollectImages, and Run() logged "Exception not properly handled"
and returned with the state never reaching Finalized: no END acknowledgement, no writer
notification, the master's .tmp left behind until the next START. On the TCP image stream the
broker then waited for the END acknowledgement indefinitely, because the writer's heartbeat
thread keeps reporting it alive whatever state it is in.
Now the same catch as at the other sites: error state, fatal ACK, END processed, Finalized.
The new StreamWriter test provokes the JSON case through a sample name that is not UTF-8 with
the file-done socket enabled, and checks the run ends Finalized with no master written.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A new page, RUGNUX_MODEL_AND_REFERENCE.md, beside the other rugnux pages in the toctree and
linked from RUGNUX.md and the --model section of RUGNUX_ADVANCED.md. It lists every option that
brings in a model or reference data (--model, --mr, the PAE file beside a prediction, -z and
--reference-column, -S, -C, --force-rotation-lattice, --report-resolution), sorts every decision
into data-only, model-informed label in <prefix>.mtz (hand past the anomalous veto, indexing
ambiguity, setting), model-driven separate files (<prefix>_mr) and report-only statistics, walks a
--model run step by step, says exactly where a model can touch the space group and what guards it
(the null, the anomalous veto, the indexing margin, the MR rules, the wrong-model battery step),
and describes the free set and model bias (a deposited model's R-free on rugnux's own test set is
not cross-validated; -z with the PDB's structure factors inherits the depositor's set). Includes the
one pre-merge use of a model: serial stills with -C/-S, where it breaks the per-image indexing
ambiguity. Documentation only; behaviour as merged in rc175 (7ed970503).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- All 84 DOIs checked (82 on Crossref, 2 DataCite): title, journal, volume, pages and first author
match; Matthews 1968 and Yeates 1997 links percent-encode their parentheses; the MOSFLM link
points at a page that answers.
- autoPROC: an explicit acknowledgement of Clemens Vonrhein (Global Phasing) for the principle that a
pipeline should verify the metadata it is given - beam centre, geometry, goniometer - and correct
it where it is wrong; Vonrhein et al. (2011) Acta Cryst. D67, 293-302 (DOI checked).
- Zhang, Sauter et al. (2006) DISTL, already credited in SpotFindingSettings.h, gets its entry.
- Molecular replacement: tNCS looked for in the native Patterson as Read, Adams & McCoy do; the same
credit at DetectTranslationalNcs (comment only).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
One page listing the CUDA engines by pipeline stage - decode and preprocessing, per-image analysis
(azimuthal and ROI integration, spot finders, extractor), pre-scan accumulators, indexing,
prediction and integration, the scaling engine, model validation and molecular replacement - each
with what it computes, how closely it agrees with the CPU, and a link to the section describing the
method. Added to the toctree after CPU_DATA_ANALYSIS_DECISIONS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- CPU_DATA_ANALYSIS_IMAGE §0: CBF byte-offset and marCCD widening are decoded on the card; the host
decoders are the CPU build's path.
- CPU_DATA_ANALYSIS_DECISIONS §14: the null runs for every model the rigid body is tried on; the
device choice is made once on the card's total (structure factors and rigid body a quarter, scale
fits an eighth) and a CUDA failure ends the validation, deciding nothing; a subgroup the model fits
is written as <prefix>_mr; new §14.10 on molecular replacement (model kind, rotation and
translation functions, likelihood, Matthews, tNCS and P1 rules, null, what it may change).
- CPU_DATA_ANALYSIS: scope and references gain molecular replacement.
- CPU_DATA_ANALYSIS_INDEXING: the near tie decided early and re-checked once every frame is in.
- RUGNUX: merging is the default and _process.h5 optional; live following of a collection with a
broker URL; --model's MR fallback.
- RUGNUX_INSTALL, RUGNUX_OVERVIEW, RUGNUX_TUTORIAL, RUGNUX_REPORT: GPU work list, <prefix>_mr outputs,
MR in --mode mx for rotation and stills, rigid body only in --mode scale.
- RUGNUX_ADVANCED: -N default as the usage states it.
- battery README: the ci tier has 28 sets.
- ModelValidation.cpp: a comment still said a GPU failure reran the validation on the CPU (comment only).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A stored sweep is all there when the run starts, so a first pass now decides a near tie on its
validation frames and the later frames together at once, instead of deciding on the validation frames,
re-checking the pass after the run and making the run again where the re-check ends elsewhere. Live
(following a collection from the broker) keeps that early path unchanged: ProcessConfig::live.
The pre-pass decisions - the rotation-scale walk's refits, the demoted metric-symmetry arm, the
integration radius - stay on the pre-pass frames offline too, with their near-tie re-checks after the
run; only the run made again decides them on all the frames (decide_all_late_, split from decide_late_).
--simulate-live <ms> reads a stored sweep as the collection it was while being written - image i
readable (i+1)*ms after the run starts, 0 = all at once - and takes the live path, so that path can be
tested and run in the battery without a broker.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Owner decision: about a second slower is preferred over 2.5-4 GB more VRAM. With one GPU the
quality guard's merge is launched ahead again, beside the post-refinement and the main engine's
host ingest, and joined before the main engine's device ingest, as before 4158c48df/095814bae;
the per-reflection placement estimate (prepass_engine_bytes_/_reflections_) and its share in the
all-observation arm's memory check go. Kept from those commits, being independent of where the
guard sits on one card: the post-refinement gather in the fit's own task, the first merge holding
its per-frame write-back (rather than waiting) for a guard on a second card, and the guard's
numbers recorded where the pass first needs them.
Measured (RTX 5080 16 GB, base = f853596, alternated, 2 runs each; machine load ~13 on 32 cores):
VRAM peak 7qij 14.7/15.2 -> 13.3/12.2 GB, 9v16 15.1/14.6 -> 12.3/12.8 GB
wall 7qij 47.1/39.0 -> 44.1/42.5 s, 9v16 40.3/37.4 -> 36.8/38.2 s (within the noise)
guard merge alone 4.4-4.7 -> 1.8-1.9 s (7qij), 3.7-3.8 -> 1.6 s (9v16)
p.hkl identical on both.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
