The intermittent "out of GPU memory" on the 3600-frame cbf.gz set (about one run in three on a 16 GB
card, base included) was caught with allocation backtraces: FitErrorModelGPU's 42.8 MB in
MergeAndStats failed while the speculative geometry probe (GPUWorkBeside) held its FFT indexers on
the same card. The engine's own arrays (RotationScaleMergeGPU Impl::Alloc) wait for such work to end
and ask again; the merge's helpers - ErrorModelGPU, WilsonOutliersGPU, PairSortGPU and the cub
scratch of the engine itself - allocated directly and failed at once.
CudaAlloc::SynchronousAfterWorkBeside: Synchronous, and where the card is full after the pool is
trimmed, it waits for the GPU work beside the main line (wait_for_gpu_work_beside, 10 min as for
the engine) and asks once more. The four merge files use it. Only the probe passes register
GPUWorkBeside and they index only, so none of them merges and nothing waits for itself. When a
buffer is allocated is all that changes; what is computed is not.
Catch2 CudaDevicePtr_AfterWorkBesideWaitsForItsMemory: with the card filled by work beside, a
Synchronous buffer fails and a SynchronousAfterWorkBeside one gets its memory once the work ends.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
determine_cutoff_radius() starts at the IT92 approximation of the radius. When the density there is
exactly the float cutoff (1e-5f) - which happens for a few B per element, e.g. a nitrogen at
B + blur = 277.64563 - neither loop runs, x1 == x2, and the interpolation returns 0/0 = NaN. Upstream
gemmi master has the same code.
A NaN radius cast to int gave MakeAtomBox() INT_MIN half-widths, so UseAtomBoxesByPlane() sized its
point buffer from a wrapped product (one point) and then wrote past it: the heap overflow (SIGSEGV, or
a crash later in FFTW) that killed molecular replacement after the rigid body of a large deposited
model, turned to imitate another crystal form, had failed (2 of 73476 atoms, in the CPU rigid body of
an MR placement). The GPU density path took the same NaN radius into its box sizes.
The outward search now steps at least once, so that case returns the point at the cutoff. Every other
B gives the same radius bit for bit (checked over ten elements, every 8th float B in 2-600 A^2: 3 NaN
before, none after, no finite radius changed).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
ZMQMessage::data() returns const uint8_t *, and GCC refuses the implicit conversion to the
region's uint8_t *. In this mode the region is only read (deserialised and republished), never
written, so the cast is safe.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A send that blocked for over a second drops that image and turns the socket non-blocking for
the rest of the run, so every later image that finds the queue full is dropped too - without a
word in the log. Log the moment it happens, once per run, with the image number.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The slots of closed files stay in `files` (empty) for the whole run, and CloseOldFiles visited
all of them on every image: a thousand slots per image for a million-image run. Keep the open
file numbers in an ordered set and walk that; being ordered, the walk ends at the first file
that is not old enough to close.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every stage between the socket and the HDF5 write was bounded by a message count and never by
bytes: ZeroMQ receive watermark 100, cbor_fifo 200, outside_fifo 1000, repub_fifo 200 - up to
some 1500 images in flight per writer process, each a whole compressed image, so 6-12 GB for a
JUNGFRAU 4M and 30-60 GB for a 9M, and the writer's memory was a quantity nobody had chosen.
ReceiveRingBuffer is a fixed ring of bytes owned by the ImagePuller base class, so the ZeroMQ
and TCP pullers share it. A message is received straight into it (TCP reads the payload into
the ring; ZeroMQ copies the message in) and travels as a shared_ptr region in ImagePullerOutput;
the deserialised image points into that region, republish sends it as is, and the bytes come
back when the last holder lets go. Space is reclaimed in arrival order: a region released out
of turn - republish finishes with messages in its own order - is marked and waits for the ones
in front of it. Because of that, what republish may hold is capped at a quarter of the ring,
and an image over the cap is not republished, exactly as one that finds the republish queue
full; otherwise a stalled republish consumer would pin the ring and slow the writer to its
rate. A message larger than the whole ring (a START or END of a very long run) goes on the heap,
so it cannot wedge the receiver.
A full ring stalls the receive thread, and what that does depends on the stream. On TCP the
pusher waits behind the writer's BUSY heartbeats and gives up only after a minute in which no
image at all was written. On ZeroMQ the pusher's send blocks for one second, then drops the
image and sends without waiting for the rest of the run - so on ZeroMQ a ring that is smaller
than the writer's stalls costs images, where the old 1500-message queues absorbed them. That
is the trade: the budget is now a number, and it has to be chosen. jfjoch_writer sets it with
-m/--rcv_ring_mb, default 2048 MB per writer process; the ZeroMQ receive watermark default goes
from 100 to 10 messages, since it now only counts what sits beyond the ring. The ring's peak
use is in the end-of-run log line, and docs/JFJOCH_WRITER.md has a Memory section with the
sizing rule.
The puller's own default is no ring at all, which keeps every other user as it was: the
broker's DECTRIS stream2 puller (JFJochServices, JFJochReceiverService) stays zero-copy with
the ZeroMQ message as the buffer, and the broker's direct HDF5 writer (HDF5FilePusher ->
FileWriter) never goes through a puller. The puller threads now also catch std::exception, as
the writer does since the previous commit: deserialising the END message of a million-image
run allocates a vector per per-image array, and a bad_alloc leaving the thread ended the process.
Tests: ReceiveRingBufferTest (reuse, wrap-around, full ring, out-of-order release, heap
fallback, zero size, stop, a region outliving its ring); ZMQImageCommTest_SmallRingBuffer (64
one-megabyte images through a three-image ring, nothing lost, peak within the bound); one of
the two writers in TCPImageCommTest_2Writers_WithAck runs on a three-image ring.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ProcessDataImage, ProcessEndMessage and NotifyReceiverOnFinalizedWrite caught only
JFJochException, while ProcessStartMessage and FinalizeDataCollection already caught
std::exception. Anything else - std::bad_alloc from a per-image array, filesystem_error from
a path check, the JSON library refusing a string that is not UTF-8 while publishing a
file-done notification - left CollectImages, and Run() logged "Exception not properly handled"
and returned with the state never reaching Finalized: no END acknowledgement, no writer
notification, the master's .tmp left behind until the next START. On the TCP image stream the
broker then waited for the END acknowledgement indefinitely, because the writer's heartbeat
thread keeps reporting it alive whatever state it is in.
Now the same catch as at the other sites: error state, fatal ACK, END processed, Finalized.
The new StreamWriter test provokes the JSON case through a sample name that is not UTF-8 with
the file-done socket enabled, and checks the run ends Finalized with no master written.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A new page, RUGNUX_MODEL_AND_REFERENCE.md, beside the other rugnux pages in the toctree and
linked from RUGNUX.md and the --model section of RUGNUX_ADVANCED.md. It lists every option that
brings in a model or reference data (--model, --mr, the PAE file beside a prediction, -z and
--reference-column, -S, -C, --force-rotation-lattice, --report-resolution), sorts every decision
into data-only, model-informed label in <prefix>.mtz (hand past the anomalous veto, indexing
ambiguity, setting), model-driven separate files (<prefix>_mr) and report-only statistics, walks a
--model run step by step, says exactly where a model can touch the space group and what guards it
(the null, the anomalous veto, the indexing margin, the MR rules, the wrong-model battery step),
and describes the free set and model bias (a deposited model's R-free on rugnux's own test set is
not cross-validated; -z with the PDB's structure factors inherits the depositor's set). Includes the
one pre-merge use of a model: serial stills with -C/-S, where it breaks the per-image indexing
ambiguity. Documentation only; behaviour as merged in rc175 (7ed970503).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
nvcc contracts a*b+c into an FMA by default, and decides it separately for every architecture it
compiles for. The same source therefore ran to different bits on a T4 (sm_75) and an RTX 5080
(sm_120), and a run that crossed a knife edge came out differently on the two machines: a rotation
set made again after a near-tie overturn merged with multiplicity 19.0 / R_meas 75.1% on the T4
server and 20.1 / 82.2% here, the delta-CC1/2 frame rejection removing 33 frames on one and 14 on
the other. A build here with only compute_75 code reproduced the server's numbers to the digit.
With --fmad=false for all CUDA sources the sm_75 and sm_120 builds give the same p.hkl on that set.
The host side has been built with -ffp-contract=off for the same reason; this extends it to the
device. Kernel time on a large HDF5 set: 48.3 -> 48.6 s (+0.7%), no measurable wall change.
This changes the bits of every GPU run, so every battery row moves; it is not an exact change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The three cytochrome c rows take P 32 2 1 (154) from their reference models, whose 2-fold holds to
measurement noise; XDS's label carried an arbitrary hand. The why-texts of 6oww, 6toc, 6pxb, 8xte
and 8xtg now quote the tools/battery/symmetry evidence; no accepted answer changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ModelValidation_ReindexesTheDataIntoTheModelsIndexing still expected no null for data already in
the model's indexing, as when the null ran only where a decision was pending. Since the unified
model validation it runs on every validation whose schedule asks for it (fit_test, the default),
and is skipped only where the verdict is another validation's; the test now checks that the null
ran and that the model fits.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A UINT16_WIDENED frame (marCCD) was page-locked where it lay with PinInputRegion, which keeps the
registration until the engine moves on to another frame. The pre-scan hands its kept frames to the
spot measurement and to the defective-pixel search at once, each with its own engines, so one
engine could unregister a frame while another engine's copy from it was still in flight. That
takes the context down ("unspecified launch failure"), seen as a rare crash at the end of the
pre-scan on a 3072x3072 marCCD sweep (1 run in 30 reproduced it).
The stored bytes now go through the engine's own page-locked staging buffer, as the CBF and BSLZ4
decoders already do. The pixels decoded are unchanged.
Test: ImagePreprocessorGPU_SharedStoredFramesManyEngines - four engines upload four shared marCCD
frames concurrently; it aborted with the launch failure before the change and passes after it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The first device decoder walked 256-byte tiles, one thread each, straight from global memory: on a
PILATUS 6M frame (~6 MB of stream) that is ~24k threads walking alone, and the two walks took
~0.5 ms of the card per frame - with the CBF decode gone from the workers the card is what bounds
the 7qij loops (kernels 67-78 % busy). Now each thread walks a 68-byte part that its block has read
into shared memory together (68, not 64, so the threads of a warp do not read the same banks), the
scan runs over tiles of 8 parts whose maps are composed first, and the write kernel stages a block's
pixels in shared memory and writes them out coalesced.
Measured on 36 real frames of 9 CBF sets (nsys, average per frame): 0.41 ms against ~0.6 ms before.
Output unchanged: the same tests, and the same 36 frames identical to the host decoder.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Every merge downloaded the fulls' frame and group keys and built both CSRs by serial counting
sorts on the host - four passes over the fulls, two of them scattering into an array as long
as the distinct hkl (15 M on 4nwv) - and uploaded them again. They are now stable radix sorts
of the resident fulls on the card, with the counts summed there; the host gets the counts per
frame and the group CSR it walks itself. The same permutations (stable sorts list each frame's
and each group's fulls in index order, as the counting sorts did): p.hkl identical on cytc
and 4nwv.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The two passes read every integrated reflection and are memory-bound once on all threads;
the ice flags are now set in the overload sweep. 7qij, 78 M reflections: 0.4 s for both
(1.2-1.3 s serial before this lever). Same counts, same flags.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
On a set too large for the quality guard's engine to sit beside the main one (8qaw: 123 M
observations; 8a1a), the main engine was ingested, found not to leave room, dropped and
ingested again after the guard - several seconds of ingest for nothing. The pre-pass's
engine merged the guard's very frames earlier in the run, so its card memory per reflection
is now kept (prepass_engine_bytes_ / prepass_engine_reflections_) and both engines are
reckoned from it before either ingests: the guard at its frames' reflections, the main
engine at all of them (observations add in proportion; rocking events and distinct hkl at
most in proportion). Beside where the two fit the card less the 3 GB reserve, otherwise
the guard first. The re-ingest path is gone.
7qij, 9v16, cytc: beside; 8qaw, 8a1a: guard first. Same merges, same numbers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The gather and the engine's ingest both read every reflection, and side by side they split
the memory bandwidth and the worker threads; the engine's ingest is what the chain waits for.
The gather now waits for it (or for the pass to end) and runs beside the first merges
instead, which leave the host mostly idle. 7qij main ingest: 2.9 -> 1.1 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The rest of the resident ingest moves to the card: the delta_phi re-solve against each
frame's smoothed lattice, the one exact-Bragg angle per rocking event, the partiality
recomputed from the smoothed mosaicity (with corr brought to it) and the histogram of the
frames an event spans. The host keeps the per-frame work (the smoothed lattices, the
mosaicity window) and reads back only corr; nothing per observation crosses back, and the
host arrays the smoothing worked on are no longer built on this path.
The integer work is the host's to the bit (the event-span histogram). The floating-point
work differs in the last bits: every rounding on the card is explicit (no contraction, so
the same bits on any GPU), the host's compiler fuses some of the same products into sums,
and erf/atan2 come from the CUDA library. This is the one place in the lever where results
change; see the hand-back report for the sets and metrics it moves.
Measured on 7qij (main engine, 78 M reflections): the whole ingest 6.5 s host + 1.3 s device
before this lever -> 2.9 s, of which 2.4 s is the upload (memory-bound, beside the
post-refinement gather) and 0.25 s this smoothing. Same space group, error model,
resolution and reflection count; IMEAN differs at the 1e-7 level except on near-zero
intensities.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Two waits on the critical path of a canonical rotation pass go:
- the post-refinement gather (2-3.5 s on a 78 M-reflection sweep) now runs in the task that
fits it, beside the chain; it reads only the reflections, and the chain waits for it before
the space-group search, the first step that re-indexes them;
- with one GPU the quality guard's merge used to have the card to itself while the main
engine waited. It now runs beside the main engine wherever both fit the card's total
memory (the main engine's DeviceBytes plus its reflection share for the guard, plus the
3 GB reserve the all-observation arm uses); otherwise it runs first as before and the
main engine is ingested again after it. The main engine's first merge holds its
per-frame write-back until the guard has read the outcomes (as the arm on a copy does),
and the guard's numbers are recorded where the pass first needs them.
Same merges, same numbers; only when the guard and the gather run changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
With the GPU resident, the host ingest swept the source reflections four times (background
and h histogram, sort keys, the in-range build, the device staging), bucket-sorted the keys
and cut the raw-hkl runs: 5-6 s of host time per engine on a 78 M-reflection sweep, run
twice beside the quality guard. Now the frames' reflections are copied once, through two
pinned buffers, to the card, where the per-frame background is summed, the reflections
are ordered by two stable radix passes ((image, index), then raw hkl - the one order the
host's total comparator gives), cut into runs by run-length encoding, filtered by the
resolution range and compacted. The observations, the runs and their order are the host
ingest's to the bit; a new GPU test checks the layout against the host's sort and cut.
The geometry smoothing still runs on the host in this step, on arrays read back.
The all-observation arm's engine is cloned after the main engine's device ingest (its
DeviceBytes are known only then) and copies the runs on the card as well. DeviceBytes now
also covers the ingest's own peak (90 bytes per reflection, 64 per run before the cut).
Measured: p.hkl and MTZ data identical on cytc and 7qij.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The two passes over every integrated reflection that open the scaling step ran on one
thread: 1.2-1.3 s on a 78 M-reflection sweep, on the critical path before the scaling
engine starts. One image per task now; the counts are integers and each reflection's
ice flag depends on itself alone, so the result is the same at any thread count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The marCCD reader read each frame's 16-bit scanlines into scratch and widened them to int32 on the
worker thread (marccd::ReadInto, ~20 % of the image-loop worker CPU on 4nwv), and the 4-byte image was
uploaded. ReadRawImage now reads the scanlines straight into the image's buffer and hands them over as
a new read-only CompressionAlgorithm::UINT16_WIDENED (mode Int32): two bytes a pixel cross PCIe, from
page-locked memory, and the preprocessing kernel widens them itself.
It is not CompressedImageMode::Uint16, which calls 65535 an error pixel; a marCCD's 65535 is its
saturation marker. So the kernel compares each pixel with int32's error marker and the int32 saturation
cap - the comparisons are made on the pixel's value, in int64, rather than in the storage type - and
decides every pixel exactly as for the int32 image the host would have widened it to. The host widening
(JFJochDecompressPtr) is the CPU path.
Test: ImagePreprocessorGPU_StoredEncodingsMatchCPU, with a frame carrying 65535 at a saturation of
65535, against the CPU preprocessor on the host-widened image.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The CBF reader decoded every frame's x-CBF_BYTE_OFFSET stream on the worker thread - 19-30 % of the
image-loop worker CPU on CBF sets (7qij, 4mtk) - and the 4-byte image was then uploaded. ReadRawImage
now hands over the stream as the file stores it, in the image's own buffer, as a new read-only
CompressionAlgorithm::CBF_BYTE_OFFSET (mode Int32, what it decodes to), and ImagePreprocessorGPU
decodes it on the card into the same int32 image and runs the same preprocessing kernel on it.
The device decode (CBFDecoderGPU) cuts the stream into 256-byte tiles. A token is 1, 3, 7 or 15 bytes,
so a tile can be entered at 15 offsets; for each one thread works out where the decode leaves the
tile, how many pixels it emitted and their sum modulo 2^32 (the walks from different entries meet
within a few bytes, after which they are one walk). An exclusive scan composing these maps gives each
tile its true entry, first pixel and running value, and a second walk writes the pixels. A stream
that ends before the image is refused as on the host.
The host decoder moves to compression/ (DecodeCBFByteOffset) and is reached through
JFJochDecompressPtr, so every host consumer - the CPU build, the azimuthal-integration worker, the
pre-scan's shadow and gain passes - decodes exactly as before, now from the stream. It is the CPU
path, not a fallback: a frame the device refuses is not retried on the host. A process file written
from CBF input stores the decoded int32 image, as it always did. .cbf.gz is still inflated on the host.
Tests: CBFDecoderGPU_MatchesHost (every token size, 0x80 inside payloads, escape runs across tiles,
trailing CIF text), CBFDecoderGPU_ShortStreamIsRefused, ImagePreprocessorGPU_StoredEncodingsMatchCPU.
Checked locally against the host decoder on 36 real frames of 9 open-arm CBF sets (signed 32-bit,
signed and unsigned 16-bit, plain and gzip): identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The rotation predictor used to download every predicted reflection into pinned memory, copy it into
the host vector, build sort keys, std::sort them, gather the 88-byte structs into scratch and copy
them back (OrderOutput), and - on a frame over output_limit - partial_sort the structs themselves
(TruncateToOutput). On a large cell that is a quarter to a third of the image-loop worker CPU
(profiles of 4nwv and 7qij: TruncateToOutput 12-16 %, OrderOutput 9-13 %, plus the memmoves).
Now the same order is built on the device with stable cub radix sorts of an index: delta_phi, then
the packed hkl (OrderOutput's h, k, l, delta_phi), then - over the cap only - partiality descending
and excitation error ascending (TruncateToOutput, whose hkl tie-break the stable sort keeps). Only
the kept rows are gathered and downloaded. Floats are keyed by an order-preserving integer map with
-0 made +0, so the key order is the comparators' order; the only possible difference from the host
is between two solutions of the SAME hkl with bit-identical partiality and excitation error, whose
relative order partial_sort leaves to its heap and the stable sort to delta_phi.
The device buffers grow with half again of slack, so a frame predicting a few more than the last
record (the supercell probe's frames above all) no longer reallocates and predicts again; the host
side is sized for what is returned, at most output_limit.
Test: BraggPredictionRotGPU_DeviceOrderMatchesHost holds the device output against OrderOutput's
order and against TruncateToOutput applied on the host, at four caps.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
One --model now does the right thing by itself:
- A prediction (AlphaFold/ModelCIF, read from the file) goes straight to molecular replacement.
- An experimental structure is placed by the rigid body as before. The null now runs on every
--model run (no NOT_TESTED): "placed" = the fit beats its null (4.51 sd) in the answer's merge
or in the twin hypothesis' merge. Placed: behaviour unchanged (hand and reindex may go into
p.mtz); a subgroup the model fits and prefers is written as <prefix>_mr.
- Not placed (or no crystal cell): molecular replacement with the experimental-model path. It may
relabel the hand into p.mtz past the anomalous veto, never reindexes; another group or the twin
subgroup goes only to <prefix>_mr. Maps, R-free and CC come from validating the MR placement with
every decision switched off; MODEL_FIT stays the rigid body's verdict.
- The twin hypothesis' merge is made in memory with a model where a metric re-ask promoted the
group or the model's group has fewer rotations than the adopted one; written and reported only
where the model chooses it.
- --mr is kept as "--model, searched for by MR straight away".
- Report: MODEL_PLACEMENT= RIGID_BODY|MR|NOT_PLACED, MODEL_DECISIONS_TAKEN gains MR_ENANTIOMORPH,
MODEL_CHOICE_FILE_PREFIX/SPACE_GROUP; the MR keys move into 5. MODEL VALIDATION; MODEL_FIT absent
where no rigid body ran. MR NOT_PLACED is never a warning.
- Battery wrong-model step: check (c) also fails a wrong model placed by MR.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The scale fit of every model-validation fit (fit_model: the real model, the indexing probe's relabellings
and every null replicate) was FitModelScale on the host - a long-double Levenberg-Marquardt per solvent grid
point, ~80% of the validation's CPU on a 400 A cell. It now runs on the GPU through ModelScaleGPU::FitSolvent,
the engine the rigid body and molecular replacement already use, from a small pool (ModelScaleGPUPool: up to
four engines with their own streams, sized on the card's total memory, at most an eighth of it). The grid,
the fits and the winner rule are FitModelScale's; the sums are taken in double on the device, so R-factors
agree with the host's to rounding. The CPU build, and a GPU build without a card, keep FitModelScale.
The change-of-basis probe gridded each candidate frame with gemmi's serial put_model_density_on_grid, once
for the data as merged and once more for each alternative indexing. It now grids each frame once, with
ModelGrid's parallel PutModelDensityOnGrid (the same grid, bit for bit), from threads outside the pool so the
gridding spreads over it, and scores every labelling on that F_calc. Exact.
4nwv (122k atoms, I222, 400 A cell): model validation 33.2 s -> 14.0 s; every logged R, sigma and decision
the same.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
During a collection every dataset tick ran JFJochViewerDatasetInfo::UpdateLabels(), which
clear()ed the plot-type combo and re-added every item, so an open dropdown lost its hovered
row and scroll position on each tick and picking a plot type was a race against the next
update. The same clear-and-refill ran per tick or per image in the calibration window's
calibration combo, the ROI list combo and the powder calibrant combo; the last two also
threw away the user's choice (the calibrant was reset to the first entry on every image).
New LiveComboBox::SetItems() takes the full item list and leaves the combo untouched when the
list is unchanged. When it does change while the popup is open, the change is held back and
applied (queued) once the popup window hides, so a click that closes the popup lands first.
A replacement keeps the selection by item data, blocks signals, and a held-back one emits
itemsReplaced() so the owner can re-plot.
The calibration window now re-shows the selected calibration on a dataset tick instead of
always reloading the pixel mask over it.
Tested offscreen: a timer fires datasetLoaded() every 25 ms while the dropdown is open; the
popup and hovered row survive, newly available indexing metrics are held back until the popup
closes, and the picked type sticks and is what the chart plots. The rc175 code fails the
hover checks of the same harness.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The option is --mr <file>. What kind of model it is is read from the file: a prediction where it
is ModelCIF (_ma_qa_metric) or has no cell and one B value per residue in 0-100 (AlphaFold PDB/mmCIF);
an experimental structure otherwise. A prediction is handled as before (pLDDT >= 70 trim, B from
pLDDT, PAE domains where the AlphaFold DB's PAE file sits beside it). An experimental structure is
taken as its polymer - no waters, ligands, hydrogens or alternative conformers, occupancy 1, isotropic
B as refined - with a coordinate error of 0.67 A (a prediction's at pLDDT 90) for sigma_A; a B column
above 100 no longer stops the search.
- The report section is MOLECULAR REPLACEMENT (EXPERIMENTAL), its keys MR_* (MR, MR_MODEL_FILE,
MR_MODEL_KIND, MR_DECISION, MR_SPACE_GROUP, MR_COPIES, MR_HAND_CHECK, MR_SECONDS, ...); the <prefix>_mr
outputs are unchanged.
- The twin hypothesis' merge that is made only for the model to test (a metric re-ask promoted the
indexed lattice's group) is written and reported only where the model chooses the subgroup.
Raw deposited coordinates of nine control sets, unconverted: every one placed and confirmed against the
deposition (all chains as one body), the groups the same as with the prediction or the prepared model.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
From the nsys/perf profiles of the copy chain (GPU busy 39% between copies, the thread mostly in host
work and syncs):
- The known-structure translation function's lists, device arrays and cuFFT plan are made once per
group (MolecularReplacementGPU::Prepare) instead of on every copy (~260 MB of uploads and a plan
per call before).
- The known copies' CopyFactors and likelihood, which rank the peaks, are taken while the GPU runs
the translation function.
- The structure-factor engines are made per entry, without one lock held across all groups' creation.
- The transform's index wrap divides only for an index outside the grid (CPU and GPU).
All exact: same arrays, same kernels, same order. 4mtk MR, alternating 3 pairs: 9.5 -> 9.0 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The Files panel never showed files written after a directory was first
listed when they came from another host (a detector writer on NFS/GPFS/
Lustre), because QFileSystemModel only learns about changes through its
inotify watching, and there was no way to refresh it by hand.
F5 (window-wide, only while the panel is visible) and a refresh button
next to the root's browse button re-list the root and every expanded
directory. A timer does the same every 3 s while the panel is visible
and no mouse button is held, but re-lists only directories whose mtime
changed since the last listing, so a steady tick costs one stat per
expanded directory.
QFileSystemModel has no rescan call; index() of a path it does not know
adds that row in place, so only the names missing from the model are
passed (index() is ~10 us a call; a CBF sweep is thousands of files).
Selection and expansion are untouched, new rows are sorted into place,
and the first visible row is kept where it was when rows land above it.
Entries deleted by another host still stay listed; local deletes are
seen by the model's own watching as before.
Measured offscreen on a 5000-file directory: unchanged tick ~1 ms,
re-listing the changed 5000-file directory ~5 ms.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The mapping is now built while the frames are read, and with every core it took them from the
readers - on a 16M EIGER the read went from 0.10 to 0.19 s. Built on AvailableCpus() minus the
readers it is back at 0.11 s. The mapping is per pixel, so the same whatever thread count builds it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Drops the rule of af796d0a6 that the merge's own group searched to fewer copies always leaves the
choice UNDECIDED: it also caught a correct SPACE_GROUP decision (a target set; the half-vector move
check already catches the wrong one it was meant for). Instead, the chosen copies carried into a
tNCS-related group rule that group out only where the chosen placement fits the data itself
(R-free below 0.50): a perfect deposited model placed in the wrong group at R-free 0.520 had ruled
out the deposited group by 0.036; it is UNDECIDED now. The carries that decide in the survey all
start from R-free 0.36-0.47 and lead by 0.04-0.10.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Completes c6912e2a0 (whose message was a placeholder). What the two commits do:
- tNCS by half a lattice vector v: copies v apart weaken the reflections with h.v half-integral, and a
placement moved by (v + o)/2 (o an origin shift of the group) fits the strong ones exactly as well.
The candidate the choice falls on is compared with every such move of itself (rigid body refined,
packing) by R over the weakened reflections only; a move better by 0.01 is taken instead, and where
the two best are within 0.01 the merge is UNDECIDED.
- With tNCS, the group the merge was made in, searched to fewer copies than the chosen group, keeps
the merge UNDECIDED (not for its enantiomorph, which the hand check decides): its first copy alone
can sit at half the vector, and no placement of the chosen group, which may sit there itself, rules
it out.
- P1: the first copy sits at the origin, which no function of its position can single out (the
translation function is flat), so one copy stands out where its rotation-function Z reaches 8; the
null then judges it as before.
Checked against the deposited coordinates (placement comparison, symmetry/origin/setting matched):
two half-vector-shifted placements that had been decided (a SPACE_GROUP in the wrong setting, an
AGREES off by a/4) are UNDECIDED; two UNDECIDED tNCS sets are now AGREES with confirmed placements;
the nine confirmed tNCS placements and the ENANTIOMORPH stay; a P1 single copy with a perfect-match
prediction is placed (confirmed), a wrong PAE-domain placement in P1 (RF Z 6.4) is not.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
