The pre-scan's spot measurement (spot width, bandwidth, powder rings) built
all of its GPU engines on card 0 from unpinned threads. Worker t now runs on
card t % gpus with pin_gpu(), as the image-loop workers do, which also binds
it to that card's NUMA node where binding is enabled. Every tier runs on
fresh threads, so each tier re-pins worker t to the card its engine was built
on, and the engines are freed on threads pinned the same way (as the first
pass's ReleaseSpotEngines does).
Worker count, width tiers (8/15/30/60 on a 60-frame sample) and the frames
read are unchanged. A frame's spots do not depend on the card that found them
(same model), and the per-frame results are stitched in sample order as
before, so the output does not depend on the number of cards.
The projection and defective-pixel passes are left on card 0: both fold
every frame into a single device accumulator (ShadowAccumulatorGPU,
HotPixelFinderGPU) that lives there.
Verified on one card: CI-tier battery p.hkl md5 and MTZ reflection data
identical to the base binary on every set.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
RugnuxFarEnd.cpp used M_PI, which MSVC does not define without
_USE_MATH_DEFINES, and broke the Windows viewer build (C2065). The
ResultReport_SupercellCheck test still looked for the supercell warning's
wording from before the doubled-cell change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
TCPStreamPusher::SendImage is called from the receiver's frame-transformation
threads. It used a 128-deep per-connection queue and waited when it was full.
With images routed per file ((n / images_per_file) % writers), every writer
receives a whole file as a burst at the full frame rate (1000 images/file at
2 kHz fills 128 in ~64 ms), so the queue filled on every burst and all
transformation threads ended up waiting on one writer. They then stopped
returning FPGA frame buffers, and the FPGA dropped packets: partial images,
images without a background estimate, data loss on every writer. The ZeroMQ
pusher has an unbounded queue and never blocks, which is why it coped.
- The per-connection queue is unbounded: every element owns a distinct
image-buffer slot, so the bound is the image buffer's slot count. When slots
run out the receiver drops only the write (GetImageSlot() == nullptr), after
analysis. Absorbing one file burst therefore needs an image buffer of about
images_per_file slots (~20 GB for a 9M at 1000 images/file); below that,
images are still dropped, now as writer-queue-full instead of packet loss.
- SendImage never blocks. A dead or wedged writer is still caught in SendAll
(15 s silence / 60 s without progress), after which WriterThread returns the
queued slots unsent.
- The 200 ms fallback in StopDataCollectionThreads is gone: at END it closed
the connection of any writer whose queue stayed full for 200 ms - a normal
state for a slow but healthy writer - discarding up to 128 queued images and
never sending END. END now follows the whole backlog.
- A per-connection enqueue mutex makes "check active, enqueue" atomic against
"clear active, enqueue the end sentinel": an image could land behind the
sentinel when a connection was torn down mid-run, leaking its slot
(CheckIfBufferReturned then threw) or being sent as stale data in the next
run. New stress test SendImageRacingEnd_NoSlotLeak (verified to fail with
the race widened and the lock removed).
- TearDownConnection marks the connection broken first, so the destructor
returns a backlog instead of sending it.
Writer side:
- jfjoch_writer -b/--rcv_buffer sets the TCP receive socket buffer (was
hardcoded to the kernel default).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The model scale searched b_sol only over 10-80 A^2. That is the range
Fokine & Urzhumtsev report for models at ordinary B, but on low-resolution
entries whose deposited B is 200-400 A^2 the fit sat on the 80 A^2 ceiling
(and k_sol on its 0.1 floor): a solvent edge far sharper than the atoms it
lies against, reaching too deep in resolution. R-free came out 0.05-0.10
above REFMAC (0 cycles) and phenix.model_vs_data on the same MTZ, model
and flags.
The ceiling is now max(80, 2 x the model's mean atomic B) (SolventBMax),
for the reported fit and for the rigid-body placement's per-zone solvent
(CPU, GPU and the host fallback alike). The grid keeps eight b_sol values
across a wider box, so it costs no more points; the default box, and so
every model with mean B <= 40 A^2, is bit-identical to before.
Measured (10 open-arm sets with --model, GPU build, vs the same base):
high-B set, 7.0 A: R-free 0.462 -> 0.364, b_sol 80 -> 379 (REFMAC 0.366,
REFMAC's own b_sol ~371; model_vs_data 0.291);
the rigid-body step now commits
high-B set, 3.9 A: 0.410 -> 0.331, b_sol 80 -> 388, k_sol 0.10 -> 0.225
(REFMAC 0.328, model_vs_data 0.314); null 10.1 -> 14.2 sigma
B~200 set, 3.2 A: 0.408 -> 0.402, b_sol 80 -> 120; enantiomorph label
decision unchanged (27.2 -> 25.8 sigma)
7 other sets (mean B 16-57): bit-identical R, k_sol, b_sol, placement.
p.hkl md5 and the MTZ reflection data are identical to the base on all 10.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In a stress run (two rugnux on one 16 GB card) the indexer's cufftPlanMany failed three times with
CUFFT_INTERNAL_ERROR (5), not CUFFT_ALLOC_FAILED, so the message carried neither the free memory
nor the remedy. Every plan failure is now reported like a failed allocation: free memory on the
card, the two usual causes and the CPU remedy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the first pass is run again at the measured beam centre and the file's centre is kept, the run
restored the experiment, spot finding, mosaicity, geometry and prepass result, but not the rest of
what the arm's pass writes: its reflection footprint replaced bragg_adaptive_ (the integration
settings of the canonical pass), and last_rotation_result_, bkg_starved_fraction_,
bragg_before_adaptive_ and supercell_settled_ stayed the arm's. A losing arm therefore changed the
result, and an arm that completed gave a different answer from one that failed: on an open-arm
tetragonal set 66132699 against 66121594 integrated reflections and a different p.hkl.
These are now restored with the rest. Verified on that set with a forced arm failure (temporary
local patch, not committed): the arm completing and the arm failing now give the same p.hkl
(99faeb99), which is also what the arm failing gave before (stress runs #0/#1); the base binary
gave 94c3afc9 when the arm completed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A pass that finds no lattice carries score -1. That sentinel was printed as "-1/60 validation
frames" and compared as a count: the axis-sign rescue adopted the opposite sign when it scored 0
frames against the file's "no lattice" (0 > -1), and the long-axis rescue could return a re-index
that indexed nothing over a standing pass that found nothing.
Both rescues now count a missing lattice as 0 frames, so an alternative needs at least one indexed
frame to win. The log says "no lattice" (Frames()) in the axis-sign, beam-centre, long-axis and
rung lines and in the near-tie lines, whose margins also count it as 0 (log only). On the base
binary's full open+inhouse battery no set adopted a sign or a long-axis re-index against a
lattice-less pass, so no result moves.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A CUDA out-of-memory in an indexer worker was turned into an IndexerResult carrying only the
message string. The rotation indexer then returned no lattice, the first pass treated that as a
genuine negative and the rescues acted on it: in a stress run (two rugnux on one card) a run had
both first-pass schemes fail with out-of-memory, the axis-sign rescue compared 0 frames at the
flipped sign with the -1 "no lattice" sentinel, adopted the flipped sign, and the beam-centre rescue
then moved the centre - a failure of the machine used as evidence about the crystal. The CUDA
category was lost on the way, so the fail-fast check (IsFatalResourceError) never saw it.
- IndexerResult::error is now the std::exception_ptr the worker caught. RotationIndexer::RunIndexing
rethrows it before touching any state, so nothing is memoised for a failed run and the exception
keeps its category: rugnux stops with the CUDA message at once (verified with a forced indexer
failure: exit 1 after 1.9 s, "CUDA (GPU) error (...)"). FirstPass::indexer_error,
RotationIndexer::GetIndexerError and RequireLattice's machine-failure branch are gone.
- The beam-centre arm in RunAllPasses rethrows fatal resource errors like its sibling catches; it
had caught the out-of-memory and carried on at the file's centre.
- IndexAndRefine keeps per-frame indexer failures apart from frames that did not index (count +
first exception). The frame stays "not attempted" as before; the receiver logs the count and the
first error at the end of a collection (no cancel), rugnux's image pass and stills geometry
refinement rethrow it and stop. A rotation-indexer failure inside IndexAndRefine (online rotation
indexing, FinalizeRotationIndexing) is recorded the same way, so a live collection is not
cancelled by it.
- A failed device allocation or cuFFT plan now says how much was asked, how much of the card is
free, and - when the card is merely full - that another program is using the GPU or the data set
needs more than the card has, with the CPU remedy; previously "Failed to allocate device memory
(out of memory)" / "Failed to create CUFFT plan with cufftPlanMany".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Narrow, guarded: the doubled pass runs only where the added class grows with resolution and is resolved; the doubled
cell is adopted only where one pseudo-translation explains the added class in every shell and the point group does not
rise. Fixes the one remaining lattice failure (half cell -> deposited cell, R-free 0.574 -> 0.275); p.hkl identical on
the ci tier, the supercell gate set, 78 further supercell-candidate sets and all private sets.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The doubled pass now runs only on resolved, rising classes (two sets of ~134 with probe logs), so the
twenty-minute model placement in a doubled cubic cell can no longer arise, and the adopted answer is
validated again (9min: R-free 0.275 against the deposited model, 0.574 in the half cell).
Final rules, measured on gate + ci tier + all sets with a de-modulated axial row + 49 sets with a
baseline supercell occupancy >= 8% + the private arm: 9min fail -> pass (P212121, doubled cell);
every other set run keeps its verdict and a bit-identical p.hkl.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A shell is read only where at least half of the class's reflections are clear of the lattice's by r3.
On a 701 A axis 26-43 of ~36000 per shell were clear; they read at the lattice's own strength, the
doubled merge (93% of predictions overlapping a neighbour) passed the one-translation test, and the
run was adopted doubled and failed. With the rule, of ~134 sets with probe logs only 9min and one
I-centred crystal reach the doubled pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A superstructure cannot be more symmetric than its own average, which the half cell's merge is. On a
private-arm crystal whose added class passed the rise and the one-translation tests, the doubled merge
came out centrosymmetric over a chiral half-cell group (point group order 8 against 4) and the run
failed against its reference.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A centred half cell (I-centred orthorhombic, its centring class the added one) was compared to the
doubled cell's sub-lattices as if primitive, so its class was never read.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
On strong data a class at one or two percent of the lattice rises significantly (0.009 to 0.059 at 4
standard errors on a ci-tier triclinic crystal, which was then adopted doubled and failed), and a half
cell missing a few percent of its intensity still describes the crystal. The doubled deposition ends
at 0.35-0.40, those classes at 0.06.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Exact: p.hkl md5 and MTZ data identical on every real set (34 open/inhouse incl. the ci tier, all 27 private);
both no-crystal controls now ~7 decades inside the bar instead of 0.3-0.9.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
nothing_1 and nothing_2 now pass the no-crystal test by about seven decades (tails 10^-2.06 and
10^-2.70 against 1e-9) instead of 0.3-0.9, so a change that moves one stray reflection no longer
flips them, and a regression that lets a chance lattice through fails CI. The ci tier is 28 sets.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
A class that does not rise keeps the half cell whatever the doubled merge shows, so the doubled pass
is no longer run for it - on a ci-tier crystal (R-centred, class at 1-2% of the lattice) that pass ran
the card out of memory and failed the run. The doubled pass also runs without --model: placing the
model in a doubled cubic crystal's 776 A R-centred cell took twenty minutes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Exact: p.hkl md5 and MTZ reflection data identical on 18 sets. 4nwv R-free 0.545 -> 0.297 and cell in the depositor's
frame; written cells corrected on 9rcs, 9b22, 9qw8, myob_x06da_powder_1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
- ModelValidation reads the model and, where it carries non-given NCS operators
(_struct_ncs_oper 'generate' / MTRIX without the given flag), expands them
(gemmi expand_ncs, vendored assembly.cpp from GEMMI 0.7.5, MPL-2.0 as the rest
of gemmi_gph). Same in the battery's model_check.py and depdata_check.py.
An icosahedral-virus entry deposited as 3 of the 45 chains of its ASU was
scored against a fifteenth of its atoms: R-free 0.545 -> 0.297.
- The indexing the model chooses (AdoptModelFrame) now moves the cell with the
indices (reseat_integrated in the main path; CellInBasis in the forecast,
twin-file and stills/scale paths). A near-symmetry law (b within 2% of c, beta
a degree off 90, a~b in P1) relabelled the reflections only, so the files were
written with two axes swapped or beta flipped relative to the indices. Where
the law moved the cell, the validation is remade on the moved cell.
Measured on 18 open/in-house sets with models: p.hkl and MTZ reflection data
bit-identical on all; cell and model R unchanged where the law is exact; the
cell now matches the deposited one on the 6 pseudo-merohedral cases (orthorhombic
a/c swap, three monoclinic beta flips, a P1 alpha/beta swap), R-free within
+-0.001 there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Most likely cause of the intermittent 'unspecified launch failure' (Xid 31, copy-engine write to an unmapped
address) seen once on 9ig7; not reproduced. Exact: p.hkl md5 and MTZ data identical on 10 sets.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
The first-pass spot engines (FirstPassRun::SpotEngine, SpotFindingWorkers::Engine)
were given back by move-assigning a fresh struct over them. A move-assignment goes
in declaration order, so it freed raw_image - whose bytes the analysis engine had
page-locked with cudaHostRegister (PinInputRegion, every uncompressed/CBF frame) -
and only then destroyed the engine, which unregistered the already freed range.
In between, the address may be handed to another thread's buffer. Measured in a
standalone test on this card and driver: a copy into a new buffer at a stale
registered address goes to the old pages (the data never arrives, no error), and
cudaHostRegister / cudaMallocHost on it fail with "already mapped"; eight threads
freeing in this order made unrelated threads' ordinary pageable copies return
wrong data, the correct order none. The engines now release themselves engine
first, buffer last.
The image pass had the same hazard where the writer still holds an image: the
worker read the next frame into a new one and dropped the old one, which the
writer then freed while the worker's engine still had it registered. The old
image is now held until the engine has pinned the new one.
This is the most likely cause of a sticky copy-engine fault (Xid 31, CE write to
an unmapped address) that ended a rotation run's scaling with "unspecified
launch failure": the speculative geometry probe released its spot engines on
many threads at once while the merges beside it were copying. That failure was
not reproduced in 9 reruns before the fix, so the link is inferred, not shown.
Merged data unchanged (p.hkl md5 and MTZ data identical to rc175 on 10 sets;
p.hkl identical with --write-process-h5 on a CBF set).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Doubled-cell hypothesis: the run now always merges the doubled cell under <prefix>_doubled first and
decides with both merges in hand. Adopted only where (1) the probe's class grows significantly with
resolution (three standard errors between the 20-8 A shell and the finest measured one) and (2) in
the doubled merge one pseudo-translation - half a cell plus eps, fitted over the class - modulates at
least half of the added class in every shell (FitDoubledClassTranslation: A sin^2(pi h.eps) + B per
shell, share 1 - B/mean). Where adopted, every <prefix>_doubled file takes the plain name and the
probed cell's file of that name moves to <prefix>_halfcell. Replaces the HALF_CELL_BREAKS_DOWN bound.
Measured on the forced doubled merges: modulated share per shell 0.80-1.0 out to 1.9 A on the crystal
deposited in the doubled cell; on the ones deposited in the half cell it falls to 0.01-0.34 by
2.1-2.8 A (the rising class with twinning), or stays at -0.3-0.48 at every resolution (three others);
the one class it fully explains does not rise (eps 0.1 of a cell, flat at 0.1 of the lattice).
Space-group search: the trim of a screw zone's largest absence exempted a violation "at its row's
mean" on the de-modulated intensity, so under a deep pseudo-translation a reflection at 0.015 of its
row (de-modulated by 1/0.002) could never be trimmed and alone decided the row against the screw.
The exemption now reads the measured intensity, at both sites.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EizBKhTqrAYago9KJG3dA3
Exact: p.hkl, MTZ and map data, p_model.pdb identical at -N 1/default/-N 8 on 9 sets (CPU build and GPU build on
the CPU path); GPU path unchanged. Indicative: 8xtg CPU validation 32-41 -> 23-25 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where the first pass recognises two twin domains on their coincidence lattice (reticular
merohedry, index n = 2 or 3) and adopts the stronger domain's cell, the 1/n of that domain's
reflections that lie on the other domain's lattice too carry both domains' intensity. The step
now records the overlap as a lattice vector t of the standing cell (h . t divisible by n), the
rotation result carries it, and the per-image integration drops those reflections. t is a
real-space vector, so it holds through every later change of basis; it is read in sixths so a
centred conventional cell (F, I, C, R) is exact. Each unique reflection keeps its equivalents
that the other domain does not overlap; only those whose every equivalent is overlapped are lost.
The two-pass quality guard compared signal counts between a header pass that integrated
everything and a refined pass that left the overlap out, and sent the run back to the
composite lattice. A pass that left an overlap of index n out now has its counts taken up by
n/(n-1), and its axial rows are not compared with a pass that did not.
Nothing changes where the step does not fire: the field stays empty, the factor is 1.
Measured (merging the existing outputs with and without the class H.(111) = 0 mod 3, against
removing a random third): spinel-twinned F-cubic private set R_meas 25.5 -> 14.3% (random
26.7%), insu_I_x06da_low_isa 24.1 -> 16.6% (random 24.2%). With this change, end to end:
- insu_I_x06da_low_isa: I23, R_meas 24.0 -> 12.7%, ISa 5.2 -> 28.8, CC1/2 0.997 -> 1.000,
completeness 99.8 -> 95.7%, d 1.45 -> 1.47 A, model R-free 0.260 -> 0.246.
- private F-cubic twin: F432, R_meas 25.5 -> 6.3%, ISa 3.9 -> 20.4, completeness 100 -> 93%,
d 3.23 -> 3.77 A (XDS reference 4.0). 432 now passes the added-operator gate on the
all-observation merge (contrast 0.75, bound 0.72) instead of being refused there (0.62) and
rescued by the twin-immune zone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
LevMarFit takes compute_wssr() in a copy (ComputeWSSR) that keeps each
reflection's exp(-0.25 * b_star.r_u_r(h)); when the step is accepted,
ComputeLMMatrices is evaluated at exactly those parameters and takes the same
values from there instead of calling exp again. Same expressions, same
parameters, same numbers.
Bench against the base FitModelScale (single thread, 48k reflections, P2_1):
1.65 -> 1.10 s for this and the previous commit together; P1, P2_12_12_1,
P6_122, I23 and the dependent n<=5 path identical to the bit and 30-40% faster.
Validation outputs (p.hkl md5, p.mtz / p_maps.mtz data, maps, p_model.pdb, log)
identical to the base: CPU build --mode scale on 5reo, myob_x10sa, 8sa8, 8xtg,
5ebi at -N 1, default and -N 8; full CPU runs of 9ea5, 6iu5, 7dkp; the GPU build
with no visible device on the five --mode scale sets.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
A pooled CudaDevicePtr frees on the thread's allocation stream, not on the engine's stream, and the
pool may hand the memory to another engine - or, past its release threshold, unmap it - as soon as
that free is reached, which on an idle allocation stream is at once. An engine destroyed with work
still queued (FFTIndexerGPU after SearchCap's last DirectionsChanged upload, a spot finder between
DetectAt and Extract, a shadow accumulator after a pending fold, any engine on an exception path)
thus had kernels or copies writing memory that was someone else's or no longer mapped. Now:
- CudaStream synchronises before cudaStreamDestroy (destructor and move-assignment), which covers
engines whose own stream is declared after their buffers (FFTIndexerGPU, the gather buffer);
- every engine holding pooled buffers and a stream (shared or own, declared before the buffers)
synchronises it in its destructor; BraggIntegrationEngineGPU also before EnsureCapacity
reallocates, where a Run that threw leaves work queued.
A GPU failure that is handled no longer hides a lost context: ShadowFinder, BeamCenterFFT, the
rigid-body pool and model validation call cuda_throw_if_context_lost() before cuda_clear_error(),
as the device-decode fallbacks already did; RotationScaleMergeGPU's Alloc does so before waiting up
to ten minutes for GPU work beside it and then reporting a lost device as out of memory; and a
failed cudaMalloc says why. BeamCenterFFT logs the failure it used to drop silently, the
speculative geometry probe logs the exception it swallowed (its GPU fault was otherwise reported by
the merge beside it, under the merge's name), and RotationScaleMergeGPU's DeviceGuard no longer
throws from its destructor.
Only synchronisation and error paths change: p.hkl md5 and the MTZ data (gemmi) are identical to
the b530c2d full battery on myob_x10sa, cytc_x10sa, 8a1a, 9gdj, 11if, kdp_x10sa_20keV and 6z9g.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Supercell probe, log-only part: the class it reads over 20-3 A is followed out to the run's own
limit (a second prediction of the same supercell on the same 60 frames, so the 20-3 A band and the
SUPERCELL_* keys are unchanged). Half-integer reflections are counted only where no lattice
reflection predicted on the same frame lies within r3, and each resolution shell is read against the
median of the six other parity classes - which a single doubling leaves empty - as an excess in units
of the lattice's intensity and as a z in the class's own standard errors, the strongest sibling's z
beside it ("Supercell probe to the limit" in the log).
Doubled-cell hypothesis: where that class stands clear of the six empty ones (measured above its
noise and at more than twice the strongest of them, in at least three shells), the run integrates and
merges the doubled cell too, forcing the last pass's indexing result with its lattice doubled along
the class (orientation, geometry and axis kept). One of the two is written as the answer, the other's
merged .mtz/.cif/.hkl beside it as <prefix>_doubled or <prefix>_halfcell; the report names both
(DOUBLED_CELL_*). Which one is the answer is a policy behind one constant: ADDED_CLASS_SIGNIFICANT
(the doubled cell wherever the class stands clear) or HALF_CELL_BREAKS_DOWN (default: only where the
class reaches a quarter of the lattice's intensity in the finest shell both are measured in).
The doubled lattice is classed with a near-zero tolerance: the default lattice search promoted a
triclinic doubled cell with an 88.2 deg angle to orthorhombic C, and no frame indexed.
Measured on forced doubled-cell merges (rc175 b530c2d base): the open-arm crystal deposited in the
doubled cell merges at R-free 0.275 against its deposited model, against 0.574 in the half cell
(odd/even 0.02 at >8 A rising to 0.65 at 1.95 A, odd half-set CC1/2 0.93 at 8-5 A). Five crystals
deposited in the half cell read classes the doubled merge holds as well (odd CC1/2 0.73-0.98 at
8-4 A): two rise with resolution like the first (to 0.4-0.5 at 2.5 A), one is strong from low
resolution (0.3-0.5), two stay flat at 0.1-0.2. Nothing measured on intensities - the trend, the
neighbour correlation, the separation from the nearest lattice reflection after the r3 filter -
separated the doubled deposition from the rising half-cell ones, so the policy is the owner's call.
Also: the SUPERCELL_POSSIBLE prose no longer says correct cells "read the same", and the warning no
longer advises -C with the doubled cell, which the indexer reduces back to the half cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
compute_model_factors' CPU path (no GPU, or the grid does not fit the card)
gridded the density with gemmi's serial put_model_density_on_grid and the mask
with SolventMasker::put_mask_on_grid. It now uses ModelGrid's
PutModelDensityOnGrid / PutMaskOnGrid with -N threads, sharing one set of orbit
leaders (the two grids are the same size) - gemmi's grids bit for bit at any
thread count, as the rigid body already uses them. The GPU path is untouched.
Exactness: p.hkl md5, p.mtz/p_maps.mtz reflection data, the three maps and
p_model.pdb identical to the base and the validation log lines identical (up to
the order of concurrent replicate lines) - CPU build, --mode scale --model on 5reo,
myob_x10sa, 8sa8, 8xtg, 5ebi at -N 1, default and -N 8; full CPU runs of 9ea5,
7dkp, 6iu5, 8tyy; the GPU build with no visible device on the five --mode scale
sets.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
All exact to the bit (bench against the previous FitModelScale on P1, P2_1,
P2_12_12_1, P4_32_12, P6_122, P2_13, I23 and the dependent n<=5 path; the
ModelScaling cases; validation outputs identical on real data):
- the start of a grid point (fit_isotropic_b_approximately) is computed from the
same |Fcalc| the fit takes, once instead of twice, and without copying the
Scaling's points per chunk;
- gemmi's Levenberg-Marquardt is followed in a copy (LevMarFit) whose
compute_lm_matrices is templated on the parameter count (2..7), so alpha/beta
stay in registers; a zero derivative row adds +0 instead of being skipped
(alpha/beta are never -0), and the full square is summed (the lower half is
gemmi's, the upper is its mirror) - branch-free and vectorised;
- grid points are scheduled one per task (ParallelFor) instead of fixed chunks;
- a fine-pass pair bit-identical to a coarse-pass pair reuses that fit.
Bench (single thread, 48k reflections, P2_1): 1.58 -> 1.33 s; ~15-25% on the
other groups.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
UseAtomBoxesByPlane handed its functor one grid point at a time, so the density
(gemmi's ExpSum/ExpAnisoSum with unsafe_expapprox) ran scalar - on the CPU path of
model validation the rigid body's density copies were 48% of all validation
CPU time (8sa8, perf). The walk over an atom's box on a plane is unchanged (same
order, same delta.x stepping) but now only collects the points within the radius
(index, d2, delta); the density functor then evaluates them a Gaussian at a time
over all points - each point still sums its Gaussians in gemmi's order from 0 -
in loops GCC vectorises (AVX2, no intrinsics), and adds them to the grid in walk
order. The mask functor zeroes the gathered points. Same grid, bit for bit, at
any thread count (ModelValidation_ParallelGriddingMatchesGemmi: iso + aniso,
five groups, 1 and 4 threads).
Measured (CPU build, --mode scale --model, shared machine, indicative): 8xtg
validation 28.1 -> 13.6 s wall, run total 665 -> 366 user-s (with the FitModelScale
commit); outputs identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
subprocess.run's timeout killed only the direct child. Under --gpulock that is flock, under
`stress --sanitizer` compute-sanitizer, and in both cases rugnux went on running on the card,
orphaned, beside whatever took the GPU lock next - found when a cut-short memcheck stress run left
its rugnux behind. Each run now gets a process group of its own, killed whole on a timeout, Ctrl-C
or SIGTERM.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
The line put the distinct-reflection count against a normal-approximation bar, which at a
handful of reflections reads the wrong way: a control the exact test rejected (5 vs 2 by
chance, tail 10^-2.70 against 10^-3) was printed 0.59 above the bar. It now prints the exact
binomial tail and the significance, both as log10, with the margin in decades and the noise as
the tail's change over one binomial standard deviation of the count. The tail is summed in logs
(ValidationEvidenceLog10Tail), so it stays finite for a real lattice; the decisions compare the
same tail in log10.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Reframe the Global Phasing / autoPROC acknowledgement as inspiration and
name the ideas: header metadata (beam centre, geometry, axis) must not be
trusted blindly (autoPROC, Clemens Vonrhein in particular); the beam
centre is determined from the images (rugnux does it its own way); and
diffraction anisotropy must be analysed and reported (STARANISO, cited
with the reference the STARANISO server recommends; no DOI exists).
One-line pointers in data-analysis §1.4 and §13.5, and both works added
to the CPU_DATA_ANALYSIS references. The Sheriff & Hendrickson, Popov &
Bourenkov and Evans & Murshudov credits for the estimators are kept.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
The de-novo FFT indexers implement the 1D-projection method of Steller,
Bolotovsky & Rossmann (1997), which was already credited; the 3D Fourier
transform indexing idea it descends from (Bricogne 1986, EEC PSD software
workshop, Phase III, LURE, p. 28 - no DOI) was not. Add it to
ACKNOWLEDGEMENT.md, the CPU_DATA_ANALYSIS references, the §5 text and the
in-source credit, and add the DOI of Powell (1999) to the MOSFLM paragraph.
3D-FFT papers (Campbell 1998, Gildea et al. 2014) are not cited: the code
does not compute a 3D transform.
Add a section acknowledging Global Phasing (Bricogne and colleagues) and
autoPROC (Vonrhein et al. 2011) as the expert-system model rugnux aims at.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
The final no-crystal test (RejectChanceLattice) used the 0.001 binomial level of the comparisons
between candidate lattices. The lattice it tests is the winner of some fifty indexing attempts,
each fitted to an accumulated cloud that already holds the long-lived reflections it then re-finds
on the validation frames, so that null is optimistic by roughly the number of attempts. The two
in-house no-crystal controls passed it by one or two distinct reflections (tails 10^-2.06 and
10^-2.70 against 10^-3).
The test now asks 1e-9 (Z ~ 6); the rescue comparisons keep 0.001. With two reflections by chance
the controls would now need 15 distinct reflections instead of 4-5. Real lattices are tens of
decades past either level (weakest seen: 10^-83).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
A planned fallback that rugnux logs with a CUDA error text - the rigid body's zone over its
engine's reserve, revalidated on the CPU, or an indexer worker out of device memory while other
jobs held the card - would have turned sets into crashes in every run. A handled error now counts
only when it names a device fault (unspecified launch failure, illegal memory access/address,
misaligned address, launch timeout, device-side assert, "CUDA device unusable"), which takes the
context down for every thread and is never a property of the data. "Processing failed" still
counts whatever its cause.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
A run that was killed by a signal, or whose log has "Processing failed", a GPU error (also one
rugnux handled and went on from), "terminate called" or a non-zero compute-sanitizer ERROR SUMMARY
is now verdict `crash` (score.crash_note), decided before anything else, so it fails even a
no-crystal control whose normal refusal exits 1. A crash counts as a failure in every pass rate,
gets a CRASHED section at the top of the report and a CRASHED line under the console table, and
makes `run` exit 1 whatever tier the set is in. Reports and compares re-read crashes from the log,
so an older run's crash shows as one: the GPU crash of 9ig7 in the last full run showed only as
"fail, exit 1: Step 8 FAILED" before - failure_note now prefers the [error] line it follows.
`battery.py stress` runs named sets --repeat times, --concurrent at once, optionally under
compute-sanitizer (memcheck/racecheck/synccheck/initcheck) or with CUDA_LAUNCH_BLOCKING=1, and
exits 1 on any crash; it reports distinct p.hkl per set and keeps out of the run list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Exact: reflection data identical to the base on 51/51 open+inhouse and 27/27 private sets. Diagnostics only;
the full battery collects their firing counts.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
# Conflicts:
# rugnux/RugnuxFirstPass.cpp
# rugnux/RugnuxFirstPass.h
Exact: MTZ data (gemmi) and p.hkl md5 identical to the base on 31 sets, -N 8 on 3; only the PASS= count in
p_report.txt drops where a pass is no longer run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
# Conflicts:
# rugnux/RugnuxScaleMerge.cpp
A crystal whose spots avoid one coset of an index-2 sub-lattice thins the overlap and the class on
neither alike, so comparing only the weaker domain's own class with the class on neither nearly
fired on a single long-axis hexagonal crystal (stopped by the congruence check alone). The class on
neither must now be four times emptier per node than the emptiest of the weaker domain's own class
and the overlap. On the evaluated private, inhouse and open sets the non-twins score at most 1.44;
the two known index-3 twins score 61 and more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Where pass 1's rotation-scale fit is a step or more (STEP_BOUND, 1 %) off the
stored angles, the walk runs on its own before the refined passes instead of
beside an output pass that a refit calls off and throws away. Scheduling only:
the run ends where the sequential walk leaves it either way (the overturned
path already re-ran the refined passes on the run itself).
Exactness E: all output files identical to rc175 e9d4f2221 on 9gdj, 9qw8,
6ttn, lalanine, 9t6s, 2 runs each.
Note: "the walk will then always refit" does not hold. Of the four battery
sets with the fit >= 1 % off (9gdj 0.980, 9qw8 0.988, 9t6s 1.012, lalanine
1.012), the stored angles stand on 9t6s and lalanine, which now walk first and
lose the overlap of the walk's two probes with the output pass (~0.5-0.8 s of
probe time); 6ttn (0.998) is below the step and unchanged.
Wall (time -v, base -> new, rep1 / rep2, shared machine):
9gdj 99.6->109.7 / 107.4->101.4 (rep1 new run noisy)
9qw8 43.1->39.2 / 34.4->33.3
9t6s 18.1->18.2 / 18.9->18.5
lalanine 20.5->19.7 / 7.9->8.3
Pre-rebase run on rc175 0b2dde214: 9gdj 99.6->93.8 and 99.6->93.4.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
Every round the geometry walk takes ends in a pass that writes the files
again, so the canonical pass before it spent its space-group search, merges,
analyses, model validation and files on an answer nobody reads. The walk's
first question needs only that pass's post-refinement and, for a move of less
than a step, two indexing probes. The speculative probe now runs both beside
the merge - at the fit's geometry as before, then at the geometry in hand,
which the walk otherwise indexed after the pass (a memo hit where the pass's
own first pass left one) - and GeometryWalkTakesRound reads the walk's answer
from the same first-pass memos the walk's probes then read. The canonical pass
asks it, without waiting, before its space-group search (stops there; not
with a _process.h5 open) and again at the superseded point before the
analyses. An answer not yet in leaves the pass to finish as before, so only
timing depends on when the probes finish; the walk itself is unchanged.
Exactness E (not F): merged data and all output files identical to rc175
e9d4f2221 on every walking set, 2 runs each; decisions identical. Fired:
before the search on 7ph1, 5m17, 8pqd; at the superseded point on 2xfw, 6zr0,
9t6s; on 6zqy the probes were not in by either point in these runs; 8xtf's
canonical pass is starved and never merges.
Wall (time -v, base -> new, rep1 / rep2, shared machine):
5m17 65.9->57.0 / 68.5->56.7
8pqd 49.8->43.0 / 50.0->43.1
7ph1 51.1->48.9 / 51.1->45.3
6zr0 31.1->22.7 / 24.4->21.1
2xfw 101.2->66.7 / 48.2->48.2 (rep1 base noisy)
6zqy 27.3->26.1 / 26.8->25.7
controls myob_x10sa, cytc_x10sa, 11if, kdp_x10sa_20keV, 8a1a: no change
beyond machine noise
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
