v1.0.0-rc.166 #76

Open

leonarski_f wants to merge 170 commits from rc166 into main

pull from: rc166

merge into: :main

:main

:rc166

:gitea-pages

:rc165

:rc164

:rc163-unshare-doc-files

:2608-rc-162

:2608-performance

:adaptive-spot-finding

:stills-partiality-postrefine

:2607-res-prediction

:2607-rc-160

:french-wilson-model-maps

:2607-win-viewer

:2607-state-machine

:2606-rc.156-fixes

:viewer-windows

:2606-viewer-processing

:2606-tcp

:2606-pixel-refine

:2606-eiger-calib

:2606-q-spacing

:2606-eiger-module-fix

:2606-force-cpu-azint

:2606-xds-plugin

:2606-azint

:2506-rot-all

:2606-sgsearch

:2605-scaling

:2605-hdf5-vds

:2605-hdf5-enospc

:2605-hdf-fixes

:2605-b-factor

:2605-speed-up-preview-start

:2604-index-and-refine-fix

:2604-cuda-cleanup

:2604-azint-mapping-parallel

:2604-more-fixes

:2604-start-speed

:2604-async-start

:2605-sigma-correction

:2604-hdf5-errors

:2604-rc.136

:2604-rc.135

:2604-processing-options

:2603-single-file

:2604-detector-parallel-logic

:2603-pipeline-upgrades

:2603-multilattice

:2603-rc.131-2

:2603-rc.131

:2603-rc.130

:2603-rc.129

:2603-ep

:2602-scaling-d-limit

:2602-residual

:2601-completness

:2601-1.0.0-rc.124

:2601-1.0.0-rc.123

:2512-scaling-merging

:2512-1.0.0-rc.122

:2512-1.0.0-rc.121

:2511-1.0.0-rc.120

:2511-1.0.0-rc.119b

:2511-1.0.0-rc.119

:2511-1.0.0-rc.118

:2511-1.0.0-rc.117

:2511-1.0.0-rc.116

:2511-1.0.0-rc.115c

:2511-1.0.0-rc.115

:2511-1.0.0-rc.114b

:2511-1.0.0-rc.114

:2511-1.0.0-rc.113

:2511-1.0.0-rc.112

:2511-1.0.0-rc.111

:2511-1.0.0-rc.110

:2511-1.0.0-rc.109

:2511-1.0.0-rc.108

:2511-1.0.0-rc.107

:2511-1.0.0-rc.106

:2511-1.0.0-rc.105

:2511-1.0.0-rc.104

:2511-rc.103

:2511-1.0.0-rc.102

:2511-viewer-enh-2

:2511-viewer-enh

:2511-eiger-mask-3

:2511-eiger-mask

:2510-viewer-improvements

:2510-viewer-3D

:2510-fpga-clamp

:2510-rc.88

:2510-release

:2509-rc.82

:2509-gitea

Conversation 0 Commits 170 Files Changed 377

+20321 -3843

leonarski_f commented 2026-09-01 07:46:38 +02:00

Owner

Copy Link

Copy Source

• rugnux --mode calibration writes <prefix>.json beside the .poni, whose dataset_settings member is a jfjoch_broker dataset_settings body as it stands.
• rugnux and jfjoch_viewer read PILATUS miniCBF sweeps natively, without conversion.
• Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants.
• rugnux measures the beam centre on every run, and indexes with it when the file's value indexes nothing.
• A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed.
• rugnux writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing.
• Significant improvements to symmetry handling in rugnux: the lattice, the point group, the setting and the systematic absences.
• The rugnux report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to.
• The rugnux report gives the twinning statistics measured before the space group was decided, beside the ones measured after.
• The rugnux report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution.
• rugnux ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have.
• Twinning is no longer reported when the L-test contradicts it.
• The rugnux report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's.
• --no-refine-tilt holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots.
• The jfjoch_viewer grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area.

* `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion. * Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants. * `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing. * A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed. * `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing. * Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences. * The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to. * The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after. * The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution. * `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have. * Twinning is no longer reported when the L-test contradicts it. * The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's. * `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots. * The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area.

leonarski_f added 97 commits 2026-09-01 07:46:39 +02:00

Test: a failed calibration now leaves the machine in Error, not Inactive ... 8999287ba4

JFJochStateMachine_CalibrationFailure was written in rc.164 against the
behaviour of the time. rc.165 moved every calibration FAILURE path -
CalibrateDetector's catch, TakeDarkMaskInternal's mask-size check and the
three ImportPedestal checks - from Inactive to Error, so that /wait_till_done
and /wait_until_running answer with the reason instead of the bodiless 502
that a deliberate Deactivate() also leaves; only a CANCELLED calibration still
ends in Inactive. The test kept asserting the old state and started failing.

Assert Error. The rest of the case is unaffected: the message and severity are
what the new SetState writes, and Start() still throws from Error because it
requires Idle, so "a data collection is refused on it" keeps its meaning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Viewer: Alt and the wheel step through the images ... a6fbd1d195

The wheel already zooms, and with Ctrl or Shift it moves the foreground; Alt
now moves through the dataset, one image per notch, wheel up forward - the
direction QAbstractSlider's own wheel handling uses, which is what the
toolbar's scrub slider follows.

The view does not know which image it is showing, so it emits the step and the
navigation toolbar applies it: the same loadImage() every other control ends
in, with the same clamp and the same Sum setting, so nothing about loading is
duplicated. An empty dataset is left alone, because loadImage(-1) does not mean
"before the first image" but "the latest one" when the viewer is following a
running collection over HTTP.

The signal is on JFJochImage, so the other views emit it too; only the
diffraction view is connected, which leaves them as they were.

Note for a desktop where Alt+wheel does nothing: many window managers grab
Alt-modified mouse events before the application sees them, and that is a
window-manager setting, not something the viewer can take back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Spot finding: the strong-pixel limit follows the detector ... ac202a55a1

An image with 65535 or more strong pixels was given up on and reported ZERO
spots - silently, no log line, indistinguishable from a frame that did not
diffract. 65535 is one pixel in 64 of the JUNGFRAU 4M the number was written
for; left fixed while the detectors grew it became one in 276 of an
18-megapixel EIGER, which a strongly diffracting crystal passes on its best
frames. On the strong rotation set just added to the battery it cost 767 of
1800 images: peakCountUnfiltered 0 and resolutionEstimate NaN across two
blocks of the sweep, the two where the crystal diffracts hardest. Make the bar
one pixel in 64 everywhere, and never below the value that stood here, so no
smaller detector loses ground. It lived in three places - the host extractor,
StrongPixelSet, and SpotExtractorGPU's buffer capacity - now one function.

The bar was there for a reason and raising it alone would not have been safe.
sparseccl walks a sliding window of the last two lines and tests every pixel in
it, which is quadratic in how many strong pixels a line pair holds: a handful
for the silicon-tracker hits upstream wrote it for, four thousand for a lit
detector line, and 76 seconds for a fully lit frame. But the pixels arrive in
raster order, so the window need not be walked at all - a pixel's earlier
8-neighbours are the one to its left and the at most three above it, which is
what the GPU extractor already finds by binary search. Keeping the previous
line's range and a forward-only cursor gives the same edge set and the same
unions in the same order, so the labels are identical, and the fully lit frame
now takes 0.16 s. Verified bit-identical on real frames, on fully dense frames,
across occupancy 1e-5 to 5e-2, and on 4000 randomised images including ones
with blank lines; SpotExtractorGPU's host-vs-device parity test passes
untouched.

ImagePreprocessorBufferGPU's gather staging was sized to the old constant, with
a comment tying it to the caller's give-up. Raising that give-up without it
would have run the gather off the end of the device buffer, so it follows the
same limit now.

Byte-identical .hkl on three battery crystals that never reach the bar. On the
strong set, with symmetry, cell and geometry pinned so only the spot list
moves: <I/sigma> better in every resolution shell, CC1/2 97.8 -> 98.5%,
R_meas 30.5 -> 28.4%, ISa 3.36 -> 3.58, indexing rate 0.772 -> 0.824.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Resolution estimate: predict past the edge of the detector ... 1388b16d7a

The spot-finding resolution estimate was clamped so it could never beat the
detector corner. On a crystal that diffracts past the corner that reports where
the DETECTOR stops, which is the one thing this number is not for - it is meant
to say how far a merge of data like these would reach, a property of the
crystal and the exposure. The clamp also hid the interesting case: an estimate
finer than what the run actually merged is the statement "this run was
detector-limited", and there was no way to make it.

The statistic already extrapolates. Its quantile sits in the middle of the
fall-off, well inside what the detector records, so it goes on measuring the
crystal's own decay when the detector cuts that decay short. Measured by
truncating the spot lists of 31 battery crystals at an artificial detector edge
and scoring the unclamped answer against each crystal's own measured CC1/2 =
0.30 crossing, it holds its 8-9% floor out to about 1.7x past the cut and only
then drifts pessimistic, which is the safe direction. Every genuinely
detector-limited crystal in the battery needs between 1.10x and 1.63x.

Against a truth corrected for censoring - the six crystals whose merge is cut
off by their own detector cannot have a measured crossing, so theirs is
extrapolated from multiplicity-corrected <I/sigma> and anchored on the 25 where
both exist: symmetric-log RMS 13.5 -> 9.4% over 37 crystals, 25 -> 28 within
0.2 A. On the six detector-limited ones 26.1 -> 9.8% and the bias goes +19 ->
-3%; on the 31 that are not, 9.23 -> 9.32%, i.e. it costs them nothing. The
0.30 tail fraction and the 2.25 reach were refit by leave-one-out against that
truth and did not move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Report the strong-pixel count off the FPGA path too ... 42add7c0e2

DataMessage::strong_pixel_count is written all the way to CBOR, to
/entry/MX/strongPixels, to the receiver plots and to the frontend - and only
the FPGA receiver ever set it. Every EIGER dataset, and every rugnux run,
stored zeros.

It is the one number that distinguishes an image the spot finder gave up on
from an image that did not diffract, and its absence is why a defect that cost
767 of 1800 images their spots looked like a crystal that stopped diffracting
for two blocks of the sweep. Fill it on the CPU/GPU path as well: the finders
know the count, the GPU extractor already had it on the device, and it rides
back with the spot count on the frame's one synchronisation, so it costs
nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

rugnux: report the geometry in the form the broker takes it in ... fe1f3ba96e

A rotation run post-refines the detector distance and the beam centre, and that
is usually the best measurement of them anyone has - but the only way back into
the instrument was to read two numbers off the report by eye and retype them
into a collection.

Write them once more, as the object jfjoch_broker takes: the four required
properties of dataset_settings in broker/jfjoch_api.yaml, spelled the way the
API spells them, on one line of valid JSON that a script can lift with a grep
and POST. The existing DETECTOR_DISTANCE and BEAM_CENTRE keys are unchanged and
stay the ones a person reads. REPORT_VERSION is not bumped: a new key breaks no
consumer of the old ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Ice rings: carry the list past 1.5 A, where ice does not stop ... 6516bc96af

The eleven measured bands end at 1.522 A because their source says so in its
own words - "pure hexagonal ice has 11 diffraction rings between 4 and 1.5 A
resolution" - and its subject was detecting ice in deposited data, not masking
it. On a detector that reaches further, the rings it does not list are the ones
left in the data: on the strong rotation set just added to the battery, 44% of
every image's spots sit in ice bands, and beyond 1.5 A the spot list is ice and
nothing else, which is why the resolution estimate read the ice rather than the
crystal.

There is nothing measured to copy below 1.522 A, so the eight added bands are
calculated. Enumerating hkl is not enough and the code already said so: ice Ih
is P6_3/mmc with O on 4f, and most of what enumeration emits is extinguished by
the OXYGEN SUBLATTICE rather than by the space group - which is why (004) at
1.830 A and (104) at 1.657 A are missing from the measured list although they
sit inside its range and its reflection conditions allow them (for (00l) the
structure factor goes as cos(2*pi*l*z), and z ~ 1/16 kills l = 4). So compute
structure factors - oxygen only, the hydrogens being half-occupancy disordered
and weak to X-rays - and keep the lines reaching 3% of the strongest. That rule
REPRODUCES THE MEASURED ELEVEN EXACTLY and every line it drops inside their
range computes to zero, which is what makes it trustworthy below 1.522 A. It
stops at 1.170 A: below that the real lines fall to 2-3% while the extinct ones
rise to about 1%, and an oxygen-only calculation cannot separate them honestly.

Every added band was independently confirmed in the data - the spot-count
histogram of the strong set peaks at each of them and is empty between - and
every line the rule calls extinct is absent there too.

Costs, measured. The bands are inert above 1.6 A: on 38 of 39 battery sets the
profile ice score does not move at all, and the one that appeared to (a jet set,
1.25 -> 2.60) does not on the peak-excluded profile the score actually uses -
that was Bragg peaks in the plain profile, which is what the peak exclusion is
for. Where a detector does reach past 1.5 A the bands cover more of reciprocal
space: unchanged at 1.6 A, +7.4 points at 1.4 A, +16.5 at 1.18 A. On the strong
set that is 17% -> 27% of reflections held out of the scale fit, and it shows:
the spot resolution estimate improves from 1.33 to 1.46 A against a truth near
1.42, while CC1/2 falls 98.5 -> 97.6% and ISa 3.58 -> 3.37. Ice handling only
runs at all on a run that trips the ice gate, so a clean crystal pays nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Spot finding: bound how large a spot may be, not how bright ... 0af109b838

A component of more than 50 connected pixels was discarded. Under the
self-calibrating threshold that is an INTENSITY CEILING, not a size bound: the
threshold is an absolute per-ring contour, so a component's area above it grows
as sigma^2*ln(A/T), without bound in the peak amplitude. Measured on the strong
rotation set, footprints run 3 px at 30-100 counts to 50 px above 10000 - a
slope of 4.4 px per ln(peak) against the fixed local-box test's 0.8, which
saturates at 13 px and never reaches the bound at all. So the brighter a
reflection, the more certainly it was thrown away: 59% of the box finder's
d>3 A spots were missing from the adaptive finder's list, including every one
of its ten strongest, at an intensity ratio of 1.085 for those that did match.
Indexed spots per image collapsed from 220 to 7.

Raise the bound to 200 - CrystFEL peakfinder8's --max-pix-count, the only
directly comparable number in the field; XDS has no such parameter and guards
on shape instead - and ask a component above 50 pixels to be COMPACT: it must
fill a fifth of the square its bounding box fits inside. A Bragg reflection is
round and fills about half of that square however bright it is; an ice arc, a
cosmic-ray track or a lit detector row fills a fifth or less, and those are what
an upper bound was ever protecting against. Below 50 nothing is asked of the
shape, so every component accepted before still is. Integer arithmetic on both
sides, and on the GPU the bounding side fits in what was padding, so the device
struct does not grow.

The shape test is what makes the raise safe. With a flat 200 alone, two battery
crystals moved: one lost a little I/sigma, and the other's de-novo lattice was
NOT MONOTONE in the bound - correct at 50, 100 and 200, wrong at 150 and at 250
and above - so 200 was partly luck. With the shape test both are unchanged to
three decimals.

De novo at defaults on the strong set: indexing rate 0.574 -> 0.804,
completeness 97.3 -> 99.6%, <I/sigma> 3.42 -> 6.07, R_meas 0.299 -> 0.249,
CC1/2 0.947 -> 0.961, ISa 3.29 -> 4.13. The full 37-crystal battery, scored per
shell, is still owed.

Also documents what StrongPixelSet::AddStrongPixel has required since the
component search became linear - pixels in raster order - and puts the existing
test's insertion order into it. Both callers scan a bitmap in ascending flat
index and always satisfied it; the test did not, and was the only thing that
did not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Spot finding: say honestly that the compactness gate is inert ... 844b461232

The commit that added it claimed the shape test was what made raising the size
bound safe - "with a flat 200 alone, two battery crystals moved; with the shape
test both are unchanged to three decimals". That is wrong. Those two runs came
from different builds, and the comparison never isolated the gate at all.

Isolated properly, gate on against gate off in ONE build: byte-identical .hkl on
three rotation crystals, the strongly diffracting one included. Not merely close
- identical, every digit of every merged reflection. On that crystal the gate
rejects 5 to 22 of the 110 to 440 oversize components a frame, so essentially
all of what that commit bought, it bought by raising the bound from 50 to 200.

Keep it, but for the reason that survives rather than the one that did not. The
raised bound admits components up to 200 pixels, and on data carrying ice arcs,
cosmic-ray tracks or a lit detector row those are what arrives; none of the
crystals measured here carry that population in quantity, which is exactly why
the gate reads inert on all of them. So it bounds the SHAPE of what the larger
size bound now lets through, and no experiment on these crystals can validate
it - an inert gate scores identically whatever its threshold.

The comment also carried a "keeps 97% of the genuine wide reflections and none
of the ice arcs" that came from a simulation over component lists, not from the
pipeline. Removed rather than restated: the pipeline measurement says no
reflection anywhere changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Adaptive spot finding: the ring threshold replaces the floor, not the local test ... 9f2696c1ac

The self-calibrating finder was meant to replace the classic finder's FIXED
PHOTON FLOOR with a per-resolution-ring threshold read off the image's own
noise. As written it replaced the local-box SNR test as well, and that is the
defect: a whole-ring threshold is an ABSOLUTE contour with no feedback from a
pixel's own surroundings, so the area a spot puts above it grows as
sigma^2*ln(peak/threshold) and never saturates.

Measured on a strongly diffracting rotation set, the detected footprint grows by
+8.05 pixels per e-fold of peak, so the brightest reflections came out as
100-500 pixel blobs and were then discarded for exceeding the size bound - every
one of the ten strongest on an image. Intersecting with the local box gives
-0.24 pixels per e-fold, the classic finder's own number to two decimals.

WHY the local box is the right partner, rather than merely the incumbent: it is
a prominence rule whose reference level is a 961-pixel mean. A spot inflates the
box's own variance and the peak divides out of the acceptance test, so it cuts
at a fixed FRACTION of the spot's own height. Referring that level to fewer
pixels makes it inherit their shot noise - at FIXED footprint, estimating the
level from 961 pixels, from 25, and from the single maximum gives centroid
residuals of 0.524, 0.539 and 0.656 - so flat growth and a stable centroid turn
out to be two ends of one dial. A contour on the bare maximum has the flattest
growth of anything tried (+0.1) and merges worst.

The two arms bind in different regimes, which is why intersecting beats choosing:
on serial stills the ring threshold is 0.6x the classic floor, on this rotation
sweep 2.3-6.0x. Stills are a strict no-op - 175 components against 175,
identical per frame - so the +40% in stills indexing that the adaptive threshold
was introduced for is untouched.

What it buys, stated as one fact rather than two. Across five geometry pins
spanning 1.1 mm it indexes the most frames of any arm tried, 0.831 against
0.803, and integrates 3.04 to 5.76% more observations - but those are the SAME
number: regressing observation count on indexing rate over four arms leaves
residuals of +/-0.7 percentage points against swings of -7 to +4.5%, so the
extra observations ARE the extra indexed frames, not better data per frame.
CC1/2, the only statistic here carrying per-observation quality, is +0.66 at one
pin and -0.06 at the other: not harmed, not improved. <I/sigma>, ISa and R_meas
cannot arbitrate on this data - across those pins each crosses zero as a
monotone function of the pin.

WHY an absolute contour indexes fewer frames, when its spot list is equal or
better on every axis measured - recall, top-1000 recall, centroid, ice fraction,
component count - is the interesting part, and it is not a detection effect at
all: ITS OWN SIZE BOUND DELETES THE BRIGHTEST REFLECTIONS ON THE FRAME. A
component is discarded because it grew past 200 px, and it grew past 200 px
because it was bright, so the deletions are drawn from the head of the indexing
budget rather than uniformly from it: they are 11x enriched in the top 250 of
the thousand spots handed to the indexer, and the bound's own real deletions sit
at MEDIAN RANK 12. Turning the bound off recovers 66% and 50% of the deficit at
the two pins, against a bar registered at 33% before the run.

Three of us dismissed this for most of a day on the grounds that the gates
delete only ~4% of what is detected. That arithmetic was right and the
denominator was wrong - a rate is not an impact when the thing being lost is
selected for the property that makes it matter.

Reworking the bound instead was measured and rejected: it recovers half the
deficit, and it cannot be done without re-admitting what the bound is for - 68
components past 200 px, of which 8 are real and 60 are junk, where the intersect
gets the 8 without the 60. The residual once the bound is off, +1.08%/+1.70%,
is the contour itself.

Component merging is ruled out separately: geometrically impossible here, 33.9
px minimum reflection separation against components spanning 10 px. So is a
ranking effect - the intersect's lead runs +0.06% at --max-spots 250, +3.46% at
1000 and +14.26% at 2000, which is backwards for a selection artefact.

Costs 0.48 ms per image in the finder, and 0.044 px of bright-spot centroid
precision - measured convention-free, by fitting a line to a reflection's own
centroid across five frames, after an XDS-referenced figure proved to be four
fifths aperture convention.

It also makes the compactness gate above it safe. On the absolute contour that
gate is net damage, deleting 37 genuine reflections per ten frames; once the
footprint stops growing nothing reaches its threshold at all.

Also fixes a real but unexercised defect in PoissonThreshold, where the exact
tail handed over to a normal approximation with a step. It changes nothing here:
the clipped ring sigma is over-dispersed 1.2-4.9x against sqrt(mu) because it
still contains diffraction, so the Gaussian arm wins every ring above mu=50 and
none of the 522 thresholds move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Space-group search: ignore the frames the crystal barely diffracted on ... df98bb8789

A crystal that repeatedly leaves the beam over a full turn was read as P2(1)
where it is P4(3)2(1)2, on data that merge at CC1/2 96-98% once that symmetry
is assumed. The cell was right, the indexing was right - 83% of images - and
every 422 operator correlation still came out between 0.08 and 0.38 against a
gate of 0.30.

The per-frame scale enters the intensities as 1/G. On this crystal the fitted
scale spans 356x between its 5th and 95th percentiles and 15% of frames sit
below a tenth of the run median, so those frames arrive amplified ten- to a
hundredfold - and their sigmas are amplified by exactly the same factor, so
nothing weighted by sigma can see it. What saves the PRODUCTION merge is
multiplicity: in the determined symmetry a reflection is measured ten or twenty
times and --reject-outliers throws the amplified observation out. The search
merges in P1, where a reflection has two or three observations and no majority
exists to call any of them an outlier, so the bad value lands in the merged
intensity at full weight and the operator correlations pay for it.

So drop those frames from the search merge only, beside the |zeta| filter and
through the same snapshot that undoes it before the production merge, which
keeps every frame. A tenth of the median because that is where the two
populations sit: over the 39-crystal battery 33 crystals have NOT ONE frame
below it, so the filter is inert on them by construction; of the six that do,
five spend 0.1-3.5% of their frames there against this crystal's 15%. A
twentieth leaves it under the gate; a fifth starts costing frames the battery
says are real, the smallest legitimate min(G)/median(G) measured being 0.070.

Battery 35/39 -> 36/39 space groups matching XDS, with 38 of the 39 rows
character-identical: the only crystal that moves is this one. On it the
operators go to 0.37-0.62, and every one improves in every resolution shell,
most at low resolution - the signature of a scale error removed rather than
noise removed. The run then merges to 1.41 A instead of 1.67, at CC1/2 97.7%
and completeness 99.7%.

Four other explanations were measured and refuted first: the search resolution
cut (the correlations are flat in resolution, and a battery crystal with a
finer detector and no cut at all gets 422 without trouble), the space group
being inherited from the pass-1 geometry (handing the refined geometry in from
the start changes nothing), the ordering of the corr snapshot around the
collapsed-scale guard (0.02 in correlation, nothing in space group), and the
ice-ring flagging (removing it entirely leaves the answer at P2(1)).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

rugnux: measure the spot budget instead of taking it as given ... 4951538368

--max-spots was a fixed 1000, and on a rotation sweep it sets the DENOMINATOR of
the per-frame acceptance gate, which admits a frame when at least a fifth of its
spots index. Detections are not all reflections: on a strongly diffracting
crystal with heavy solvent background there are 2372 a frame, 1416 of them on
ice bands, and only 178 index - so a budget that takes essentially the whole
list puts the entire frame population on the gate (median indexed fraction
0.271, tenth percentile 0.218) and the run integrates 83% of its images. The
same run with a smaller budget gains 211 frames and loses none, and the frames
it gains are the ones with the MOST detections. That is how a larger budget
integrates fewer images.

So measure it: over the first pass's validation frames, with the sweep's lattice
known, tally each spot by rank as +1 if it lies on the lattice and take the
budget at

    argmax over N of  n_indexed(N) - 0.20 * n_counted(N)

which rises exactly while spots at that depth index better than the gate's own
floor and falls after. The 0.20 is that floor, not a new constant - it is lifted
out of the function-local it already lived in. It means: as deep into the
intensity-ordered list as the image is still showing reflections of THIS crystal.

The argmax alone would not do. Under the null that spots index at the same rate
at every depth the tally is a driftless random walk, whose maximum is positive
whatever the data, so a bare argmax shortens every dataset. The budget therefore
has to clear the walk's own noise: the quantity it acts on is the fall from the
peak to the end of the list, which is that walk read backwards, and the
reflection principle gives its null law in closed form -
P(fall > z*sqrt(g(1-g)T)) = 2(1-Phi(z)). That already pays for the search over
ranks, so nothing further is owed to multiple comparisons. One false cut in a
thousand measurements - a twelfth of one over a 39-crystal two-pass corpus -
fixes z at 3.29. The level was chosen before the rule was written and was not
revisited afterwards.

Battery, same build, one changed default: 36/39 space groups in both arms, none
lost, and 36 of the 39 crystals BIT-IDENTICAL. Two move materially - the strong
crystal by +17.4% observations and +35.9 points of CC1/2 at 1.58 A, its indexing
rate 83.4 -> 95.1%, and another by +93.5% observations with completeness
76.6 -> 98.4%. The significance requirement is what makes that list clean: it
removed the one crystal the unguarded rule regressed, and with it two of the
five gains, whose peaks do not clear 3.29 sigma on 60 frames. The lever for
those is the sample and not the threshold - power grows as the root of the frame
count while the bar stays where it is.

Rotation only, and the first-pass lattice search always sees the full list: a
fixed --max-spots 250 would have been much cheaper to write and fails a battery
crystal outright, starving the de-novo FFT into a wrong cell that indexes 7 of
60 frames. Stills never reach the code path - verified bit-identical merged
reflections on a serial set - and --max-spots N still pins it, as does the
library default the broker and the FPGA use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

Indexing: let the FFT search reach past 500 A, and stop clipping the peak background ... e78fbd55b7

fft_max_unit_cell_A was both the default and an enforced check_max, so 500 A was the longest
basis vector the FFT could ever return: FFTIndexer sizes its projected histogram from that value
and the transform's last usable bin IS that length. Of the PDB's 206950 X-ray entries, 1091
(0.53%) have an axis longer than that and were unindexable by construction. The accepted range
now goes to 1200 A, which leaves 8. The DEFAULT is unchanged at 500 - the histogram is sized from
the value in use, so nothing pays for the wider range unless a caller asks for it.

The peak picker's running-mean background was truncated at the ends of the spectrum rather than
slid inward, so a peak within bg_half (~15 A) of either end - which is exactly where the longest
cells sit - was judged on a one-sided background, biasing its prominence by however much the
spectrum sloped there. Keep the window a constant width and slide it. Both bounds stay
monotonically non-decreasing in j, so the GPU kernel's running sum is still valid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

reader: read a PILATUS miniCBF sweep natively, without libcbf ... dc16a00271

Most facilities still archive rotation data as a directory of miniCBF frames, which until now had
to be converted to HDF5 before rugnux could see it. Nothing in that format needs a CIF parser or a
library: it is an ASCII header, four separator bytes, then one byte-offset compressed image, and
every value the reader wants sits on a "# " comment line or a MIME line.

MiniCBF holds the format itself - header parse and the byte-offset decoder, which is a running
value with deltas stored smallest-container-first. Verified byte-exact against dxtbx on PILATUS 6M,
6M-F, 300K, silicon and CdTe sensors, and three sensor thicknesses.

JFJochCBFReader is a sibling of JFJochHDF5Reader under the JFJochReader base. NAMING ANY FRAME
READS ITS WHOLE SWEEP: the sweep is identified by the template (prefix + digit count) the named
frame belongs to, not by "every .cbf in the directory", so a directory holding two sweeps does not
splice two crystals together. Naming a directory takes the sweep with the most frames in it.

Images decode on demand, one per call, so any number of workers can read at once - there is no
global lock as there is on the HDF5 path, HDF5 not being thread-safe. A raw CBF carries no analysis
results, so the dataset it builds is the geometry, the mask and nothing else, exactly as a plain
DECTRIS file with no /entry/MX gives.

Two header quirks are handled because real files have them: the sensor material is written
"Silicon" where the rest of the code compares against "CdTe", and the thickness unit is sometimes
omitted. Headers are not a fixed size either - one set carries 6335 bytes - so the parse runs to
the binary separator rather than over a fixed prefix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rugnux: accept a CBF sweep, decide the rotation axis sign, and search as far as a given cell needs ... 84d14696b1

Three changes to the rotation path, all in the same first-pass block.

Input. Rugnux now holds a JFJochReader rather than the concrete HDF5 reader, so a sweep can come
from a directory of miniCBF as well as from a master file; the CLI picks by looking at the file.
Only the _process.h5 image links need the concrete reader and they ask for it by dynamic_cast -
there is nothing for a virtual dataset to point at in a directory of CBFs, so that file writes its
images instead of linking to them. --mode scale refuses a CBF sweep, there being no stored
reflections to re-scale.

Search bound. A reference cell longer than the FFT bound cannot be represented by the search meant
to find it, so the run reported "found no lattice" - strictly worse than not giving the cell at
all. Raise the bound to cover a given cell (margin 1.3, as DIALS uses when a cell is known), and
raise it in the long-axis rescue too, where it costs nothing: that pass already coarsens the
resolution to 3.5 A, which shrinks the transform by 1.75x, so spending it back on reach leaves the
rescue at 1.14x the standard pass. The rescue's constrained re-index gets a pool wide enough to
hold what the coarse pass recovered, which it did not before. ROTATION ONLY - stills fire the FFT
once per image across every worker and routinely run with a known cell, so widening the transform
there would cost the whole serial run for nothing. Clamped rather than throwing: the rescue can
recover an implausible axis, and a bad candidate must not take the run down with it.

Axis sign. The rotation-axis sign is a convention the input often cannot settle - a miniCBF header
names the axis but gives no direction, and an NXmx vector is only meaningful together with the
detector orientation, which nothing here reads. It is decidable from the data, though: the wrong
sign does not index at all (measured on one sweep, changing nothing else: every frame indexed one
way, none the other). So after a poor first pass, try the other sign and keep whichever indexes
more validation frames - the same count the scheme choice already uses, so no new metric and no new
threshold. It flips the AXIS, not the angles, because prediction reads the axis too; the decision
then holds for the rest of the run. Runs before the long-axis rescue, since with the sign wrong
every candidate lattice is wrong. Only after a poor pass, so a correctly-signed file costs nothing,
and spot finding is not repeated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

compression: read plain LZ4 (HDF5 filter 32004), and name the filter when one is unsupported ... a7615c8778

DECTRIS Eiger firmware 1.x compressed its images with the HDF Group's plain LZ4 filter rather than
bitshuffle, and files from that era are still what a repository hands you. Read-only: nothing here
produces it, and the new enumerator goes last so the existing values do not move, the enum being
part of the CBOR stream.

The framing is the same as bitshuffle's - a 64-bit big-endian total size, a 32-bit big-endian block
size in bytes, then each block prefixed by its 32-bit big-endian compressed size - so only the
unshuffle step differs. Verified on a real chunk: the declared total matched width*height*4 exactly
and the block walk consumed the chunk to the byte. It cannot reuse the bitshuffle path, which
requires the block to be a multiple of BSHUF_BLOCKED_MULT elements: these files put the whole image
in ONE block, which is not.

The unsupported-filter message now names the filter it found and the ones it knows. Before, an
Eiger 1.x file did not report a codec problem at all - it reached spot finding with nothing decoded
and failed as "0 spots from 60 images" then "found no lattice", which reads as a crystallography
failure rather than a format one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

reader: fall back to the pre-NXmx key names, so an Eiger 1.x master opens ... df3696e6e2

Firmware 1.x writes the same three values under different names. Try the NXmx spelling first and
the old one only if it is absent:

    detector/distance         <- detector/detector_distance
    detector/saturation_value <- detectorSpecific/countrate_correction_count_cutoff
    sample/transformations    <- sample/goniometer

This cannot change what a current file reads: every modern Eiger master carries BOTH spellings.
Measured on thirteen masters from ten facilities, firmware release-2020.2.1 through
release-2024.1.1 - all of them write detector_distance and countrate_correction_count_cutoff beside
the NXmx names, and a goniometer group beside the transformations one.

The goniometer is the one that matters. A goniometer is only ever set from that one group, so a
file whose axes are somewhere else was not an error - it was read as STILLS, silently, and the run
completed with the wrong answer. The old layout also tags no axis with transformation_type and
gives no vector, both of which ReadAxis required, so absence now means two different things by
layout: in a transformations group it still means "not an axis" (that is how AXISNAME_end and the
width scalars are skipped), while in the legacy group every leaf IS an axis and the companions are
recognised by name instead. A missing direction defaults to the one every DECTRIS master since has
written and says so in a warning rather than assuming it silently; a wrong guess there does not
index, so it is visible, and the rotation first pass tries the opposite sign anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

reader: match the legacy companion suffixes exactly, not any underscore ... fa4aa30a1e

The legacy DECTRIS layout stores five companion scalars beside each goniometer
axis - AXIS_end, _start, _increment, _range_average, _range_total - and only the
bare name is the axis itself. The first cut rejected _end and _range, which left
_start and _increment to be offered to ReadAxis as axes in their own right.

Matching the full suffix set, anchored at the end of the name, also keeps a
genuine two_theta axis readable, which a "contains an underscore" test would not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Scaling: make Run() idempotent, and re-calibrate the promotion it was hiding ... dce8d877d0

RotationScaleMerge::Run() resumed its alternating-least-squares loop from
wherever the previous call left off instead of restarting from the ingested
data. A de-novo run calls it four times, so pass N was scaled at 3N iterations,
not 3. Two identical calls on the same object produced merges correlating at
0.847 with ZERO bit-identical intensities; they are now 68010 of 68010
identical.

Where that mattered is narrower than it sounds, and worth stating exactly. In a
determined point group the fit is stationary from iteration 2 - over the battery
the collapsed-scale guard drops 377 frames at three iterations and 397 at
twelve, and 30 of 39 crystals drop nothing at any count - so production merges
were barely touched, 38 of 39 rows character-identical. But the SEARCH merges in
P1, where a reflection has two or three observations, and there the fit never
reaches a fixed point at all: the same guard drops 417 frames at one iteration
and 1101 at three. A factor 1.05 against a factor 2.6, same code. So the defect
lived in the one arm whose output the user cannot recover from downstream - the
arm that picks the space group.

Fixing it moves the statistics the promotion gates are calibrated on. The
population as a whole does not shift (median change 0.9-2.7% over 280 matched
candidates) but the tails do, p90 by 8-14%, and that is exactly where a
promotion is decided. On the two crystals that decide, the H statistic's two
populations SWAP ORDER: a genuine tetragonal 4->422 goes 1.685 -> 1.819 while a
trigonal 3->32 twin law goes 1.906 -> 1.665. No H bound exists any more that
keeps the genuine case and refuses the twin - the noise the loop was adding had
been acting as a brake.

Of the four statistics only b and R_meas still order the two correctly. Set
max_systematic_b_ratio 1.90 -> 1.78, by a rule written down before the
population statistic existed: take the statistic with the largest
spurious_min/genuine_max on the population its gate actually sees, put the bound
at the geometric midpoint. That is sqrt(1.711 * 1.853) = 1.78.

Battery 36/39 space groups matching XDS, the same as before the scaling fix and
one better than the scaling fix alone; the three mismatches are the known
references where XDS is wrong. The refusal now reaches the user by name.

The bound's margin is 4% either side - SMALLER than the p90 shift this very
commit produced in that statistic. It separates the two crystals that exist and
is not robust to another input shift, which is written into the header rather
than left to be discovered. Two hazards go with it: re-measuring it must hold
the scaling correction surfaces fixed, since changing the iteration count
re-rolls their cross-validated on/off decision on 7 of 39 crystals for ~0.3 in
ISa, all-or-nothing and non-monotone; and --scaling-iterations stays at 3 on
both arms, confirmed rather than assumed (production 2->3 moves <I/sigma>
+1.38% and 3->6 only +0.13%; the search gives the identical space group on all
39 crystals at 2 and at 3, and loses one at 1).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FBumeJVx4oeXxiBRpkrE5H

rugnux: fail on an exhausted GPU instead of dropping the images it could not process ... 92ad304f15

A per-image worker caught every std::exception and continued to the next image. That is right for
one undecodable image and wrong for a resource fault: a CUDA out-of-memory says nothing about the
image that happened to be in flight and everything about the machine, and it recurs on the next one.
The run therefore logged an error per image, skipped each of them, and exited 0 with a dataset that
was silently short - the merged numbers all moved, and nothing in the exit code or the summary said
so. Observed with several processes sharing one card: two arms of the same comparison lost 21 and 15
images, by different amounts, so the arms no longer saw the same data.

IsFatalResourceError rethrows for std::bad_alloc and for a JFJochException of category GPUCUDAError
or MemAllocFailed, and every per-image catch consults it first. There are ten of them, not the three
the obvious grep finds: besides the analyze/integrate/load sites in the two image loops, the
first-pass spot read swallows the same way at Warning severity, the pre-scan swallows it in three
places, and one geometry-refinement site caught it with no log line at all. The first-pass one
matters most - those frames build the reciprocal-space cloud the indexer runs on, so losing them
thins the input that decides whether the crystal indexes, and it does not change IMAGES_PROCESSED,
so a frame-count audit cannot see it.

Every one of these loops joins with future::get() or ParallelFor's RunTasks, both of which hold the
first exception and rethrow it at the join, so the throw propagates and nothing terminates.
JFJochException gained a Category() accessor for the test.

Verified by squeezing the card to 804 MB free and running at -N 4: the run now exits 1 with zero
dropped-image lines, where the same test on the previous binary produced the silent skips above.
The broker and receiver paths are deliberately untouched - dropping a frame is the better failure
there than killing a live acquisition.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

indexing: tell an indexer that failed apart from one that found nothing ... 6319d5c600

IndexerThreadPool collapsed two different outcomes into the same empty reply. A worker that threw
set result = nullptr, and so did a dispatch that never found a free worker; both then became a
default-constructed IndexerResult, indistinguishable downstream from an indexer that ran and found
no lattice. The first says nothing about the frame at all - the indexer never looked, and it will
fail again on the next one - while the second is a real negative result about the crystal.

The visible cost was the advice a failed run gave. With the card full, rugnux printed "Indexer
thread 0 failed: CUDA (GPU) error" and then ended with "Two-pass rotation indexing found no lattice.
Check the beam centre (--beam-x / --beam-y), raise --max-spots ..." - sending the operator to look
at geometry that was never wrong, for a machine that was simply out of memory. None of those
remedies can help when the frames were not examined.

IndexerResult gains an optional error, set by the worker and by the pool's own catch; the remaining
nullptr path keeps its meaning of "not attempted" and deliberately carries no error.
RotationIndexer records it and exposes GetIndexerError(), and rugnux's first pass branches on it at
the throw site, naming the resource failure instead.

The error travels as DATA through image_analysis/ rather than as an exception, because
RotationIndexer::RunIndexing() is on the online path - IndexAndRefine calls it on a schedule from
the broker and the receiver, where dropping a frame is the right failure and killing a live
acquisition is not. Only rugnux, which owns the "this run is over" decision, turns it into one. The
failure result is byte-identical to the default-constructed one it replaces, so any_executed and the
online frame-drop behaviour are unchanged.

The new IndexerError category exists because the category is only the display prefix on what() -
Category() is read nowhere - and the old line read "Processing failed: Input parameter invalid" for
a GPU fault, which is the same defect one layer up. SpotFinderError is the precedent.

msg.indexing_result is deliberately left alone: of its three states, "not attempted" is the honest
one for a frame the indexer never examined, and asserting false would be the same collapse again.

Verified on a squeezed card (246 MB free, -N 4): exit 1, zero dropped frames, the new message, and
no mention of --beam-x. On a free card, against the previous binary at -N 6 on the same input, the
logs differ only in paths and timing - same space group, cell and merge statistics. Targeted Catch2
cases pass, including three that drive the pool through the online path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

reader: open a master that states its goniometer axes in one place and their angles in another ... 1ea6ba25be

A hybrid master carries an NXmx /entry/sample/transformations group holding one EMPTY SUBGROUP per
axis - the direction as a vector attribute, no NX_class, no units, no angles - beside a legacy
/entry/sample/goniometer group holding all the angles. Opening it failed outright with "Cannot open
HDF5 dataset /entry/sample/transformations/omega": the existing legacy fallback keys off
Exists("/entry/sample/transformations"), which is true here, so it never fired, and the axis stub
was then opened as if it were the angle dataset.

Present is not the same as usable. GoniometerGroup now takes transformations only if it holds at
least one DATASET, and ReadAxis asks IsDataSet rather than Exists, so a member that is not a dataset
can no longer be read as one. HDF5Object gains that predicate, in the style of the neighbouring
Exists.

The stub is the load-bearing half, not merely the thing that crashed. In the legacy branch, before
falling back to the assumed (-1,0,0), the reader now looks for the NXmx stub and takes its stated
vector. With it the axis is (0,-1,0) and the run indexes 60/60 validation frames; with the stub
deleted the assumption applies and the same file indexes 0/60 on both schemes and both signs, and
the run stops with no lattice. So without this half the fix would have turned "cannot open" into
"found no lattice" - a differently shaped failure, not a success. The direction stated here is 90
degrees from the assumption, not merely its negation, which the rotation first pass could have
recovered on its own.

The vector size check moved out of the first branch so it now covers every path that produces one.

Verified: the file processes end to end, 100% of frames indexed, cubic cell 105.87 against a
deposited 105.88 (0.009%), space group reported as I23 or I213 - correctly refusing to choose, since
the reflections that separate them are extinguished by the I-centring and were never measured. A
both-layout master reprocesses unchanged (100% indexed, P212121). [HDF5] passes: 2194 assertions in
91 cases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

lattice search: reject an impossible character, and supplement every angle ... 10873eee7a

Two defects in the Bravais character walk, both of which silently cost symmetry.

An impossible character MATCHED. The walk computes acos(cond_F/sqrt(A*B)); when the character is
geometrically impossible the argument leaves [-1,1], acos returns NaN, and the acceptance test
fabs(NaN - actual) > tol is FALSE - so the character is taken. One corpus crystal matched a
monoclinic-C character whose implied cos(gamma) is 1.086.

The type-boundary retry covered beta only. All three angles carry the Niggli boundary, and negating
two basis vectors supplements the OTHER two, so each angle needs its own flip. A crystal whose
reduced gamma sits at 89.900 degrees needs the alpha/beta flip and never got it.

Measured on 84369 exact lattices spanning all 14 Bravais classes: nothing is lost in any class, and
orthorhombic-I recovery rises from 60.3% to 87.2%. Over 440000 random and perturbed cells the first
change only ever demotes a monoclinic-C match to triclinic and the second only ever promotes out of
triclinic - it is a strict superset of the beta-only retry. On the 94-dataset corpus, two crystals
gain their correct point group and none regresses.

Both regression tests fail without the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

symmetry: score screw absences per zone, and choose centring by likelihood ... 8634e9a72b

Two places where the absence tests threw away information they already had.

The screw evidence was POOLED across axial rows, so one unmeasured row vetoed a confirmed one. A
crystal reading h: +18/+24 and k: -6/+8 answered a silent P222 rather than "2_1 along a confirmed, b
undetermined". Scoring each zone separately, and letting a zone abstain, is what POINTLESS has always
done; the conditions are independent, so their log-likelihoods add.

Centring was decided on a COUNT of net absences while screws used a Beta-tail likelihood. On one
crystal that made F222 (1662 absent, 356 violations, absent class at 34% of present) beat C222_1
(1098 absent, 15 violations, 0.4%) purely because 1306 > 1083. Giving centring the same likelihood
separates them by 1717 nats - F222 -357.9, C222_1 +1359.3, I222 -787.6 - and needs no bound:
min_absent_observed and max_absent_present_ratio are untouched, and nothing was tuned to this case.
The deposited structure for that crystal is C222_1.

Corpus of 94 datasets, both arms at -N 6: 78 comparable, 73 bit-identical, point group correct on 68
against 66, space-group number on 54 against 51, nothing regressed. The 73 bit-identical results are
also the determinism control - two binaries cannot agree byte-for-byte that often by chance.

An earlier ranking key summed each group's single gating number and demoted a P2_12_12_1 to P2_12_12,
because pooling three genuine screws reads weaker than two when the third row is shallower. Summing
the per-zone log-likelihoods is what fixes that, and the whole corpus was re-run on the corrected
binary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

rugnux: decide the space group after the second pass, not the first ... 4ef6bd9952

Pass 1 searched for the space group on the geometry the run started with, stashed the answer, and
reinstated it before pass 2's merge - so the decision was made on the worse of the two passes and
then fed forward as a constraint. Four guards existed to reconcile it with what pass 2 later found.

Now pass 1 does not search at all. It keeps its first merge, which is where mosaicity smoothing and
geometry post-refinement already happen, and skips the in-symmetry re-merge that existed only to feed
the search. prepass_merge_sg_, prepass_promoted_point_group_, the reinstatement, the
primitive-to-conventional reindex it needed, and lattice_conflicts_with_prepass_sg all go.

The obstacle was the pass-adoption guard, which compared completeness and CC1/2 across what could now
be two different space groups. It compares each pass's FIRST merge instead - P1 on both sides, full
range, before correction surfaces - which pass 2 produces anyway.

One guard had to be replaced rather than removed. The index-time veto is re-keyed from pass 1's GROUP
to its LATTICE and made one-directional: a centred pass-1 lattice against a primitive pass-2 one.
Without it a C2 crystal fell to P1, and the obvious narrower fix - exempting a pass-1 P1 - would not
have caught it, because that crystal's pass-2 lattice is monoclinic-P rather than triclinic. A
centred lattice and its primitive sub-cell share a primitive volume, so the supercell arm cannot see
that demotion.

Corpus of 93 datasets, both arms at -N 6, every log validated: 78 comparable, 65 with a byte-identical
p.hkl. Two crystals gain their reference point group, one gains its screw axis, one is lost. Per
shell, over the 72 unchanged crystals, the median move is +0.25 points of R_meas and zero in
<I/sigma>, CC1/2 and completeness. Load-matched compute is +0.4%, and the header-geometry fallback
fires three times in the old arm against once in the new.

What is lost is one crystal whose 222 the operator correlations still confirm: the merge chi-squared
ratio moves 3% across a fixed bound at the refined geometry. A second crystal in the XDS harness
crosses the H bound the same way. Both bounds are in SearchSpaceGroup and are recorded elsewhere as
miscalibrated; recalibrating them is deliberately left to its own change.

The comment at the long-axis rescue is corrected while here: the resolution setting only sizes the
FFT histogram, and coarsening moves the true axis DOWN the direction ranking - 1996 to 16361 of
16384 - rather than making it more robust as the comment claimed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

reader: open a master whose beamline writes standards-correct NXmx differently ... 60c15d6931

Four defects, hit in sequence, that between them stopped eight masters from one beamline before any
geometry question was reached. The files are correct NeXus; the reader was assuming one writer's
conventions.

ReadScalar demanded rank 0. NXmx puts no rank on distance, saturation_value, two_theta or det_z, and
these files write them as shape (1,). Any dataspace holding exactly one element is now accepted; a
genuine vector is still refused.

frame_time was read unconditionally and NXmx does not require it.

A virtual-dataset source filename of "." was resolved as a relative path, giving <dir>/. - but "."
is HDF5's spelling for THIS file, and these masters compose /entry/data/data as a virtual dataset
over datasets in themselves that are external links to the data files.

The virtual source's DATASET PATH was parsed and then ignored in favour of a hardcoded
/entry/data/data, while these data files keep their images at the root. Fixing that exposed a fifth:
the positional-read fast path used the master's own path, but a dataset reached through an external
link lives in another file and a chunk address is an offset into THAT file.

An audit of all 38 masters in the non-SLS corpus finds exactly these eight need the change and the
other 30 need nothing. Corpus A/B over 73 dataset pairs, base and patched back to back: 64
byte-identical results, 9 identical failures, none differing. The blast radius is bounded by
construction - JFJochReader is linked only by rugnux and jfjoch_viewer, and the one shared header
this touches only ever accepts more, so nothing that opened before can read differently.

With it the eight files read completely: 12850 images, no decode errors, seven of them spanning two
source datasets through the master's own external links.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

reader: take a miniCBF rotation axis from the goniometer the header states ... 367eba55b1

Every miniCBF sweep was handed the same hardcoded axis regardless of what its header said, and
Chi/Kappa/Phi/Omega were not parsed at all - there were no members for them. A sweep collected on a
tilted chi cradle therefore ran with an axis that is 54.7 degrees wrong.

The header angles and their increments are now read, the scanned axis is identified from the
non-zero increment (the name is consulted only when no increment is stated, which is what absorbs the
five different spellings the corpus contains, including one file that states no axis name at all),
and a phi scan composes the head chain. An omega scan returns the base axis untouched, because a
fixed chi cannot tilt the axis it hangs from.

-9999 is a sentinel meaning "not set", not an angle. It is treated as absent, so it can never reach
the geometry.

The direction and sense are not invented: these files append an imgCIF _axis loop stating their own
vectors, and SOURCE with GRAVITY fix the imgCIF-to-internal transform, which independently reproduces
the transform this repository already documents for NXmx. Under it the file's own stated phi axis is
exactly the composed one, to four decimals.

Driving the real reader over all 39 corpus sweeps, 37 return the previous axis bit-identically -
including every sweep carrying a large fixed chi, every sentinel header and every axis-name spelling.
Only the two genuine phi scans move, and an unrelated rotation dataset is unchanged end to end.

This is necessary but not sufficient for the one dataset that motivates it: with the axis corrected
it still does not index, because that detector is also mounted rotated 90 degrees in its own plane,
which the reader does not yet read. Compensating both takes its phi sweep from no indexed validation
frames to 90.89% indexed and a complete merge, which is what shows this half is load-bearing. The
detector mount and the two-theta swing belong to the detector-frame work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

rugnux: expose the indexed-spot gate and the third detector rotation ... 9a66a21a10

--min-indexed-spots sets how many spots must lie on a candidate lattice for a frame to count as
indexed. It is what the rotation first pass scores candidate lattices on, and it was reachable only
by editing the source. The default of 9 is unchanged; on a sparse pattern, lowering it to 6 takes the
first pass from 31 of 60 validation frames to 38.

--rot3 completes the set beside --rot1 and --rot2. rugnux --mode calibration WRITES a .poni file
containing all three, and without this flag a refined third rotation could not be given back - it had
to be transcribed by hand and its convention undone. With it, a detector mounted rotated in its own
plane is expressible from the command line.

Also corrects the description of fft_high_resolution_A in the API, which claimed to be the highest
resolution of spots USED. It filters nothing - every spot is projected whatever its resolution. It
sizes the projection histogram, and with it the transform. Implementing the documented behaviour was
tried separately and makes its own motivating crystal strictly worse, so the code is right and the
description was wrong. (The generated Python client doc carries the same text and will pick this up
at the next regeneration.)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

indexing: refuse a coplanar candidate, and search the plane normal when the shortlist is flat ... 4e8db41785

Three related changes to the FFT candidate path, batteried together because they touch the same
function.

A COPLANAR CANDIDATE REACHED REFINEMENT. ReduceResults filtered triples on lengths and angles only -
the 30-150 degree bound admits any flat combination - and there was no volume test. On one dataset 41
of 5535 candidates had |V|/abc below 0.05, with a clean decade gap to the next, and three of them
reached the optimizer. UnitCell is float, and for a cell that flat the metric determinant is around
1.5e-7, so float32 gets its sign wrong 19% of the time where float64 never does. The guard against a
negative argument to sqrt then CREATES the singularity it was meant to prevent: it puts c in the a-b
plane, the reciprocal volume is 1/0, and the residual is 0 times infinity. Ceres reported a
not-a-number Jacobian and wrote several hundred lines of solver output per failed solve.

VolumeFraction() is |V|/(|a||b||c|), rejected below 0.02 - about 1.1 degrees off flat, ten times below
the flattest real candidate observed and a thousand times above where float loses the sign. It is
enforced at the producer and at the two optimizer entry points. Note the existing sanity checks use
ABSOLUTE volume, which a 320 cubic-angstrom flat cell passes. The same reciprocal-volume division is
now guarded at the two remaining sites that share the pattern.

A SHORTLIST CONFINED TO ONE PLANE cannot close a cell, and the row it is missing is the plane normal.
That is detected from the scatter-matrix eigenvalue ratio - measured, degenerate clouds score 2e-5 to
3.3e-4 against 0.026 or more for every non-degenerate one, a factor of eighty - and one further
transform is spent with the same direction count inside a three-degree cap about the normal, so the
plan and buffers are untouched. More directions cannot substitute: at the exact true direction the
long axis ranks 1422 of 16384 by prominence while the shortlist cut is four times higher. Ranking, not
sampling, is the obstacle. A four-fold denser grid was measured and rejected - it reaches the same
answer to three decimal places and takes a run from 2.5 to 8 GB of device memory.

fft_min_unit_cell_A is reachable as --fft-min-unit-cell and is lowered automatically by -C, mirroring
how the maximum is already raised. The default of 10 is unchanged: a lower floor admits spurious
sub-cells on protein data, and over 73 protein runs the floor was never lowered while the sibling
maximum did fire twice, so the path is live and correctly inert.

Corpus of 93 datasets, both arms, one build: 72 bit-identical on report content and p.hkl checksum, 13
failing identically, and the count of working datasets rises by one. The volume guard fires on 58 of
93 and 47 of those stay bit-identical - it fires constantly and almost never changes an answer, which
is what it should do. Solver chatter falls from 919 lines across three datasets to none. The cap
fires on 4 of 93, none of them in the in-house or private arms.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

rotation indexing: refine twelve candidate lattices instead of four ... 172a845cbb

The first pass refined only min(candidates, 4). The comment at the selection already conceded that
the pre-refinement indexed fraction is an unreliable discriminator, and on one dataset degenerate
cells occupied three of those four slots - so the right cell, ranked fifth, was never refined. The
preceding commit's volume guard frees those slots but does not widen them.

This is a one-for-one trade, measured by rebuilding with the old value to confirm the attribution.
It gains one dataset: R_meas 27.2% to 23.8%, with mean I over sigma better in EVERY shell at
identical shell edges, so it is not a resolution ramp. It costs another: a clean abort becomes a
wrong four-fold supercell in P1 at 36% completeness.

That is the same shape of trade already accepted for the plane-normal cap search in the preceding
commit - an honest failure becoming a wrong answer - on a different dataset. It is committed
separately so it can be reverted on its own if the maintainer weighs this instance differently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

viewer: open a miniCBF sweep, the same way rugnux already does ... 84efb0666c

The native miniCBF reader added for rugnux is a plain JFJochReader, so the viewer
only needed to be told which one to open a file with. JFJochImageReadingWorker held
a concrete JFJochHDF5Reader; it now holds both readers and a JFJochReader* pointing
at whichever the open file needs, chosen by JFJochCBFReader::CanRead. Naming any
frame opens the whole sweep - the reader's own template matcher decides which frames
belong to it, so a directory holding two sweeps or XDS auxiliary files is not spliced
together. The previous file is closed before the new one is opened, so switching
format between HDF5 and CBF in one session leaves nothing behind.

Everything the viewer draws already goes through JFJochReader and
JFJochReaderDataset, so nothing else had to move. The two HDF5-only features stay on
the HDF5 reader: calibration images, and the reprocessing snapshots, which are
metadata read back over the same images the reader is serving and have nothing to
attach to for a directory of CBFs. A raw sweep therefore shows geometry and pictures
with an empty run list, which is the state a live HTTP stream is already in.

Reprocessing jobs run on a CBF sweep too - JFJochProcessController opens its own
reader the same way, and Rugnux has handled a CBF source since the reader landed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

docs: credit the public diffraction data the pipeline is tested against ... 9d3c2787f8

Jungfraujoch is developed at one facility, so the reader and the reduction
pipeline are exercised on data collected elsewhere - other detectors, other
file formats, other conventions. That data was collected and published by
other people, and until now nothing in the repository said so.

NON_SLS_TEST_DATA lists all 51 datasets: the DOI to cite for each, the
repository it came from, and the experiment as deposited. Beamline,
resolution, space group and cell are the values deposited with the
corresponding PDB entry, read from the RCSB data API - not results measured
here; no quantity produced by this software appears on the page. The detector
is read out of the image file instead, because the detector named in a PDB
entry is often only approximate, and the twelve cases where the two disagree
are listed rather than silently reconciled.

ACKNOWLEDGEMENT gains a section for the repositories themselves, with the
IRRMC, SBGrid and Zenodo citations and the PDB citation for the metadata.
Every DOI on both pages was resolved before it was written down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

geometry: hold the detector plane as axis vectors, and give the discrete part its own home ... fed077e683

The detector plane was three PONI angles and nothing else, so the two things it
cannot express - an image mirrored in Y, and one mounted at a multiple of 90
degrees - had no home at all. They are now the DetectorOrientation carried by the
detector setup, composed with the PONI rotation into one orthogonal matrix whose
columns ARE the fast axis, the slow axis and the sample->PONI normal:

    lab = R(rot1, rot2, rot3) * Delta * ( (x-bx)*p , (y-by)*p , distance )

GetFastAxis/GetSlowAxis/GetNormalAxis read those columns and DetectorAxes() sets
the plane from them, decomposing back to the angles; PoniRotMatrix and
PoniAnglesFromMatrix are the conversion in both directions, exact on the canonical
branch (rot2 in [-pi/2, pi/2]) and with a stated convention at gimbal lock. The
angles stay stored rather than re-derived, so a geometry given as angles is
written back as the same angles, to the bit.

Delta is never inferred. In particular an arbitrary rot3 is NOT decomposed into a
quarter turn plus a residual: rot3 is a fitted quantity, and a least-squares step
must not be able to turn the stored image. It is set only where something states
it - the detector setup, --detector-mirror-y / --detector-quarter-turns, or the
value a file this system wrote records - and defaults to the identity, which makes
the whole change a no-op for every existing detector and every existing file.

It is a different setting from DetectorSetup::mirror_y, which flips the MODULE
LAYOUT while an image is assembled and so decides what the stored pixels are.
Merging the two would apply the mirror twice for every modular detector, or change
the pixel content of every file written; both are ruled out. The new one earns its
keep exactly where the old one is a no-op: a detector whose image arrives already
assembled has no layout to flip.

Both generators are signed permutations of the in-plane offset, so they preserve
the distance from the PONI. That is why almost nothing downstream changes:
everything needing an azimuth already goes through LabCoord, and everything that
does not needs only a radius. The two hand-written copies of the rotation -
XtalResidual and RingOptimizer - take the discrete part as four constants next to
cos_rot3/sin_rot3, since it acts in the detector frame where rot3 acts in the
laboratory and cannot be folded into it. RingOptimizer needs it despite being a
radial fit: it fits the tilt, and the discrete part changes which way the tilt
tips a ring.

Carried as two optional CBOR keys and two detectorSpecific datasets, both
back-compatible; the NXmx module axis vectors and the translation direction stop
being hardcoded and are computed from it, reproducing today's values exactly at
the identity. GetPoniRotMatrix is renamed GetDetectorMatrix, because it is no
longer only the PONI rotation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

reader: take the image orientation from the module directions the NXmx file states ... 08f57235d6

NXmx says how the stored image sits in the detector plane, in the NXdetector_module
fast_pixel_direction and slow_pixel_direction vectors. rugnux read neither, and
assumed every detector was mounted the way this system mounts its own. A facility
that bolts its detector a quarter turn round therefore produced a geometry that
was wrong by 90 degrees, which no amount of refinement recovers: the run indexed
nothing and reported that it could find no lattice.

The two vectors are read, taken from McStas into the internal frame, and matched
against the eight discrete image orientations. An exact match is adopted; anything
else is left alone, because an orientation that is not discrete is a continuous
rotation of the detector in its own plane and cannot be told apart from the tilt by
looking at the module. Every file examined here is exactly discrete.

Measured over the 94-dataset battery: 78 masters state no module vectors at all and
12 state the standard ones, so the change can reach exactly 4. Three of those go
from "Nothing was integrated" to a complete merged result with no flags -
43/93/92% indexed, CC(1/2) 0.957/0.984/0.998 - and are the sets that until now
needed a hand-passed --rot3 pi/2. The fourth states a half turn, which the existing
rotation-axis sign rescue already absorbed, and is unchanged to every digit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW

docs: add the eight new public datasets to the non-SLS test list ... bec2a10a3a

Three IUCrData Raw Data Letters (Zenodo, CC-BY-4.0) and five SBGrid Data
Bank depositions (CC0) have been added to the data the pipeline is
exercised on. Same rules as the rest of the page: the source is the
repository and its own citable DOI, every one of which was resolved before
it was written down; beamline, resolution, space group and cell are the
values deposited with the PDB entry; the detector is read out of the image
files. None of the new detectors disagree with their PDB entry.

Two of them do not fit the page's one-row-per-sweep shape, so the shape is
described rather than flattened. The 6R72 Zenodo record holds two complete
360-degree collections on one crystal - a helical one that produced the
deposited structure and a low-dose one that has no PDB entry - and both are
listed, sharing a DOI, with the second in the no-PDB-entry table. The three
CHESS depositions are 4-11 wedges of 50 degrees per crystal plus a rotation
taken with the crystal translated out of the beam, tabulated in a new
section.

One SBGrid deposition is named but not in the table: its images are 1995 CCD
TIFFs, a format the reader does not support, so it is not processed here and
saying so is more useful than leaving it out.

ACKNOWLEDGEMENT gains the new per-repository counts and a pointer to the
Raw Data Letter citations; TESTS points at the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

reader: read a third-party NXmx master, and stop a broken one reading as empty ... c9ca7e424a

A valid NXmx master written outside the DECTRIS toolchain could not be opened. Measured on a
Diamond-written master of a 360 deg EIGER 16M sweep, where the images and the meta file are pure
DECTRIS and only the master is third-party - which is why the two sides disagree on units at all.
Five independent things, of which two were silent:

* The image size came from detectorSpecific/x_pixels_in_detector, a DECTRIS extension rather than
  NXmx, so a third-party writer has no reason to emit it. It now comes from the image array's own
  shape, as it already did for a VDS master.
* Lengths were assumed to be metres and the units attribute was never read. A pixel size, sensor
  thickness or distance stated in millimetres - correct NXmx - was silently a factor of a thousand
  out. The unit is now read; an undeclared one still means metres, an unknown one is refused.
* The detector distance can sit in NXinstrument rather than in NXdetector; that is now the last
  fallback after the NXmx and the firmware-1.x spellings.
* A pixel mask that is an external link into a file not holding it passed the Exists() check and
  then threw on the open. Whether the array is there is now decided by opening it.
* Each data file was re-opened and searched for /entry/data/data, ignoring the path the master's
  own link names. A master linking to a plain /data therefore found no images at all - and that
  was a warning and exit code 0 over a sweep sitting right there, not an error. The link is now
  taken at its word, and a master that links to data files but yields no images is an error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

tests: open a third-party NXmx master, in the shape a Diamond-written one takes ... 8e9ca1f6d2

Covers all five ways such a master differed from a DECTRIS one - lengths in millimetres, no
detectorSpecific, the distance one level up in NXinstrument, a pixel_mask linked into a file not
holding it, and per-file links naming a plain /data - and the reader's refusal to report a master
whose data files cannot be read as a dataset with no images.

Verified to fail on the parent commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

tests: give both beam-centre estimators a case with the detector tilted ... 3ddfbb2da9

No test anywhere set a detector tilt, so PoniRot1/2 were zero in every one of them and the PONI and
the direct beam sat on top of each other. That matters because the conversion between the two is
used three times in the spot estimator - to centre the vote, to start each tooth's refinement, and
to turn the answer back out of the spindle frame - and with the two centres coincident it is the
identity, so its sign was unobservable. Verified by flipping it: with DirectBeamOffset negated, the
ten pre-existing beam-centre cases all still pass and only the new one fails.

The tilt used here puts the direct beam about 12 px from the PONI, twenty-four times the tolerance
asserted, and the case also pins that the tilt is not read as a spindle azimuth - what the fit sees
of the detector belongs to the detector.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

beam centre: run the four consistency restarts at once ... 97f57a430d

FindBeamCenterFromSpotSymmetry calls Estimate() five times - once for the answer and once from each
of four starts 25 px away - and three quarters of each of those is an 861-point brute-force grid
over the spindle. So making the uncertainty gate live was paid for by multiplying the estimator by
five, which is the whole of the pre-scan's cost.

The four restarts share nothing: each takes its own copy of the geometry and only reads the spots.
What is wanted from them is a max, which is order-independent, so running them concurrently gives
the same number. Measured on three rotation crystals, the committed centre, the reported sigma and
the fitted spindle angles are unchanged to every printed digit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: say what the file's beam centre looks like, and when no beam stop is found ... d1818b195e

Four things a run can say about the beam centre before it reads a single frame. All of them are
free, so none of them is behind a flag.

Two are worth acting on. A centre that lands on a masked pixel is wrong about something - a real
beam does not sit on a dead pixel or in a module gap - and over the 29 non-SLS datasets whose
outcome is on record it fires on 4, of which 3 went wrong, against 2 false alarms in 42 in-house
masters. And a beam-stop pre-scan that finds exactly zero pixels is not a beamline without a beam
stop: the flood starts from seeds within 4 px of the ASSUMED centre, so a centre far enough out puts
every seed in a module gap. That one fires on 3 of 29 foreign datasets, all 3 of which had a bad
outcome, with no false alarm in 39 in-house sweeps.

The other two are provenance and are said rather than warned about: a centre equal to the geometric
detector centre, or a whole number of pixels in both coordinates, was typed rather than measured.
They are true of 29 of 42 in-house masters and those are out by a median 3.5 px, but as predictors
of harm they sit at or below the base rate, so they gate nothing. They earn their line because the
spot-symmetry search reaches only about +-55 px: a placeholder tens of pixels out is unreachable by
construction, all five of its searches agree, and the sigma comes back small.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

beam centre: gate on what the geometry needs, and on the move being significant ... 52e9e9da2b

The accept bound was one flat 1.0 px constant applied at three points. It was calibrated on a
244 mm / 0.95 A / 0.15 mm geometry, and the displacement the J0 smearing law actually allows runs
from 0.6 px to 9 px over the in-house and non-SLS corpora - a fifteen-fold spread - so on a loose
geometry the constant is nine times too tight and throws away answers that are perfectly usable.
The bound is now the LARGER of the constant and what the geometry asks, which is one-sided by
construction: nothing today's gate accepts can be lost.

What the geometry asks is printed and never tested against. It says how wrong the header may be;
the estimator's sigma says how well the estimator knows its own answer, and gating one on the other
rejects a centre correct to 0.03 px because its sigma was 0.98. It is not a relevance floor either
- measured on real data, a 0.12 px change of centre, 0.03x of what the law asks across the spindle,
is the difference between the deposited cell and a halved axis - so the printed line says so.

Added alongside: the move has to be worth making. Under three times the estimator's own sigma it is
not a measurement of anything, and a centre that is not moved cannot push the two-pass loop off its
fixed point. On the 39 rotation regression crystals this adopts 38 and keeps the header on one.

Two things that were happening silently now say so: the move resolved across the spindle, which is
the only component the law is about, and the fall-through itself. Below about 220 deg of sweep the
spot symmetry never clears the bound and the background answers instead - correctly, to within
0.23 px of the full-sweep centre at every span from 120 to 220 deg - and until now the log did not
say the spot arm had even been tried.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: name the beam centre when two candidate cells are axis harmonics ... 4d2bb98a13

The first pass already has the evidence and says nothing about what it means. When two schemes come
back with primitive volumes related by a small integer - or by sqrt(3), which is the hexagonal
harmonic and is not a whole number - one of them is the other's axis harmonic, and what decides
between them is the beam centre to a fraction of a pixel.

The mechanism is that the J0 law is about the FFT AMPLITUDE, which is translation-invariant. A
centre error along the spindle translates the derotated cloud rigidly, so the transform cannot see
it, the peaks stay sharp, and the lattice fit that follows - whose origin is the beam, and which is
not translation-invariant - commits with confidence to a sub-multiple. Measured on a deposited
dataset: 0.12 px of centre, 0.09 px of it across the spindle against a 2.78 px need, is the whole
difference between the deposited cell and a halved axis, and the halved one is the answer that
indexed MORE frames - 100.00 % against 99.23 %.

So the message is deliberately not conditioned on any indexing-quality signal, and it is taken over
the whole pair rather than only over the tie: on this failure the frame count points the wrong way,
which is exactly why it cannot arbitrate. Nothing is decided differently; the integer-subcell
tie-break below is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: try the beam centre as an indexing hypothesis after a failed first pass ... a2f451cf12

A beam-centre error is not repairable downstream. It is fixed in the lab frame, so accumulating a
sweep smears every reciprocal-lattice point around a circle and the FFT amplitude at an axis of
length a is multiplied by J0(2 pi delta p a/(D lambda)); past the first zero the true axis is gone
and its harmonic wins. But it is decidable from the data, on exactly the count the scheme choice
and the axis-sign rescue already use: the right centre indexes and the wrong one does not.

So after a first pass that indexes fewer than half the validation frames, step the centre a pixel
at a time and keep the first rung that clears the same majority the other guards test. It runs only
after a pass that has already failed, so a run that works never pays for it, and spot finding is not
repeated - the cache is keyed by image and the spot positions do not depend on the centre. On the
prototype this rescued 2 of 12 failing non-SLS datasets and declined cleanly on the other 10.

BOTH detector directions are searched, and that is the part to keep. Measured by injection on two
crystals: across the spindle the cell stays right and the indexed fraction collapses, 99 % to 25 %,
so that direction announces itself; along the spindle - the one the J0 derivation calls free,
because the FFT amplitude is translation-invariant - the run holds 96-100 % indexed and quietly
adopts a 2x, 3x or sqrt(3) supercell. Searching only the perpendicular direction would search the
failure that already shows.

The step is one pixel, flat. Deriving it from the J0 law was tried and is wrong: the only cell
available at that point is the one the FAILED pass returned, and on a dataset whose failed cell was
a small spurious sub-cell the formula asked for a 6 px step, which steps over the lobe it is looking
for. Taking any improvement rather than requiring a majority was also tried and is wrong: where no
centre works, the ladder wandered to the far end of its range on 10/60 against 5/60.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: say where the scattered background puts the beam, on every run ... 0644404286

A beam centre in a file is the metadata field everyone jokes about and nobody measures. rugnux
already has an estimator that can measure it - the isotropy of the scattered background, whose
leverage is the curvature of the water ring - and it already builds, on every run, the projection
that estimator needs: --detect-beam-stop makes one to find the shadow. So the measurement is a fit
over a mean image that already exists, no frames of its own, and it can simply be done always.

--beam-center-check, on by default, does that and reports it: what the file claims, what the
background says, how far apart the two are, how well the fit knows its own answer, and how right
this particular geometry needs the centre to be. That last term is what makes the line a verdict
rather than a pixel count - 3 px is nothing at 100 mm and a lost lattice at 500 mm. On the two
public depositions in the test corpus whose header is the geometric centre of the detector, the line
reads 73.49 px and 70.96 px against a 1.57 px and 1.82 px tolerance, on runs that until now said
nothing at all about it.

Nothing is committed. The run keeps the centre it was given, and where the fit cannot place the
centre - a flat background has nothing to separate a radial shift from an amplitude - it says so
rather than guessing. Measured on a 0.3 Mpx detector the fit takes 8 ms.

The fall-through in --estimate-beam-center now reads this same result instead of repeating the fit;
it is the same call on the same projection with the same mask, so that flag's answer is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: index the measured beam centre too, and say when the two answers differ ... 60f6a46b25

The estimate the previous commit makes is free, and a first pass re-uses spots it has already found,
so running the first pass a second time at the measured centre costs about what one rung of
--beam-center-search costs. That buys the one comparison an on-failure trigger structurally cannot
make. A centre wrong ACROSS the spindle announces itself - the indexed fraction collapses - but a
centre wrong ALONG it does not: the run holds 96-100 % of frames indexed and quietly returns a 2x,
3x or sqrt(3) axis harmonic. Nothing fails, so nothing fires. Indexing both centres and comparing
the two lattices is what can see that at all.

What must not arbitrate the two is the indexed frame count. Acceptance is a fractional-Miller test,
so a cell twice as long has to place every spot twice as accurately to score the same; measured on
real data, two centres 0.12 px apart gave the deposited cell at 99.23 % and a halved axis at
100.00 %, and the wrong answer indexed better. A rule of the form "take the centre that indexes
more" picks wrong in exactly the case the comparison exists for.

So only what needs no arbiter is decided. The file's centre indexing nothing where the measured one
indexes a majority is not a comparison, it is a run that produced nothing and now does: take the
measured centre. The two agreeing on the lattice is reported and nothing else - it is a free
statement that the header is good enough for this crystal, which is most runs and is worth saying.
A disagreement between two passes that both worked is reported with both cells, their primitive
volumes and the ratio, flagged when that ratio is an axis harmonic, and left undecided.

Runs before the blind ladder, so a measured hypothesis is tried before any grid, and only where the
move exceeds three times the fit's own sigma - below that the two centres are one measurement twice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: ask the rotation-axis sign at the measured centre, and say when only the symmetry differs ... d8816169e1

Two corrections to the check, both from measuring it on the non-SLS corpus.

Where the file's centre indexes nothing and the measured one does not either, the rotation-axis sign
is now asked again AT the measured centre. The two unknowns are discrete and coupled: on the two
public depositions in the corpus whose header beam centre is the geometric centre of the detector,
the file is 73 px out AND its axis sign is the opposite of the one that indexes. The sign rescue
therefore asks its question at a centre 73 px wrong (0/60) and puts the sign back, and the centre is
then asked at the wrong sign (0/60). Each error hides the other and the run produces nothing. Asking
the pair takes both runs to 100 % indexed on the deposited lattice - 199.40 67.11 against a
deposited 199.54 67.15, and 208.77 67.20 against 208.77 67.22 - at the cost of one more first pass
on a run that has already failed twice.

Asked here rather than by moving the check ahead of the sign rescue, which was tried: that also
works, but it moves the beam centre of three datasets whose only fault is the axis sign, for no
benefit. Where nothing works the file's centre is put back, so a run that fails for another reason
fails at the geometry it was given - 6yqf, whose spindle is along the detector's slow axis, and
7atg both come out exactly as they did before.

And the beam centre decides the primitive VOLUME, not the Bravais class: a centre error along the
spindle makes the FFT take an axis harmonic, which changes the volume by an integer factor or by
sqrt(3), while a class differs for a reason that has nothing to do with the centre. The two are now
reported separately. Both "disagreements" in the corpus are of the second kind - primitive volumes
agreeing to 0.16 % and 0.57 % while one pass reads R-centred trigonal or P orthorhombic and the
other stays triclinic - and in both the file's centre is the one that finds the symmetry and the
run's cell matches the deposited one to 0.23 % or better. The cell was never in question, and the
strong warning now fires zero times in 32 datasets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: index both beam centres always, without a significance gate first ... bf41b1fab1

The second first pass was skipped where the measured centre sat within three times the fit's own
sigma of the file's - a quarter of the non-SLS corpus, eight of thirty-two runs. That is a decision
taken at a threshold before the evidence is in, and the evidence here costs a median 0.55 s: the
spots are already found and cached, and their positions do not depend on the centre, so a second
first pass is a median 20 % of what the first one costs.

Worse, the gate removed exactly the case worth asking about. A centre error ALONG the spindle does
not fail - it holds 96-100 % of frames indexed and quietly returns an axis harmonic - and the scale
it turns on is a fraction of a pixel: measured on real data, 0.12 px of centre is the whole
difference between the deposited cell and a halved axis. No sigma small enough to gate on would
notice that, so the gate could only ever hide the question, never answer it.

The three-sigma test stays where it belongs, in the report: it says whether the difference is a
measurement or noise, which is what a reader wants to know. It no longer decides whether to look.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: write the unmerged MTZ by default ... 5f47d73cc1

The unmerged export is how a run's observations reach the rest of the field. It is what aimless,
pointless and careless read, it is what a head-to-head against another program's answer runs
through, and handing several of these files to pointless is the only way to merge sweeps rugnux
does not combine itself. Behind a flag it reached only the people who already knew the flag
existed, which is the shape of defect the "rugnux <file> with nothing else" direction asks to
design out. --no-export-unmerged turns it off, and the regression batteries now pass it: the
objection to defaulting it on was their disk and time cost, not the product's.

Measured on three rotation crystals of 900, 1800 and 3600 frames: the file is 21, 86 and 26 MB -
two to five times the merged .mtz, .cif and .hkl put together - and the write costs 0.24, 2.29 and
1.11 s, 5.5 %, 12.2 % and 3.1 % of each run's own wall time. The cost is linear in the number of
observations and independent of how long processing took, so it is the largest fraction of the
runs rugnux finishes fastest, not of the longest sweeps. A run's .hkl, .mtz and .cif are
byte-identical with the export on and off.

--export-unmerged-partials stays off: it is a second, larger artefact and a separate question.

The viewer's reprocessing dialog ties the export to its merged-output switch, so it appears beside
the merged files rather than beside a job that only asked for the per-image _process.h5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

reader: place a detector swung out on a 2theta arm where the file says it stands ... 5c44e544dc

Chemical crystallography reaches high angle by swinging the detector out on a
2theta arm. Both readers had the number and neither used it: the miniCBF header's
Detector_2theta was parsed into a struct member nothing ever read, and on the NXmx
side the rotation was in the depends_on chain, which was not followed at all. A
sweep taken at 30 degrees was therefore processed with its detector plane 30
degrees from where it stood, and nothing indexed.

The geometry could already express it, and needed no change: the arm turns the
detector about the sample, so the distance is still measured along the detector
normal and the beam centre is still the point of normal incidence - which is
exactly the PONI convention, and a swung detector is one PONI rotation. What moves
is the direct beam, by distance*tan(2theta), off the beam centre and often off the
detector.

NXmx is the harder half, because the swing has no field of its own: it is one
rotation in the chain the detector's position depends on, and "two_theta" is only
one beamline's name for that dataset. So the chain is followed and its rotations
composed, rather than a field of one name being looked for - each transformation
states its vector in the frame of the one it depends on, which is why the product
is the whole placement. Translations are skipped; they are the distance and the
beam centre, which the file states separately in the square-on frame. Vectors come
from McStas through the same 180-degree turn about z the module directions already
use, a proper rotation, so an axis carried through it turns the same way.

The three rotations a file this system writes ARE that chain, and are also read as
the PONI angles - so those three paths are skipped, or every tilted file we have
ever written would come back tilted twice. That is the one way this change could
have broken existing data, and the test for it writes a tilted file and reads it
back.

For miniCBF the arm turns about the base spindle axis: on the four-circle geometry
those headers describe the two are one axis, and the imgCIF axis table such a
header carries states them with the same vector. Both now come from one constant,
so a later correction to the frame moves them together.

Measured. On a swung NXmx sweep the chain gives rot2 = -0.34907 rad for the 20
degrees it states, and the sweep goes from "nothing was integrated" to 25000
reflections at 82.2% completeness and CC(1/2) 0.9993, in the same space group and
the same cell to 0.03 A as the square-on sweep of that crystal; the opposite sign
indexes nothing. A miniCBF sweep at 30 degrees goes the same way, to 0.585 A, and
a second sweep of that crystal at 55 degrees reaches 0.476 A and reproduces the
cell again - with a low-resolution limit of 2.36 A rather than 13 A, which is what
a detector swung that far records. On all of them post-refinement recovers the
header's own beam centre and distance, and the beam stop shadow sits within four
pixels of where the swung geometry puts the direct beam, 417 and 537 pixels from
where the unswung one does. Seven sets whose detector is square to the beam, three
of them carrying a chain whose 2theta is zero, are byte-identical in .hkl, .mtz,
.cif and the image statistics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: keep the P1 merge, so a wrong space group is recoverable ... dc71cb7299

A de-novo rotation run merges in P1 to search for the symmetry, adopts a
group, and then overwrites that merge with the in-symmetry one. If the
adopted group is wrong the user has no route back: every file the run
wrote, and every statistic in them, is computed in the group that was
assumed, so nothing in the output says the choice was wrong and the only
way to a different answer is to process the images again.

Merge the same integration once more in P1 after the run's own files are
written, and put it beside them as <prefix>_P1.mtz. From it the space
group can be re-determined and the data re-merged, re-solved or
re-refined in any subgroup - measured end to end on three crystals:
POINTLESS reads the file, recovers the group, and AIMLESS re-merges it,
reproducing rugnux's own merged intensities at CC 0.9965 where the two
agree. On one of the three it recovered the deposited/XDS group where
this run had under-called the screw axis.

The merge is the full one - correction surfaces fitted, ice rings and
near-tangential observations kept, whole resolution range - not the
deliberately degraded merge the space-group search itself runs on, and
it is what `rugnux --mode scale -S P1` produces from a _process.h5. That
route already existed but needs a _process.h5, which a merging run does
not write, so it only helped a user who had foreseen the problem.

Every de-novo rotation run writes the file, including one whose search
concluded P1 and where it therefore repeats the merged output byte for
byte. Whether a file exists must not depend on what the pipeline
decided: a script harvesting results would otherwise have to reproduce
the search's decision to know whether to expect it, and a missing file
would not separate "the run chose P1" from "the run failed".

A user-fixed -S writes nothing, and that condition is not a pipeline
decision. With a group fixed, prediction rejects that group's centring
absences (IndexAndRefine.cpp:499-506), so those reflections are never
integrated; a P1 merge built from such a run would be missing whole
centring classes and would mislead rather than merely be smaller.

Cost, median of five paired runs read off the log timestamps (the box is
shared, so end-to-end wall time is noise): +0.32 s of 4.2 s, +2.18 s of
31.4 s, +0.19 s of 17.9 s, +0.66 s of 15.9 s - 1 to 8% of a run. The
file is 1.6x the merged MTZ on a monoclinic crystal and 30x on a cubic
one, where the merged MTZ is tiny; it is well under the unmerged export
in every case measured, and 0.02 to 0.6% of the raw dataset.

Rotation only for now: the stills merge re-fits per-image scales and
per-reflection partialities onto the integration outcomes, and
_unmerged.mtz is written from those afterwards, so on stills this extra
merge alters a file that is the run's own output. Fixing that means
running the cross-check below the unmerged export, which needs the merge
lambda hoisted out of its block; deferred, since rotation is what the
online pipeline processes. On rotation the merged .mtz, .cif, .hkl,
_image.dat, _unmerged.mtz and _unmerged_partials.mtz are all
byte-identical with and without this change, including on a run whose
search returns P1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

reader: take a miniCBF's mounting from the imgCIF axis table its header states ... a27c4cf26f

A miniCBF header states three things about how the instrument is put together
that the reader was assuming instead: which laboratory direction the image's
columns run along, which its rows run along, and which the spindle turns about.
Some beamlines append a CBF template block holding the full imgCIF axis table,
which says all three outright.

Two instruments in the corpus are not what was assumed, in two different ways.
One mounts its detector a quarter turn round, so the image's columns run
vertically. Another turns its spindle about the VERTICAL, with the image mounted
the usual way; its table says so, and its "# Oscillation_axis" line says so a
second way, by naming the image direction the spindle runs along rather than a
vector. Either error leaves the spindle 90 degrees from the image. That is not a
sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers
it: all three affected sweeps indexed nothing usable.

So the table is read. The element axes give the image orientation, matched against
the eight discrete mountings exactly as the NXmx module directions already are -
the match itself moves to DetectorOrientation, so both readers share one
definition rather than two copies. The goniometer axis with no parent gives the
spindle DIRECTION; its sign stays the rescue's business, which is the part a
convention can legitimately differ on. The detector axis with no parent gives the
2theta arm, replacing the assumption that the arm shares the spindle's axis - the
one header stating both states them with the same vector, so this changes no
answer, only what it rests on. imgCIF's frame differs from the internal one by a
half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already
records from the other side.

Where a header carries no table, a "+SLOW" on the Oscillation_axis line still
says the spindle runs along the image's slow direction. That is the only thing one
of the three affected sets says about it. The axis NAME on that line stays
unusable - the header that carries both says "X.CW" where its own table says Y -
but the direction token is not: where both are present they agree, which is what
makes reading it evidence rather than a guess.

Also: naming a frame with no directory at all now finds its sweep. parent_path()
of a bare filename is empty and iterating an empty path finds nothing, so running
from inside the data directory reported that no images were found.

Measured, with nothing on the command line. The vertical-spindle protein set goes
from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from
deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its
companion from the same detector, which has no table and only the +SLOW token,
goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the
right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The
quarter-turned set's three sweeps, at three arm positions, now all index without
the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six
miniCBF sets that state no table and no +SLOW - including one whose
Oscillation_axis line names an axis in a third dialect - are byte-identical in
.hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the
shared orientation matcher moving nothing on that path either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

docs: write down the three directions this work is heading in ... 7528bd8761

Three preferences that have been decided in conversation and applied to
several changes already, but were nowhere a new reader would find them.

They are directional, not absolute. Spending compute to buy quality is
now affordable where it once was not, so the question to ask of a slower
design is what it buys. Deciding late beats deciding at a threshold,
because a gate calibrated on one population refuses another - measured
repeatedly in the lattice and symmetry code. And the target is a bare
invocation with no flags, because a capability nobody knows to switch on
has not solved the problem.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

reader: read the detector arm and the mounting each file states ... 65dfdb1324

A detector swung out on a 2theta arm was read as if it stood square on,
and a miniCBF written with a vertical spindle or a quarter-turned image
was read with the standard mounting assumed. Both are stated in the file
and both were ignored: the NXmx depends_on chain was never followed, and
Detector_2theta was parsed into a field nothing read.

Recovers three datasets that produced no usable lattice, and the
small-molecule sweeps at 30 and 55 degrees.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

beam centre: measure it on every run, and try it when the header fails ... f75cf72d1a

The header beam centre is wrong by more than the geometry absorbs on
two thirds of foreign depositions, and nothing measured it. The solvent
ring already gives it away: the background projection the beam-stop
pre-scan builds is enough to fit the centre, so the measurement is a fit
over an array the run has already paid for.

It is reported on every run and committed on none. The first pass runs
again at the measured centre and the two lattices are compared; the
measured centre is taken only where the header indexes nothing, and a
disagreement is reported rather than resolved, because the only arbiter
available at that stage is the frame count and it is inverted.

Recovers two depositions whose header is the geometric detector centre.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

rugnux: write the unmerged MTZ by default ... 123dc22be2

A run that does not ask for it still has to hand its data to aimless,
pointless or careless eventually, and the file it needs is one a user
had to know a flag to get. It is the largest a run produces, but it is
3.4 to 5.8 per cent of the images it replaces - and the better it is,
the less anyone needs to keep those.

The batteries pass --no-export-unmerged: they process the whole corpus
and would write a few gigabytes nobody reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

# Conflicts:
#	docs/CHANGELOG.md

rugnux: also merge in P1, so a wrong space group is recoverable ... d5818713cb

A rotation run that determines its own space group now also writes
<prefix>_P1.mtz: the same observations, the same scaling, merged in P1.
If the group was wrong there is no route back today except processing
the images again, and the P1 data settle it - re-merge, re-solve or
re-refine in any subgroup.

Written whether or not the search adopted P1, because a file whose
presence depends on what the pipeline decided cannot be harvested by a
script. A run given -S writes nothing: its group's centring absences
were never integrated, so the P1 merge would be missing whole classes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

# Conflicts:
#	docs/CHANGELOG.md
#	rugnux/rugnux_cli.cpp

calibration: an option to hold the detector tilt fixed ... 03456dcddd

--mode calibration fits five parameters - beam centre, distance and the two
PONI tilts - and a program that cannot express a tilted detector has nowhere to
put the last two. Dropping them after the fact is worse than never fitting
them: the centre and the distance of a tilted fit have already absorbed the
tilt, so the flattened geometry is right nowhere.

rugnux --no-refine-tilt, the "Refine detector tilt" tick box on the viewer's
Calib page and RingOptimizer's refine_tilt argument hold rot1/rot2 at the value
the geometry came in with and fit the remaining three. That is the best
flat-detector answer, and the one such a program would refine to itself.

Measured on a five-distance calibrant series. At short distance the tilt is
real and reproducible - three independent fits agreeing to 0.01 deg, radial rms
1.4 -> 0.4 px - and its direct beam agrees with the background beam-centre
estimator to 0.05 px, so the tilted model is the physically right one. The
pinned fit then displaces the centre 2.6 px to absorb the tilt and lands within
0.03 px of the same place at every distance.

Past ~300 mm, where only two rings reach the detector, the tilt is instead
under-determined: it comes out with the opposite sign to every short-distance
fit and drags the PONI 28 px while the rms does not move (0.960 against 0.962).
The existing degeneracy guard only fires on a strictly single ring, so it does
not catch that; declining a tilt that does not pay for itself in rms is left
for a separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5

docs: the open test battery, updated - 82 datasets, and not only non-SLS ... 9924dd9fc3

The public data the pipeline is exercised on has grown from 59 datasets to 82,
77 of them with a released PDB entry and released structure factors, so the
page that credits the depositors and carries the DOI to cite for each is
brought up to date with what is actually run.

Renamed from NON_SLS_TEST_DATA to EXTERNAL_TEST_DATA, because the old title
stopped being true: a few of the sets were collected at SLS beamlines, where
the data are still written by someone else's detector and someone else's
acquisition system. What the battery tests is foreign files, not a foreign
facility.

Also rewritten from the current archives rather than the earlier sample:

- Multi-collection archives: eleven are not a single continuous rotation, not
  four. Seven IRRMC archives hold more than one collection; one sweep is kept
  in six of them, and both are kept in the one whose two sweeps are at
  different wavelengths. A repository project page is not a reliable guide
  here - one describes a 900-frame sweep its own tarball does not contain.
- Detector labels: 76 rows can be compared against the PDB entry. Seven
  genuinely conflict, and one of those the file settles outright - pixel
  count, pixel size, sensor thickness and firmware string agree with an
  EIGER2 9M against the entry's PILATUS4 4M. A further 29 differ only in how
  much they state, which is not a conflict.
- One archive ships 30 placeholder files named like images that are 64-byte
  text; named on the page so a reader that globs the directory is not
  surprised by them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5

docs: bring the changelog and the method notes up to rc.166 ... fb18457e6f

The rc.166 changelog was missing fourteen user-visible changes and carried
rationale and measurements that belong here instead. Added: the native miniCBF
sweep reader, the third-party and firmware-1.x NXmx masters, the plain LZ4
filter, the image orientation taken from the file's module direction vectors,
the two beam-centre flags and their rescues, and the FFT reach past 500 A.
Trimmed the rest to one line each, moving the numbers out of the user-facing
file.

RUGNUX.md described the input as a single Jungfraujoch master file, which it
has not been since this branch; it now covers the foreign and legacy masters,
the accepted compression filters and the miniCBF sweep, including how a sweep
is collected from one named frame. Six options existed with no entry in the
table - --beam-center-check, --beam-center-search, --fft-min-unit-cell,
--min-indexed-spots, --rot3 and --no-p1-crosscheck - and -C now moves both FFT
cell bounds, which was not written down anywhere.

CPU_DATA_ANALYSIS.md carried two statements this branch made false: 7.5 still
said pass 2 reuses pass 1's space group, and 13.1 still said centrings are
ranked by net absence count. Both now describe what the code does - the group
is determined after pass 2, and centrings are ranked by the same Beta-tail
likelihood the screw test uses. Also documents the per-zone screw scoring, the
coplanarity volume-fraction guard, the plane-normal transform, the FFT cell
bounds and the twelve refined candidates.

The miniCBF reader implements the x-CBF_BYTE_OFFSET scheme and reads the imgCIF
axis table from the specification alone. No CBF code is vendored or linked, so
there is no licence obligation, but reimplementing a published specification
carries one of credit: ACKNOWLEDGEMENT.md gains a section and the two
algorithms carry a one-line reference each. Both DOIs were resolved before
being written.

Rottger's initial was wrong where this branch first cited it - K, not A.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5

reader: a link to a file that is not there is not a dataset that exists ... 6ed4ea541e

Every DECTRIS Eiger master links saturation_value, pixel_mask,
bit_depth_readout and serial_number into a companion <prefix>_meta.h5, and
that file is routinely not kept when a dataset is archived or deposited. The
existence test asked only whether the LINK was written, which it is, so every
optional-field guard in the reader answered yes and the read that followed
threw. A deposited Eiger 16M set could not be opened at all, over values the
reader was perfectly prepared to do without.

Exists() now asks the second question too - whether the object the link names
can be reached - so an orphaned link reads as absent and the fallbacks behind
it do their job.

The saturation value is then allowed to be missing outright, because on such a
file it is: neither the NXmx name nor the DECTRIS one is readable, and there is
no third place to look. Left unset, GetSaturationLimit() falls back to the
container's own overflow. That is the safe direction - it can only fail to call
a pixel saturated, where too LOW a value drops the whole reflection and
silently removes the strongest data - and the run says out loud that nothing
will be called saturated.

The set that could not be opened now processes to 2.17 A against a deposited
2.40 A, in the deposited space group, with a cell agreeing to 0.08%. Output is
byte-identical on datasets that already opened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 0637979f6d95b446406ab70d1f3982841d195b36)

symmetry: carry the space group as the group, not as its number ... 26fc4b02b3

The adopted space group travelled the pipeline as a bare int and was rebuilt
downstream with find_spacegroup_by_number, which returns the reference setting.
So every setting a number cannot name was destroyed one line after it was
determined: P 1 1 2 came back as P 1 2 1, I 1 1 2 as C 1 2 1, R 3:R as R 3:H.

DatasetSettings now holds the gemmi::SpaceGroup itself, DiffractionExperiment
exposes it as GetGemmiSpaceGroup() / GetSpaceGroupOrP1(), and everything that
used to take an int - HKLKeyGenerator (its int constructor is gone, so the
compiler finds the callers), the merge, the R-free flags, French-Wilson, the
reindexing ambiguity, the completeness enumeration, the MTZ and mmCIF exports,
the model validation - takes the group. -S keeps the setting the symbol names
rather than reducing it to a number.

The end message carries both spellings and a reader prefers the name, since
only the name keeps the setting while the number is what a reader written
before the name understands. It carries them over CBOR too: the determined
group was never serialised at all, so a group rugnux chose reached the master
file only when the same process wrote it, and an online writer fell back to
whatever the user had supplied at the start. Both keys are optional additions,
so an older reader skips them and a newer one reads an older sender.

On disk the master's /entry/sample/space_group carries the extended
Hermann-Mauguin name and is what the reader takes the group from, so a setting
survives a _process.h5 and the --mode scale that re-reads it; the number stays
beside it and is the fallback for files written before. Every one of the 230
reference settings the old writer could produce reads back as itself, so older
files are unaffected.

Stage A and Stage B of the search still enumerate reference settings only, so
this determines no group differently today - it is what the enumeration needs
before it can be widened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

symmetry: enumerate the settings the cell can host, not only the reference ones ... 493be2cf19

The space-group search offered a candidate only if gemmi calls it the reference setting. A setting
is a statement about direction, so that restricted the search to the axes the convention chose: a
crystal whose 2-fold lies on c had no rung between P1 and 222 and fell to P1, and one whose screws
lie on b and c was reported as the group with a single screw on c - the wrong group, not a lower
one, because the candidate that predicts a subset of the real absences and nothing else wins on no
evidence at all.

Both stages now enumerate more, under refusals rather than thresholds.

Stage B offers the non-reference settings of the chosen point group. Their rotation set is equal to
the chosen one, not merely contained in it, so this cannot raise the symmetry; what it adds is a
screw or a centring on the axis the data show it on. A candidate is offered only if the cell's own
metric admits the rotations its setting names, and one predicting exactly the absences another
candidate already predicts is dropped as the same hypothesis under a second name. A non-reference
candidate whose centring class this merge does not contain is refused outright: the reference path
may adopt an untested centring because the caller's centred-lattice re-test backs it, and a
non-reference setting has no such backing.

Stage A offers the rotation sets no reference setting carries - the a-unique and c-unique monoclinic
2-folds, and the two rhombohedral-axes trigonal groups - and only those, so every point group
reachable before is still reached by the same group in the same setting. A rung reached only that
way may be ADOPTED but does not judge anything else: it is skipped when the reference chi^2 is
formed and when a higher promotion's parents are collected. Without that it made higher promotions
strictly harder - a promotion answers to the most damning of its parents, and offering two more
order-2 subgroups of 222 refused 222 and 432 on crystals that had them.

The run report gains SPACE_GROUP_NAME beside SPACE_GROUP_NUMBER, since the number alone does not
say which axes a group's symmetry lies on, and a group that is not a reference setting is printed as
its extended Hermann-Mauguin name. The two-arm reconciliation compares point groups by that name
rather than by number for the same reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f

symmetry: say when the cell metric hosts more symmetry than the group adopted ... d35e8f680f

Every symmetry under-call in the corpus has the same signature: a lattice whose
metric carries rotations the adopted group does not, with nothing in the run
saying so. The user is left to notice that a P1 answer sits on a cell whose
axes are equal and whose angles are 60 degrees.

So a run that determines its own group now compares the two and says what it
sees. It decides nothing - no threshold, no promotion, no demotion, no
reprocessing - and the message says as much, because a pseudo-symmetric metric
is ordinary and only the intensities can settle whether the extra rotations are
real. Measured over 83 crystals it fires on 19 and covers 6 of the 7
point-group under-calls; on the crystal whose deposited group is F 4 3 2 and
which rugnux reports as P 1, the line reads "the cell metric is cubic - it
admits 24 rotations where P 1 has 1".

The metric symmetry is Le Page's, taken through GEMMI's implementation, at an
obliquity of 1 degree - the middle of the band over which the answer is stable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

symmetry: take the point group from whichever merge found it, the absences from all observations ... b31fb0a55c

The search runs on two merges - one holding every observation, one holding only
the well-measured ones - and the second was allowed to report but never to
decide. That cost real symmetry: over the corpus the two arms disagree on eight
crystals and the arm that found the LARGER point group is right on all eight. A
promotion is refused by the most damning statistic it can be shown, so only one
of the two arms has to be starved for a genuine operator to be thrown away, and
which arm that is depends on the crystal.

But the two halves of the answer do not come from the same place. Absences live
in the weak reflections the filter discards, so adopting the filtered arm whole
recovers the point group and loses the screw - a P6_5 crystal comes back as P6.
So the arms are split rather than ranked: the point group is taken from
whichever supports the larger one, and its absences are then judged here, on
every observation. SearchSpaceGroupOptions::fixed_point_group was written for
exactly this and had no caller.

Measured on the deposited arm: a trigonal-3 answer becomes P 6_1 against a
deposited P 6_5 (an enantiomorph pair, so correct), and a monoclinic answer
becomes P 2 2 2 against a deposited P 2_1 2_1 2. The screw survives the split,
which it did not when the filtered arm was adopted whole. No crystal that was
already right moved, and the over-call column is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

lattice: judge a structurally-zero scalar product against the size of the cell ... 931d7acc7c

The Niggli type of a reduced cell is the sign of its three scalar products, and
gemmi's reduction was asked to decide those signs against an absolute tolerance
of 1e-9 while the products themselves are 10^3 to 10^5 A^2 and the cell is held
in float. A product that is structurally zero therefore arrives carrying about
1e-4 A^2 of rounding and is read as definitely signed. The reduction lands on
the wrong side of the type-I/type-II boundary, the character written for the
other side matches nothing, and the lattice comes back with less symmetry than
it has.

It is not a corner case. Take an exactly body-centred tetragonal lattice with
c > a*sqrt(2) and merely ROTATE IT IN SPACE: 38 of 60 rotations lose the
4-fold, and it comes back C-centred. Every lattice this code classifies is a
refined, rotated one; the two tetragonal-I cases already in the tests are
axis-aligned, which is the one corner where the rounding vanishes.

The tolerance is now scaled by the cell's own magnitude. Over 14845 exact
lattices that puts it on a plateau three decades wide - demotions 461 -> 31 -
with the over-call count unchanged at every point of it, and with the three
fixes that follow the plateau is four decades wide and the demotions are zero.
The constant is 100x the one Grosse-Kunstleve et al. give because theirs is
calibrated for a double-precision cell and this one is float: measured, their
value recovers 6% of these lattices and this one 93%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit b82f217719450e1e46e94be5b5a43ed8ffbf2ca4)

lattice: restore the character that names a centred monoclinic mI reduced form ... 527a5187f4

ITA character 43 was absent from the table. It is the type-II reduced form of a
centred monoclinic lattice with no length equality - mC, mI, mA and mF are one
Bravais lattice in four settings, and this row names the one whose conventional
cell comes out I-centred. With the row missing such a cell reaches character 44
and is reported as triclinic, losing its centring outright. It accounted for 30
of the 31 demotions left after the two fixes before this one, and for 4.1% of
random centred-monoclinic lattices.

Both of its conditions are equalities on scalar products - International Tables
gives them as 2|D+E+F| = A+B and |2D+F| = B - so the three angle tests are
vacuous for it and the second equality is what selects it. cond_2DF was declared
in the character struct and never tested against anything, because until now no
row used it.

Audited over 14000 exact lattices, the row fires 35 times: 32 are centred
monoclinic lattices it recovers correctly and 3 are triclinic cells it promotes,
all three of which Le Page promotes at the same tolerance. No over-call is
attributable to it. The row is taken from International Tables rather than
derived - no single integer matrix of determinant 2 covers more than 68% of
these cells - and the test checks the conventional cell it produces has two
right angles and twice the primitive volume.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 4c2baf0cc3b2177196a9448df06e24b5446de96a)

lattice: transpose the change of basis to a primitive cell, as everywhere else ... e0ff51e2a0

gemmi states centred_to_primitive as an operator on COORDINATES and
CrystalLattice::Multiply combines BASIS VECTORS, so the matrix has to be
transposed on the way in - as it already is at both of the other places a gemmi
Op::Rot reaches Multiply, one of them in this same file.

A, B, C, I and F are symmetric matrices, so for them the transpose is a no-op
and the omission never showed. R and H are not. Measured on an R-centred
hexagonal lattice, ToPrimitive('R') returned 59.5 81.7 43.3 / 145.6 124.5 46.7
where the rhombohedral primitive cell is 49.3 49.3 49.3 / 60.9 60.9 60.9. What
hid it is that a determinant is unchanged by transposition, so the VOLUME came
out right - and most callers only take the volume.

It is not only cosmetic: the result feeds the re-seating path that puts a
lattice into a space group the user fixed by hand, so an R-centred lattice was
handed the classifier a "primitive" cell that is not that lattice - broken for
exactly the centring whose setting most needs re-seating.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 6ca00e927d664e870515c04164defa81d8a18725)

symmetry: a zone whose absences were never measurable must not outrank a dead one ... 8d719c1383

sum_u is a sum of max(0, E^2) over a zone's predicted-absent reflections, in
units of their control class's mean, so it is EXACTLY zero when every one of
them merged non-positive. The Beta tail then diverges and the clamp that caught
it made each such reflection worth about 690 nats.

That was harmless while the value only had to clear a bound of 20. Since the
absence evidence became the primary sort key, summed across zones with no
minimum count per zone, it has ranked the candidates - and it ranked them
backwards. A zone of two absences that were never measurable scores 1378 nats
where a genuine zone of six absences at 1% of its own row scores 22, so the
candidate claiming a screw on an UNMEASURED row beat the one whose rows are
actually dead, by sixty times. The comment on the ranking claimed an extra
condition "LOSES the zone's evidence when the row is not dead"; an unmeasurable
row could not lose, it won outright.

No measurement places a merged intensity at exactly zero, so sum_u is floored at
a thousandth of the control mean per absent reflection. The constant is an order
of magnitude below the precision any real merge reaches - a thousandth of the
row mean needs I/sigma ~ 1000 against that row, where ISa tops out near 40 - so
it can only remove the singularity, never suppress evidence a measurement could
have produced. Measured across the realistic range it changes no genuine zone at
all (22.0, 34.7 and 65.4 nats, unchanged to four figures) and takes the
unmeasurable ones to 13 and 36, below and around the bound respectively.

AbsenceEvidence moves to the header, where the comments already named it, so the
ordering it has to satisfy can be tested directly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

reader: refuse a CBF that is not a detector image instead of misreading it ... 7c3ab72fed

Three ways a miniCBF opened silently wrong.

A byte-offset CBF with no PILATUS header at all was accepted: Count_cutoff then
defaulted to 0, SaturationLimitFromValue(0) is 1, and every pixel at or above one
count was flagged saturated - the integration accept gate drops the whole
reflection, so the run comes out empty for a reason nothing reports. The pixel
size defaulted to 0 with no validation anywhere downstream, which collapses every
resolution, every scattering vector and the beam centre in millimetres. XDS
writes its correction files in exactly this shape, so this is not hypothetical.
A pixel size is now required to claim the file at all, and a missing Count_cutoff
leaves the saturation limit unset - falling back to the container's own overflow,
which can only fail to call a pixel saturated - with a warning saying so.

And the header captures are character classes, not number grammars: "[\d.eE+-]+"
matches a bare "." and "(\d+)" matches a digit string too long for int64.
std::stod and std::stoll answer both with a raw std:: exception, which escaped
the format probe - CanRead catches JFJochException only - so merely LOOKING at a
corrupt file threw out of the viewer's open path. A header field that does not
parse is now reported as a malformed header.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

rugnux: keep a rotation-axis sign the first pass corrected ... 5e3a7d113c

The two-pass post-refinement snapshots the goniometer before pass 1 and restores
it after, to undo the start/stride shift a sub-range run applies inside the pass.
But the rotation-axis sign rescue also runs inside the pass, and adopts the
opposite axis when the file's own indexes almost nothing - so the restore threw
that away, handed pass 2 the sign that had already failed, and applied pass 1's
fitted rotation scale to the axis it was not fitted on.

It self-heals: pass 2's rescue fires again, and a genuinely wrong sign indexes
almost no frames, so the run reaches the same answer having paid for one
redundant first pass. But the correctness of that rests entirely on "a wrong sign
always scores below half the validation frames", which nothing states and nothing
tests. The restore now keeps the axis direction and reverts only the angles,
which is what it was for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

rugnux: a space group that names twice the cell is a signal to halve it ... 3a30dc8ab5

C 1 1 2 and C 1 1 2_1 are the only two monoclinic settings whose change of
basis from the reference has determinant 1/2, and the reason is physical:
with unique axis c the C-centring vector lies wholly in the oblique a-b
plane and reduces away. Such a group is never a distinct Bravais centring -
it is a primitive lattice written on twice the cell - and it wins the
candidate table only because the table is scored on that doubled cell,
where half the reflection positions are absent by construction.

Refusing to enumerate those settings was measured and is worse: the
volume-preserving alternative keeps the by-construction absences as real
observations and merges no better than P1. So keep the enumeration as it
is and act on the win instead: halve the cell, re-classify in the group's
own system, carry the reflections across (the invented positions map to
non-integral indices and are dropped, which is what the centred candidate
was already doing) and search again there.

Measured on one rotation crystal: C 1 1 2_1 on 68.6 73.0 170.3 90 90 90
becomes P 1 2_1 1 on 50.1 170.3 50.1 90 93.5 90, exactly the deposited
setting, with every merge statistic unchanged (R_meas 0.1209, CC1/2 0.9979,
d_min 1.238) and ISa 18.27 -> 18.31. Stable across +/-0.5 mm of detector
distance. Eight other crystals are bit-identical, and across 112 stored
battery reports exactly one adopts a volume-doubling setting.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

release: 1.0.0-rc.166 ... 55d9e2536f

The changelog entry for this release is six lines, not the fifty-four the
development log had accumulated: what a user of rc.166 gets is native miniCBF
input, masters from other facilities opening, the beam centre measured on every
run, a 2theta-swung detector placed where the file says, the unmerged MTZ and a
P1 merge written by default, and a substantially reworked symmetry
determination. The per-change detail is in the commits.

update_version.sh regenerates the three clients from broker/jfjoch_api.yaml, so
this also carries the one API description that had drifted from the generated
code - fft_high_resolution_A, which sizes the FFT's projection histogram and
does not filter spots by resolution. Everything else in the regenerated tree is
the version string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

report: give the resolution the CC1/2 fit reached, not only the one it was cut at ... 6d37b31948

The automatic cutoff fits the CC1/2 fall-off, takes the crossing of the target
(0.30 by default) and then deliberately keeps one more shell. Only the extended
limit was reported, as INCLUDE_RESOLUTION_RANGE, and that is the number a reader
takes for "the resolution of this dataset" - so the report quoted a limit that is
generous on purpose as if it were the measurement.

The generosity itself is right and stays: a shell that is included can still be
downweighted or dropped by refinement, while one that was truncated cannot be put
back. What was missing is the other number. ComputeCCHalfLogisticCutoff now also
returns the crossing, ApplyResolutionCutoff passes it out, and the report prints
it as FITTED_RESOLUTION beside the range the reflections were written to, with a
sentence saying which is which. The log line names both as well.

Nothing about the cut, the merge or the written reflections changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: confirmed cells for three small-molecule sets; drop the unprocessable CCD one ... 756c3a22bb

The four Diamond I19 small-molecule datasets had no reference of any kind, so a run
on them could not be scored at all. Three now have one, each from a published
structure with its DOI verified against Crossref; the fourth has a published space
group but no numeric cell anywhere, which is recorded as such.

Also removes the "obtained but not in the table" section. That dataset is stored as
CCD TIFFs the reader does not support, so it is not processed here and adds nothing
to a page about what the test corpus contains.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

viewer: draw grid scan cells in the proportion of the scan steps ... d189faeec7

A grid scan with, say, a 20 um step in x and a 5 um step in y was drawn as a
square grid, so the picture had nothing to do with the shape of the area that
was scanned. The cells now carry that proportion.

The anisotropy lives only in the view transform: JFJochImage gains a
pixel_aspect_ (the drawn height of one pixel in units of its width, 1.0 for a
detector image), the initial fit fits the image as if it were that much taller
and then puts the factor back into the transform, and the grid view sets it
from |step_y / step_x|. Scene coordinates stay one unit per cell, so the mouse
mapping, the selected-image box, the pixel labels and the ROI code need no
change and the uniform wheel zoom keeps the proportion.

Measured on an 8x4 grid driven offscreen: m11/m22 comes out 4.000 for steps
20/5 um, 0.250 for 5/20, 12.333 for 37/3 and exactly 1.000 - the old
behaviour - for a square step. Every cell centre still maps back through
mapToScene to its own cell in all four cases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hcoh6VNrmSswjfeMeDQqWP

comment: say what the refined geometry in the end message actually is ... 6491345fea

Two things the field names hide, both verified in code and both able to mislead a
reader into publishing a wrong geometry.

The beam centre written here is the PONI - the foot of the perpendicular from the
sample to the detector plane - so on a tilted detector it is not where the direct
beam lands. That point is D*tan(rot)/pixel away and is GetDirectBeam_pxl().

And the tilt is not written only for the record, which the old comment implied by
mentioning nothing but the _process.h5: IndexAndRefine puts it on
outcome.experiment, which is the geometry Bragg prediction and integration then
run at. It is fitted with the distance frozen and nothing checks the covariance,
so the split between the PONI and the tilt is not reproducible between passes even
though their sum - the direct beam - is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

report: the twinning measured before the search, and the strong-direction limit ... c25418319a

Three additions to the report, none of which changes a number the run computes.

The twinning statistics printed today are measured in the Laue class the run
ADOPTED, so where the search promoted the point group they can say no more than
"no twin law exists inside the class it chose" - and a twin is precisely what
would have caused that promotion. The uncontaminated pair was already computed on
the subgroup merge the search was given and reached only a log line; it is now
printed as L_TEST_MEAN_ABS_L_BEFORE_SEARCH and SECOND_MOMENT_I_BEFORE_SEARCH,
beside the post-adoption pair rather than instead of it, since the two answer
different questions.

ANISOTROPY_D_MIN_BEST names the finest of the three principal limits so it can be
grepped, and a paragraph distinguishes the three resolutions the report now
carries: what was written, where the isotropic fit crossed, and how far the
crystal reaches where it reaches furthest. It is read off the fitted tensor rather
than off the cut, so it can land either side of the other two - measured on one
dataset at 2.01 A against a 1.81 A cut - and the text points at
ANISOTROPY_D_MIN_CENSORED for when it is the edge of the data instead.

A warning when CC1/2 falls and then climbs again, which means the fall-off the cut
is read off does not describe the data. A climb counts only above CC1/2 0.30 and
only if it exceeds 0.05: past that the number is oscillating in noise beyond any
cut this run would take. It advises no re-cut - the generous cut is deliberate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

rugnux: rank screw candidates without the control count ... 71cca91ec1

A screw's control class is the complement of its absent class on one axial row, so the two move
together: a candidate that predicts more of the row absent leaves fewer reflections to be judged
against. The Beta tail AbsenceEvidence computes grows with that control count, so the stricter
candidate was charged for the very reflections it correctly called extinct, and a group whose
predicted-absent class is a strict superset of another's - with the extra reflections equally dead -
could score LOWER than the group that explains only part of the row.

Two mechanisms, one dataset each.

  * The control-count minimum gated the row MEAN, which is the scale the zone's evidence is stated
    in, on the same count as the row MEDIAN, which is a violation threshold. A candidate could
    therefore forfeit a whole zone by being right: on a tetragonal 4_1/4_3 wedge, thirteen 00l
    reflections measured at 0.1% of the two l = 4n beside them scored ZERO because only two control
    reflections were left, while the nine of them a 4_2 also predicts absent scored 38.7. The mean
    now stands on two, the median still on three.

  * The Beta tail is replaced, for a SCREW zone only, by its b -> infinity limit - the same
    statistic with the control count dropped. A p-value computed against each candidate's own null
    is not one scale across candidates; what is left is the likelihood ratio of the absent class
    against Wilson at the row's own mean, which is a sum over reflections and therefore comparable.
    Asymptotically it is n_absent * (log(1/ubar) - 1), so an equally dead superset can no longer
    score lower. Measured on a tetragonal 4_1/4_3 crystal: 29 dead 00l against 8 control read 47.7
    nats where a subset of 19 of them against 18 control read 55.5.

The centring statistic is untouched: its control is the whole present population, not the complement
of a claim on one row, so neither mechanism applies to it.

Measured. Analytic superset monotonicity, on a 176368-point grid with no calibration and no corpus:
1.29% violations -> 0.00%, exactly monotone wherever both candidates keep a control class. Over 112
stored merges (open, in-house and private arms) exactly ONE decision changes and it is a gain, a
4_2 -> 4_1/4_3 swap on a deposited 4_3; the screw column over the deposited arm goes 33/41 -> 34/41
with zero screw over-calls before and after, and the four centring outcomes are bit-identical. An
eight-wedge reproducer whose 00l row a strict candidate eats goes from two mis-calls to none,
end to end. The refused side moves the safe way: 103 zones the old statistic scored negative go
lower, none turns positive, and the minimum evidence of an adopted screw rises 20.4 -> 23.8 against
an unchanged bound of 20.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

twinning: do not report a twin the L-test rules out ... ba24946c0f

The two indicators can only move one way under twinning - <|L|> down from 0.500
toward 0.375, the second moment down from 2.0 toward 1.5 - but they were combined
with an OR, so a narrow intensity distribution could report a twin on its own while
the L-test said the opposite.

That is not a marginal disagreement. It fires on small-cell rotation data, where a
twin fraction of 0.50 was reported for a crystal whose <|L|> was 0.63: a value no
twin can produce, and one that points at the opposite situation - a structure whose
intensities behave centric. Suppress the flag when <|L|> sits at or above its
untwinned value; the two indicators are otherwise combined exactly as before.

Measured over 113 stored reports: two flip, both of them wrong today, and nothing
else moves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

calibration: move the powder fit into image_analysis/geom_refinement ... f34d9fe62b

CalibrateFromProfile/CalibrateFromSpots/WritePoniFile sat in rugnux/, so the
only way to reach them was to link the Rugnux library - which drags in
JFJochWriter and gemmi. Nothing in them needs either: the includes are all
common/ and image_analysis/geom_refinement/, next to the RingOptimizer and
RingsFromProfile they call.

Moving them to image_analysis/geom_refinement/PowderCalibration.{h,cpp} puts
the powder fit beside the rest of the geometry refinement and makes it
reachable from anything that already links JFJochImageAnalysis - the receiver
and so the broker included, which is what an online geometry calibration would
need.

Pure move: the file contents differ from their previous form only in the
include paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: report what the ring fit knows about its own answer ... fba5435c38

The ring fit reported the SCATTER of its measurements (rms, and the beam-centre
standard error that follows from it) but nothing about how well the fit pinned
each parameter. Those two part company exactly where a calibration is worth
doubting: as the rings run out, the tilt and the beam centre stop being
separable - both displace a ring's radius as cos(phi) and only the way that
amplitude scales with radius tells them apart - so the fit can sit tightly on
the few points it has while being free to spend tens of pixels of beam centre
on a tilt the data do not support.

Take the covariance of the converged problem from Ceres and report it. Measured
on a LaB6 distance series, the fitted tilt is 50 sigma at 110 mm and 0.1 sigma
at 500 mm, where only two rings reach the detector; at 500 mm the fit quotes its
own beam centre to +-180 px and its tilt to +-2.9 deg on a 0.35 deg value, and
the correlation between them is 1.000. Nothing acts on this yet - it is printed
so the next change can gate on it.

Ceres returns the bare (J'J)^-1 of an unweighted problem, so it is scaled by
chi2 per degree of freedom; that leaves the sigmas in pixels, mm and radians
whatever unit the residual is stated in. Cost is one 5x5 SVD per run, below the
noise of the surrounding I/O.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: fix four ways the powder fit quietly loses its input ... 5a80d2df53

None of these changes the answer on a well-separated cubic standard - the LaB6
distance series is bit-identical by both methods - but each one is a case where
input is dropped or mis-assigned without saying so.

The circumcentre vote grid was a fixed 4000x4000 box, and the caller never
passed anything else. That allocated 128 MB whatever the detector, and on a
detector larger than 4000 px in either direction it put the beam centre outside
the grid, so every vote was discarded and the guess failed with "Beam center
not found". Span the spots' own bounding box instead: a powder ring encloses
its centre, so that is where the answer has to be. uint32 votes while there -
the most any bin can take is C(500,3).

Spots were assigned to the FIRST calibrant ring within a fixed 0.1 1/A, not the
nearest. Silver behenate's orders sit 0.108 1/A apart and hexagonal ice has
three rings inside 0.06, so for those two standards the window reaches the
neighbour and every point lands on the lower-q ring of the pair, biasing the
distance. Take the nearest ring, and clamp the window to half the gap to the
neighbour - which is what the profile path already did inline, now shared as
RingMatchWindow and covered by a test that checks it actually narrows on the
crowded standards and not on LaB6.

A profile bin no pixel fell in is NaN. SectorPeakQ dropped such a sector by
accident, through NaN comparisons falling false; check the four background bins
and return explicitly.

Ice-ring handling is switched off in calibration mode. Flagged spots are sorted
last by the spot budget and so discarded first, which for --calibrant ice
throws away exactly what is being calibrated on.

The two per-ring std::cout lines in GuessGeometry are gone: a library has no
business writing to a terminal, and constructing a Logger to keep them would
emit a version banner from inside a fit. What matched belongs in the result
struct, which the quality gating still to come needs anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: -C overrides the calibrant, with absences from -S ... d275bbcae7

The five named standards are a convenience, not the limit of what a powder
calibration can be run against. A unit cell given with -C now IS the standard
in --mode calibration, and its rings are enumerated from that cell.

A cell alone does not give a ring list, though: the centring and any glide
decide which hkl the lattice actually diffracts into, and the fit pairs the
innermost OBSERVED ring with the innermost LISTED one - so a list opening with
a reflection that is not there scales the whole calibration by the ratio
between them. Where -S is given, the absences come from the space group itself
via gemmi, which covers centring, glides and screws in one mechanism rather
than the three hand-written conditions the built-in table uses. The two agree
exactly on LaB6 and CeO2, which is the cross-check that says the gemmi route is
safe to hand a user's cell.

They do NOT agree on silicon, and the test now pins that: Fd-3m's symmetry
absences are only the F centring, while silicon's 222 and its relatives are
extinguished by its two-atom basis - a structure-factor absence, not a symmetry
one, so no symmetry handler can know it. gemmi offers 24 rings where the
diamond condition gives 18. The extra rings do not move the first one, so the
distance is not scaled, but they are rings with no intensity offered to the
matcher - which is why --calibrant si stays, and why the usage text says -S
supplies symmetry absences only.

Without -S the cell is taken as primitive and the log says "assumed primitive".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: take the detector distance from the rings, not from the header ... a5f416fcdc

A powder calibration is run because nobody is sure the header is right, and the
header's distance was the one number the fit could not survive being wrong
about. The ring search is local - each ring is looked for inside a window a few
pixels of radius wide - so a distance more than a percent or two out puts every
ring outside its own window, and the fit then converges on whatever background
fluctuation each window contains. It does not fail: a 110 mm exposure told the
detector was at 150 mm reported 149.8 mm, with 146 ring points and exit 0. Only
its residual said anything, 5.3 px against 0.4 px, and nothing read it.

Measure the distance from the rings instead. The peaks of the azimuthally
averaged profile give ring RADII, and a radius does not depend on the assumed
distance at all - bin i holds the pixels at one particular radius whatever q
that radius was called - so the radii are a property of the image. Against the
calibrant's d-spacings, r = D tan(2 asin(lambda/2d)) then has one unknown. It is
scanned rather than solved because the pairing of observed rings to d-spacings
is unknown too, and the winning basin is solved in closed form. Nothing here
reads the header distance except to bin the profile; it needs only the
wavelength, the pixel size and the detector's extent.

A powder pattern has genuine distance aliases, so one answer is not enough. A
cubic primitive standard puts its rings at radii proportional to sqrt(N), and
scaling the distance by sqrt(2) maps ring N onto ring 2N - most of the comb
still lands on peaks. Measured: the 110 mm exposure with a 115 mm header scored
its best at 156.5 mm, which is 110*sqrt(2). No adjustment of the score removes an
alias the lattice really has, so the scan hands back the few best distances and
each is fitted, the header among them as one hypothesis of several. The residual
then separates them - 0.4 px against 5.2 px on that case - subject to an attempt
explaining a comparable share of the pattern first, because a start so wrong
that one ring point survives leaves a residual of exactly zero.

Each attempt re-extracts at the geometry it converged to and fits again. The
seed is measured from blended peaks and is good to about a per cent, close
enough to converge from but far enough to sit every search window a few pixels
off its ring, and an off-centre window takes its background off the ring's own
flank. Nothing is re-read from disk, so the loop is free.

Measured on the LaB6 distance series. A 110 mm dataset now recovers 110.03-110.17
mm from any header between 25 and 1200 mm, against +-2 mm before. All five
datasets recover their own distance from a fixed wrong 250 mm header. With
correct headers, four of the five are bit-identical to before and the 500 mm one
moves by a single ring point - the two-ring fit whose tilt is 0.1 sigma anyway.
Run time is unchanged at 0.62 s.

The residual is larger on a run whose header was wrong (1.1 px against 0.4 px on
the 110 mm case), because the profile was still binned at the wrong distance and
its radial sampling is correspondingly coarse. The geometry is right; only the
scatter about it is inflated. Re-running with the recovered distance recovers
the residual too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: only report a tilt the fit actually measured ... 01ad1e1743

The detector tilt is refined by default, and on a pattern that cannot separate
it from the beam centre the fit returns one anyway - there was nothing to stop
it. Both displace a ring's radius as cos(phi), and only how that amplitude grows
with the ring's radius tells them apart, which takes two well-sampled rings. At
500 mm on the LaB6 series only two rings reach the detector and the outer one is
barely there: the tilt came out at the opposite sign to every shorter distance,
dragged the PONI 28 px, and bought a residual of 0.960 px against 0.962 pinned.
The covariance says so plainly - 0.1 sigma, and a beam centre quoted to +-180 px.

So ask it. A tilt is kept only where the fit had it free AND it stands at least
three times its own uncertainty; otherwise rot1/rot2 go back to the header's
values and the beam centre and distance are refitted around them. Over the
series the tilt stands at 50, 33, 15 and 8 sigma at 110 to 300 mm and 0.1 at
500 mm, so any threshold between 2 and 5 gives the same verdict on all five -
this says which regime a fit is in, not where a line was drawn. The declined
500 mm fit lands on a direct beam of 773.53 px, against 773.56 for the pinned
fit measured independently.

It is a rejection criterion and nothing more. Clearing it does not certify a
tilt: that estimator is limited by systematics rather than by this sigma, and a
coherent half-pixel error in the ring positions fakes a tilt of the usual size
while leaving sigma small. The report says "refined", never "verified".

Writing the gate turned up a related fault in the pass loop. RingOptimizer pins
the tilt by itself when every point it is given lies on one ring, and on a
barely-sampled pattern a later pass lands in exactly that state - which froze
the tilt at whatever the FIRST pass had produced and returned it with sigma
zero, an unmeasured tilt wearing the appearance of a fixed one. The gate reads
that as "not measured" and refits pinned, which is why it is stated over the
geometry that gets reported rather than over what the last fit happened to do.
Both paths are covered, profile and spots; the spots path was reporting a
refined tilt as declined for the same reason.

Two things measured and NOT taken:

A robust loss. A Cauchy loss scaled to the previous pass's median residual
changed nothing on the series - rms 0.415 to 0.421 at 110 mm, no case improved,
every direct beam within 0.06 px. Ring points are per-sector peaks that already
had to stand 3 sigma clear of their own background, so there are no gross
outliers left to reject. Recorded at the call site rather than left as an unused
option.

A quality gate that refuses a bad calibration. Three candidate signals, all
measured against naming the wrong standard on LaB6 data: sigma(PONI) does not
see it at all (0.52-0.65 px, indistinguishable from healthy); the residual only
half sees it (3.2-3.6 px wrong against 0.4-1.0 right, but a correct run from a
wrong header sits at 1.0-2.4 and would be caught too); and the seed's match
score is dominated by how many rings the calibrant lists, scoring 0.29 for a
perfect LaB6 fit against 0.21 for a wrongly named silicon. None of the three
separates, so no gate is shipped. What the run does say is the recovered
distance against the header, and a wrong standard moves that to 446 mm on a
110 mm exposure - unmissable, and the operator's call rather than a threshold's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

report: the detector tilt, and the two points the beam centre is not ... 9267bd67de

BEAM_CENTRE is the PONI - the foot of the perpendicular from the sample - so on a
tilted detector it is not where the direct beam lands. The report printed only that
one number and never mentioned the tilt at all, so the two points were
indistinguishable to a reader. On a detector at 85 mm with a 0.22 degree tilt they
are 8.3 px apart.

Print DETECTOR_TILT (rot1/rot2/rot3, degrees, as the run used them - nothing is
refined here) and DIRECT_BEAM beside BEAM_CENTRE, and say which is which.

JFJOCH_DATASET_SETTINGS was the more serious half: it exists to carry a geometry
back into the instrument, and dataset_settings carries poni_rot1/2/3_rad, but the
block omitted them - so it described a FLAT detector, silently, on every tilted
setup. The rotations now ride with it when they are non-zero; they stay out when
they are zero, because the API's own default is 0.0 and the shorter block means the
same thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

calibration: take the beam centre from the rings too ... c84b91be8a

The header's beam centre was the last input the ring fit had to be roughly
right about. Each ring is looked for in a window a few pixels of radius wide,
and a centre wrong by (dx, dy) puts a ring at a different q in every sector, so
past about ten pixels the ring leaves that window over much of the turn - and
the fit then reads its cos(phi) signal off whichever sectors are left, which are
the ones where the signal is weakest. A 20 px error ended 31 px wrong.

The rings answer this without a calibrant and without a distance. A powder ring
is a conic centred on the beam, so a wrong centre makes EVERY ring's radius
oscillate once per turn by the same amount: r(phi) = R + dx cos(phi) + dy
sin(phi), solved directly and pooled over every ring the profile shows, with
each ring searched about its own measured radius rather than about where a
standard says it should be.

Using it needs the extraction to follow the rings sector by sector, which is
what ProfileRingTrack now does - exactly, and in all five parameters at once,
by walking the ring in the geometry believed true and asking the binned geometry
what q and azimuth it would have given each point. That replaces the
flat-detector distance correction it grew out of.

Following the rings is not free, and the reason is worth stating: a window that
moves with phi makes every systematic of the peak finder - where the background
line is taken, how the centroid sits in the window - vary with phi as well, and
phi is exactly the axis the beam centre is read off. Measured, it costs rms
0.415 -> 0.525 px on a good 110 mm fit, and 0.831 when the window follows the
fitted tilt too. So a second measurement is taken with a window that is the same
in every sector - the binned geometry with only its DISTANCE replaced, which is
phi-independent by construction - and both are offered to the same rule that
ranks everything else here. Acquire by following, measure by holding still.

The seeded centre is likewise a hypothesis and not a belief. It reads a
once-per-turn wobble, and a tilt puts a term of that shape there too - one that
grows as the radius squared, where a centre error does not - so pooling the
rings absorbs part of the tilt into the centre. Believed outright it made a good
110 mm fit worse; offered as an alternative start it costs one more fit and
needs no rule about when it applies. It is skipped entirely below a pixel, where
it is not a different hypothesis at all, which keeps a well-headed run at 0.71 s.

Measured on the 110 mm LaB6 exposure, whose true PONI is 765.90: a header centre
20 px out now lands within 0.5 px, where before it landed 31 px away. All five
datasets are unchanged from their correct headers, and the distance still
recovers from any header between 25 and 1200 mm.

The limit is now understood rather than merely reached. Past a few pixels the
azimuthally averaged profile stops showing rings: a ring tracing r(phi) piles up
density where that turns round, so it averages into the two HORNS of the
sinusoid, at R-|d| and R+|d|. The radius finder reports two rings where there is
one, and the gap between them is 2|d| - the search window shrinks to exactly the
offset it was meant to span. That caps recovery at roughly half the ring
spacing, about 20 px here and failing by 40. Beyond it nothing is left in an
azimuthally binned profile, and --calibration spots, which works from the spot
positions themselves, is the method that still can.

One pre-existing limit measured and NOT introduced here: a wrong distance
together with a centre more than about 5 px out fails, because the centre error
splits the radius list the distance search reads. The committed code before this
change fails identically on those cases.

Also fixed: fit_from now takes a whole geometry rather than a distance, and the
declined-tilt refit was inheriting rot1/rot2 from it - pinning the tilt at
exactly the unvalidated value the gate had just rejected. Same fault the gate
exists to catch, one level up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: let the spots vote on the geometry in ring mode too ... 49dd01331b

The rings and the spots fail in different regimes, which is the whole reason to
carry both. The header is right on a well-configured instrument and is the thing
a calibration is run to check. The profile's own estimates are exact while the
error stays small and stop meaning anything beyond that - past about half a ring
spacing each ring reaches the azimuthally averaged profile as two horns rather
than one peak, and the distance search reads a list of horns as a list of rings.
The circle through the spots reads nothing from the header at all: measured on a
110 mm LaB6 exposure, --calibration spots returns the same geometry from a header
400 px out in the centre AND eight times out in distance.

So ring mode now finds spots as well - half a second - and offers what they make
of the geometry as one more starting hypothesis, fitted like the others with the
residual left to choose. It is added whole rather than as a centre alone:
GuessGeometry votes for the circle centre, clusters the radii into rings and
takes the distance from the innermost, and those two belong together. Taking only
its centre would not have helped, because the distance candidates come from a
profile averaged about the header's centre, and a ring smeared over hundreds of
pixels cannot be un-smeared by reading its bins differently.

Measured on the 110 mm exposure, whose true PONI is 765.90 at 110.03 mm. A header
40 px out in the centre now lands within 0.6 px (it landed 41 px away before).
The cases with BOTH wrong, which failed before this and equally before the beam
centre work, now come out: 250 mm / 780 px gives 110.041 mm and 766.10 px against
42.3 mm and 782.7; 250 mm / 867 px gives 110.065 and 765.48. All five datasets
are unchanged from their correct headers and the distance still recovers from any
header between 25 and 1200 mm.

Past about a hundred pixels nothing rescues ring mode, and the reason is the
profile rather than the seeding: binned about a centre that far out it shows each
ring smeared across its own sectors, so even started from the exact answer the
fit leaves rms 4.8 px and drifts. Re-binning would fix it and would need the
images read a second time; --calibration spots, which never touches the profile,
already covers it.

That regime is now visible rather than silent. The spots' beam centre is printed
beside the fitted one as a cross-check - two methods sharing no assumption, so a
reader can see at a glance whether they agree. It costs nothing, the spots having
been found already, and it separates cleanly: 0.3-0.4 px on the good runs against
201.4 px on the exposure whose profile could not be fitted at all. Reported as a
fact and not gated on, since at long distance both methods weaken together and
the honest thing is to show the number (5.2 px at 300 mm, 12.6 at 500).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: re-integrate the images about the geometry that was fitted ... 16b368739e

Everything the ring fit reads was binned with the geometry the run started
from, and a calibration is run precisely because that geometry is in doubt.
Binning is not something a later fit can undo: which pixel landed in which
(q, phi) bin was decided when the images were read. Get the distance wrong and
the radial sampling is the wrong scale - a run recovering 110 mm from a 250 mm
header ended at rms 1.07 px where the same data from a right header gives 0.42.
Get the centre wrong and every ring is smeared across its own sectors, and past
about a hundred pixels the fit leaves rms 4.8 px even when it is started from
the exact answer, because there is nothing left in the profile to fit.

So integrate the images a second time, binned about what was fitted, and fit
that. A calibration run is a handful of images and the answer is worth far more
than the extra read.

Re-binning about the FIRST fit is not enough on its own. Where the beam centre
was badly out that fit is itself wrong, and binning about it digs deeper - rms
3.75 -> 5.66 on an exposure 200 px off. The spots' geometry is the one that does
not degrade there, having never read the header, so both are tried where they
differ and whichever comes back better is kept. On that exposure the first
candidate gives 288 points at rms 5.66 and the second 369 at 0.81.

The second pass is taken only if it is actually better, by the same rule that
ranks everything else here - at least half as many ring points and a smaller
residual - and it is skipped altogether when the fitted geometry moves a ring by
less than the radial width of one profile bin, since re-binning would then put
every pixel back where it already is. On the 110 mm exposure with its own header
that is 1.77 px against a 1.84 px bin, so the run is untouched and its answer
bit-identical.

Measured on the 110 mm LaB6 exposure, true PONI x 765.90 at 110.03 mm. The
calibration is now independent of the header it was given:

  beam-x out by  +20 +40 +100 +200 +400 px -> 765.9-766.6, 110.02-110.05 mm
  250 mm header, beam-x 780  -> 765.998 at 110.037, rms 0.394
  250 mm header, beam-x 867  -> 765.723 at 110.038, rms 0.413
  900 mm header, beam-x 1167 -> 766.496 at 110.038, rms 0.804
   30 mm header, beam-x 967  -> 766.486 at 110.039, rms 0.807

All of those failed before this, most of them catastrophically. The residual
inflation is gone with them: the 250 mm case now leaves 0.394 px, better than
the same data from its own correct header.

The four other datasets are unchanged where the re-bin fires, and slightly
better where it does: rms 0.433 -> 0.423 at 150 mm and 0.555 -> 0.550 at 200 mm,
with the geometry moving under a tenth of a pixel. A run that needs the second
pass costs about twice one that does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

calibration: write the result as JSON, not only as a PONI file ... 8691bf4876

A calibration run produced one file, and it was a pyFAI PONI - which pyFAI and
its neighbours read and nothing in this system does. Carrying the answer back
into the instrument meant a person reading numbers off a printed report and
retyping them into a dataset_settings body, and the report is where the two
points a "beam centre" can mean are easiest to confuse.

So write <prefix>.json beside it. Its "dataset_settings" member holds the
geometry under the property names broker/jfjoch_api.yaml gives them and holds
nothing else, so it is a valid dataset_settings body as it stands:

    curl -X POST -H 'Content-Type: application/json' \
         -d "$(jq -c .dataset_settings det.json)" http://broker:5232/start

beam_x_pxl is the PONI, as everywhere here. The three poni_rot*_rad ride along
whenever any is non-zero and are left out when all are zero: a body without them
does not leave the tilt unstated, it states a FLAT detector, so they travel
together or not at all - the same rule the report's JFJOCH_DATASET_SETTINGS
block already follows.

The "calibration" member holds what the run knows about that geometry: the
residual, the fit's own sigmas and the correlation between the tilt and the beam
centre, whether the tilt cleared its significance test or was declined and
pinned, where the direct beam lands, and where the spots independently put the
beam. A calibration that has gone wrong looks exactly like one that has not until
those are read, and a machine-readable file that carried only the geometry would
be the easiest possible way to feed a bad one into an instrument.

Tested against the model generated from the spec rather than against a list of
field names written out by hand: the file's dataset_settings member is parsed
into org::openapitools::server::model::Dataset_settings and validated, so a
field renamed or newly required in jfjoch_api.yaml fails here rather than at
someone's POST. The tilted and untilted branches are both covered, and the
PONI/direct-beam distinction is asserted rather than assumed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

api: define the powder calibration result as calibration_output ... 6fe3f30ab4

The JSON a calibration writes was a shape invented at its writer, described
only by the comments around it. That is enough for a file somebody reads with
jq and not enough for anything else: a client cannot type it, and an endpoint
returning it later would have to declare the shape a second time and keep the
two in step by hand.

So declare it where every other shape in this system is declared.
calibration_output holds dataset_settings and a calibration member; the latter
is calibration_quality, which nests calibration_fit_sigma and
calibration_spot_check. The descriptions carry what a reader has to know to use
the numbers rather than only what they are named - that beam_x_pxl is the PONI
and the direct beam is elsewhere, that the rotations travel together because a
body omitting them states a flat detector, that a tilt below about three sigma
was declined and pinned, and that the two correlations approach 1 as the tilt
stops being separable from the beam centre.

Nothing references it yet. It is declared now because /powder_calibration will
return exactly this, and because the file rugnux already writes is decodable
today: jfjoch_client's CalibrationOutput.from_dict reads it as it stands, with
o.calibration.fit_sigma.correlation_beam_x_rot1 and the rest typed.

Generated clients regenerated from the spec, as the spec requires: the C++
server model (four new pairs under broker/gen/model), the TypeScript frontend
client, and broker/redoc-static.html. Both regenerations are purely additive -
no existing generated file changed except to export the new names. The python
client regenerates from the same spec and is gitignored.

The test now validates the WHOLE file against the generated Calibration_output
rather than only its geometry member against Dataset_settings, so the quality
block is under the same contract: a field renamed or newly required in
jfjoch_api.yaml fails here rather than at a client.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

api: name the calibration schemas powder_, not calibration_ ... bd6e933f04

"Calibration" already means something else in this broker: the JUNGFRAU pedestal
and gain measurement, which owns /statistics/calibration, calibration_statistics
and JFJochState::Calibration. Four schemas called calibration_* would have sat
beside it meaning a completely different procedure, and the confusion is cheapest
to remove now, before an endpoint returns them.

calibration_output, calibration_quality, calibration_fit_sigma and
calibration_spot_check become powder_calibration_output,
powder_calibration_quality, powder_calibration_fit_sigma and
powder_calibration_spot_check. All four, not only the outer one - the collision
is in the word, and half a rename would read as though the inner three belonged
to the other kind of calibration.

Rename only. Every client regenerated from the spec: the C++ server model, the
TypeScript frontend client, redoc-static.html, and the python client, which is
gitignored but was checked to still decode the file rugnux writes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

docs: publish the powder calibration schemas in the python client docs ... 7c00731992

docs/python_client is tracked and is what readthedocs publishes, but it is
generated - update_version.sh copies it out of python-client after regenerating
the client - so adding schemas to the spec left it four files behind. The
README's model list did not name them and their pages did not exist.

Copied the way update_version.sh copies them. The four new pages and four lines
in the README, nothing else: the generator's README is built from the spec, so
the stale Calibration* pages it had left on disk under the old names are absent
from it and were removed before copying rather than published alongside their
replacements.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NfuDvf5ipV3Hi8TiCUKD27

report: the measured tilt, and one geometry instead of two ... dcfcff5bc4

Adds REFINED_DETECTOR_TILT - rot1/rot2 in degrees as rotation indexing measured
them - so the value can be checked against a powder calibration, which measures the
tilt from ring shape and is the better instrument for it. rot3 is not carried: a
rotation about the beam is an exact null of a single-axis rotation experiment, so
the fit cannot move it and printing it would suggest otherwise.

Also repairs two defects in the commit that added DETECTOR_TILT, found in review.

The geometry was taken from the caller's copy of the experiment, which is the state
BEFORE the run, while BEAM_CENTRE and DETECTOR_DISTANCE come from result.used_*,
which is the state after - so on a rotation two-pass the report mixed a post-refined
centre with the file's rotations and DIRECT_BEAM was off by the whole post-refinement
shift. JFJOCH_DATASET_SETTINGS carried the same mixture, which is a geometry no pass
ever ran at. The tilt and the direct beam are now taken from experiment_ alongside
the other three, so the block describes one geometry.

And the prose claimed the tilt "is not refined here". It is: rotation indexing
refines rot1/rot2 on every rotation run. What is true is that the result is never
written back onto the geometry, which is why it needs a key of its own.

M_PI -> PI (common/JFJochMath.h). rugnux is in the forced-Windows viewer-only set
and M_PI is not portable to MSVC; the file did not include <cmath> either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

calibration: --no-refine-tilt holds the header tilt, it does not zero it ... efca844bce

GuessInitialGeometry resets rot1/rot2/rot3 to zero before seeding, and the branch
that puts the header's tilt back was guarded by `refine_tilt && !tilt_refined`. With
--no-refine-tilt the tilt is never free, so tilt_refined is false, so the guard is
false, so the restore never ran - and the fit reported a zero tilt while the CLI
printed "held at the header value".

The guard only needs !tilt_refined: a tilt that was declined and a tilt nobody asked
to refine both want the header put back and the beam centre and distance refitted
around it. That is what the branch already does.

Measured on a LaB6 sweep with a tilt imposed on the command line and asked to be
held, --calibration spots:

  before  Rot1= +0.0000 deg  (+0.001400 rad from the header)
  after   Rot1= -0.0802 deg  (+0.000000 rad from the header)

The rings path reaches this only when the spot-derived start wins, which is why it
does not show on a file whose profile start is chosen; the spots path always did.
A calibration whose header tilt is already zero is unaffected - checked, byte-equal
distance either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

post-refine: bound the beam move on the measurement, not on the header ... f98442077e

PostRefine step B refuses a fitted beam centre more than 15 px from the geometry the run started
from. Two things were wrong with that and they compounded.

The 15 px was BOTH the Ceres search box and the acceptance threshold, so a fit that wanted to move
further was pinned at the box face, landed at exactly hypot >= 15.0, and was then refused for being
there. The gate never saw the fit it was judging. Measured on one crystal at three detector
distances, the fit came back at 15.128, 15.133 and 15.130 px - the box corner, three times.

And it was measured from the header, which is the value most worth correcting exactly when it is
most wrong. The run already knows better: the default beam-centre check fits the centre from the
isotropy of the scattered background on every rotation run, reported it, and then discarded it.

The bound is now measured from whichever of the nominal centre and that measurement is nearer -
accept a move that is small relative to something independent, rather than small relative to the
header alone - and the search box is widened to match, so the gate decides rather than the box. The
measurement is used only where it was precise enough to be adopted as a centre in its own right; a
run without one is bounded exactly as before. Step B also now says WHICH of its four tests refused a
fit, since all four leave the same geometry behind.

Measured over 113 rotation datasets against the same corpus, one binary per arm, with a control arm
that reproduced the stored battery on 112 of 112 reports:

  the bound fired and refused a real correction   1   6ukf
  the bound fired and caught a runaway            0
  the bound never fired                         112

On 6ukf the header centre is 22.1 px from what the background fit measures. The old fit, pinned at
the box, improved the held-out positional residual 9-fold and was refused; the new fit lands 0.46 px
from the measurement and improves it 418-fold. ISa 4.42 -> 6.71, R_meas 18.5 -> 12.8 %, CC1/2 0.987
-> 0.995, d_min 1.119 -> 0.973 A; at matched resolution, 1.68 A goes I/sigma 8.9 -> 15.1 and R_meas
28.8 -> 13.4 %. The refined cell moves from 0.4 % off the deposited one to under 0.1 %.

It survives perturbation, and inverts it: over +-0.5 mm of detector distance the corrected run is
flat (ISa 6.62 / 6.71 / 6.94) while the uncorrected one is bistable (4.02 / 4.42 / 6.90). The change
removes an instability rather than exploiting one.

No space group anywhere in the corpus moves, and no other dataset's report changes at all - 112 of
113 are identical. The distance half of the gate is untouched: it is separately argued, and it is
what legitimately refuses the one dataset whose distance wants to move 1.05 %.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: one changelog line for the geometry reporting work ... 5ce7eb3014

Three lines were appended over the course of the work - the tilt and direct beam in
the report, the measured tilt beside them, and the post-refine beam bound. They are
one change to a user: the report now describes one geometry and says which point is
which, and the bound that governs it is measured against the run's own measurement
rather than against the file it is meant to correct.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

leonarski_f added 8 commits 2026-09-01 15:21:07 +02:00

rugnux: the two-pass guard says when it has no completeness, instead of printing 0.0% ... 8935a59721

The guard that judges the post-refined pass against the header-geometry pass reads completeness off
each pass's space-group search merge. That merge does not count possible reflections, so the quotient
is 0.0 on both sides and every de-novo rotation report has been printing

    completeness 0.0% vs 0.0%, CC1/2 before corrections 0.994 vs 0.993

as though a comparison had happened. On the 28-dataset in-house battery that is all 28 runs. The
number is not just uninformative, it is a test that did not run: "completeness above 100% means the
cell is wrong" is the guard's wrong-cell arm, and it cannot fire against a constant zero, so only the
CC1/2 arm decides - which is computed on whatever data the pass kept, so a pass that discards a large
part of the sweep can post an equal CC1/2 and win.

Carry a measured flag beside the number. The arm is skipped and the log and PASS_DECISION say
"completeness not measured" where there is nothing to read. No behaviour changes: the arm could not
fire before either. The next commit gives the merge the count so it can.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

rugnux: record what the two-pass completeness arm can decide, and what three stand-ins could not ... 04d823135e

Only under -S does the arm read a completeness with headroom. De novo the search merge is in P1 and
does not count its possible reflections, so there is no number and CC1/2 is the only live term.
Three ways to give the arm one back were measured on 28 rotation crystals and none lands, so say so
at the guard rather than leave the next reader to rediscover them:

* Count the possible reflections on the search merge too. It is affordable - 5.2 ms against a 38 s
  run - but a healthy crystal reads 27-43% of the P1 hemisphere, so the 100.5% bound is never
  approached and no decision changes. It would only add a figure to the user's report that reads as
  the dataset's completeness while being a fraction of a different group's asymmetric unit.
* Observation count. The two search merges are built in the same terms, so pass 2 retaining under 90%
  of pass 1's observations looks like the signal the completeness ratio stood in for. It never fires -
  bit-identical on all 28, including the crystal whose pass 2 predicted 44% fewer partials. Predicted
  partials and observations that survive into the merge are not the same population.
* Completeness ratio. Same idea one level up, and it does fire - on the wrong crystals. This merge is
  deliberately never resolution-cut, so the possible list grows with whatever range the pass reached
  and a pass that predicts FINER scores as one that lost the sweep. At a 0.90 bound it reverted two
  crystals indexing at 100%, taking one of them from 0.07% to 0.29% cell deviation and ISa 19.5 to
  17.1, and rescued nothing.

Bringing the wrong-cell detector back for de-novo data means putting it on the final merge or
retiring the arm and saying so; neither is done here.

No test covers the decision line: Rugnux_Rotation does not reach this guard - the two passes in its
log are the indexer's, not RunAllPasses' - and there is no fixture for building a RotationScaleMerge,
so an assertion placed there would never execute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: name the in-house SLS 2.0 data beside the public sets ... 0e245d28cf

The acknowledgement read as though the whole test corpus came from other people's beamlines. It does
not - in-house data collected at SLS 2.0 is tested alongside it. One clause, at the top of the
section; everything that follows still concerns the public data, which is the part that carries an
obligation to cite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

cbor: carry the direct beam on the start message ... 336fafffd0

beam_center_x/y is the PONI - the foot of the perpendicular from the sample - so on a tilted
detector it is not where the beam lands, and the two are distance*tan(tilt)/pixel apart (~8 px on
a real in-house setup). A consumer that wants the beam position, which is what most of them mean,
had to redo the tilt arithmetic or get it wrong.

Add optional direct_beam_x/direct_beam_y to StartMessage, filled from the geometry's own
GetDirectBeam_pxl() so there is no second formula, and put them at the top level of the CBOR start
map beside beam_center_x/y. Optional keys, so a consumer that does not know them skips them.

No _pxl suffix: the stream2 neighbours (beam_center_x, pixel_size_x, detector_distance) carry none
and the units are in docs/CBOR.md. The API's calibration schema spells it direct_beam_x_pxl because
its neighbours there are beam_x_pxl - same quantity, each matching its own neighbourhood.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

writer: the direct beam beside the beam centre in the NXmx master ... 26ba019752

Write direct_beam_x/direct_beam_y, in pixels, into
/entry/instrument/detector/detectorSpecific - snake_case and unsuffixed like the rest of that
Dectris-style group, with a units attribute like the beam_center_x it must be read against.

Computed inside Metrology from the very locals that produce the beam_center_x/y and rot1/2/3
written a few lines away - refined when the offline analysis refined them, the StartMessage values
otherwise - and from the same start.detector_distance the translation vector uses. The file
therefore cannot disagree with itself: one composed geometry feeds all of them. Provenance differs
by producer (the broker's master states the user's geometry, a rugnux _process.h5 a measured one),
the field does not.

The test writes a real master in all three NXmx layouts at a non-zero tilt and reads the datasets
back, and requires the tilt to have moved the point, so a writer that stored the PONI would fail it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the beam centre is the PONI, the direct beam is the beam ... 1a25d396a8

Document direct_beam_x/direct_beam_y in the CBOR start-message table and in the HDF5
detectorSpecific catalogue, and add the section that says what the distinction is: the PONI is the
foot of the perpendicular, the direct beam is where the beam lands, they are
distance*tan(tilt)/pixel apart, and XDS ORGX/ORGY wants the second. Also record that the same field
is a statement about the user's geometry in a broker master and a measurement in a rugnux
_process.h5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

report: say what the measured detector tilt is worth, and what it is not ... 51e90f5549

The report invited the reader to check REFINED_DETECTOR_TILT against a powder
calibration. Measured over two detectors' worth of crystals, that check
misleads: the rotation fit does one outer round, so it leaves its starting
value by only a small and crystal-dependent fraction of the distance to the
calibrated value, and on the worse of the two detectors the per-crystal median
sits an order of magnitude further from the powder answer than either method's
uncertainty. A user comparing one run against their own calibration would
conclude the calibration was wrong.

What survives the aliasing is the direct beam, which is already printed beside
it, and the MEDIAN of the tilt over several crystals on one detector - enough
to show up a placeholder or a stale value in the file, not enough to replace a
calibration. Say that, and say not to feed one run's value back into the
instrument.

The conditioning law (VIF = 4.40 tan(2theta_95)^-1.66) was tested as a per-run
gate and does NOT order the errors - the crystals whose fit stays put are found
in the best-conditioned band as often as the worst - so no threshold is added
here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: one line per behaviour change in the rc.166 changelog, named by program ... 5869a27fe2

The section had grown to eighteen entries, several of them describing the same report or carrying
detail that belongs in a commit message. Consolidated to thirteen one-liners, each naming the program
it concerns - rugnux, jfjoch_writer, jfjoch_broker, jfjoch_viewer - so a reader can find what changed
in the part they use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

leonarski_f added 38 commits 2026-09-02 09:22:45 +02:00

geometry: restrain the spindle-parallel tilt, the same gauge the beam prior already restrains ... e441434644

The beam prior at XtalOptimizer already treats the beam component parallel to the spindle as the gauge
direction of a single-axis rotation experiment and restrains it toward the value it was handed. The
detector tilt is that same gauge described a second time, and it was left free with a flat +-3 deg box.

The correspondence is not an observation about one beamline, it follows from the convention:
PoniRotMatrix builds the detector matrix as R(-rot3,z)R(-rot2,x)R(rot1,y), which puts the direct beam
at (beam_x - rot1*D/pixel, beam_y - rot2*D/pixel). rot1 IS beam_x written as an angle and rot2 IS
beam_y, component for component. So on a horizontal spindle the gauge tilt is rot1 and rot2 is
refined; on a vertical spindle it is the other way round.

With only one end of the alias restrained, a beam-centre error the prior refuses to absorb reappears
as an angle: measured elsewhere in this campaign, refined rot1 tracks the starting beam centre at
0.072 deg per pixel against a geometric 0.080, while the refined beam never leaves its anchor by more
than 0.24 px. Restraining both ends, in the same direction and to the same three-pixel budget, leaves
the determined component - the direct beam, and the tilt perpendicular to the spindle - alone.

Both restraints now take their direction from one projection of the spindle onto the detector plane
rather than from two independent snaps to whichever of X/Y dominates, so they cannot disagree, and a
spindle at any angle is handled. On a spindle along a detector axis the projection is the snap.

geometry: take the gauge direction in the frame each parameter lives in, and tighten the tilt's budget ... c0bb159c98

Two corrections to the restraint added in the previous commit, both measured.

The beam prior compared the goniometer vector's LABORATORY components against an index into the
PIXEL-frame beam centre. det_matrix is PoniRotMatrix * DetectorOrientation::Matrix(), so on a quarter
turn of 1 or 3 the pixel X axis is the laboratory Y axis and the comparison picks the wrong component -
pinning the determined one and freeing the gauge one, which is worse than having no prior. Two datasets
in the corpus are in that state. The direction now comes from projecting the spindle onto the pixel
axes' own laboratory images, which is exact for any orientation, any tilt and a spindle at any angle,
and equal to the old comparison when the orientation is the identity. The tilt keeps the spindle's
laboratory components, because rot1 and rot2 are laboratory rotations applied outside that orientation
matrix - same physical direction, each in the frame its own parameters live in.

And the tilt's restraint is three times tighter than the beam's, in shared direct-beam pixels, because
the data determine the SUM of the two: at equal budgets the shift splits evenly and the reported tilt
still followed the starting beam centre at 41% of one-for-one. At a one-pixel budget it follows at 8%,
from 79% before this work, while the free component moves by 0.0035 deg over the same eight-pixel swing
and the indexing rate does not change.

The comment says plainly what the change does not do: it does not make the tilt accurate.

geometry: say that the gauge argument is local to the fit that co-refines the orientation ... 657b6c5d1a

The restraint is correct here because the crystal orientation is free alongside the tilt and absorbs
the spindle-parallel difference. A stage that freezes the orientation has no such compensator, so
whether that component is measurable THERE is a separate question; the comment no longer reads as a
claim about the tilt everywhere.

geometry: make which tilt component is restrained two numbers rather than a shape ... d32c8d526b

Both directions of the tilt now go through the same restraint, with a budget each in shared
direct-beam pixels; zero means free. Swapping which component is held, or holding both, is a change to
two constants. The perpendicular budget is zero today, which is the arrangement the measurements
support: it is the component whose conditioning tracks the 2theta the fit reaches, while the parallel
one's does not move with 2theta at all - a conditioning number that ignores the data is the prior
talking.

This is a clarity change, not a precision one, and the record should not read otherwise: the standing
'freeing the tilt inflates both beam components' cost is the perpendicular component's doing, not the
parallel one's, so restraining the parallel one recovers essentially none of it. What it buys is that a
beam-centre error is no longer reported as an angle.

refine: fit the direction of the goniometer axis, not its length ... 9c19646f6a

The residual applies angle_rad * |rot_vec| and rot_vec is a free three-vector, so the first pass has
been fitting a goniometer rotation SCALE nobody asked for. GoniometerAxis::Axis() then normalises it
away on write-back, and RotationIndexer scores the candidate with the normalised axis - so the cell
that won the fit is judged under a rotation model the fit did not use. Measured over 43 rotation
datasets: the length reaches 1.2%, and the fit-vs-score disagreement a median 0.124 deg and up to
6.19 deg of goniometer angle, against rocking widths of 0.05-0.36 deg. That score picks the lattice
class, which nothing later revisits.

The fitted length is not a usable measurement of anything either: on synthetic data it recovers 54%
of a known scale error, repeated first passes on one dataset disagree with each other in sign, 26 of
43 datasets disagree with themselves, and on the one dataset with a proven 1.3% stage fault it comes
out negative. It is absorbing other systematics. The rotation scale is measured properly, once, with
cross-validation and gates, in PostRefine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

rugnux: carry the tilt across the two-pass handoff, and post-refine about it ... 8650ac1c59

Post-refinement was handed the HEADER geometry while the outcomes it refines had been integrated at
the first pass's own, and only three numbers came back - beam_x, beam_y, distance. The tilt was
dropped at both ends, so the pass that ships started at the header tilt with a beam that step B had
moved to absorb a tilt the images do not have. On one in-house mounting the first pass ends at
rot1 -0.2995 deg and the shipped value is -0.0992, against a powder calibration of -0.282: the two
geometries the two passes use differ by the whole of the tilt error, inside a single run.

Both ends move together, and they have to. Step B holds the tilt fixed while it fits the beam and the
distance, so the beam it returns is only meaningful about the tilt it was given: handing it the refined
tilt without carrying that tilt forward, or carrying the tilt forward under a beam fitted about the
header's, each describe a geometry that never existed. The carrier goes from three floats to five and
the header snapshot the quality guard reverts to grows with it, so a rejected second pass still returns
to the geometry the file states.

This is not a tilt measurement and does not make one: rot1 remains the gauge direction of a single-axis
rotation experiment. It only stops the two passes of one run from working in two different geometries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

writer: the direct beam is absent where there is no geometry, not fatal ... 4026ebc6ab

Computing it needs a DiffractionGeometry, and that refuses a detector distance under 1 mm or a pixel
size of zero. A start message can legitimately carry neither: the writer's own pre-flight check asks
it to prove it can create the files for a dataset that never describes a detector, and the master
write then threw where it used to succeed - Preflight_TCP fails at rc.166 and passes at rc.165.

The dataset is a convenience for whoever reads the file later, so where the geometry is not there to
compute it from it is simply not written. Everything else in the master is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

integration: the observed centroid is the signal's, not the disk's ... ecb0571d8d

The centroid was a first moment of the RAW counts over the signal disk, so it weighted signal plus
background. The background is flat over a disk centred on the PREDICTION, which makes its own centroid
the prediction exactly: it adds nothing to the displacement and everything to the denominator, and the
measured offset comes out shrunk by I/(I + n*bkg).

That factor is worst where the background dominates the signal, which is at high resolution - so the
one consumer of this quantity, the geometry post-refinement, fits the beam centre and the detector
distance on displacements that are systematically too small, by a factor that varies with resolution.
An estimator whose bias depends on the very coordinate it is correcting.

Subtracting a flat pedestal from a first moment is exact, and the background is not known until the
ring has been read, so the positions of the pixels behind the intensity sum are accumulated alongside
it and the correction is applied afterwards: sum(x*(px-bkg)) = sum(x*px) - bkg*sum(x). Both engines,
identically. Where nothing rises above background there is no signal centroid to compute and the raw
one is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

rugnux: the two-pass supercell guard compares primitive cells, not settings ... ffce27ac33

The guard that catches the bistable supercell collapse between the two rotation passes took the
volume of each pass's CONVENTIONAL cell. A centred conventional cell is an exact integer multiple
of its primitive one - C and I twice, R three times, F four times - so two settings of the same
lattice differ by exactly that factor, and the guard read a change of setting as a supercell. It
then forced pass 1's result, and with it pass 1's lower symmetry, on a pass that had found the
same lattice in a better one. Ten of 116 rotation datasets tripped it, every one at an exact
centring multiplicity: seven at 2.00x, one at 3.06x, one at 3.99x. The last is an F-centred
lattice; it was held in P1 where the second pass had found it centred orthorhombic.

The two structurally identical guards inside RunPipeline already convert with ToPrimitive first,
and their comments say why. This one could not: ProcessResult carried the consensus cell with no
centring beside it, so at the comparison there was nothing to convert with. The centring is now
carried alongside the cell, set at each of the four places the cell is - the finalized rotation
lattice, the reindex into a user-fixed group's setting, the reduction of a doubled cell, and the
committed higher-symmetry reindex - and the guard converts both sides before comparing. Nothing
else reads it; every other consumer of the cell is unchanged.

On the F-centred dataset the run goes from P1 at 2.232 A to I 2 2 2 at 2.077 A, multiplicity
1.86 to 7.07.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

predict: integrate every centring node, so a fixed space group costs no data ... fb07263025

With -S the prediction rejected the fixed group's centring absences, so those reflections were never
integrated. Two things followed, and only the second was known.

The P1 cross-check was withheld on such a run, because a P1 merge missing whole centring classes is
misleading rather than merely small - 50% of the nodes on an I lattice, 75% on F, 67% on R. That was
the documented reason and it was right.

The unknown one is that it cost intensity accuracy. Every predicted reflection marks its signal region
so a neighbour's background ring can exclude it (BraggIntegrationEngineCPU, the reflection mask); an
unpredicted node is an unclaimed patch of detector, and the neighbouring reflections sweep those pixels
into their background and over-subtract - worst at high angle, where the background dominates. On a
fixed F-centred group that is three quarters of the nodes: measured against the de-novo run of the same
data, <I/sigma> 16.07 against 17.21, CC1/2 0.9862 against 0.9895, ISa 10.93 against 11.92.

Predicting them costs nothing downstream, because both merges already decide absence against the group
they are merging in: the run's own merge drops them again, and the P1 cross-check keeps them because P1
has none. One integration, two correct merges. The -S output becomes byte-identical to the de-novo run
on the five crystals measured, which is the point - pinning a group should not change the answer - and
such a run can never be slower than de novo, since it predicts the same reflections and additionally
skips the space-group search.

The de-novo path is untouched by construction: it already predicted in P.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

unmerged MTZ: a batch header for every image the sweep spans ... b33cae9068

The header set was built from the batches that produced an observation, so an
image that indexed nothing left a hole in the phi series. AIMLESS starts a new
run at such a discontinuity: measured on a 360 degree sweep with 67 unindexed
images it made 23 runs of one sweep, its scale model diverged, Rmeas overflowed
to -1266 and the result no longer correlated with rugnux's own merge (Pearson
0.0018). Renumbering the batches contiguously does not help - the split is on
phi, not on numbering - so the fix is a header per image over the span the
observations cover.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

unmerged MTZ: export the events the merge would keep, not every event ... 762c093b10

The export declared its rows FULL - LDTYPE=2 in the batch header, M=0 in
M/ISYM - while filtering them on --min-partiality alone (0.02), where the 3D
combine also applies --min-captured-fraction (0.7 on rotation) to the same
summed event. AIMLESS reads the FULL declaration, reports "all runs have only
fulls" and ignores FRACTIONCALC, so an event that caught a twentieth of its
rocking curve entered the merge whole, with a small sigma, and was weighted
heavily. There was no cut for the reader to make: the column that would let it
make one is the one the reading program has been told to ignore. Dropping those
events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from
0.985 to 0.993.

FRACTIONCALC itself is unchanged, values above 1 included. Each part's
partiality is the erf pair the predictor computed on that frame, from that
frame's own refined lattice and mosaicity, so the parts of one event do not tile
the rocking curve exactly - the offset steps by the wedge to within 12% of it,
and the sums land in a peak at 1.000 whose 95th percentile is 1.09. It is an
honest estimate of a captured fraction, and it is not the number rugnux scales
on, so clamping it would only hide the spread.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

merged MTZ: FreeR_flag in the convention its own column name carries ... 47ec939c39

The column was written with 1 for the test set and 0 for the working set - the
phenix/CNS numbering - under FreeR_flag, which is CCP4's column name. CCP4's own
freerflag writes 0..19 with 0 as the test bin, and REFMAC5's default FREE 0
reads it that way, so REFMAC5 stopped on every file we wrote: "more than half of
reflections are in free R set", then "Cannot switch free R flag", exit 1.
phenix auto-detects either numbering with equal confidence (measured on both,
score 3 each), and rugnux's own reference-MTZ reader already takes flag 0 as the
test set, so 0 = test is the numbering that works everywhere and the one the
rest of the code assumes.

This changes every merged .mtz we write: a script that reads FreeR_flag == 1 as
the test set has to be inverted. The mmCIF's _refln.status_free is a separate
item with its own convention and is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

-S: check the metric the fixed group needs, not only its centring ... 247130a841

Both guards on a user-fixed space group keyed on the centring differing, so an
axis permutation within the same centring passed: a run that indexed
a=37.909 b=78.031 c=77.594 and was told -S P41212 - which needs a=b with the
4-fold along c - merged through operators that do not act on its own indices,
and --mode scale on it reported COMPLETENESS= 195.3, an arithmetically
impossible number, without complaint.

MetricViolation asks the setting-independent question instead: a group's
rotations must leave the cell's metric tensor invariant. The re-seating arm now
runs on either failure, so a permuted cell can be reindexed into the setting the
group needs rather than merely refused; the refusal arm catches what re-seating
could not fix. --mode scale never reaches either arm, so it gets the same test
where it fixes its cell and its group, which is the only place an impossible
completeness could still be produced.

The tolerance is a refusal bound, so it sits above what a correct answer
reaches: over 113 corpus runs every group determined from its own cell scores
under 0.032 and the permuted case scores 0.764.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

report: name the groups the data could not separate, as keys ... 200a2d53b7

SPACE_GROUP_NAME is a scalar and reads like a determination, and the docs
nominate a grep of it as the interface. Where several groups predict the same
absences it is one of them, picked by convention: the report said so in prose
and nothing machine-readable carried it, so a script recorded a coin flip as an
answer. Refining against a deposited model in the wrong enantiomorph gives
R = 0.549.

Four keys, none of which change which group is adopted: the alternatives the
data cannot separate; whether the enantiomorph was determined, given, or is
undetermined - decidable from the group number alone, so it is answered even
where the search did not run; and the higher point group whose promotion was
refused, with its reason, which until now existed only as prose.

REPORT_VERSION is 5. No existing key changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the change of hand is a label, only the ambiguity choice reindexes ... d3879d0112

Both pages said in one place that adopting a model's enantiomorph moves no
reflection, and in another that it is applied to the merged reflections and
exchanges the Bijvoet mates. The code (ModelValidation.cpp, AdoptModelFrame)
does the former: the model's group is a label on the written files, I(+)/I(-)
stay as measured, and only the merohedral-ambiguity reindex touches reflection
indices. Say that once, consistently, in CPU_DATA_ANALYSIS.md §14.5 and in the
two RUGNUX.md passages that carried the wrong reading.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: name the beam parameter as the PONI and state the laboratory frame ... 694e687fce

Section 1.1 called (x_beam, y_beam) the direct-beam position while constructing
it as the point of normal incidence; on a tilted detector the two differ by
D*tan(tilt), and a reader loading a header direct beam into it would be wrong by
several pixels. Name it as the PONI (the system-wide convention,
DETECTOR_GEOMETRY.md), say which point the construction pins, and state the
laboratory frame's axes and handedness, which the rotation-sign, R-centring and
Bijvoet-hand discussions all silently depended on. Two-theta is computed as a
two-argument arctangent; say so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: pin the sign convention of the back-rotation in 7.3 ... f68fd3d7af

R(phi) rotates the observations by +phi about the stored axis, which makes it
the inverse of the crystal's own rotation - the reading under which the formula
as written is correct, and the same convention that has the exported MTZ batch
axis and the XDS echo negated. Said explicitly, so a reimplementation cannot
take R(phi) for the crystal rotation and land at 2*phi.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: fix the sigma-ratio, the profile-fit term and the outlier cut as written ... 14b79b0314

The 1.109 in 9.2 is the variance ratio of the shipped 4/6/13 stencil (45/408
pixels), not the sigma ratio the sentence attached it to, and the effect is a
bound attained on weak reflections, not uniform. The profile-fit background
term is (sum P/v / sum P^2/v)^2 var(b), matching the code; the undefined w is
gone. The 13.3 refit cut is N^2 times the model variance
(RotationScaleMerge.cpp), not N*sigma^2. The sigma floor is 1 count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: state the q convention at every numeric q and on the q flags ... 9814846ebd

Every q in the pipeline is 2*pi/d (Definitions.h, the azint bin mapping), but
the numbers in 3.3, 7.6 and 10.10 and the --azim-* flags never said so, and a
reader taking q = 1/d would set --azim-q-spacing or --azim-max-q wrong by
2*pi. The 7.6 ice triplet is spelled out so 'within 0.06 of one another' reads
as the adjacent-ring spacing it is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the twinning exemption list, and why -1, 2/m and mmm are not on it ... e94548c614

The stated criterion (a merohedral twin law exists) formally exempts every
holohedral Laue class, but the code (TwinningAnalysis.cpp) deliberately keeps
the low-symmetry ones eligible because pseudo-merohedral twinning through a
special metric cannot be excluded there. Say both halves, and give the
reference values of the statistics so the mmCIF numbers can be read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: disambiguate the reused symbols and the bandwidth definition ... 32adb882eb

sigma_bw is one physical smear written in reciprocal units in 8.2/11.1 and in
pixels in 9; the two Delta-phi of the partiality formula are named; 13.5's
|s| = 1/d is reconciled with the s = sin(theta)/lambda of 10.6/14.2; bandwidth
is the rms spread, with the FWHM-input conversion (/2.355,
BraggIntegrationEngine.cpp) stated. The ice-extinction clause now covers (104),
not only (00l), and the AIMLESS <I/sigma> = 2 constant is named as AIMLESS's
default rather than pointed at a criterion this project does not use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the direction grid bounds the longest findable axis; say so in 5.3 ... 40403d7beb

The ranking-not-sampling observation was measured on the coplanar-shortlist
rescue and is scoped to it now. The angular-resolution bound
theta < d_min/(2a) means the shipped 16384-direction grid resolves axes only
to roughly 120-150 A, far below the 1200 A the accepted maximum admits, and
the failure mode is a plausible sub-cell, not a refusal - the reader raising
fft_max_unit_cell alone deserved to know it cannot work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the quick start tells the truth about inputs, outputs and the cutoff ... 40cb5cd6db

A default rotation run writes seven files, not five - the two the list omitted
are most of the bytes. The input is any NXmx/EIGER master or miniCBF sweep,
which the page said only 350 lines later after twice implying Jungfraujoch
data only. The CC1/2-0.30 resolution trim moves up to the quick start, and
_image.dat's columns are finally named (ScalingResult.cpp writes a # header).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: a default rotation run already writes real anomalous data ... b1ae1ece67

The -A row and FRIEDELS_LAW=TRUE together read as 'without -A your Friedel
pairs were averaged', which is false: the rotation merge always keeps the
Bijvoet split and the default .mtz/.hkl carry it (WriteReflections.cpp,
BuildMergedRows). Say what -A actually changes - the counting basis and the
error model - and give the merged MTZ's exact column labels, which scripting
against phenix needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: hand the output onward - downstream commands and the three blocking facts ... 87b1402c50

The page named phenix, REFMAC, POINTLESS, AIMLESS, careless and SHELXC a dozen
times without one command line, and left unstated the three facts those
programs stop on: the free-flag convention (0=work 1=free; REFMAC needs
FREE 1), the phenix label choice the double intensity array forces, and the
unmerged file's header symmetry and sort order (determined group, sorted
H K L M/ISYM BATCH - WriteReflections.cpp sorts it). A new 'Taking the data
onward' section carries the worked lines, the careless column renames
(BGVAR is a variance), and the Phaser SGALTERNATIVE keywords for the
enantiomorph the report leaves open. The POINTLESS series trap now names
ALLOW OUTOFSEQUENCEFILES instead of telling users to touch their data, the
report-grep block warns that SPACE_GROUP_NAME carries one member of an
enantiomorphic pair by convention, and the P1 cross-check's free set is
declared to be its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the FFT axis ceiling, one-sweep inputs, exit status and _anom.ccp4 ... 86f8edfabc

The longest FFT axis (500 A, no flag; -C moves it) was implied twice and never
stated, and its failure mode is a plausible sub-cell rather than a refusal.
One input is one sweep - said affirmatively where inputs are described instead
of in an aside about pointless. A default 50 A low-resolution cut discards
real reflections on a very large cell; the option row says so. Exit status is
documented for scripts (0 = completed, non-zero = stopped), and the
--model output list gains _anom.ccp4, which was written but undocumented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: credit the L test, FFT indexing, TORO, Niggli, peakfinder8 and SparseCCL ... 6081b6bc43

Six methods the pages name or describe carried no citation: Padilla & Yeates
(the L test), Steller, Bolotovsky & Rossmann (the projection/FFT autoindexing
MOSFLM implements), TORO (what ffbidx implements), Krivy & Gruber and the
ITA lattice-character table (the reduction and Bravais assignment), Cheetah's
peakfinder8 (the per-ring background statistics of the adaptive finder) and
Hennequin et al.'s SparseCCL (already credited to traccc, now also to its
authors). Each gets its ACKNOWLEDGEMENT.md paragraph, a References entry in
CPU_DATA_ANALYSIS.md, and a one-line credit at the algorithm. The
Sheriff & Hendrickson / Popov & Bourenkov entry is re-scoped so each claim
sits on the paper that supports it - P&B 2003 is titled, and credited for the
sigma-aware anisotropy estimation its statistic modelling contains, not for
the tensor and its constraints. All DOIs verified against the publishers;
the SparseCCL DOI resolves to IEEE document 9049184 (IEEE blocks content
scraping, so verified by the resolved document id plus two independent
sources).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: put the generated python-client reference behind one landing page ... 784cf87cc8

The OpenAPI Python client owned the sidebar: DefaultApi's 128 method anchors
plus the 64 hidden-glob model pages were 195 of its 256 entries, because
sphinx_material's globaltoc includes hidden toctrees by default. A new
PYTHON_CLIENT.md landing page carries the links and a hidden glob toctree, and
globaltoc_includehidden is off, so every generated page is still built and
reachable (verified: 64 model pages + DefaultApi render, zero warnings) while
the sidebar drops to 61 entries. docs/review/ joins exclude_patterns so a
local, gitignored review report can never again be rendered into the
published site.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: split RUGNUX.md into one page per job, and put rugnux first ... 376aa0a2e0

The 1274-line page becomes a landing page (quick start, the page map, where it
fits) plus seven pages a reader can answer one question from: installing,
what rugnux reads, running it, integration with other programs, the results
report, advanced usage, and powder calibration. Content is moved, not
rewritten - only the connective sentences at each page top are new. Every
internal anchor is remapped to its new page and every inbound link
(DEPLOYMENT, TOOLS, HDF5, CPU_DATA_ANALYSIS) updated; the built site has zero
Sphinx warnings and an anchor check over the rendered HTML finds no dead
link. index.rst leads with the rugnux group, then acquisition, FPGA,
reference and project.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: split the data-analysis reference into four parts along the pipeline ... 20f869c0b8

CPU_DATA_ANALYSIS.md becomes a short landing page (scope, part map,
references) over four parts in pipeline order - images to spots (0-3),
indexing and geometry (4-7), integration/scaling/merging (8-12), space group
and validation (13-14). Pure moves: the section numbering is continuous and
unchanged, since the rest of the documentation and the source cite sections
by number. Inbound topical links now land on the right part; the build has
zero warnings and the rendered-HTML anchor check finds no dead link.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: an overview page - what a rugnux run does, in order ... 64182bb295

The page that did not exist: one paragraph per stage from opening the file to
the written reflections, each linking into the data-analysis reference part
that carries the depth, with the stills differences at the end. First entry
after the landing page, and the landing page says to read it first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the phenix label override, for the mmCIF as well as the MTZ ... ee4c23b9a2

Both merged formats hit the same refusal - the file carries the mean and the
Bijvoet pairs - but the label vocabularies differ per format and the MTZ
incantation fails on the mmCIF with a fresh error. Give both measured
commands, and say it is one behaviour in two formats, not a difference
between our files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: credit DENZO/SCALEPACK and MOSFLM for the 2D-then-merge architecture ... 67bf380401

Both programs were acknowledged for specifics (profile-fit variances, FFT
autoindexing, post-refinement practice) but not for the paradigm rugnux's
rotation pipeline is built on: integrate each image in 2D, then combine the
partials into fulls, as against XDS's 3D profiles. One paragraph names it;
the citations were already on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: one changelog line for the rugnux documentation work ... 889c9b6cbb

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: point the two source comments at the pages their sections moved to ... d6c6547e0e

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: keep the analysis landing page and drop a trailing transition ... 34e1ec8fb9

The four-way split leaves CPU_DATA_ANALYSIS.md as the landing page the toctree points at, and one
part ended on a horizontal rule, which docutils refuses at the end of a document.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

viewer: label the merge plot over the range it is drawn on ... 4bb3d44983

The merge-statistics window asked for an absolute y-axis - CC1/2 and CCref on
0..100, everything else from 0 - by setting the range on the chart's value axis
after JFJochSimpleChartView::UpdateData had already built the chart. The visible
tick labels are a separate QCategoryAxis whose entries UpdateData had generated
from the range of the data, and those entries were not rebuilt. So the plot was
drawn over 0..100 while the labels down its left edge covered only 91.5..100 and
crowded into the top tenth; the numbers matching the drawn range appeared only
on the right-hand grid axis, which the same call had made visible. CC1/2 showed
it worst because its range is the narrowest, but every metric was affected.

Give UpdateData the range instead, so the ticks and both axes come from one
number. The side-panel azimuthal-integration chart passes no range and renders
pixel-identically.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

leonarski_f added 27 commits 2026-09-02 15:22:54 +02:00

calibration: a fit that hands back the file's own tilt is not a calibration ... ef5da29319

A powder calibration is run because the file's geometry is in doubt, so a fit that
quietly returns part of that file has answered nothing - and it is indistinguishable
from one that worked, down to the residual and the sigmas arranged around it.

On one of four LaB6 exposures of one detector the tilt came out at 2.93x its own
sigma, a hundredth under the significance gate, so it was declined and pinned - at the
master's hardcoded rot1 -0.08, rot2 -0.22 deg. That is eight times the tilt just
refused, on no evidence, and worth 10 px of PONI at 190 mm. rugnux printed it to four
decimal places, wrote the .poni, and exited 0.

Judge the result on provenance instead of on any residual: a geometry is a measurement
only if every parameter in it came from this data. Two ways out of the fits do not
qualify - a covariance that never conditioned, so the fit cannot say what it
determined, and a declined tilt pinned at a non-zero value from the file. A declined
tilt over a file stating no tilt still qualifies, because reporting no tilt is then
exactly what was measured; so does --no-refine-tilt, because a hold that was asked for
is a stated choice and not a silent substitution.

No single number separates the four. rms is 2.465 px against 1.44-1.64; the
significance of all four lies between 2.93 and 4.47, so the gate is nearly a coin flip
at these distances and moving it would only recalibrate on one population; and the
failed fit has the TIGHTEST parameter sigmas of the set, because pinning the tilt
removes the tilt/centre correlation that inflates a good fit's. The spot cross-check
reads 13.5 px against 0.98-2.66, but 10.4 px of that is the pinned tilt moving the
PONI - the same defect one step downstream, not independent evidence.

On a failure rugnux says so, writes no .poni - a PONI file states where the detector is
and has no field in which to say it does not know - writes the JSON with converged
false and the reason beside it, and exits non-zero. The re-binning pass now prefers a
converged refit over a non-converged one whatever its residual, so a tilt an earlier
pass measured is not what a later one gets pinned at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

writer: peakCountUnfiltered in the master, like its four siblings ... 7890149ad6

/entry/MX/peakCountUnfiltered was written only by the data-file plugin.
The four other per-image spot counts are written to both the data files
and the master, so a reader holding just the master got every count
except the unfiltered one.

It was not only a missing dataset. HDF5MetadataSource already reads
/entry/MX/peakCountUnfiltered from the master and falls back to
/entry/MX/nPeaks when it is absent - and nPeaks is the number of spots
*stored* for an image, i.e. after the spot budget truncates. On a VDS
master the fallback therefore substituted the post-filter count for the
unfiltered one silently, and the two differ precisely on the images that
hit the budget.

EndMessage::spot_count is the right member: it and the data-file
plugin's spot_count_total both come from DataMessage::spot_count (the
end-message copy via ScanResultElem::spot_count), and CountSpots() sets
that from spots.size() before FilterSpotsByCount() applies the budget -
"spots found before filtering", as docs/HDF5.md already described it.

The CBOR end block already carried spot_count on both encode and decode,
so no message or protocol change was needed. SaveVectorIfMissing keeps
the NXmxIntegrated case correct, where the data-file plugin has already
written the dataset into the same file.

Verified by writing files in all three formats and dumping them: the new
dataset appears in the legacy, VDS and integrated masters and matches its
sibling peakCountLowRes in value, datatype and read-back precedence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

api: record the beam size at the sample, and write it where NXmx puts it ... 8a04773d1d

dataset_settings gains beam_size_x_um and beam_size_y_um, the horizontal and
vertical size of the X-ray beam where it meets the sample. They follow the same
route total_flux takes - OpenAPI, DatasetSettings, the CBOR start message, the
HDF5 master, and back out of a stored file - and nothing consumes them; this is
metadata a beamline can state and a downstream program can read.

NXmx puts this in the application definition rather than the base class: not
NXbeam's extent (rank 2, nP x 2, per scan point, always FWHM of a rectangular
aperture) but NXmx's own incident_beam_size, a recommended rank-1 two-element
array in the order x, y. Both are live and neither is deprecated, so the choice
matters; the MX definition wins in an MX file. Written as one array with a
units attribute of "m", like every other length in the master, so the settings
hold micrometres and FillMessage converts once.

The unit table of ReadLength_m becomes LengthUnitFactor so the array read can
share it: a master written elsewhere may state this in millimetres.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

api: chi is a reported angle, not a travel limit ... aa6073d41f

The Smargon at the SLS 2.0 MX beamlines reports its own axis positions with
readout noise, so a chi parked at zero comes back as about -1e-7 degrees. The
schema bounded chi_deg to [0, 90], so an ordinary "chi is at zero" setup was
refused - and the same holds at the other end of the arc, where a chi parked at
90 reads just above it. Both end-stops are exactly where a static positioner is
left, so widening the range would only move the problem.

The bound bought nothing. Chi never enters any computation: OpenAPIConvert puts
it in SmargonPosition, DiffractionExperiment::BuildTransformationChain hands it
to a rotation transformation verbatim, HDF5NXmx writes it and HDF5MetadataSource
reads it back. No downstream reads its sign, and a rotation is defined for any
angle. phi, the sibling angle in the same object with the same semantics, has
never been bounded. What was left was a restatement of a hardware travel limit
that the goniometer enforces itself, and its only observable effect was to
refuse a value the instrument genuinely reported.

It was also not enforced where a server-side check would matter: the
cpp-pistache-server generator does not recurse into a nested object model, so
Dataset_settings::validate never calls the Smargon model's. The rejection was
raised by the generated python and TypeScript clients, which do check.

Regenerated the C++ server model, the TypeScript client and redoc-static.html;
python-client is gitignored and comes from gen_python_client.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the python client reference, for the two schemas that gained fields ... 90d0d3c3f9

docs/python_client/docs is generated from the API spec and copied in by
update_version.sh, so it goes stale between releases. The calibration
convergence gate and the beam size both added properties without it; this is
the same generator run those commits should have carried.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

mmcif: the free set is _refln.status = f, not a column of our own ... 7b1caa6ce5

The merged mmCIF carried the R-free flag in `_refln.status_free` as 1/0 and
wrote `o` into `_refln.status` on every row. `_refln.status_free` is not in the
PDBx/mmCIF dictionary - it is absent from mmcif_pdbx v4.0, v5.0, v5.288 and
v5.362, from CCP4's and phenix's shipped copies, and its wwPDB item page is a
404 - and no deposited structure-factor file uses it. `_refln.status` is the
item that carries the free set, with `f` for a test reflection and `o` for a
working one; on a deposition that also carries `_refln.pdbx_r_free_flag` the
two agree exactly.

Measured on a real merged file this run wrote:

  CCP4 cif2mtz   refuses the file outright - "Unexpected context type for
                 category REFLN" from its dictionary-validating parser, exit 1,
                 a 12-byte truncated MTZ. Dropping the non-dictionary column is
                 what fixes it: the same file without it converts.
  gemmi          converts, but its cif2mtz spec knows only `status` and
                 `pdbx_r_free_flag`, so FreeR_flag comes out 1 everywhere and
                 the free set is silently lost - R-free would then be computed
                 on the working set.
  phenix         worked, but only by a filename heuristic matching the words
                 "status" and "free".

Writing `f` while keeping the extra column is worse than either, because phenix
then finds two candidate free-set arrays and refuses the file, so the column
goes in the same change. After it, all three read the same 5% test set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

report: name the direction the anisotropy warning is about ... 3be79fb1b6

The warning said the diffraction limit "runs from 3.34 to 2.45 A depending on
direction" and stopped there, so a reader was told the crystal is anisotropic
and given no direction to act on. It fires on 19 of 113 datasets in the battery.

The eigenvectors are already measured and already written to the merged mmCIF
as `_reflns.pdbx_aniso_B_tensor_eigenvector_N_ortho`; they had simply never
reached the human report. The tensor is fitted on s = frac.mat * (h,k,l), so in
that Cartesian frame a*, b*, c* are the rows of frac.mat and naming the axis is
one dot product per eigenvector. The label is exact in every Laue class that
has a free tensor direction except triclinic, and the cosine is printed so a
loose fit shows as one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

grid scan: the snake reverses on the acquisition row, not the display row ... 92a615085d

GetElementPosFast_step took the snake parity from GetElementPosSlow_step,
which is the DISPLAY row: it is the acquisition row r = image / n_fast,
flipped to (n_slow-1) - r when the slow step is negative. So for a negative
slow step the parity it hands back is parity(n_slow-1) XOR parity(r), and
with an even n_slow that is inverted on every row - the whole raster comes
out mirrored along the fast axis. An odd n_slow leaves it correct, so the
same scan collected with 20 or 25 images disagreed about where image 0 sat:
n_fast=5, fast +1.5 um, slow -2.5 um, snake on gives images 0..4 at fast
index 4,3,2,1,0 with 4 rows and 0,1,2,3,4 with 5 rows. A positive slow step
was correct at both counts, and so was every non-snake configuration.

Snake means the stage reverses direction on alternate rows in acquisition
order, so the parity has to come from the acquisition row. Taking it from
image_number / n_fast directly makes the fast index independent of the slow
axis and of the row count, and drops the call into the display-row function
that caused the coupling. vertical_scan only relabels which axis is fast, so
it was wrong in exactly the same way and is fixed by the same line.

Affected files: written by an affected build, with snake on, a negative
grid slow step (step_y for a horizontal scan, step_x for a vertical one),
and an even number of rows. Their /entry/sample/transformations/grid_scan_x
or _y is mirrored along the fast axis, as was the grid map in the frontend
and the viewer - both mirrored together, which is why neither showed it.

Tests: the interaction of snake with the step signs was never asserted, only
each in isolation, so add a table over snake x {+,- fast step} x {+,- slow
step} x {even, odd row count} x {horizontal, vertical} asserting positions,
plus a case running one affected configuration through GetXContainer_m /
GetYContainer_m and Rearrange. Every pre-existing assertion is unchanged and
still passes; only the four negative-slow, even-row cells of the product
move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

grid scan: say how the stationary spindle angle is stated, and pin it ... 54f2de31bb

A grid scan is a set of stills at a stationary spindle, and the angle it stood
at is what relates one grid to another taken elsewhere on the circle. Users were
finding an all-zero omega in the file and concluding the angle could not be
recorded at all.

It already can, and has since the goniometer and the grid scan stopped being
alternatives: send the axis with step 0 and its start angle, and that angle is
written per image into the NXmx sample chain, read back by reader/, and taken by
dials.import as a set of stills. Measured on a generated 12-image grid: the
placeholder file carries omega = 0 x 12, the same file with the axis sent at
step 0 carries omega = 90 x 12, and dials.import reports "still: 1, sweep: 0"
for both. Nothing in the code needed changing, so nothing was; what was missing
was that nobody could tell, and that no test held the behaviour down.

So: the API and the HDF5 documentation now say it in as many words, and three
tests pin the three legs the value crosses - the OpenAPI request (which used to
drop the grid scan whenever an axis was present, unpinned until now), the CBOR
start message, and the file round trip.

Also corrects a claim two comments and the HDF5 page were making. NXmx can
express "no rotation" perfectly well - a sample may depend_on "." - so the
placeholder is not there for the standard's sake. It is there because dxtbx
cannot read a sample chain of translations alone: strip the rotation axis from a
grid scan master and dials.import dies in get_dxtbx_goniometer with a matmul
dimension mismatch. Recorded so nobody removes the placeholder on the strength
of the standard.

One thing the change does not fix, because it cannot: a stationary angle is
invisible to DIALS when a grid scan is present. dxtbx picks the first varying
axis as the scan axis, which is a grid translation, so the oscillation reads
(0, 0); and with exactly one rotation axis in the chain it builds a single-axis
goniometer whose fixed rotation is the identity, never consulting the angle. The
same angle IS visible when it is the only candidate (oscillation reads (90, 0))
or when a Smargon head puts a second rotation axis in the chain (the setting
rotation then carries it). The value is in the file and correct either way.

indexing: decide an axis harmonic on which cell explains more of the spots ... 476a849c0e

When two first-pass schemes return cells whose primitive volumes differ by a small
integer, the validation-FRAME count cannot tell them apart: a spurious axis multiple
indexes every frame its true sub-cell does, so both reach 60/60 and the count
saturates. The rule that then decided the pair was unconditionally against the larger
cell, so on a crystal with a real pseudo-translation the true cell could not win in
any scheme order.

Ask the same question at the granularity where it does not saturate: how many of the
validation frames' SPOTS does each cell account for? That comparison leans towards the
smaller cell by construction, and needs no threshold to do so. Acceptance is a
fractional-Miller test, so multiplying an axis by n multiplies that axis's residual by
n: the larger cell places every shared reflection n times less accurately than the
sub-cell does, and loses outright the spots that sit in the tolerance margin. The only
thing that can pay for that loss is the class of reflections the larger cell ADDS -
empty for a spurious multiple, the superstructure's satellite rows for a real one - so
the larger cell wins the count only when the extra periodicity is really there. The
count is taken over the validation frames' whole spot lists, which reach far deeper
into each frame's intensity distribution than the first pass's own accumulation cap,
and a superstructure layer is faintest exactly where that cap cuts.

Measured over the five crystals of this corpus where the two schemes return an
integer-related pair, the larger cell accounts for 1.37x and 1.98x the spots on the two
whose true axis was being halved, and 0.30x, 0.36x and 0.71x on the three where the
doubling is spurious. All five come out right: the two keep the true cell and its
deposited space group, the three reproduce the answer the old rule gave, to the digit.
(One of the three has a bistable first pass - four builds give three answers, one of
them without this change at all - so it is not evidence either way; the other two are
reproducible.)

What the added class holds is computed and reported next to the decision, because it
is the physics the count is a consequence of. It is deliberately NOT thresholded, and
that is the part of this that took the measuring. Refuted along the way:

- An occupancy floor, which is how this was first written. Over the five crystals the
  arbiter is asked about, the emptiest index-n class reads 48.2, 41.6, 37.3, 25.4 and
  7.1 %. The two the larger cell should win are the 41.6 and the 37.3, so the three it
  should lose bracket them on both sides, and a real superstructure elsewhere on the
  corpus reads 3.4 %, below all five. Recomputing the same statistic on the merged
  intensities over a sweep of I/sigma cuts leaves the ordering unchanged, so this is a
  continuum and not two populations: no floor separates them, and no amount of extra
  data would. The bimodality a floor needs was an artefact of a calibration set that
  contained no failure.
- Requiring the two cells to stand in a genuine sub/super-lattice relation, the change
  of basis being integral. Measured, all five pairs are index-n relations to within
  0.016 of an integer - the volume ratio is not the weak link.
- Deciding it on the merge, by integrating and merging both cells. The worst failure
  does announce itself there (CC1/2 0.9994 -> 0.9566, ISa 18.6 -> 1.4), but it costs a
  second full integrate-and-merge, and the successes lose 13-30 % of their ISa where
  another failure loses 27 %, so the metric does not separate them either.
- Requiring the sub-lattice class to be the STRONGER of the two, which is the right
  mechanism but the wrong observable at this point in the run: the sign it turns on
  lives in integrated intensities, and the spot finder reports no spot at all where a
  class is absent, so at first pass the same ratio reads 0.90 against 0.39 and 0.27.
  The ordering survives, the sign does not.

The occupancy is maximally wrong on the worst failure because that cell is not a
superstructure at all. A beam-centre error along the spindle translates the derotated
cloud rigidly, and a lattice shifted by half a spacing is indexable only on a doubled
axis - the shift needed scales as 1/(2L), so a long axis is the easy one to
half-offset. In that doubled setting the even class is empty and the zero layer reads
a negative mean intensity, which no crystal can do, while the odd class carries
everything. Such a cell fits no index-n sublattice at all, so its added class reads the
chance value (n-1)/n, the largest the occupancy can take, and the occupancy test
reports the artefact as more real than any genuine superstructure. The spot count sees
it for what it is, at 0.30x.

So the beam-centre warning below is now suppressed only when the larger cell WINS.
Declining it is a fall back to the default, and that warning - which names the beam
centre and offers --estimate-beam-center - is then the most useful thing the run can
say; on the half-offset mode it is the correct diagnosis.

The same question asked unconditionally of the committed cell, the halved-axis probe,
is not included. Measured over the committed cells of 29 crystals, 19 of 189 axes read
above the 2 % floor it would have used and the largest read 30 %, and those largest
readings are on crystals whose committed cell is already wrong - where doubling an axis
is the wrong response. It rescues one crystal whose superstructure layer reads 3.4 %.
That is not worth the rest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

indexing: an axis harmonic is a small multiple, and the Wilson B is XDS's ... 5cc2f811b3

Two unrelated notes, one on each side of what the merge reports.

The axis-harmonic precondition accepted any near-integer volume ratio. Its window
is absolute in a unit integer spacing, so a ratio between two UNRELATED lattices
passes it about a third of the time whatever the multiple is, and at 124x the
pair is not a cell and its harmonic in any sense. That branch was harmless while
the rule was "always take the smaller cell"; deciding the pair on the evidence
makes it reachable, so it is now bounded at 4.

Measured over the corpus: of 44 firings, all 36 at n <= 4 read 1.1 to 51.3
points BELOW the chance occupancy (n-1)/n, so an index-n sub-lattice really
exists in each of them; all 8 above it - two datasets, both decided by 8 to 34
sigma, so not a margin problem - read within 7.8 points OF chance, so none does.
The largest n at which a real sub-lattice was ever seen is 3, and no dataset
moves either way.

Separately, WILSON_B and _reflns.B_iso_Wilson_estimate are fitted to log<I>
directly, which is what XDS's Wilson line does and is not what TRUNCATE,
ctruncate or phenix.xtriage do - they divide out Sigma = sum f^2(s) first.
Leaving Sigma in the slope inflates B by 2-8 A^2 on our own merges, and against
those two programs on the same files this number runs 10-36 A^2 high, in the
same direction every time. The docs said "the analogue of XDS's Wilson-line B",
which is accurate but easy to read past; they now say the two conventions are
not comparable and which one this is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

model validation: place the model, then weight the maps by sigma_A ... 6b738713e8

--model re-fractionalized the model into the data cell and then left it there.
On a non-isomorphous pair that is a placement error, not a cell error: the box
is squeezed, the body inside it is not moved. Six parameters now recover it -
an angle-axis rotation about the model's centroid and a translation, refined
over a 6 / 4.5 / 3.5 A ladder, the scale (k_overall, anisotropic B, k_sol,
b_sol) re-fitted at every evaluation so the target measures the placement and
not the scale. The refinement sees only the working reflections and the step is
committed only if R-free, on the free set it never saw, drops; otherwise the
model goes back where it was read.

Measured on merged lysozyme data against a deposited lysozyme model whose cell
differs by 3.4% in c: R-work 0.559 -> 0.400, R-free 0.591 -> 0.383. Over the
same 3.5 A range the external arbiter (REFMAC rigid body through dimple) works
in, 0.524 -> 0.330 against REFMAC's 0.522 -> 0.355, and the recovered movement
agrees with REFMAC's to 0.25 deg and 0.03 A (3.05 deg / 1.04 A vs 2.76 / 0.98).
2.4 s of added wall clock, 234 structure-factor evaluations.

The map coefficients become 2mFo-DFc and mFo-DFc. sigma_A is estimated by
maximum likelihood per resolution shell on the free reflections only, with the
number of shells taken from the size of the free set so no shell is thin;
centric and acentric reflections carry their own likelihoods, and a centric
reflection's bias-free coefficient is mFo. Cross-checked against CCP4 SIGMAA
on the same reflections: mean FOM 0.404 against its 0.396, with the same
per-shell structure. The figure of merit is written to _maps.mtz so the
weighting can be undone.

The per-shell scaling refusal in fit_model stands - Fobs is never rescaled and
the R-factors are untouched - but m and D are per-dataset, so maps from one
campaign are no longer scaled identically. That is argued at the code and in
docs/CPU_DATA_ANALYSIS_DECISIONS.md 14.4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: worked Phaser and SHELX routes, and the MTZ base dataset they need ... d5cf3a6e58

Both programs were run on merged rugnux output and the commands are the ones
that worked, error messages included.

Phaser takes myrun.mtz with no LABIN, picking IMEAN/SIGIMEAN and the cell,
group and resolution out of the file; MODE MR_AUTO already searches both hands
of an enantiomorphic pair and returned the hand opposite the one in the header
on a tetragonal test. SGALTERNATIVE SELECT ALL covers the screw variants the
report lists as indistinguishable, MODE CCA prints the list without searching,
and a wrong point group has to be re-merged instead. mmCIF is refused by 2.8.3
in both the CCP4 and the phenix build; gemmi cif2mtz gives the same solution.

SHELXC reads only myrun.hkl - it refuses an MTZ and exits 0 while doing it -
and needs CELL and SPAG repeated, since HKLF 4 carries no metadata. A default
rotation merge already writes the Bijvoet split, so no flag is needed and
--no-export-unmerged hides nothing. On a 5 keV cubic sweep SHELXD separated a
space-group pair the merged intensities could not (CFOM 69.4 against 52.0) and
SHELXE solved it, 42.9 % against 15.3 % autotrace CC between the two hands; a
tetragonal sweep at lower completeness and multiplicity did not separate at
all, and that is recorded too.

Verifying it exposed one defect. The merged MTZ was written with its data
dataset at id 0, the id MTZ reserves for HKL_base, so mtzlib dropped the
wavelength and mtzinfo reported the 1.54187 A default - the wrong edge for
anything taking f'/f'' from the file. Writing HKL_base first, as the unmerged
writer already does, puts the real wavelength back; Phaser's solution is
unchanged either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

model: --model is a hypothesis, and it decides nothing until it fits ... c999883eb1

A model supplied with --model rewrote the space group of every reflection
written out on the strength of its file having parsed. Measured on a rotation
dataset merged in P4(1)2(1)2: an unrelated protein and the correct model
rigidly rotated 90 degrees each produced a .mtz, .cif and _unmerged.mtz byte
for byte identical to what the crystal's own model produced - relabelled
P4(3)2(1)2 - at R-free 0.601 and 0.674 against the correct model's 0.591, with
no warning. The adoption was pure space-group-number arithmetic and ran before
the model had been fitted at all.

A model changes exactly two things on the rotation path, and both rewrite the
data: the enantiomorph label and the merohedral indexing. Both now wait for the
fit. Everything else a model produces - R-factors, maps, the rigid-body
placement - is a statement about the MODEL, cannot corrupt a reflection, and is
computed and reported either way.

The gate is not a threshold on R, because no threshold works: the classical
acentric random value is 0.586 at unit scale but 0.550 at the R-minimising
scale, observed nulls land at 0.599-0.615, and the value moves with the model's
atom count and B-factors as much as with the data. Instead the same model is
re-oriented at random about its own centroid five times and run through the
identical path - same scaling, same rigid-body placement, same R - and the real
fit is asked how far above that distribution it sits. R-work carries the
decision: nothing is refined against the working set here, and it has 12615
reflections to R-free's 709. Measured on the case above: the crystal's own model
+15.0 sigma, the unrelated protein +1.8, the 90-degree rotation -1.0, and the
two rejected runs now write files byte-identical to a run with no model.

The nulls are rigid-body refined like the real fit, or the comparison would be
between a placed model and unplaced nulls. That is what the null costs: about
12 s for the five replicates, on a --mode scale run that merges in 2 s.

The merohedral margin gets the same treatment - a random placement also picks a
winner, and measured, by a comparable lead - and the candidate operators are now
enumerated from the DATA's space group. Taking them from the model's enumerated
zero operators wherever the two groups differ, which is exactly the case the
probe exists for: a model in P4(3)2(1)2 against data merged in P4(3) probed
nothing at all, and now probes the twin law.

The anomalous difference map is the only measurement here sensitive to the hand
- inverting the model through the origin moves R-work by less than 1e-4, since
|F(h)| of the inverted structure is |F(-h)| - so where it says the hands
disagree it vetoes the adoption outright, fit or no fit.

Report: MODEL_FIT, MODEL_FIT_SIGMA and the null beside it, MODEL_DECISIONS_TAKEN,
the indexing margin against its null, and MODEL_VALIDATION= PERFORMED as the
counterpart of the failure line. SPACE_GROUP_ENANTIOMORPH= DETERMINED_FROM_MODEL
becomes ASSUMED_FROM_MODEL and is written only where the model was accepted:
nothing here measured the hand, the model asserted it. That is a reason code
changing name and meaning, so REPORT_VERSION is 6.

Stills are untouched: the per-image indexing hand a model can set at integration
time is not reachable on the rotation path and is not gated here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

model: build the null only where the model claims something ... 4af23e9b27

The null costs about 12 s - five replicates, each a full fit and a rigid-body
placement - and it gates exactly two decisions: the enantiomorph label and the
merohedral indexing. A model that claims neither, already in the group the data
were merged in on a crystal with no indexing ambiguity, has nothing for the null
to arbitrate, and paying for it there is 12 s spent gating a decision nobody is
making. That is the isomorphous fragment-screening run - hundreds of crystals of
one form against one apo model, a map wanted seconds after the last image - and
it is the case that has to be fast.

So the null runs only when a decision is pending: an enantiomorph candidate
exists, or the indexing probe returned a winner that is not the identity. The
probe is a handful of scaling fits and had to run first anyway, so the question
is answered before the expensive part starts.

Measured on a rotation dataset, --mode scale --model, interleaved on a loaded
machine: the isomorphous case is 3.6-4.7 s before this whole line of work and
3.8-4.7 s with the gate, against 16.5 s while the null ran unconditionally. The
case that still pays is unchanged at 17 s, and the three wrong-model cases give
bit-identical verdicts and bit-identical files to before.

The rigid-body refinement of the replicates stays: 3 of 5 commit a placement, so
without it the null would score a weaker procedure than the one it judges.

MODEL_FIT gains a third value, NOT_TESTED, alongside ACCEPTED and REJECTED - the
question was never put, which is not the data answering it badly - and the
MODEL_FIT_NULL_* and MODEL_FIT_SIGMA keys are absent with it rather than
reporting a null that was never built. REPORT_VERSION stays 6: MODEL_FIT is
itself new in this release, so its vocabulary has never shipped and a consumer
cannot be relying on the two-value form.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: the phenix label line, and why the MTZ carries no DANO ... 0eb9fb8a8b

Two measurements, neither of which changes what we write.

phenix's "Multiple equally suitable arrays" is a tie between the two
INTENSITY arrays, IMEAN and I(+)/I(-); iotbx scores F and F(+)/F(-) below
them, so writing amplitudes as well is not what causes it. ctruncate's own
output ties in the same place, so this is what phenix does with a CCP4 merged
file rather than something rugnux does to a user. Column order, dataset and
project names, and dropping the amplitudes all leave the tie exactly where it
was; only removing one of the two intensity arrays clears it, and removing the
Bijvoet columns would take the SHELX route with it. So the file stays as it is
and the page now carries the label line per program, both formats, including
the quoting the anomalous one needs. A program that asks iotbx for anomalous
data by preference - hyss, find_peaks_holes, molprobity, the autosol import -
needs nothing at all, which is now said as well.

The page also said the MTZ form of that message names no choices. It names
both, exactly as the mmCIF form does; corrected.

DANO/SIGDANO stay out. Against a ctruncate file, DANO is F(+)-F(-) and SIGDANO
is the quadrature sum of the two sigmas, bit-identical on every reflection, so
the pair is a restatement of columns we already write - and the quadrature sum
is the convention whether or not the mates share a scale model. Every consumer
in the documented routes takes the Bijvoet columns directly, CCP4's own bp3 and
afro ask for them in preference to F/DANO, and adding the pair costs 12.5% of
the merged file while changing nothing phenix or Phaser sees. fft's anomalous
Fourier is the one caller with no other spelling; the ctruncate command that
makes it is now on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

indexing: run the short-axis hypothesis by default ... b9443b4daf

The pass was landed behind RUGNUX_SHORT_AXIS_PASS so that a battery arm could
measure it before it ran on every dataset. This turns it on and keeps the
variable as an escape hatch, spelled the other way round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

model: run the null's replicates at once ... 53a8c428c3

The replicates are independent by construction - the same model under a
different random rotation, scored the same way, nothing flowing between
them - so each takes a copy of the model and of its structure factors and
they run concurrently on the run's own -N budget. The real model is not
moved by any of them, which also drops the save/restore/recompute the
serial loop needed to put it back.

Measured on an idle machine, decision-pending --mode scale --model:
the null 10.0 s -> 2.5 s (median of 5), the whole run 13.9 s -> 6.5 s.
The speedup saturates at 4.0x, not 5x, because the replicates cost
1.58-2.31 s each (how many placement evaluations an orientation needs)
and the null now costs the slowest one; sum/max on the serial run is
4.13x, so 97% of what is there to get. More threads than replicates buy
nothing.

The rotations are drawn up front, in order, from the same fixed seed, so
replicate i gets the same orientation whatever order the threads run in.
A verdict that depended on the interleaving would not be a measurement.
Verified: every reflection file, map, map-MTZ and report is byte-identical
to the serial build's, on the accepted, both rejected and both no-decision
cases, and identical again with -N 1.

Thread safety was read out of the vendored gemmi source, not assumed. Each
replicate owns its DensityCalculator, SolventMasker, Scaling and Structure;
their statics are the const IT92 tables (constant-initialised, never
written on these paths), pocketfft is built with POCKETFFT_CACHE_SIZE 0 and
POCKETFFT_NO_MULTITHREADING so it holds no plan store, Scaling's
Levenberg-Marquardt is stack-local per call, and nothing on these paths
writes into the Structure it was handed. Ceres defaults num_threads to 1,
so the rigid-body solve adds no threads of its own.

Peak RSS is unchanged - the merge, not the null, is the high-water mark.
Five concurrent replicates add 83 MB over the serial null on a 78x78x37 A
cell at 1.56 A, ~17 MB each, which is the two grids at d_min/(2*rate).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

model: nine null replicates, now that they cost the slowest and not the sum ... 8a0bbae3c4

The verdict is (real - mean)/sd of the null sample, so its reliability is set by
how well the SPREAD is pinned, not the mean. The relative error on an sd from n
draws is 1/sqrt(2(n-1)), and a sample that happens to come out narrow is exactly
what turns a model that does not fit into one that appears to.

Measured on the case that sits nearest the gate - an unrelated protein, 1.7-1.8
sigma against a threshold of 3 - the chance of it reading over the gate on a
different seed is 31% at n=3, 17% at n=5 and 6% at n=9.

Nine is affordable only because the replicates now run concurrently: the null
costs the slowest of them rather than their sum, and the slowest of nine is
barely above the slowest of five. Measured end to end on the accepted case, 5.98 s
against 6.46 s for five - inside the run-to-run scatter, so the four extra
replicates are free.

Verdicts unchanged, on more evidence: correct model ACCEPTED at +17.69 sigma
(+14.96 at n=5), an unrelated protein REJECTED at +1.70, the same model rotated
90 degrees REJECTED at -0.37. Both rejected runs still write .mtz, .cif, .hkl and
_unmerged.mtz byte-identical to a run with no model at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

model validation: write the model as it was placed ... 28a58e94b9

`--model` re-fractionalizes the model into the data cell and then places it as
one rigid body, but the placed coordinates never reached disk. On a lysozyme
sweep against a non-isomorphous deposited model the move is 3.058 deg and
1.035 A, so a user overlaying their input model on rugnux's maps was out by
exactly that, and no file on disk corresponded to the maps at all.

`<prefix>_model.cif` is that file: the input's chains, residues, ligands,
waters, B-factors, occupancies and anisotropic Us, at the coordinates the maps
were computed from. `<prefix>.cif` is already the merged reflections, hence the
suffix.

The cell and space group come from the same two values WriteReflections is
given - the unit cell and DiffractionExperiment::GetSpaceGroupOrP1() after
AdoptModelFrame has settled the enantiomorph - so the coordinate file and the
.mtz beside it always agree. Taking them from the input model would not: with
data merged in P4(1)2(1)2 and a P4(3)2(1)2 model, the written reflections take
the model's group, which is neither the data's original label nor, when the
model is rejected, the model's own.

Written whenever the maps are, not only where the rigid-body step was
committed. The model is re-fractionalized and may be relabelled whatever the
placement decided, so an unmoved model is still not the input file; and a model
the null rejected is scored, placed and mapped like any other - the negative
result, and the case where the density is most worth looking at.

Nothing in the tree could write coordinates: gemmi_gph declared to_mmcif.hpp
but src/to_mmcif.cpp had been trimmed from the vendored subset. Both it and
to_pdb.cpp (to_mmcif.cpp calls its use_hetatm) are vendored from the same
gemmi 0.7.5 the rest of gemmi_gph comes from, unmodified, and every header they
include was already there. Same package, same MPL-2.0, same LICENSE.txt already
collected into licenses/gemmi.txt and already listed against `gemmi_gph/` in
THIRD_PARTY_NOTICES.md, so no new row and no new licence text.

Verified end to end: read back with gemmi the file differs from the
re-fractionalized input by exactly the reported 3.058 deg / 1.035 A with 0.0000 A
rms about that rigid move, and REFMAC5 at zero cycles against rugnux's own .mtz
starts at R-free 0.3667 where rugnux reports 0.3826 - against 0.5832 for the
unplaced input model, where rugnux reports 0.5911 before the placement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

tests: the merged MTZ's base dataset, which nothing covered ... 46c50ac744

WriteReflections had no test at all, and the defect it hid was invisible to two
of the three libraries that read an MTZ: with the data dataset written at id 0 -
the id reserved for HKL_base - gemmi and iotbx still returned the real
wavelength, and only CCP4's mtzinfo fell back to the 1.54187 A Cu K-alpha
default. iotbx reading such a file reports two datasets both numbered 0.

The test writes a merged MTZ through the real writer at a wavelength well away
from that default, reads the file back, and asserts the layout the report
depends on: HKL_base at id 0, the data dataset at id 1 carrying the wavelength,
and every data column owned by the data dataset rather than the base - that
ownership is what the reported wavelength is read off. Cell and space group are
checked with it, since they travel in the same header.

Verified against the pre-fix writer (base absent, H K L added on the data
dataset): it fails on the dataset count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N

docs: say what the MTZ base-dataset fix actually changed, and what it did not ... 63a2f6744a

The page said the fix makes mtzinfo and mtzdmp report the collection wavelength,
which implies mtzdmp was wrong before. It was not. Verified across 226 files from
two battery arms, one built before the fix and one after: every pre-fix merged
MTZ reports 1.54187 under mtzinfo and the true wavelength under mtzdmp, truncate,
ctruncate, gemmi, iotbx and phenix.xtriage. No program was found whose output or
behaviour differs between the two files - cad silently repairs the layout on the
way through. So the fix buys a conformant file and a correct mtzinfo line, not a
rescued phasing run, and the f'/f'' consequence is the risk it removes rather
than a measured effect.

Also records that <prefix>_unmerged.mtz still reads 1.54187 under mtzinfo and is
not a regression: its columns sit on HKL_base deliberately, which is what
POINTLESS expects, and the per-batch wavelength AIMLESS and POINTLESS actually
read is correct.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

maps: the map file states the group the reflections are in, and the wavelength ... 0903a8eee3

Two things wrong with <prefix>_maps.mtz beside <prefix>.mtz.

It took the space group from the MODEL, unconditionally. That was invisible while
a model's hand was always adopted; now that a model decides nothing until it beats
its own null, a rejected model that asserts the other enantiomorph leaves the two
files disagreeing - the reflections in the group the data were merged in, the map
coefficients labelled with the model's. An enantiomorphic pair indexes identically,
so the coefficients are the same numbers either way and only the label moves, but
the two groups have different screw translations: a reader that expands symmetry
out of the map file was doing it in the wrong group. It now takes the group the
same way AdoptModelFrame does a few lines below - the model's only where the hand
was adopted, the data's otherwise.

And it carried no wavelength at all, DWAVEL 0.00000 on both datasets, because
nothing in ValidateAgainstModel knew it; it is now passed in from the experiment
the two callers already hold.

Measured on one rotation data set: with the model accepted both files read
P 43 21 2, with an unrelated model rejected both read P 41 21 2, and both carry
the collection wavelength.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

model: say what NOT_TESTED means, which is not that the groups agree ... 4f84b86276

The log line and the report both said a not-tested model "is already in the space
group the data were merged in". That is one way to claim nothing, not the only
one, and it is false for the rest: the gate is "no enantiomorph candidate and the
indexing winner is identity", and a model whose group differs from the data's
without being its enantiomorphic partner sets no candidate either.

Measured: data merged in P 43, model in P 43 21 2 - different groups, no
enantiomorphic relation between them - and the run printed that the model was
already in the data's group.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

model validation: review fixes - the run's output, the origin gauge, and two claims ... b2c0a5ea48

From two independent code reviews of the model-validation work. Nothing here
changes a verdict: the acceptance set still reads +17.69 / +1.70 / -0.37 sigma and
the rejected runs still write files byte-identical to a run with no model.

THE RUN'S OUTPUT. Model validation runs BEFORE the reflection files are written,
and two paths through it could throw: the null's replicates (rotated models fed to
a scaling path that fails outright on data it cannot pair up - the adversarial
input for it), and the mmCIF coordinate writer. Either would have taken the .mtz,
.cif, .hkl and _unmerged.mtz with it, after the merge had already been paid for. A
null that cannot be built is a question that could not be put, which is the
NOT_TESTED state this design already has; a coordinate file that cannot be written
is a lost convenience. Both now degrade instead of aborting.

THE ORIGIN GAUGE. Translating the whole cell content along a free-origin direction
- all three in P1, the unique axis in a polar group - leaves every |F| exactly
unchanged. The code said the LM damping and the R-free gate made that harmless
between them. Neither does: the gate is a function of |F| and is blind to exactly
this, and the gauge column of the Jacobian is not zero but noise divided by the
difference step. It is now projected out after every zone, against the group's own
common fixed subspace. P2_1 alone is a large share of deposited structures, and
the reported shift was partly fiction in every one of them.

TWO CLAIMS THAT WERE FALSE. The report told the user R-work carries the decision
"because nothing was refined against it", six lines from where six placement
parameters are refined against it; the real argument is that the null is placed
the same way, so the optimism is common-mode and cancels. And the constant's own
comment quoted a +4 vs +1 sigma gap where the measurement is +17.7 vs +1.7.

Also: the map file's phase columns are back to [0, 360), the convention they
carried before sigma_A weighting; with no data space group the map file follows
the reflections into P1 rather than taking the model's group; the sigma is floored
against a near-zero null spread rather than only an exactly-zero one; the model is
restored whether or not the solver reported a usable answer; and the zone list is
the ladder walked rather than the ladder planned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

review fixes: the calibration accept test, a NaN warning, and two grid defects ... ab4a89e0a9

From the code review of the writer, API and geometry work.

The powder calibration's re-bin accept test was one-directional. The best-of fold
twelve lines above it is symmetric on purpose - a fit that is not a measurement
never beats one that is, whatever its residual - but the final test applied the
residual clauses regardless, so a CONVERGED re-fit could lose to a non-converged
first pass on rms alone. That is exactly the motivating case: a first pass that
declined the tilt and pinned the header's. It now matches the fold.

The anisotropy warning seeded max_element and min_element on an array that can
hold NaN, and every comparison against NaN is false, so both returned the same
entry and the sentence named one direction twice with nan for both limits. The
non-finite entries are skipped, and where fewer than two directions have a limit
the warning states the anisotropy without naming one.

The grid-scan plot added a SIGNED half-step to a grid laid out in |step|, so with
a negative step every cell centre landed one whole step outside the extent - the
same family as the snake-parity fix, in the same configurations.

And one guaranteed no-op: /entry/MX/imageScaleFactor was written twice.

Changelog: the short-axis pass had no entry at all, and the mmCIF one carried its
rationale.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

indexing: the short-axis pass stays out of a given cell, and four stale comments ... 847e44d718

From the code review of the indexing work.

The short-axis pass took its settings from experiment_, which never carries the
max bound RaiseFFTBoundForKnownCell raises for a given cell - that runs on a local
copy. So with -C naming a cell longer than the default search, the short pool could
not represent the cell that was asked for and could only return a sub-multiple of
it; a sub-multiple indexes every frame its parent does, which is the premise of
this whole comparison, so it would tie and be adopted over the user's own cell. It
now skips a run with a known cell entirely, which is also where it had least to
offer: the same call has already lowered the FLOOR against that cell for the
standard pass, so the two schemes were returning the same lattice and paying two
extra FFTs for it.

Four comments asserted rules the code no longer follows. The one directly above
the decision still said the larger cell is taken as spurious "regardless of scheme
order" - the opposite of what the arbiter does - and a dead assignment below it
stated the same thing a second time. The added-class enumeration is described as
exhaustive; it covers the index-n sublattices with a CYCLIC quotient, which is all
of them at n = 2 and n = 3 but not at n = 4, where a supercell doubling two axes
has a (Z/2)^2 quotient and reads at the chance value instead. That is a report and
decides nothing, but it is quoted in the argument for bounding n at 4, so the
bound rests on thinner evidence at n = 4 than it appears to.

And std::rint where the identical loop in IndexAndRefine uses it and says why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Some checks are pending

Hide all checks

Build Packages / DIALS test (push) Waiting to run

Details

Build Packages / XDS test (durin plugin) (push) Waiting to run

Details

Build Packages / XDS test (JFJoch plugin) (push) Waiting to run

Details

Build Packages / XDS test (neggia plugin) (push) Waiting to run

Details

Build Packages / Generate python client (push) Waiting to run

Details

Build Packages / Build documentation (push) Waiting to run

Details

Build Packages / Create release (push) Waiting to run

Details

Build Packages / Unit tests (pull_request) Waiting to run

Details

Build Packages / build:windows:cuda (pull_request) Waiting to run

Details

Build Packages / build:viewer-tgz:cpu (pull_request) Waiting to run

Details

Build Packages / build:viewer-tgz:cuda (pull_request) Waiting to run

Details

Build Packages / build:rugnux-tgz (x86_64) (pull_request) Waiting to run

Details

Build Packages / build:rugnux:windows (pull_request) Waiting to run

Details

Build Packages / build:rugnux:aarch64 (cross) (pull_request) Waiting to run

Details

Build Packages / build:rpm (rocky8_nocuda) (pull_request) Waiting to run

Details

Build Packages / build:rpm (rocky9_nocuda) (pull_request) Waiting to run

Details

Build Packages / build:rpm (ubuntu2204_nocuda) (pull_request) Waiting to run

Details

Build Packages / build:rpm (ubuntu2404_nocuda) (pull_request) Waiting to run

Details

Build Packages / build:rpm (rocky8_sls9) (pull_request) Waiting to run

Details

Build Packages / build:rpm (rocky9_sls9) (pull_request) Waiting to run

Details

Build Packages / build:rpm (rocky8) (pull_request) Waiting to run

Details

Build Packages / build:rpm (rocky9) (pull_request) Waiting to run

Details

Build Packages / build:rpm (ubuntu2204) (pull_request) Waiting to run

Details

Build Packages / build:rpm (ubuntu2404) (pull_request) Waiting to run

Details

Build Packages / DIALS test (pull_request) Waiting to run

Details

Build Packages / XDS test (durin plugin) (pull_request) Waiting to run

Details

Build Packages / XDS test (JFJoch plugin) (pull_request) Waiting to run

Details

Build Packages / XDS test (neggia plugin) (pull_request) Waiting to run

Details

Build Packages / Generate python client (pull_request) Waiting to run

Details

Build Packages / Build documentation (pull_request) Waiting to run

Details

Build Packages / Create release (pull_request) Waiting to run

Details

Build Packages / Unit tests (push) In progress

Details

Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 10m31s

Details

Build Packages / build:rpm (ubuntu2404_nocuda) (push) In progress

Details

Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 16m43s

Details

Build Packages / build:rpm (rocky8_sls9) (push) In progress

Details

Build Packages / build:viewer-tgz:cpu (push) Successful in 17m52s

Details

Build Packages / build:rpm (rocky9_sls9) (push) In progress

Details

Build Packages / build:viewer-tgz:cuda (push) Successful in 20m22s

Details

Build Packages / build:rpm (rocky8) (push) In progress

Details

Build Packages / build:windows:nocuda (push) Successful in 21m42s

Details

Build Packages / build:rugnux:windows (push) In progress

Details

Build Packages / build:windows:cuda (push) Successful in 23m10s

Details

Build Packages / build:windows:nocuda (pull_request) In progress

Details

Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 25m49s

Details

Build Packages / build:rpm (rocky9) (push) In progress

Details

Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 26m55s

Details

Build Packages / build:rpm (ubuntu2204) (push) In progress

Details

Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 31m22s

Details

Build Packages / build:rpm (ubuntu2404) (push) In progress

Details

You are not authorized to merge this pull request.

This pull request can be merged automatically.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin rc166:rc166

git checkout rc166

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

appleb_m (appleb_m) buntschu_d (buntschu_d) duan_j (duan_j) glettig_w (glettig_w) gotthard_g (gotthard_g) leonarski_f (leonarski_f) mx-gitea perl_d (perl_d) takaba_k (takaba_k)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mx/Jungfraujoch#76