dataset_settings gains beam_size_x_um and beam_size_y_um, the horizontal and
vertical size of the X-ray beam where it meets the sample. They follow the same
route total_flux takes - OpenAPI, DatasetSettings, the CBOR start message, the
HDF5 master, and back out of a stored file - and nothing consumes them; this is
metadata a beamline can state and a downstream program can read.
NXmx puts this in the application definition rather than the base class: not
NXbeam's extent (rank 2, nP x 2, per scan point, always FWHM of a rectangular
aperture) but NXmx's own incident_beam_size, a recommended rank-1 two-element
array in the order x, y. Both are live and neither is deprecated, so the choice
matters; the MX definition wins in an MX file. Written as one array with a
units attribute of "m", like every other length in the master, so the settings
hold micrometres and FillMessage converts once.
The unit table of ReadLength_m becomes LengthUnitFactor so the array read can
share it: a master written elsewhere may state this in millimetres.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
/entry/MX/peakCountUnfiltered was written only by the data-file plugin.
The four other per-image spot counts are written to both the data files
and the master, so a reader holding just the master got every count
except the unfiltered one.
It was not only a missing dataset. HDF5MetadataSource already reads
/entry/MX/peakCountUnfiltered from the master and falls back to
/entry/MX/nPeaks when it is absent - and nPeaks is the number of spots
*stored* for an image, i.e. after the spot budget truncates. On a VDS
master the fallback therefore substituted the post-filter count for the
unfiltered one silently, and the two differ precisely on the images that
hit the budget.
EndMessage::spot_count is the right member: it and the data-file
plugin's spot_count_total both come from DataMessage::spot_count (the
end-message copy via ScanResultElem::spot_count), and CountSpots() sets
that from spots.size() before FilterSpotsByCount() applies the budget -
"spots found before filtering", as docs/HDF5.md already described it.
The CBOR end block already carried spot_count on both encode and decode,
so no message or protocol change was needed. SaveVectorIfMissing keeps
the NXmxIntegrated case correct, where the data-file plugin has already
written the dataset into the same file.
Verified by writing files in all three formats and dumping them: the new
dataset appears in the legacy, VDS and integrated masters and matches its
sibling peakCountLowRes in value, datatype and read-back precedence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
A powder calibration is run because the file's geometry is in doubt, so a fit that
quietly returns part of that file has answered nothing - and it is indistinguishable
from one that worked, down to the residual and the sigmas arranged around it.
On one of four LaB6 exposures of one detector the tilt came out at 2.93x its own
sigma, a hundredth under the significance gate, so it was declined and pinned - at the
master's hardcoded rot1 -0.08, rot2 -0.22 deg. That is eight times the tilt just
refused, on no evidence, and worth 10 px of PONI at 190 mm. rugnux printed it to four
decimal places, wrote the .poni, and exited 0.
Judge the result on provenance instead of on any residual: a geometry is a measurement
only if every parameter in it came from this data. Two ways out of the fits do not
qualify - a covariance that never conditioned, so the fit cannot say what it
determined, and a declined tilt pinned at a non-zero value from the file. A declined
tilt over a file stating no tilt still qualifies, because reporting no tilt is then
exactly what was measured; so does --no-refine-tilt, because a hold that was asked for
is a stated choice and not a silent substitution.
No single number separates the four. rms is 2.465 px against 1.44-1.64; the
significance of all four lies between 2.93 and 4.47, so the gate is nearly a coin flip
at these distances and moving it would only recalibrate on one population; and the
failed fit has the TIGHTEST parameter sigmas of the set, because pinning the tilt
removes the tilt/centre correlation that inflates a good fit's. The spot cross-check
reads 13.5 px against 0.98-2.66, but 10.4 px of that is the pinned tilt moving the
PONI - the same defect one step downstream, not independent evidence.
On a failure rugnux says so, writes no .poni - a PONI file states where the detector is
and has no field in which to say it does not know - writes the JSON with converged
false and the reason beside it, and exits non-zero. The re-binning pass now prefers a
converged refit over a non-converged one whatever its residual, so a tilt an earlier
pass measured is not what a later one gets pinned at.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merge-statistics window asked for an absolute y-axis - CC1/2 and CCref on
0..100, everything else from 0 - by setting the range on the chart's value axis
after JFJochSimpleChartView::UpdateData had already built the chart. The visible
tick labels are a separate QCategoryAxis whose entries UpdateData had generated
from the range of the data, and those entries were not rebuilt. So the plot was
drawn over 0..100 while the labels down its left edge covered only 91.5..100 and
crowded into the top tenth; the numbers matching the drawn range appeared only
on the right-hand grid axis, which the same call had made visible. CC1/2 showed
it worst because its range is the narrowest, but every metric was affected.
Give UpdateData the range instead, so the ticks and both axes come from one
number. The side-panel azimuthal-integration chart passes no range and renders
pixel-identically.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The four-way split leaves CPU_DATA_ANALYSIS.md as the landing page the toctree points at, and one
part ended on a horizontal rule, which docutils refuses at the end of a document.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both programs were acknowledged for specifics (profile-fit variances, FFT
autoindexing, post-refinement practice) but not for the paradigm rugnux's
rotation pipeline is built on: integrate each image in 2D, then combine the
partials into fulls, as against XDS's 3D profiles. One paragraph names it;
the citations were already on the page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both merged formats hit the same refusal - the file carries the mean and the
Bijvoet pairs - but the label vocabularies differ per format and the MTZ
incantation fails on the mmCIF with a fresh error. Give both measured
commands, and say it is one behaviour in two formats, not a difference
between our files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The page that did not exist: one paragraph per stage from opening the file to
the written reflections, each linking into the data-analysis reference part
that carries the depth, with the stills differences at the end. First entry
after the landing page, and the landing page says to read it first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
CPU_DATA_ANALYSIS.md becomes a short landing page (scope, part map,
references) over four parts in pipeline order - images to spots (0-3),
indexing and geometry (4-7), integration/scaling/merging (8-12), space group
and validation (13-14). Pure moves: the section numbering is continuous and
unchanged, since the rest of the documentation and the source cite sections
by number. Inbound topical links now land on the right part; the build has
zero warnings and the rendered-HTML anchor check finds no dead link.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The 1274-line page becomes a landing page (quick start, the page map, where it
fits) plus seven pages a reader can answer one question from: installing,
what rugnux reads, running it, integration with other programs, the results
report, advanced usage, and powder calibration. Content is moved, not
rewritten - only the connective sentences at each page top are new. Every
internal anchor is remapped to its new page and every inbound link
(DEPLOYMENT, TOOLS, HDF5, CPU_DATA_ANALYSIS) updated; the built site has zero
Sphinx warnings and an anchor check over the rendered HTML finds no dead
link. index.rst leads with the rugnux group, then acquisition, FPGA,
reference and project.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The OpenAPI Python client owned the sidebar: DefaultApi's 128 method anchors
plus the 64 hidden-glob model pages were 195 of its 256 entries, because
sphinx_material's globaltoc includes hidden toctrees by default. A new
PYTHON_CLIENT.md landing page carries the links and a hidden glob toctree, and
globaltoc_includehidden is off, so every generated page is still built and
reachable (verified: 64 model pages + DefaultApi render, zero warnings) while
the sidebar drops to 61 entries. docs/review/ joins exclude_patterns so a
local, gitignored review report can never again be rendered into the
published site.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Six methods the pages name or describe carried no citation: Padilla & Yeates
(the L test), Steller, Bolotovsky & Rossmann (the projection/FFT autoindexing
MOSFLM implements), TORO (what ffbidx implements), Krivy & Gruber and the
ITA lattice-character table (the reduction and Bravais assignment), Cheetah's
peakfinder8 (the per-ring background statistics of the adaptive finder) and
Hennequin et al.'s SparseCCL (already credited to traccc, now also to its
authors). Each gets its ACKNOWLEDGEMENT.md paragraph, a References entry in
CPU_DATA_ANALYSIS.md, and a one-line credit at the algorithm. The
Sheriff & Hendrickson / Popov & Bourenkov entry is re-scoped so each claim
sits on the paper that supports it - P&B 2003 is titled, and credited for the
sigma-aware anisotropy estimation its statistic modelling contains, not for
the tensor and its constraints. All DOIs verified against the publishers;
the SparseCCL DOI resolves to IEEE document 9049184 (IEEE blocks content
scraping, so verified by the resolved document id plus two independent
sources).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The longest FFT axis (500 A, no flag; -C moves it) was implied twice and never
stated, and its failure mode is a plausible sub-cell rather than a refusal.
One input is one sweep - said affirmatively where inputs are described instead
of in an aside about pointless. A default 50 A low-resolution cut discards
real reflections on a very large cell; the option row says so. Exit status is
documented for scripts (0 = completed, non-zero = stopped), and the
--model output list gains _anom.ccp4, which was written but undocumented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The page named phenix, REFMAC, POINTLESS, AIMLESS, careless and SHELXC a dozen
times without one command line, and left unstated the three facts those
programs stop on: the free-flag convention (0=work 1=free; REFMAC needs
FREE 1), the phenix label choice the double intensity array forces, and the
unmerged file's header symmetry and sort order (determined group, sorted
H K L M/ISYM BATCH - WriteReflections.cpp sorts it). A new 'Taking the data
onward' section carries the worked lines, the careless column renames
(BGVAR is a variance), and the Phaser SGALTERNATIVE keywords for the
enantiomorph the report leaves open. The POINTLESS series trap now names
ALLOW OUTOFSEQUENCEFILES instead of telling users to touch their data, the
report-grep block warns that SPACE_GROUP_NAME carries one member of an
enantiomorphic pair by convention, and the P1 cross-check's free set is
declared to be its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The -A row and FRIEDELS_LAW=TRUE together read as 'without -A your Friedel
pairs were averaged', which is false: the rotation merge always keeps the
Bijvoet split and the default .mtz/.hkl carry it (WriteReflections.cpp,
BuildMergedRows). Say what -A actually changes - the counting basis and the
error model - and give the merged MTZ's exact column labels, which scripting
against phenix needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
A default rotation run writes seven files, not five - the two the list omitted
are most of the bytes. The input is any NXmx/EIGER master or miniCBF sweep,
which the page said only 350 lines later after twice implying Jungfraujoch
data only. The CC1/2-0.30 resolution trim moves up to the quick start, and
_image.dat's columns are finally named (ScalingResult.cpp writes a # header).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The ranking-not-sampling observation was measured on the coplanar-shortlist
rescue and is scoped to it now. The angular-resolution bound
theta < d_min/(2a) means the shipped 16384-direction grid resolves axes only
to roughly 120-150 A, far below the 1200 A the accepted maximum admits, and
the failure mode is a plausible sub-cell, not a refusal - the reader raising
fft_max_unit_cell alone deserved to know it cannot work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
sigma_bw is one physical smear written in reciprocal units in 8.2/11.1 and in
pixels in 9; the two Delta-phi of the partiality formula are named; 13.5's
|s| = 1/d is reconciled with the s = sin(theta)/lambda of 10.6/14.2; bandwidth
is the rms spread, with the FWHM-input conversion (/2.355,
BraggIntegrationEngine.cpp) stated. The ice-extinction clause now covers (104),
not only (00l), and the AIMLESS <I/sigma> = 2 constant is named as AIMLESS's
default rather than pointed at a criterion this project does not use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The stated criterion (a merohedral twin law exists) formally exempts every
holohedral Laue class, but the code (TwinningAnalysis.cpp) deliberately keeps
the low-symmetry ones eligible because pseudo-merohedral twinning through a
special metric cannot be excluded there. Say both halves, and give the
reference values of the statistics so the mmCIF numbers can be read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Every q in the pipeline is 2*pi/d (Definitions.h, the azint bin mapping), but
the numbers in 3.3, 7.6 and 10.10 and the --azim-* flags never said so, and a
reader taking q = 1/d would set --azim-q-spacing or --azim-max-q wrong by
2*pi. The 7.6 ice triplet is spelled out so 'within 0.06 of one another' reads
as the adjacent-ring spacing it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The 1.109 in 9.2 is the variance ratio of the shipped 4/6/13 stencil (45/408
pixels), not the sigma ratio the sentence attached it to, and the effect is a
bound attained on weak reflections, not uniform. The profile-fit background
term is (sum P/v / sum P^2/v)^2 var(b), matching the code; the undefined w is
gone. The 13.3 refit cut is N^2 times the model variance
(RotationScaleMerge.cpp), not N*sigma^2. The sigma floor is 1 count.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
R(phi) rotates the observations by +phi about the stored axis, which makes it
the inverse of the crystal's own rotation - the reading under which the formula
as written is correct, and the same convention that has the exported MTZ batch
axis and the XDS echo negated. Said explicitly, so a reimplementation cannot
take R(phi) for the crystal rotation and land at 2*phi.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Section 1.1 called (x_beam, y_beam) the direct-beam position while constructing
it as the point of normal incidence; on a tilted detector the two differ by
D*tan(tilt), and a reader loading a header direct beam into it would be wrong by
several pixels. Name it as the PONI (the system-wide convention,
DETECTOR_GEOMETRY.md), say which point the construction pins, and state the
laboratory frame's axes and handedness, which the rotation-sign, R-centring and
Bijvoet-hand discussions all silently depended on. Two-theta is computed as a
two-argument arctangent; say so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both pages said in one place that adopting a model's enantiomorph moves no
reflection, and in another that it is applied to the merged reflections and
exchanges the Bijvoet mates. The code (ModelValidation.cpp, AdoptModelFrame)
does the former: the model's group is a label on the written files, I(+)/I(-)
stay as measured, and only the merohedral-ambiguity reindex touches reflection
indices. Say that once, consistently, in CPU_DATA_ANALYSIS.md §14.5 and in the
two RUGNUX.md passages that carried the wrong reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
SPACE_GROUP_NAME is a scalar and reads like a determination, and the docs
nominate a grep of it as the interface. Where several groups predict the same
absences it is one of them, picked by convention: the report said so in prose
and nothing machine-readable carried it, so a script recorded a coin flip as an
answer. Refining against a deposited model in the wrong enantiomorph gives
R = 0.549.
Four keys, none of which change which group is adopted: the alternatives the
data cannot separate; whether the enantiomorph was determined, given, or is
undetermined - decidable from the group number alone, so it is answered even
where the search did not run; and the higher point group whose promotion was
refused, with its reason, which until now existed only as prose.
REPORT_VERSION is 5. No existing key changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both guards on a user-fixed space group keyed on the centring differing, so an
axis permutation within the same centring passed: a run that indexed
a=37.909 b=78.031 c=77.594 and was told -S P41212 - which needs a=b with the
4-fold along c - merged through operators that do not act on its own indices,
and --mode scale on it reported COMPLETENESS= 195.3, an arithmetically
impossible number, without complaint.
MetricViolation asks the setting-independent question instead: a group's
rotations must leave the cell's metric tensor invariant. The re-seating arm now
runs on either failure, so a permuted cell can be reindexed into the setting the
group needs rather than merely refused; the refusal arm catches what re-seating
could not fix. --mode scale never reaches either arm, so it gets the same test
where it fixes its cell and its group, which is the only place an impossible
completeness could still be produced.
The tolerance is a refusal bound, so it sits above what a correct answer
reaches: over 113 corpus runs every group determined from its own cell scores
under 0.032 and the permuted case scores 0.764.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The column was written with 1 for the test set and 0 for the working set - the
phenix/CNS numbering - under FreeR_flag, which is CCP4's column name. CCP4's own
freerflag writes 0..19 with 0 as the test bin, and REFMAC5's default FREE 0
reads it that way, so REFMAC5 stopped on every file we wrote: "more than half of
reflections are in free R set", then "Cannot switch free R flag", exit 1.
phenix auto-detects either numbering with equal confidence (measured on both,
score 3 each), and rugnux's own reference-MTZ reader already takes flag 0 as the
test set, so 0 = test is the numbering that works everywhere and the one the
rest of the code assumes.
This changes every merged .mtz we write: a script that reads FreeR_flag == 1 as
the test set has to be inverted. The mmCIF's _refln.status_free is a separate
item with its own convention and is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The export declared its rows FULL - LDTYPE=2 in the batch header, M=0 in
M/ISYM - while filtering them on --min-partiality alone (0.02), where the 3D
combine also applies --min-captured-fraction (0.7 on rotation) to the same
summed event. AIMLESS reads the FULL declaration, reports "all runs have only
fulls" and ignores FRACTIONCALC, so an event that caught a twentieth of its
rocking curve entered the merge whole, with a small sigma, and was weighted
heavily. There was no cut for the reader to make: the column that would let it
make one is the one the reading program has been told to ignore. Dropping those
events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from
0.985 to 0.993.
FRACTIONCALC itself is unchanged, values above 1 included. Each part's
partiality is the erf pair the predictor computed on that frame, from that
frame's own refined lattice and mosaicity, so the parts of one event do not tile
the rocking curve exactly - the offset steps by the wedge to within 12% of it,
and the sums land in a peak at 1.000 whose 95th percentile is 1.09. It is an
honest estimate of a captured fraction, and it is not the number rugnux scales
on, so clamping it would only hide the spread.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The header set was built from the batches that produced an observation, so an
image that indexed nothing left a hole in the phi series. AIMLESS starts a new
run at such a discontinuity: measured on a 360 degree sweep with 67 unindexed
images it made 23 runs of one sweep, its scale model diverged, Rmeas overflowed
to -1266 and the result no longer correlated with rugnux's own merge (Pearson
0.0018). Renumbering the batches contiguously does not help - the split is on
phi, not on numbering - so the fix is a header per image over the span the
observations cover.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
With -S the prediction rejected the fixed group's centring absences, so those reflections were never
integrated. Two things followed, and only the second was known.
The P1 cross-check was withheld on such a run, because a P1 merge missing whole centring classes is
misleading rather than merely small - 50% of the nodes on an I lattice, 75% on F, 67% on R. That was
the documented reason and it was right.
The unknown one is that it cost intensity accuracy. Every predicted reflection marks its signal region
so a neighbour's background ring can exclude it (BraggIntegrationEngineCPU, the reflection mask); an
unpredicted node is an unclaimed patch of detector, and the neighbouring reflections sweep those pixels
into their background and over-subtract - worst at high angle, where the background dominates. On a
fixed F-centred group that is three quarters of the nodes: measured against the de-novo run of the same
data, <I/sigma> 16.07 against 17.21, CC1/2 0.9862 against 0.9895, ISa 10.93 against 11.92.
Predicting them costs nothing downstream, because both merges already decide absence against the group
they are merging in: the run's own merge drops them again, and the P1 cross-check keeps them because P1
has none. One integration, two correct merges. The -S output becomes byte-identical to the de-novo run
on the five crystals measured, which is the point - pinning a group should not change the answer - and
such a run can never be slower than de novo, since it predicts the same reflections and additionally
skips the space-group search.
The de-novo path is untouched by construction: it already predicted in P.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The guard that catches the bistable supercell collapse between the two rotation passes took the
volume of each pass's CONVENTIONAL cell. A centred conventional cell is an exact integer multiple
of its primitive one - C and I twice, R three times, F four times - so two settings of the same
lattice differ by exactly that factor, and the guard read a change of setting as a supercell. It
then forced pass 1's result, and with it pass 1's lower symmetry, on a pass that had found the
same lattice in a better one. Ten of 116 rotation datasets tripped it, every one at an exact
centring multiplicity: seven at 2.00x, one at 3.06x, one at 3.99x. The last is an F-centred
lattice; it was held in P1 where the second pass had found it centred orthorhombic.
The two structurally identical guards inside RunPipeline already convert with ToPrimitive first,
and their comments say why. This one could not: ProcessResult carried the consensus cell with no
centring beside it, so at the comparison there was nothing to convert with. The centring is now
carried alongside the cell, set at each of the four places the cell is - the finalized rotation
lattice, the reindex into a user-fixed group's setting, the reduction of a doubled cell, and the
committed higher-symmetry reindex - and the guard converts both sides before comparing. Nothing
else reads it; every other consumer of the cell is unchanged.
On the F-centred dataset the run goes from P1 at 2.232 A to I 2 2 2 at 2.077 A, multiplicity
1.86 to 7.07.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The centroid was a first moment of the RAW counts over the signal disk, so it weighted signal plus
background. The background is flat over a disk centred on the PREDICTION, which makes its own centroid
the prediction exactly: it adds nothing to the displacement and everything to the denominator, and the
measured offset comes out shrunk by I/(I + n*bkg).
That factor is worst where the background dominates the signal, which is at high resolution - so the
one consumer of this quantity, the geometry post-refinement, fits the beam centre and the detector
distance on displacements that are systematically too small, by a factor that varies with resolution.
An estimator whose bias depends on the very coordinate it is correcting.
Subtracting a flat pedestal from a first moment is exact, and the background is not known until the
ring has been read, so the positions of the pixels behind the intensity sum are accumulated alongside
it and the correction is applied afterwards: sum(x*(px-bkg)) = sum(x*px) - bkg*sum(x). Both engines,
identically. Where nothing rises above background there is no signal centroid to compute and the raw
one is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Computing it needs a DiffractionGeometry, and that refuses a detector distance under 1 mm or a pixel
size of zero. A start message can legitimately carry neither: the writer's own pre-flight check asks
it to prove it can create the files for a dataset that never describes a detector, and the master
write then threw where it used to succeed - Preflight_TCP fails at rc.166 and passes at rc.165.
The dataset is a convenience for whoever reads the file later, so where the geometry is not there to
compute it from it is simply not written. Everything else in the master is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Post-refinement was handed the HEADER geometry while the outcomes it refines had been integrated at
the first pass's own, and only three numbers came back - beam_x, beam_y, distance. The tilt was
dropped at both ends, so the pass that ships started at the header tilt with a beam that step B had
moved to absorb a tilt the images do not have. On one in-house mounting the first pass ends at
rot1 -0.2995 deg and the shipped value is -0.0992, against a powder calibration of -0.282: the two
geometries the two passes use differ by the whole of the tilt error, inside a single run.
Both ends move together, and they have to. Step B holds the tilt fixed while it fits the beam and the
distance, so the beam it returns is only meaningful about the tilt it was given: handing it the refined
tilt without carrying that tilt forward, or carrying the tilt forward under a beam fitted about the
header's, each describe a geometry that never existed. The carrier goes from three floats to five and
the header snapshot the quality guard reverts to grows with it, so a rejected second pass still returns
to the geometry the file states.
This is not a tilt measurement and does not make one: rot1 remains the gauge direction of a single-axis
rotation experiment. It only stops the two passes of one run from working in two different geometries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The residual applies angle_rad * |rot_vec| and rot_vec is a free three-vector, so the first pass has
been fitting a goniometer rotation SCALE nobody asked for. GoniometerAxis::Axis() then normalises it
away on write-back, and RotationIndexer scores the candidate with the normalised axis - so the cell
that won the fit is judged under a rotation model the fit did not use. Measured over 43 rotation
datasets: the length reaches 1.2%, and the fit-vs-score disagreement a median 0.124 deg and up to
6.19 deg of goniometer angle, against rocking widths of 0.05-0.36 deg. That score picks the lattice
class, which nothing later revisits.
The fitted length is not a usable measurement of anything either: on synthetic data it recovers 54%
of a known scale error, repeated first passes on one dataset disagree with each other in sign, 26 of
43 datasets disagree with themselves, and on the one dataset with a proven 1.3% stage fault it comes
out negative. It is absorbing other systematics. The rotation scale is measured properly, once, with
cross-validation and gates, in PostRefine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both directions of the tilt now go through the same restraint, with a budget each in shared
direct-beam pixels; zero means free. Swapping which component is held, or holding both, is a change to
two constants. The perpendicular budget is zero today, which is the arrangement the measurements
support: it is the component whose conditioning tracks the 2theta the fit reaches, while the parallel
one's does not move with 2theta at all - a conditioning number that ignores the data is the prior
talking.
This is a clarity change, not a precision one, and the record should not read otherwise: the standing
'freeing the tilt inflates both beam components' cost is the perpendicular component's doing, not the
parallel one's, so restraining the parallel one recovers essentially none of it. What it buys is that a
beam-centre error is no longer reported as an angle.
The restraint is correct here because the crystal orientation is free alongside the tilt and absorbs
the spindle-parallel difference. A stage that freezes the orientation has no such compensator, so
whether that component is measurable THERE is a separate question; the comment no longer reads as a
claim about the tilt everywhere.
Two corrections to the restraint added in the previous commit, both measured.
The beam prior compared the goniometer vector's LABORATORY components against an index into the
PIXEL-frame beam centre. det_matrix is PoniRotMatrix * DetectorOrientation::Matrix(), so on a quarter
turn of 1 or 3 the pixel X axis is the laboratory Y axis and the comparison picks the wrong component -
pinning the determined one and freeing the gauge one, which is worse than having no prior. Two datasets
in the corpus are in that state. The direction now comes from projecting the spindle onto the pixel
axes' own laboratory images, which is exact for any orientation, any tilt and a spindle at any angle,
and equal to the old comparison when the orientation is the identity. The tilt keeps the spindle's
laboratory components, because rot1 and rot2 are laboratory rotations applied outside that orientation
matrix - same physical direction, each in the frame its own parameters live in.
And the tilt's restraint is three times tighter than the beam's, in shared direct-beam pixels, because
the data determine the SUM of the two: at equal budgets the shift splits evenly and the reported tilt
still followed the starting beam centre at 41% of one-for-one. At a one-pixel budget it follows at 8%,
from 79% before this work, while the free component moves by 0.0035 deg over the same eight-pixel swing
and the indexing rate does not change.
The comment says plainly what the change does not do: it does not make the tilt accurate.
The beam prior at XtalOptimizer already treats the beam component parallel to the spindle as the gauge
direction of a single-axis rotation experiment and restrains it toward the value it was handed. The
detector tilt is that same gauge described a second time, and it was left free with a flat +-3 deg box.
The correspondence is not an observation about one beamline, it follows from the convention:
PoniRotMatrix builds the detector matrix as R(-rot3,z)R(-rot2,x)R(rot1,y), which puts the direct beam
at (beam_x - rot1*D/pixel, beam_y - rot2*D/pixel). rot1 IS beam_x written as an angle and rot2 IS
beam_y, component for component. So on a horizontal spindle the gauge tilt is rot1 and rot2 is
refined; on a vertical spindle it is the other way round.
With only one end of the alias restrained, a beam-centre error the prior refuses to absorb reappears
as an angle: measured elsewhere in this campaign, refined rot1 tracks the starting beam centre at
0.072 deg per pixel against a geometric 0.080, while the refined beam never leaves its anchor by more
than 0.24 px. Restraining both ends, in the same direction and to the same three-pixel budget, leaves
the determined component - the direct beam, and the tilt perpendicular to the spindle - alone.
Both restraints now take their direction from one projection of the spindle onto the detector plane
rather than from two independent snaps to whichever of X/Y dominates, so they cannot disagree, and a
spindle at any angle is handled. On a spindle along a detector axis the projection is the snap.
The section had grown to eighteen entries, several of them describing the same report or carrying
detail that belongs in a commit message. Consolidated to thirteen one-liners, each naming the program
it concerns - rugnux, jfjoch_writer, jfjoch_broker, jfjoch_viewer - so a reader can find what changed
in the part they use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The report invited the reader to check REFINED_DETECTOR_TILT against a powder
calibration. Measured over two detectors' worth of crystals, that check
misleads: the rotation fit does one outer round, so it leaves its starting
value by only a small and crystal-dependent fraction of the distance to the
calibrated value, and on the worse of the two detectors the per-crystal median
sits an order of magnitude further from the powder answer than either method's
uncertainty. A user comparing one run against their own calibration would
conclude the calibration was wrong.
What survives the aliasing is the direct beam, which is already printed beside
it, and the MEDIAN of the tilt over several crystals on one detector - enough
to show up a placeholder or a stale value in the file, not enough to replace a
calibration. Say that, and say not to feed one run's value back into the
instrument.
The conditioning law (VIF = 4.40 tan(2theta_95)^-1.66) was tested as a per-run
gate and does NOT order the errors - the crystals whose fit stays put are found
in the best-conditioned band as often as the worst - so no threshold is added
here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Document direct_beam_x/direct_beam_y in the CBOR start-message table and in the HDF5
detectorSpecific catalogue, and add the section that says what the distinction is: the PONI is the
foot of the perpendicular, the direct beam is where the beam lands, they are
distance*tan(tilt)/pixel apart, and XDS ORGX/ORGY wants the second. Also record that the same field
is a statement about the user's geometry in a broker master and a measurement in a rugnux
_process.h5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Write direct_beam_x/direct_beam_y, in pixels, into
/entry/instrument/detector/detectorSpecific - snake_case and unsuffixed like the rest of that
Dectris-style group, with a units attribute like the beam_center_x it must be read against.
Computed inside Metrology from the very locals that produce the beam_center_x/y and rot1/2/3
written a few lines away - refined when the offline analysis refined them, the StartMessage values
otherwise - and from the same start.detector_distance the translation vector uses. The file
therefore cannot disagree with itself: one composed geometry feeds all of them. Provenance differs
by producer (the broker's master states the user's geometry, a rugnux _process.h5 a measured one),
the field does not.
The test writes a real master in all three NXmx layouts at a non-zero tilt and reads the datasets
back, and requires the tilt to have moved the point, so a writer that stored the PONI would fail it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
beam_center_x/y is the PONI - the foot of the perpendicular from the sample - so on a tilted
detector it is not where the beam lands, and the two are distance*tan(tilt)/pixel apart (~8 px on
a real in-house setup). A consumer that wants the beam position, which is what most of them mean,
had to redo the tilt arithmetic or get it wrong.
Add optional direct_beam_x/direct_beam_y to StartMessage, filled from the geometry's own
GetDirectBeam_pxl() so there is no second formula, and put them at the top level of the CBOR start
map beside beam_center_x/y. Optional keys, so a consumer that does not know them skips them.
No _pxl suffix: the stream2 neighbours (beam_center_x, pixel_size_x, detector_distance) carry none
and the units are in docs/CBOR.md. The API's calibration schema spells it direct_beam_x_pxl because
its neighbours there are beam_x_pxl - same quantity, each matching its own neighbourhood.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The acknowledgement read as though the whole test corpus came from other people's beamlines. It does
not - in-house data collected at SLS 2.0 is tested alongside it. One clause, at the top of the
section; everything that follows still concerns the public data, which is the part that carries an
obligation to cite.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Only under -S does the arm read a completeness with headroom. De novo the search merge is in P1 and
does not count its possible reflections, so there is no number and CC1/2 is the only live term.
Three ways to give the arm one back were measured on 28 rotation crystals and none lands, so say so
at the guard rather than leave the next reader to rediscover them:
* Count the possible reflections on the search merge too. It is affordable - 5.2 ms against a 38 s
run - but a healthy crystal reads 27-43% of the P1 hemisphere, so the 100.5% bound is never
approached and no decision changes. It would only add a figure to the user's report that reads as
the dataset's completeness while being a fraction of a different group's asymmetric unit.
* Observation count. The two search merges are built in the same terms, so pass 2 retaining under 90%
of pass 1's observations looks like the signal the completeness ratio stood in for. It never fires -
bit-identical on all 28, including the crystal whose pass 2 predicted 44% fewer partials. Predicted
partials and observations that survive into the merge are not the same population.
* Completeness ratio. Same idea one level up, and it does fire - on the wrong crystals. This merge is
deliberately never resolution-cut, so the possible list grows with whatever range the pass reached
and a pass that predicts FINER scores as one that lost the sweep. At a 0.90 bound it reverted two
crystals indexing at 100%, taking one of them from 0.07% to 0.29% cell deviation and ISa 19.5 to
17.1, and rescued nothing.
Bringing the wrong-cell detector back for de-novo data means putting it on the final merge or
retiring the arm and saying so; neither is done here.
No test covers the decision line: Rugnux_Rotation does not reach this guard - the two passes in its
log are the indexer's, not RunAllPasses' - and there is no fixture for building a RotationScaleMerge,
so an assertion placed there would never execute.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The guard that judges the post-refined pass against the header-geometry pass reads completeness off
each pass's space-group search merge. That merge does not count possible reflections, so the quotient
is 0.0 on both sides and every de-novo rotation report has been printing
completeness 0.0% vs 0.0%, CC1/2 before corrections 0.994 vs 0.993
as though a comparison had happened. On the 28-dataset in-house battery that is all 28 runs. The
number is not just uninformative, it is a test that did not run: "completeness above 100% means the
cell is wrong" is the guard's wrong-cell arm, and it cannot fire against a constant zero, so only the
CC1/2 arm decides - which is computed on whatever data the pass kept, so a pass that discards a large
part of the sweep can post an equal CC1/2 and win.
Carry a measured flag beside the number. The arm is skipped and the log and PASS_DECISION say
"completeness not measured" where there is nothing to read. No behaviour changes: the arm could not
fire before either. The next commit gives the merge the count so it can.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Three lines were appended over the course of the work - the tilt and direct beam in
the report, the measured tilt beside them, and the post-refine beam bound. They are
one change to a user: the report now describes one geometry and says which point is
which, and the bound that governs it is measured against the run's own measurement
rather than against the file it is meant to correct.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
