When two first-pass schemes return cells whose primitive volumes differ by a small
integer, the validation-FRAME count cannot tell them apart: a spurious axis multiple
indexes every frame its true sub-cell does, so both reach 60/60 and the count
saturates. The rule that then decided the pair was unconditionally against the larger
cell, so on a crystal with a real pseudo-translation the true cell could not win in
any scheme order.
Ask the same question at the granularity where it does not saturate: how many of the
validation frames' SPOTS does each cell account for? That comparison leans towards the
smaller cell by construction, and needs no threshold to do so. Acceptance is a
fractional-Miller test, so multiplying an axis by n multiplies that axis's residual by
n: the larger cell places every shared reflection n times less accurately than the
sub-cell does, and loses outright the spots that sit in the tolerance margin. The only
thing that can pay for that loss is the class of reflections the larger cell ADDS -
empty for a spurious multiple, the superstructure's satellite rows for a real one - so
the larger cell wins the count only when the extra periodicity is really there. The
count is taken over the validation frames' whole spot lists, which reach far deeper
into each frame's intensity distribution than the first pass's own accumulation cap,
and a superstructure layer is faintest exactly where that cap cuts.
Measured over the five crystals of this corpus where the two schemes return an
integer-related pair, the larger cell accounts for 1.37x and 1.98x the spots on the two
whose true axis was being halved, and 0.30x, 0.36x and 0.71x on the three where the
doubling is spurious. All five come out right: the two keep the true cell and its
deposited space group, the three reproduce the answer the old rule gave, to the digit.
(One of the three has a bistable first pass - four builds give three answers, one of
them without this change at all - so it is not evidence either way; the other two are
reproducible.)
What the added class holds is computed and reported next to the decision, because it
is the physics the count is a consequence of. It is deliberately NOT thresholded, and
that is the part of this that took the measuring. Refuted along the way:
- An occupancy floor, which is how this was first written. Over the five crystals the
arbiter is asked about, the emptiest index-n class reads 48.2, 41.6, 37.3, 25.4 and
7.1 %. The two the larger cell should win are the 41.6 and the 37.3, so the three it
should lose bracket them on both sides, and a real superstructure elsewhere on the
corpus reads 3.4 %, below all five. Recomputing the same statistic on the merged
intensities over a sweep of I/sigma cuts leaves the ordering unchanged, so this is a
continuum and not two populations: no floor separates them, and no amount of extra
data would. The bimodality a floor needs was an artefact of a calibration set that
contained no failure.
- Requiring the two cells to stand in a genuine sub/super-lattice relation, the change
of basis being integral. Measured, all five pairs are index-n relations to within
0.016 of an integer - the volume ratio is not the weak link.
- Deciding it on the merge, by integrating and merging both cells. The worst failure
does announce itself there (CC1/2 0.9994 -> 0.9566, ISa 18.6 -> 1.4), but it costs a
second full integrate-and-merge, and the successes lose 13-30 % of their ISa where
another failure loses 27 %, so the metric does not separate them either.
- Requiring the sub-lattice class to be the STRONGER of the two, which is the right
mechanism but the wrong observable at this point in the run: the sign it turns on
lives in integrated intensities, and the spot finder reports no spot at all where a
class is absent, so at first pass the same ratio reads 0.90 against 0.39 and 0.27.
The ordering survives, the sign does not.
The occupancy is maximally wrong on the worst failure because that cell is not a
superstructure at all. A beam-centre error along the spindle translates the derotated
cloud rigidly, and a lattice shifted by half a spacing is indexable only on a doubled
axis - the shift needed scales as 1/(2L), so a long axis is the easy one to
half-offset. In that doubled setting the even class is empty and the zero layer reads
a negative mean intensity, which no crystal can do, while the odd class carries
everything. Such a cell fits no index-n sublattice at all, so its added class reads the
chance value (n-1)/n, the largest the occupancy can take, and the occupancy test
reports the artefact as more real than any genuine superstructure. The spot count sees
it for what it is, at 0.30x.
So the beam-centre warning below is now suppressed only when the larger cell WINS.
Declining it is a fall back to the default, and that warning - which names the beam
centre and offers --estimate-beam-center - is then the most useful thing the run can
say; on the half-offset mode it is the correct diagnosis.
The same question asked unconditionally of the committed cell, the halved-axis probe,
is not included. Measured over the committed cells of 29 crystals, 19 of 189 axes read
above the 2 % floor it would have used and the largest read 30 %, and those largest
readings are on crystals whose committed cell is already wrong - where doubling an axis
is the wrong response. It rescues one crystal whose superstructure layer reads 3.4 %.
That is not worth the rest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
A grid scan is a set of stills at a stationary spindle, and the angle it stood
at is what relates one grid to another taken elsewhere on the circle. Users were
finding an all-zero omega in the file and concluding the angle could not be
recorded at all.
It already can, and has since the goniometer and the grid scan stopped being
alternatives: send the axis with step 0 and its start angle, and that angle is
written per image into the NXmx sample chain, read back by reader/, and taken by
dials.import as a set of stills. Measured on a generated 12-image grid: the
placeholder file carries omega = 0 x 12, the same file with the axis sent at
step 0 carries omega = 90 x 12, and dials.import reports "still: 1, sweep: 0"
for both. Nothing in the code needed changing, so nothing was; what was missing
was that nobody could tell, and that no test held the behaviour down.
So: the API and the HDF5 documentation now say it in as many words, and three
tests pin the three legs the value crosses - the OpenAPI request (which used to
drop the grid scan whenever an axis was present, unpinned until now), the CBOR
start message, and the file round trip.
Also corrects a claim two comments and the HDF5 page were making. NXmx can
express "no rotation" perfectly well - a sample may depend_on "." - so the
placeholder is not there for the standard's sake. It is there because dxtbx
cannot read a sample chain of translations alone: strip the rotation axis from a
grid scan master and dials.import dies in get_dxtbx_goniometer with a matmul
dimension mismatch. Recorded so nobody removes the placeholder on the strength
of the standard.
One thing the change does not fix, because it cannot: a stationary angle is
invisible to DIALS when a grid scan is present. dxtbx picks the first varying
axis as the scan axis, which is a grid translation, so the oscillation reads
(0, 0); and with exactly one rotation axis in the chain it builds a single-axis
goniometer whose fixed rotation is the identity, never consulting the angle. The
same angle IS visible when it is the only candidate (oscillation reads (90, 0))
or when a Smargon head puts a second rotation axis in the chain (the setting
rotation then carries it). The value is in the file and correct either way.
GetElementPosFast_step took the snake parity from GetElementPosSlow_step,
which is the DISPLAY row: it is the acquisition row r = image / n_fast,
flipped to (n_slow-1) - r when the slow step is negative. So for a negative
slow step the parity it hands back is parity(n_slow-1) XOR parity(r), and
with an even n_slow that is inverted on every row - the whole raster comes
out mirrored along the fast axis. An odd n_slow leaves it correct, so the
same scan collected with 20 or 25 images disagreed about where image 0 sat:
n_fast=5, fast +1.5 um, slow -2.5 um, snake on gives images 0..4 at fast
index 4,3,2,1,0 with 4 rows and 0,1,2,3,4 with 5 rows. A positive slow step
was correct at both counts, and so was every non-snake configuration.
Snake means the stage reverses direction on alternate rows in acquisition
order, so the parity has to come from the acquisition row. Taking it from
image_number / n_fast directly makes the fast index independent of the slow
axis and of the row count, and drops the call into the display-row function
that caused the coupling. vertical_scan only relabels which axis is fast, so
it was wrong in exactly the same way and is fixed by the same line.
Affected files: written by an affected build, with snake on, a negative
grid slow step (step_y for a horizontal scan, step_x for a vertical one),
and an even number of rows. Their /entry/sample/transformations/grid_scan_x
or _y is mirrored along the fast axis, as was the grid map in the frontend
and the viewer - both mirrored together, which is why neither showed it.
Tests: the interaction of snake with the step signs was never asserted, only
each in isolation, so add a table over snake x {+,- fast step} x {+,- slow
step} x {even, odd row count} x {horizontal, vertical} asserting positions,
plus a case running one affected configuration through GetXContainer_m /
GetYContainer_m and Rearrange. Every pre-existing assertion is unchanged and
still passes; only the four negative-slow, even-row cells of the product
move.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The warning said the diffraction limit "runs from 3.34 to 2.45 A depending on
direction" and stopped there, so a reader was told the crystal is anisotropic
and given no direction to act on. It fires on 19 of 113 datasets in the battery.
The eigenvectors are already measured and already written to the merged mmCIF
as `_reflns.pdbx_aniso_B_tensor_eigenvector_N_ortho`; they had simply never
reached the human report. The tensor is fitted on s = frac.mat * (h,k,l), so in
that Cartesian frame a*, b*, c* are the rows of frac.mat and naming the axis is
one dot product per eigenvector. The label is exact in every Laue class that
has a free tensor direction except triclinic, and the cosine is printed so a
loose fit shows as one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The merged mmCIF carried the R-free flag in `_refln.status_free` as 1/0 and
wrote `o` into `_refln.status` on every row. `_refln.status_free` is not in the
PDBx/mmCIF dictionary - it is absent from mmcif_pdbx v4.0, v5.0, v5.288 and
v5.362, from CCP4's and phenix's shipped copies, and its wwPDB item page is a
404 - and no deposited structure-factor file uses it. `_refln.status` is the
item that carries the free set, with `f` for a test reflection and `o` for a
working one; on a deposition that also carries `_refln.pdbx_r_free_flag` the
two agree exactly.
Measured on a real merged file this run wrote:
CCP4 cif2mtz refuses the file outright - "Unexpected context type for
category REFLN" from its dictionary-validating parser, exit 1,
a 12-byte truncated MTZ. Dropping the non-dictionary column is
what fixes it: the same file without it converts.
gemmi converts, but its cif2mtz spec knows only `status` and
`pdbx_r_free_flag`, so FreeR_flag comes out 1 everywhere and
the free set is silently lost - R-free would then be computed
on the working set.
phenix worked, but only by a filename heuristic matching the words
"status" and "free".
Writing `f` while keeping the extra column is worse than either, because phenix
then finds two candidate free-set arrays and refuses the file, so the column
goes in the same change. After it, all three read the same 5% test set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
docs/python_client/docs is generated from the API spec and copied in by
update_version.sh, so it goes stale between releases. The calibration
convergence gate and the beam size both added properties without it; this is
the same generator run those commits should have carried.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Smargon at the SLS 2.0 MX beamlines reports its own axis positions with
readout noise, so a chi parked at zero comes back as about -1e-7 degrees. The
schema bounded chi_deg to [0, 90], so an ordinary "chi is at zero" setup was
refused - and the same holds at the other end of the arc, where a chi parked at
90 reads just above it. Both end-stops are exactly where a static positioner is
left, so widening the range would only move the problem.
The bound bought nothing. Chi never enters any computation: OpenAPIConvert puts
it in SmargonPosition, DiffractionExperiment::BuildTransformationChain hands it
to a rotation transformation verbatim, HDF5NXmx writes it and HDF5MetadataSource
reads it back. No downstream reads its sign, and a rotation is defined for any
angle. phi, the sibling angle in the same object with the same semantics, has
never been bounded. What was left was a restatement of a hardware travel limit
that the goniometer enforces itself, and its only observable effect was to
refuse a value the instrument genuinely reported.
It was also not enforced where a server-side check would matter: the
cpp-pistache-server generator does not recurse into a nested object model, so
Dataset_settings::validate never calls the Smargon model's. The rejection was
raised by the generated python and TypeScript clients, which do check.
Regenerated the C++ server model, the TypeScript client and redoc-static.html;
python-client is gitignored and comes from gen_python_client.sh.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
dataset_settings gains beam_size_x_um and beam_size_y_um, the horizontal and
vertical size of the X-ray beam where it meets the sample. They follow the same
route total_flux takes - OpenAPI, DatasetSettings, the CBOR start message, the
HDF5 master, and back out of a stored file - and nothing consumes them; this is
metadata a beamline can state and a downstream program can read.
NXmx puts this in the application definition rather than the base class: not
NXbeam's extent (rank 2, nP x 2, per scan point, always FWHM of a rectangular
aperture) but NXmx's own incident_beam_size, a recommended rank-1 two-element
array in the order x, y. Both are live and neither is deprecated, so the choice
matters; the MX definition wins in an MX file. Written as one array with a
units attribute of "m", like every other length in the master, so the settings
hold micrometres and FillMessage converts once.
The unit table of ReadLength_m becomes LengthUnitFactor so the array read can
share it: a master written elsewhere may state this in millimetres.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
/entry/MX/peakCountUnfiltered was written only by the data-file plugin.
The four other per-image spot counts are written to both the data files
and the master, so a reader holding just the master got every count
except the unfiltered one.
It was not only a missing dataset. HDF5MetadataSource already reads
/entry/MX/peakCountUnfiltered from the master and falls back to
/entry/MX/nPeaks when it is absent - and nPeaks is the number of spots
*stored* for an image, i.e. after the spot budget truncates. On a VDS
master the fallback therefore substituted the post-filter count for the
unfiltered one silently, and the two differ precisely on the images that
hit the budget.
EndMessage::spot_count is the right member: it and the data-file
plugin's spot_count_total both come from DataMessage::spot_count (the
end-message copy via ScanResultElem::spot_count), and CountSpots() sets
that from spots.size() before FilterSpotsByCount() applies the budget -
"spots found before filtering", as docs/HDF5.md already described it.
The CBOR end block already carried spot_count on both encode and decode,
so no message or protocol change was needed. SaveVectorIfMissing keeps
the NXmxIntegrated case correct, where the data-file plugin has already
written the dataset into the same file.
Verified by writing files in all three formats and dumping them: the new
dataset appears in the legacy, VDS and integrated masters and matches its
sibling peakCountLowRes in value, datatype and read-back precedence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
A powder calibration is run because the file's geometry is in doubt, so a fit that
quietly returns part of that file has answered nothing - and it is indistinguishable
from one that worked, down to the residual and the sigmas arranged around it.
On one of four LaB6 exposures of one detector the tilt came out at 2.93x its own
sigma, a hundredth under the significance gate, so it was declined and pinned - at the
master's hardcoded rot1 -0.08, rot2 -0.22 deg. That is eight times the tilt just
refused, on no evidence, and worth 10 px of PONI at 190 mm. rugnux printed it to four
decimal places, wrote the .poni, and exited 0.
Judge the result on provenance instead of on any residual: a geometry is a measurement
only if every parameter in it came from this data. Two ways out of the fits do not
qualify - a covariance that never conditioned, so the fit cannot say what it
determined, and a declined tilt pinned at a non-zero value from the file. A declined
tilt over a file stating no tilt still qualifies, because reporting no tilt is then
exactly what was measured; so does --no-refine-tilt, because a hold that was asked for
is a stated choice and not a silent substitution.
No single number separates the four. rms is 2.465 px against 1.44-1.64; the
significance of all four lies between 2.93 and 4.47, so the gate is nearly a coin flip
at these distances and moving it would only recalibrate on one population; and the
failed fit has the TIGHTEST parameter sigmas of the set, because pinning the tilt
removes the tilt/centre correlation that inflates a good fit's. The spot cross-check
reads 13.5 px against 0.98-2.66, but 10.4 px of that is the pinned tilt moving the
PONI - the same defect one step downstream, not independent evidence.
On a failure rugnux says so, writes no .poni - a PONI file states where the detector is
and has no field in which to say it does not know - writes the JSON with converged
false and the reason beside it, and exits non-zero. The re-binning pass now prefers a
converged refit over a non-converged one whatever its residual, so a tilt an earlier
pass measured is not what a later one gets pinned at.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merge-statistics window asked for an absolute y-axis - CC1/2 and CCref on
0..100, everything else from 0 - by setting the range on the chart's value axis
after JFJochSimpleChartView::UpdateData had already built the chart. The visible
tick labels are a separate QCategoryAxis whose entries UpdateData had generated
from the range of the data, and those entries were not rebuilt. So the plot was
drawn over 0..100 while the labels down its left edge covered only 91.5..100 and
crowded into the top tenth; the numbers matching the drawn range appeared only
on the right-hand grid axis, which the same call had made visible. CC1/2 showed
it worst because its range is the narrowest, but every metric was affected.
Give UpdateData the range instead, so the ticks and both axes come from one
number. The side-panel azimuthal-integration chart passes no range and renders
pixel-identically.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The four-way split leaves CPU_DATA_ANALYSIS.md as the landing page the toctree points at, and one
part ended on a horizontal rule, which docutils refuses at the end of a document.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both programs were acknowledged for specifics (profile-fit variances, FFT
autoindexing, post-refinement practice) but not for the paradigm rugnux's
rotation pipeline is built on: integrate each image in 2D, then combine the
partials into fulls, as against XDS's 3D profiles. One paragraph names it;
the citations were already on the page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both merged formats hit the same refusal - the file carries the mean and the
Bijvoet pairs - but the label vocabularies differ per format and the MTZ
incantation fails on the mmCIF with a fresh error. Give both measured
commands, and say it is one behaviour in two formats, not a difference
between our files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The page that did not exist: one paragraph per stage from opening the file to
the written reflections, each linking into the data-analysis reference part
that carries the depth, with the stills differences at the end. First entry
after the landing page, and the landing page says to read it first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
CPU_DATA_ANALYSIS.md becomes a short landing page (scope, part map,
references) over four parts in pipeline order - images to spots (0-3),
indexing and geometry (4-7), integration/scaling/merging (8-12), space group
and validation (13-14). Pure moves: the section numbering is continuous and
unchanged, since the rest of the documentation and the source cite sections
by number. Inbound topical links now land on the right part; the build has
zero warnings and the rendered-HTML anchor check finds no dead link.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The 1274-line page becomes a landing page (quick start, the page map, where it
fits) plus seven pages a reader can answer one question from: installing,
what rugnux reads, running it, integration with other programs, the results
report, advanced usage, and powder calibration. Content is moved, not
rewritten - only the connective sentences at each page top are new. Every
internal anchor is remapped to its new page and every inbound link
(DEPLOYMENT, TOOLS, HDF5, CPU_DATA_ANALYSIS) updated; the built site has zero
Sphinx warnings and an anchor check over the rendered HTML finds no dead
link. index.rst leads with the rugnux group, then acquisition, FPGA,
reference and project.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The OpenAPI Python client owned the sidebar: DefaultApi's 128 method anchors
plus the 64 hidden-glob model pages were 195 of its 256 entries, because
sphinx_material's globaltoc includes hidden toctrees by default. A new
PYTHON_CLIENT.md landing page carries the links and a hidden glob toctree, and
globaltoc_includehidden is off, so every generated page is still built and
reachable (verified: 64 model pages + DefaultApi render, zero warnings) while
the sidebar drops to 61 entries. docs/review/ joins exclude_patterns so a
local, gitignored review report can never again be rendered into the
published site.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Six methods the pages name or describe carried no citation: Padilla & Yeates
(the L test), Steller, Bolotovsky & Rossmann (the projection/FFT autoindexing
MOSFLM implements), TORO (what ffbidx implements), Krivy & Gruber and the
ITA lattice-character table (the reduction and Bravais assignment), Cheetah's
peakfinder8 (the per-ring background statistics of the adaptive finder) and
Hennequin et al.'s SparseCCL (already credited to traccc, now also to its
authors). Each gets its ACKNOWLEDGEMENT.md paragraph, a References entry in
CPU_DATA_ANALYSIS.md, and a one-line credit at the algorithm. The
Sheriff & Hendrickson / Popov & Bourenkov entry is re-scoped so each claim
sits on the paper that supports it - P&B 2003 is titled, and credited for the
sigma-aware anisotropy estimation its statistic modelling contains, not for
the tensor and its constraints. All DOIs verified against the publishers;
the SparseCCL DOI resolves to IEEE document 9049184 (IEEE blocks content
scraping, so verified by the resolved document id plus two independent
sources).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The longest FFT axis (500 A, no flag; -C moves it) was implied twice and never
stated, and its failure mode is a plausible sub-cell rather than a refusal.
One input is one sweep - said affirmatively where inputs are described instead
of in an aside about pointless. A default 50 A low-resolution cut discards
real reflections on a very large cell; the option row says so. Exit status is
documented for scripts (0 = completed, non-zero = stopped), and the
--model output list gains _anom.ccp4, which was written but undocumented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The page named phenix, REFMAC, POINTLESS, AIMLESS, careless and SHELXC a dozen
times without one command line, and left unstated the three facts those
programs stop on: the free-flag convention (0=work 1=free; REFMAC needs
FREE 1), the phenix label choice the double intensity array forces, and the
unmerged file's header symmetry and sort order (determined group, sorted
H K L M/ISYM BATCH - WriteReflections.cpp sorts it). A new 'Taking the data
onward' section carries the worked lines, the careless column renames
(BGVAR is a variance), and the Phaser SGALTERNATIVE keywords for the
enantiomorph the report leaves open. The POINTLESS series trap now names
ALLOW OUTOFSEQUENCEFILES instead of telling users to touch their data, the
report-grep block warns that SPACE_GROUP_NAME carries one member of an
enantiomorphic pair by convention, and the P1 cross-check's free set is
declared to be its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The -A row and FRIEDELS_LAW=TRUE together read as 'without -A your Friedel
pairs were averaged', which is false: the rotation merge always keeps the
Bijvoet split and the default .mtz/.hkl carry it (WriteReflections.cpp,
BuildMergedRows). Say what -A actually changes - the counting basis and the
error model - and give the merged MTZ's exact column labels, which scripting
against phenix needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
A default rotation run writes seven files, not five - the two the list omitted
are most of the bytes. The input is any NXmx/EIGER master or miniCBF sweep,
which the page said only 350 lines later after twice implying Jungfraujoch
data only. The CC1/2-0.30 resolution trim moves up to the quick start, and
_image.dat's columns are finally named (ScalingResult.cpp writes a # header).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The ranking-not-sampling observation was measured on the coplanar-shortlist
rescue and is scoped to it now. The angular-resolution bound
theta < d_min/(2a) means the shipped 16384-direction grid resolves axes only
to roughly 120-150 A, far below the 1200 A the accepted maximum admits, and
the failure mode is a plausible sub-cell, not a refusal - the reader raising
fft_max_unit_cell alone deserved to know it cannot work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
sigma_bw is one physical smear written in reciprocal units in 8.2/11.1 and in
pixels in 9; the two Delta-phi of the partiality formula are named; 13.5's
|s| = 1/d is reconciled with the s = sin(theta)/lambda of 10.6/14.2; bandwidth
is the rms spread, with the FWHM-input conversion (/2.355,
BraggIntegrationEngine.cpp) stated. The ice-extinction clause now covers (104),
not only (00l), and the AIMLESS <I/sigma> = 2 constant is named as AIMLESS's
default rather than pointed at a criterion this project does not use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The stated criterion (a merohedral twin law exists) formally exempts every
holohedral Laue class, but the code (TwinningAnalysis.cpp) deliberately keeps
the low-symmetry ones eligible because pseudo-merohedral twinning through a
special metric cannot be excluded there. Say both halves, and give the
reference values of the statistics so the mmCIF numbers can be read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Every q in the pipeline is 2*pi/d (Definitions.h, the azint bin mapping), but
the numbers in 3.3, 7.6 and 10.10 and the --azim-* flags never said so, and a
reader taking q = 1/d would set --azim-q-spacing or --azim-max-q wrong by
2*pi. The 7.6 ice triplet is spelled out so 'within 0.06 of one another' reads
as the adjacent-ring spacing it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The 1.109 in 9.2 is the variance ratio of the shipped 4/6/13 stencil (45/408
pixels), not the sigma ratio the sentence attached it to, and the effect is a
bound attained on weak reflections, not uniform. The profile-fit background
term is (sum P/v / sum P^2/v)^2 var(b), matching the code; the undefined w is
gone. The 13.3 refit cut is N^2 times the model variance
(RotationScaleMerge.cpp), not N*sigma^2. The sigma floor is 1 count.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
R(phi) rotates the observations by +phi about the stored axis, which makes it
the inverse of the crystal's own rotation - the reading under which the formula
as written is correct, and the same convention that has the exported MTZ batch
axis and the XDS echo negated. Said explicitly, so a reimplementation cannot
take R(phi) for the crystal rotation and land at 2*phi.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Section 1.1 called (x_beam, y_beam) the direct-beam position while constructing
it as the point of normal incidence; on a tilted detector the two differ by
D*tan(tilt), and a reader loading a header direct beam into it would be wrong by
several pixels. Name it as the PONI (the system-wide convention,
DETECTOR_GEOMETRY.md), say which point the construction pins, and state the
laboratory frame's axes and handedness, which the rotation-sign, R-centring and
Bijvoet-hand discussions all silently depended on. Two-theta is computed as a
two-argument arctangent; say so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both pages said in one place that adopting a model's enantiomorph moves no
reflection, and in another that it is applied to the merged reflections and
exchanges the Bijvoet mates. The code (ModelValidation.cpp, AdoptModelFrame)
does the former: the model's group is a label on the written files, I(+)/I(-)
stay as measured, and only the merohedral-ambiguity reindex touches reflection
indices. Say that once, consistently, in CPU_DATA_ANALYSIS.md §14.5 and in the
two RUGNUX.md passages that carried the wrong reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
SPACE_GROUP_NAME is a scalar and reads like a determination, and the docs
nominate a grep of it as the interface. Where several groups predict the same
absences it is one of them, picked by convention: the report said so in prose
and nothing machine-readable carried it, so a script recorded a coin flip as an
answer. Refining against a deposited model in the wrong enantiomorph gives
R = 0.549.
Four keys, none of which change which group is adopted: the alternatives the
data cannot separate; whether the enantiomorph was determined, given, or is
undetermined - decidable from the group number alone, so it is answered even
where the search did not run; and the higher point group whose promotion was
refused, with its reason, which until now existed only as prose.
REPORT_VERSION is 5. No existing key changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both guards on a user-fixed space group keyed on the centring differing, so an
axis permutation within the same centring passed: a run that indexed
a=37.909 b=78.031 c=77.594 and was told -S P41212 - which needs a=b with the
4-fold along c - merged through operators that do not act on its own indices,
and --mode scale on it reported COMPLETENESS= 195.3, an arithmetically
impossible number, without complaint.
MetricViolation asks the setting-independent question instead: a group's
rotations must leave the cell's metric tensor invariant. The re-seating arm now
runs on either failure, so a permuted cell can be reindexed into the setting the
group needs rather than merely refused; the refusal arm catches what re-seating
could not fix. --mode scale never reaches either arm, so it gets the same test
where it fixes its cell and its group, which is the only place an impossible
completeness could still be produced.
The tolerance is a refusal bound, so it sits above what a correct answer
reaches: over 113 corpus runs every group determined from its own cell scores
under 0.032 and the permuted case scores 0.764.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The column was written with 1 for the test set and 0 for the working set - the
phenix/CNS numbering - under FreeR_flag, which is CCP4's column name. CCP4's own
freerflag writes 0..19 with 0 as the test bin, and REFMAC5's default FREE 0
reads it that way, so REFMAC5 stopped on every file we wrote: "more than half of
reflections are in free R set", then "Cannot switch free R flag", exit 1.
phenix auto-detects either numbering with equal confidence (measured on both,
score 3 each), and rugnux's own reference-MTZ reader already takes flag 0 as the
test set, so 0 = test is the numbering that works everywhere and the one the
rest of the code assumes.
This changes every merged .mtz we write: a script that reads FreeR_flag == 1 as
the test set has to be inverted. The mmCIF's _refln.status_free is a separate
item with its own convention and is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The export declared its rows FULL - LDTYPE=2 in the batch header, M=0 in
M/ISYM - while filtering them on --min-partiality alone (0.02), where the 3D
combine also applies --min-captured-fraction (0.7 on rotation) to the same
summed event. AIMLESS reads the FULL declaration, reports "all runs have only
fulls" and ignores FRACTIONCALC, so an event that caught a twentieth of its
rocking curve entered the merge whole, with a small sigma, and was weighted
heavily. There was no cut for the reader to make: the column that would let it
make one is the one the reading program has been told to ignore. Dropping those
events moves AIMLESS's Rmerge at 1.8 A from 1.353 to 0.694 and CC(1/2) from
0.985 to 0.993.
FRACTIONCALC itself is unchanged, values above 1 included. Each part's
partiality is the erf pair the predictor computed on that frame, from that
frame's own refined lattice and mosaicity, so the parts of one event do not tile
the rocking curve exactly - the offset steps by the wedge to within 12% of it,
and the sums land in a peak at 1.000 whose 95th percentile is 1.09. It is an
honest estimate of a captured fraction, and it is not the number rugnux scales
on, so clamping it would only hide the spread.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The header set was built from the batches that produced an observation, so an
image that indexed nothing left a hole in the phi series. AIMLESS starts a new
run at such a discontinuity: measured on a 360 degree sweep with 67 unindexed
images it made 23 runs of one sweep, its scale model diverged, Rmeas overflowed
to -1266 and the result no longer correlated with rugnux's own merge (Pearson
0.0018). Renumbering the batches contiguously does not help - the split is on
phi, not on numbering - so the fix is a header per image over the span the
observations cover.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
With -S the prediction rejected the fixed group's centring absences, so those reflections were never
integrated. Two things followed, and only the second was known.
The P1 cross-check was withheld on such a run, because a P1 merge missing whole centring classes is
misleading rather than merely small - 50% of the nodes on an I lattice, 75% on F, 67% on R. That was
the documented reason and it was right.
The unknown one is that it cost intensity accuracy. Every predicted reflection marks its signal region
so a neighbour's background ring can exclude it (BraggIntegrationEngineCPU, the reflection mask); an
unpredicted node is an unclaimed patch of detector, and the neighbouring reflections sweep those pixels
into their background and over-subtract - worst at high angle, where the background dominates. On a
fixed F-centred group that is three quarters of the nodes: measured against the de-novo run of the same
data, <I/sigma> 16.07 against 17.21, CC1/2 0.9862 against 0.9895, ISa 10.93 against 11.92.
Predicting them costs nothing downstream, because both merges already decide absence against the group
they are merging in: the run's own merge drops them again, and the P1 cross-check keeps them because P1
has none. One integration, two correct merges. The -S output becomes byte-identical to the de-novo run
on the five crystals measured, which is the point - pinning a group should not change the answer - and
such a run can never be slower than de novo, since it predicts the same reflections and additionally
skips the space-group search.
The de-novo path is untouched by construction: it already predicted in P.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The guard that catches the bistable supercell collapse between the two rotation passes took the
volume of each pass's CONVENTIONAL cell. A centred conventional cell is an exact integer multiple
of its primitive one - C and I twice, R three times, F four times - so two settings of the same
lattice differ by exactly that factor, and the guard read a change of setting as a supercell. It
then forced pass 1's result, and with it pass 1's lower symmetry, on a pass that had found the
same lattice in a better one. Ten of 116 rotation datasets tripped it, every one at an exact
centring multiplicity: seven at 2.00x, one at 3.06x, one at 3.99x. The last is an F-centred
lattice; it was held in P1 where the second pass had found it centred orthorhombic.
The two structurally identical guards inside RunPipeline already convert with ToPrimitive first,
and their comments say why. This one could not: ProcessResult carried the consensus cell with no
centring beside it, so at the comparison there was nothing to convert with. The centring is now
carried alongside the cell, set at each of the four places the cell is - the finalized rotation
lattice, the reindex into a user-fixed group's setting, the reduction of a doubled cell, and the
committed higher-symmetry reindex - and the guard converts both sides before comparing. Nothing
else reads it; every other consumer of the cell is unchanged.
On the F-centred dataset the run goes from P1 at 2.232 A to I 2 2 2 at 2.077 A, multiplicity
1.86 to 7.07.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The centroid was a first moment of the RAW counts over the signal disk, so it weighted signal plus
background. The background is flat over a disk centred on the PREDICTION, which makes its own centroid
the prediction exactly: it adds nothing to the displacement and everything to the denominator, and the
measured offset comes out shrunk by I/(I + n*bkg).
That factor is worst where the background dominates the signal, which is at high resolution - so the
one consumer of this quantity, the geometry post-refinement, fits the beam centre and the detector
distance on displacements that are systematically too small, by a factor that varies with resolution.
An estimator whose bias depends on the very coordinate it is correcting.
Subtracting a flat pedestal from a first moment is exact, and the background is not known until the
ring has been read, so the positions of the pixels behind the intensity sum are accumulated alongside
it and the correction is applied afterwards: sum(x*(px-bkg)) = sum(x*px) - bkg*sum(x). Both engines,
identically. Where nothing rises above background there is no signal centroid to compute and the raw
one is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Computing it needs a DiffractionGeometry, and that refuses a detector distance under 1 mm or a pixel
size of zero. A start message can legitimately carry neither: the writer's own pre-flight check asks
it to prove it can create the files for a dataset that never describes a detector, and the master
write then threw where it used to succeed - Preflight_TCP fails at rc.166 and passes at rc.165.
The dataset is a convenience for whoever reads the file later, so where the geometry is not there to
compute it from it is simply not written. Everything else in the master is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Post-refinement was handed the HEADER geometry while the outcomes it refines had been integrated at
the first pass's own, and only three numbers came back - beam_x, beam_y, distance. The tilt was
dropped at both ends, so the pass that ships started at the header tilt with a beam that step B had
moved to absorb a tilt the images do not have. On one in-house mounting the first pass ends at
rot1 -0.2995 deg and the shipped value is -0.0992, against a powder calibration of -0.282: the two
geometries the two passes use differ by the whole of the tilt error, inside a single run.
Both ends move together, and they have to. Step B holds the tilt fixed while it fits the beam and the
distance, so the beam it returns is only meaningful about the tilt it was given: handing it the refined
tilt without carrying that tilt forward, or carrying the tilt forward under a beam fitted about the
header's, each describe a geometry that never existed. The carrier goes from three floats to five and
the header snapshot the quality guard reverts to grows with it, so a rejected second pass still returns
to the geometry the file states.
This is not a tilt measurement and does not make one: rot1 remains the gauge direction of a single-axis
rotation experiment. It only stops the two passes of one run from working in two different geometries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The residual applies angle_rad * |rot_vec| and rot_vec is a free three-vector, so the first pass has
been fitting a goniometer rotation SCALE nobody asked for. GoniometerAxis::Axis() then normalises it
away on write-back, and RotationIndexer scores the candidate with the normalised axis - so the cell
that won the fit is judged under a rotation model the fit did not use. Measured over 43 rotation
datasets: the length reaches 1.2%, and the fit-vs-score disagreement a median 0.124 deg and up to
6.19 deg of goniometer angle, against rocking widths of 0.05-0.36 deg. That score picks the lattice
class, which nothing later revisits.
The fitted length is not a usable measurement of anything either: on synthetic data it recovers 54%
of a known scale error, repeated first passes on one dataset disagree with each other in sign, 26 of
43 datasets disagree with themselves, and on the one dataset with a proven 1.3% stage fault it comes
out negative. It is absorbing other systematics. The rotation scale is measured properly, once, with
cross-validation and gates, in PostRefine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
Both directions of the tilt now go through the same restraint, with a budget each in shared
direct-beam pixels; zero means free. Swapping which component is held, or holding both, is a change to
two constants. The perpendicular budget is zero today, which is the arrangement the measurements
support: it is the component whose conditioning tracks the 2theta the fit reaches, while the parallel
one's does not move with 2theta at all - a conditioning number that ignores the data is the prior
talking.
This is a clarity change, not a precision one, and the record should not read otherwise: the standing
'freeing the tilt inflates both beam components' cost is the perpendicular component's doing, not the
parallel one's, so restraining the parallel one recovers essentially none of it. What it buys is that a
beam-centre error is no longer reported as an angle.
The restraint is correct here because the crystal orientation is free alongside the tilt and absorbs
the spindle-parallel difference. A stage that freezes the orientation has no such compensator, so
whether that component is measurable THERE is a separate question; the comment no longer reads as a
claim about the tilt everywhere.
Two corrections to the restraint added in the previous commit, both measured.
The beam prior compared the goniometer vector's LABORATORY components against an index into the
PIXEL-frame beam centre. det_matrix is PoniRotMatrix * DetectorOrientation::Matrix(), so on a quarter
turn of 1 or 3 the pixel X axis is the laboratory Y axis and the comparison picks the wrong component -
pinning the determined one and freeing the gauge one, which is worse than having no prior. Two datasets
in the corpus are in that state. The direction now comes from projecting the spindle onto the pixel
axes' own laboratory images, which is exact for any orientation, any tilt and a spindle at any angle,
and equal to the old comparison when the orientation is the identity. The tilt keeps the spindle's
laboratory components, because rot1 and rot2 are laboratory rotations applied outside that orientation
matrix - same physical direction, each in the frame its own parameters live in.
And the tilt's restraint is three times tighter than the beam's, in shared direct-beam pixels, because
the data determine the SUM of the two: at equal budgets the shift splits evenly and the reported tilt
still followed the starting beam centre at 41% of one-for-one. At a one-pixel budget it follows at 8%,
from 79% before this work, while the free component moves by 0.0035 deg over the same eight-pixel swing
and the indexing rate does not change.
The comment says plainly what the change does not do: it does not make the tilt accurate.
The beam prior at XtalOptimizer already treats the beam component parallel to the spindle as the gauge
direction of a single-axis rotation experiment and restrains it toward the value it was handed. The
detector tilt is that same gauge described a second time, and it was left free with a flat +-3 deg box.
The correspondence is not an observation about one beamline, it follows from the convention:
PoniRotMatrix builds the detector matrix as R(-rot3,z)R(-rot2,x)R(rot1,y), which puts the direct beam
at (beam_x - rot1*D/pixel, beam_y - rot2*D/pixel). rot1 IS beam_x written as an angle and rot2 IS
beam_y, component for component. So on a horizontal spindle the gauge tilt is rot1 and rot2 is
refined; on a vertical spindle it is the other way round.
With only one end of the alias restrained, a beam-centre error the prior refuses to absorb reappears
as an angle: measured elsewhere in this campaign, refined rot1 tracks the starting beam centre at
0.072 deg per pixel against a geometric 0.080, while the refined beam never leaves its anchor by more
than 0.24 px. Restraining both ends, in the same direction and to the same three-pixel budget, leaves
the determined component - the direct beam, and the tilt perpendicular to the spindle - alone.
Both restraints now take their direction from one projection of the spindle onto the detector plane
rather than from two independent snaps to whichever of X/Y dominates, so they cannot disagree, and a
spindle at any angle is handled. On a spindle along a detector axis the projection is the snap.
The section had grown to eighteen entries, several of them describing the same report or carrying
detail that belongs in a commit message. Consolidated to thirteen one-liners, each naming the program
it concerns - rugnux, jfjoch_writer, jfjoch_broker, jfjoch_viewer - so a reader can find what changed
in the part they use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
The report invited the reader to check REFINED_DETECTOR_TILT against a powder
calibration. Measured over two detectors' worth of crystals, that check
misleads: the rotation fit does one outer round, so it leaves its starting
value by only a small and crystal-dependent fraction of the distance to the
calibrated value, and on the worse of the two detectors the per-crystal median
sits an order of magnitude further from the powder answer than either method's
uncertainty. A user comparing one run against their own calibration would
conclude the calibration was wrong.
What survives the aliasing is the direct beam, which is already printed beside
it, and the MEDIAN of the tilt over several crystals on one detector - enough
to show up a placeholder or a stale value in the file, not enough to replace a
calibration. Say that, and say not to feed one run's value back into the
instrument.
The conditioning law (VIF = 4.40 tan(2theta_95)^-1.66) was tested as a per-run
gate and does NOT order the errors - the crystals whose fit stays put are found
in the best-conditioned band as often as the worst - so no threshold is added
here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
