Full open and in-house battery with the REFMAC model check on 3770f42c9 (run
20261005-2102_3770f4_rc174-final-refmac) against the rc.173 battery; commentary on verdicts,
R-free, resolution, processing time and small molecules, and the time (all sets, and zoomed on
0-60 s), R-free and resolution figures.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
paper_plots.py --time-arms (default open, so the paper figures do not change) widens the time
histogram to e.g. the open and in-house arms; time_zoom is the same histogram over 0-60 s in 5 s
bins, with the sets beyond it counted in the label. Also brings 8xtg's accepted-alternative note in
open.json up to the zone read at the other twin laws' fraction.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
CPU_DATA_ANALYSIS_IMAGE §1.4 now follows the decision flow in Rugnux.cpp:
header provenance lines, the background measurement on every run (capture
+ walk, what the check lines mean), the second first pass and its five
outcomes, the merge-judged arms (metric-symmetry, lean-depth and the
same-lattice inner-shell CC1/2 < 0.9 retry), the search after a failed
pass, the post-refinement bound, --estimate-beam-center's acceptance rule,
stills, report keys and options. Removes the stale "where both work and
disagree, neither is chosen" and the "centroid of the radial background
profile" fall-through.
RUGNUX_ADVANCED: --beam-center-check lists every adoption route and that it
still runs with --beam-x/--beam-y; --beam-center-search re-finds spots per
rung (the table said spot finding was not repeated); --estimate-beam-center
states its acceptance rule. OVERVIEW and TUTORIAL brought in line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The root README described only data acquisition. The install page now
lists the pre-scan and the correction surfaces among what runs on the GPU,
and says that -march changes speed and not results (floating-point
contraction is off project-wide).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Both DOIs were resolved against doi.org/Crossref (title, volume and pages
match), so the "DOI not verified" notes go. The analysis reference's
References list gains Wilson / Howells-Phillips-Rogers and Baur & Kassner,
whose methods 13.1 now describes (CLAUDE.md: a method described there is
listed there as well as in ACKNOWLEDGEMENT.md).
Every other DOI in ACKNOWLEDGEMENT.md, the CPU_DATA_ANALYSIS* pages, the
RUGNUX* pages, README.md and the battery README resolves.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Space group (13.1): every setting of a chosen rotation set is offered;
screws are claimed at 8 nats, with how that bound was set; the P1 search
merge is cut where a monotone fit of <I/sigma> drops under 1; the
operator-homogeneity gate reads the R gap between a subgroup and the rest;
a lattice-holohedral group is refused when merging under it narrows the
L-test like a twin.
Outliers (13.3): the rotation median is weighted at the expected
intensity, and the band's systematic part is multiplicative, with its
spread measured on the merge's own fulls.
Indexing: a sparse lattice is integrated on every frame; pass 1's lattice
is a hypothesis against a different re-index, and a pass sent back reuses
pass 1's indexing. Image: the flipped axis sign must beat chance; the
same-lattice beam-centre retry on an inconsistent merge.
Integration: five correction surfaces, not four; the .hkl carries no
amplitudes; the fulls' scale is smoothed differently from the partials'.
Tutorial: the correction surfaces are cross-validated on the half-set
correlation (as the reference says), not on an R_meas-like deviation.
Overview: the footprint, the 5 A indexing hypothesis, the centre of
symmetry, and the GPU pre-scan.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The P1 cross-check (p_P1.mtz, twin-immune zones, finalist ledger) was merged
after the reflection files were written, serially on the critical path of a
run without --model: in-house protein sets wait 0.3-1.5 s for it after tail
work (tNCS, twinning, Wilson, anisotropy, writing the merged files) that it
shares no data with, and on the largest sets the merge itself is 16-32 s.
It is now started as soon as the final merge is settled and taken where it is
used. What tied it to the main thread was the space group: the engine read it
from the shared experiment, which the caller flipped to P1 around the merge.
Run now takes the group explicitly (RotationScaleMerge::Group() at the 13 sites
that read the experiment's group), so the experiment is left alone. Nothing
between the launch and the join calls the engine, changes the outcomes it was
ingested from, or changes the experiment; the engine writes no per-frame scale
back for this merge. The merge is therefore the one made before, on the same
engine state. The engine's log goes to a held buffer replayed at the join, so
the log keeps one order.
With --model the same merge already ran beside the validation; that is now the
same path, taken before the validation's relabelling is applied.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The band spread (the weighted median |ln(I / median)| over the merge's fulls,
added with the measured band) sorted every usable full on one thread, in every
merge with outlier rejection. Equal pairs are the same two doubles, so
ParallelSort returns the serial sequence byte for byte and the median's prefix
sum runs in the same order. Measured side by side on the same input in every
merge of 6 in-house GPU runs: bytes identical in all calls; a thaumatin run
0.261 -> 0.089 s, a cytochrome C run 0.130 -> 0.040 s, summed over its merges.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
ComputeAsuGroups sorted one key per distinct raw hkl on one thread, once per
merge (6-8 merges a run). Ties are now broken on the run index, which makes the
order total, so ParallelSort returns the same sequence; and the result never
depended on the order within a tie (every run in it reduces to the same ASU
reflection). Measured on the GPU build, both sorts side by side on the same
input in every merge of 6 in-house runs: keys identical in all calls; a cytochrome C
run 0.230 -> 0.058 s, a thaumatin run 0.186 -> 0.058 s, summed over its merges.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The first non-search merge on an ingest was made both ways (partials scaled, and
the scale from the fulls alone) and PartialScalingWins picked one. When the prior
is to scale the partials and that merge resolves its error model, the decision is
the prior whatever the other arm says, so the fulls-alone merge was a whole final
merge thrown away - 16% of all scaling time over the battery (238 of 258
decisions), e.g. ~1.5 s of a 22 s GPU run on an in-house cytochrome C set, 8-13 s on
the largest open-arm sets.
That arm is now made only when the partials' merge fails to resolve its error
model (then the old sequence runs unchanged) or when the prior is the fulls
(small molecules; unchanged). The partials' arm starts from g_partial = 1, the
only state the fulls-alone arm left for the arm after it (corr is restored,
g_full reset by the combine, device g/scaled cleared per call, the sweep /
rejection / damage diagnostics reset where they are measured), so the merge is
bit-identical to the one the pair returned.
The log's decision line no longer quotes the fulls-alone ISa when that merge is
not made.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
- OBSERVATIONS_REJECTED / _WILSON / _OVERLOAD: what each counts and how
they relate (the overload count is not part of OBSERVATIONS_REJECTED).
- CENTRE_STATISTICS_REFLECTIONS and the three per-statistic _F keys are
named; NOT_APPLICABLE is stated as the code decides it (any of the three
outside -0.3..1.3, or the control below 0.5).
- BUILD_CXX_FLAGS no longer says that -march moves the results: the build
turns floating-point contraction off and the indexer's reductions no
longer depend on SIMD width, so CPU levels agree. The RUGNUX_GIT and
BUILD_CXX_FLAGS keys were also listed twice in the header paragraph.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Running Rugnux gains a "Small-molecule data" section: what the default run
does about short axes, wide and split spots, sparse patterns, overloads and
glide / centre-of-symmetry space groups, the two settings worth checking
(polarization on a laboratory source, the resolution cut), and what the
unmerged HKLF 4 file holds. Rugnux with other programs gains a worked
SHELXT -> SHELXL route (instruction file built from the report keys, what
SHELXL does with the unmerged file). The SHELXT/SHELXL commands were run on
a battery set's output to check they work as written.
The quick start's file list now includes <prefix>_detector.jpg, which every
default rotation run writes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
What Rugnux reads now describes the d*TREK vocabulary the SMV reader
learned in this release (pixel size, image directions, 2theta arm,
wavelength, encoded overflows), and no longer says that SMV never states a
saturation value: CCD_IMAGE_SATURATION / SATURATED_VALUE are read where
present.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The viewer has its own libjpeg-turbo and libtiff; Qt's copies only risk being mixed into the
static link. Takes effect when the build images are rebuilt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the static Qt of the Linux viewer, QJpegPlugin brings Qt's bundled libjpeg-turbo
(3.1.4, JPEG_LIB_VERSION 80) next to ours from preview/ (3.2.0, ABI 62). Both export the
same jpeg_* symbols and the linker silently kept Qt's, so WriteJPEGToMem - compiled against
our headers - failed with "Wrong JPEG library version". In the viewer that aborted an
in-process rugnux run at the end, where _detector.jpg is collected. The rc173 artifact shows
it directly: jpeg_CreateCompress checks for 80 and the only libjpeg version string is 3.1.4.
The viewer now excludes Qt::QJpegPlugin and the unused Qt::QTiffPlugin (which bound to
whichever libtiff the link offered), and its clipboard copy and "Save image as JPEG" encode
through WriteJPEGToMem instead of QImage::save. The Linux viewer CI job checks the binary
holds no Qt JPEG/TIFF handler and exactly one libjpeg-turbo.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The in-house cytochrome C crystals are bovine heart cytochrome c in a form
with no deposited entry: P 32 2 1, three molecules per asymmetric unit (two
related by a 33% pseudo-translation 0.30 0.70 0.33), c 86.0-88.6 between
crystals. Neither 8RKP (cytochrome c', P 62 2 2: Laue -3m1 data, 6/mmm CC 0.5)
nor 6FF5 (H32: R-forbidden reflections as strong as allowed) is this form.
Phaser with 6FF5 places three copies in P 32 2 1 (LLG 11375 against 10483 in
P 31 2 1; refined R-free 0.310 against 0.345); no CB density at 60/89 says
bovine. Each set gets its own model, refined with phenix.refine against its
XDS data with rugnux's free flags and stored beside the data as
reference_model.cif: one model placed in the other crystals loses ~5 points of
R-free to the 3% spread in c. Refined R-free 0.340 / 0.296 / 0.286.
A set's `model` may now name an mmCIF file beside its data; rfree_deposited
is that file's own _refine R-free.
rugnux --model R-free (6FF5 -> own model): cytc_x10sa 0.505 -> 0.357,
cytc_x06da_1 0.486 -> 0.335, cytc_x06da_2 0.479 -> 0.306; anomalous mean
sigma at the irons 1.7 -> 2.7, 3.3 -> 5.4, 4.1 -> 6.3. Processing unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
6iu9 (deposited P3_1, merohedrally twinned at 0.4 or more by its 321 law: H 0.097, CC 0.79) went
from P3_1 to P3_121 when the penalised per-frame scale smoother (495f6a97b) landed; the ingest
partiality fix alone (7d6c201e9) keeps P3_1. Bisected on the two binaries: smoother-only 6iu9 P3_121,
partiality-only P3_1. The smoother only removed the luck: the P3_1 answer rested on the
Lorentz-filtered arm's added-operator R contrast reading 0.71 against a bound of 0.72 (the
all-observation arm already passed 32 at 0.82); better scaling moved it to 0.87, no gate refused, and
the twin-immune zone - which reads -401 nats, acentric - is only consulted where a gate fired.
A near-perfect twin passes every agreement gate by construction (H ratio 0.97 here); only the zone
can refuse it. It was confined to refusals because a genuine trigonal crystal read -46 nats with no
gate firing; on the battery 5uth (genuine P3_121, twinned by a 622 law, other-law CC 0.14) reads
-206. The reason: the centric density was untwinned, but a twin by a law OTHER than the zone's
operators reaches the zone as it reaches every reflection, so a genuine zone reads 0.81 at a = 0.2.
The operators' own twin law cannot reach their zone, so only the centric side needs it.
- TwinningAnalysis: the zone's centric density is the twinned one at a caller-given fraction
(weighted sum of two chi^2_1, via exp(-y) I0(y)); the control's calibration expectation is taken
at the same fraction (-KL(acentric || centric_a), -0.130 at a = 0 as before). At a = 0 nothing
changes.
- SearchSpaceGroup: twin_fraction_outside = the fraction implied by the strongest CC of a lattice
rotation outside the group, relative to the group's own mean CC, through rho = 2a(1-a)/((1-a)^2+a^2);
exported for the adopted group and used by Rugnux's TwinZoneVerdict and the zone report.
- New Stage A test: a candidate that does not hold every lattice rotation, on a P1 merge whose <|L|>
is in the partial-twin band [0.375, 0.44), is refused when the zone over one of its index-2
subgroups reads acentric by 20 nats (the TwinZoneVerdict bound), unless zones_ambiguous. The band's
lower end is the L-test gate's: below it something else compresses the zones too (a pseudo-cubic
small-molecule set, cuhf2, read its 422 zone at 0.64 beside a control at 0.58 and was turned to
P222 without it).
Zone evidence (calibrated, at the other-law fraction): 6iu9 32 -230 / -143 (refused, P3_1);
5uth +209 / +33 (P3_121 kept); 8xtg +324 / +65; 6vww P6 +20 / +8.5; 7k1l P6 +8 / +7.5.
Battery (open+inhouse, targeted, against 20261004-2350 all2-full; only beyond-noise change listed):
- 6iu9: fail -> pass, P3_121 -> P3_1, R_meas 17.4% -> 14.4%, ISa 4.6 -> 5.4, R-free 0.318 -> 0.327.
- unchanged: 6iu5, 6iu6, 6iu8 (P3_1), 5uth, 5j23, insu_H_x06da_twin/notwin, 6vww, 7k1l, 8xte, 8xtg,
9i80, 4bwl, 2wnq, 8c3e, 2wnn, 2xfw, 5ebi, 6p8j, 6z9g, 6rlr, 6toc, 3mc4; 321/312 controls 5lzl,
6w4h, 9gqg, 6pxb, 9z72; all 14 small-molecule sets (cuhf2 checked again after the band).
- private subset (8 sets): no change beyond noise.
Tests: [twinning] (new: zones read at the other twin laws' fraction), SearchSpaceGroup*.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The gate that refuses a point group whose own operators "disagree among
themselves" read worst-agreeing over best-agreeing operator R. That divides
by an extreme order statistic - the pole the contrast gate was introduced to
remove - so a genuine group whose operators merely spread in a continuum is
refused as soon as one operator happens to agree unusually well.
That is what refused 432 on a private cubic F432 crystal after the overload
drop (37a8c8e24; bisected: that commit alone gives F23, b126b9fc1 does not
change the search). The crystal's intensity-weighted R is dominated by a few
dozen strong low-order reflections. Dropping the overloaded ones (whose
flank-extrapolated values disagreed ~2x with their mates) raised every
operator CC and made the cleanest 432 operator cleaner still, while
anisotropic systematic error keeps the genuine 23 operators spread ~2x
among themselves: worst-over-best went 2.68 -> 4.0 against the 3.2 bound.
H ratio (1.25) and contrast (0.78) passed throughout.
A mixture of real and false operators is specific: the real ones form a
subgroup (the true group's intersection with the candidate) and everything
outside it is false. The statistic is now the largest factor by which every
operator outside some subgroup reads worse than every operator inside it -
a gap, found by walking the operators sorted by R and reading each prefix
that closes into a group. It is never above worst-over-best, so nothing
admitted before can be refused; the bound stays 3.2. Read off the operator
tables of the all2 full battery and run 3: genuine adopted groups reach at
most 2.5 (the cubic crystal 1.8-2.0), the 5ebi 622 mixture reads 3.6
(was 6.0), 9hnc's false 422 3.4, and 2wnn's false 222 3.8 (unchanged, still
refused by this gate).
Targeted runs vs all2-full (f64f76): the private cubic set F23 -> F432
(pass); every other set identical to the last digit - five other private sets; open 5epe 8sqt 3inp 9fhc 6oel 7orr
8tyy 3ky7 6h5t 6moj 2wnn 5ebi 9hnc; lyso_x06da_ref, lyso_x06da_5keV,
myob_x06da_split; all small-molecule sets (citric R1 .0374, aspirin
.0363/.0358, hepes .0310, yag .1021, kdp, lcystine, cytidine, dnba,
lalanine, metformin, nidppe, cuhf2); and the high-spread trigonal/
tetragonal/hexagonal panel (insu_H twin/notwin, insu_I weak/low_isa, 6v2r,
8xtg, 5j23, 6iu6, 6iu8, 3mc4, 6vww, 7k1l, 9i80, 6pxb, 9hs7).
SearchSpaceGroup* and [SearchSpaceGroup] test cases pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Open set 5ebi (deposited P2_1, a = c and beta = 120: a monoclinic crystal on a hexagonal metric,
pseudo-merohedrally twinned, <|L|> 0.433) regressed from P2_1 to C222_1 with dq-sm2-rest baf2017c9.
Of that commit's two changes it is the sub-lattice ask's R-contrast operator test, not the hot-patch
mask: 5ebi is a CCD (.img) sweep, so HotPixelFinder never runs on it. Without the R contrast every
metric two-fold that passes min_operator_cc was kept, LePage closed them into the whole hexagonal
metric and no sub-lattice ask was made. With it the false hexagonal two-folds drop out, the two
in-plane two-folds of the C-centred orthohexagonal cell survive (CC 0.97, R 0.061 against 0.045 for
the genuine screw: contrast 0.96), and the ask on that sub-lattice adopted C222_1. The main search
had refused 622 on the added-operator R (contrast 0.70 against 0.72), but the ask's own search
cannot see that: the gates there are relative and the twin law passes them on its own. Stage A's
L-test-under-merge gate does not reach it either - C222_1 is not holohedral on a hexagonal metric -
and read per operator it does not separate it: merging under the in-plane two-fold moves <|L|> only
0.425 -> 0.418 (14% of the gap to 0.375).
Fix: where a twin gate of the search in hand refused a higher group, an ask that would promote
re-opens what that gate answered, so it is arbitrated as every other promotion over a refusal is -
by TwinZoneVerdict on the operators it adds over the group in hand (that group carried into the
ask's setting). Acentric with margin and the group in hand stands. 5ebi: zone 222 over 2,
n 1865, calibrated -23.9 nats -> twin law, P 1 21 1 kept. cuhf2 (the case baf2017c9 fixed) has no
twin-gate refusal in hand, so its zone is not read and P422 stands.
Targeted battery vs the rc174-all2 full run (20261004-2350_f64f76_all2-full), same base:
reg4-v1 (32 sets: 5ebi, cuhf2, sub-lattice sets 6iu6 6iu8 6iu9 6z8o 8t7r 9ea5, twinned/pseudo-
merohedral 9hnc 3meb 2wnn 2xfw 2wnz 4bwl 2wnq insu_H_x06da_twin, controls lyso_x06da_ref
thau_x10sa_0p1deg 5reo, all 14 small-molecule sets): only 5ebi changes - C2221 -> P1211 (pass),
ISa 10.9 -> 14.8, R_meas 12.3% -> 10.2%, R_free (deposited model) 0.571 -> 0.537, cell dev 40% ->
0.05%. Every other set identical in space group, ISa, R_meas, R_free, d_min and SHELXL R1.
reg4-v1b (the 12 other open/in-house sets whose run adopts a metric or sub-lattice ask): identical.
Private arm (the 7 sets whose mask changed with baf2017c9 and the 4 that take an ask): identical.
Still failing as before: 4bwl, 2wnq (near-perfect twins, C222_1 via the metric ask with no refusal
in hand), 6iu9, 9hnc, kdp.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Each lysozyme, thaumatin, insulin, cytochrome c and myoglobin set names the PDB
entry of its crystal form in a `model` field; the runner fetches it like an
open-arm deposition and passes it to --model, so the row carries rfree,
rmodel_shell_scaled, rfree_deposited/rfree_ratio and the new anom_sigma
(ANOMALOUS_SCATTERER_MEAN_SIGMA) - an external yardstick beside the XDS
statistics. Reported only: the arm's scoring is unchanged, and the processing is
identical with and without the model (19 sets vs all2-full: same space group
modulo the model's enantiomorph label, d_min, R_meas, CC1/2, ISa, cell and
REFRES table). depdata_check stays open-arm only.
Models, chosen by cell/space group and confirmed by rugnux --model:
4AXT lysozyme (6G8A, the coordinates beside the data, has c 2% short: R-free
+0.04), 5A47 thaumatin (6G89 +0.06), 2BN3 cubic and 4E7T rhombohedral insulin
(bovine; pig 9INS +0.07, human 1MSO +0.08), 4DC8 horse heart myoglobin (b 28.8,
not sperm whale's 30.8; 5ZZE equal), 6FF5 bovine cytochrome c - our crystals
are a primitive relative of its H32 form (pseudo-centring TNCS), so R-free
~0.49 but the heme iron is the top anomalous peak; 8RKP, the cytochrome c'
beside the data, does not fit at all (R-free 0.63, nothing at the iron).
Run 20261005-0735_2624a6_inhouse-model (19 sets): R-free 0.18-0.30 on the
good sets, 0.37-0.57 on the ISa~2 powder/split myoglobin sweeps and ~0.49 on
cytochrome c; anom_sigma rises with wavelength (thaumatin 2.3 at 16 keV, 6.7
at 6.5 keV, 8.9 at 3.8 keV).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
ANOMALOUS_SCATTERER_MEAN_SIGMA= is the anomalous difference map, read at every
model atom from phosphorus up (S of Met/Cys, metals, Cl, I) with the same cubic
interpolation as ANOMALOUS_SITE_*, averaged in map sigma; ANOMALOUS_SCATTERERS=
is their count. One number for the anomalous signal of a merge that does not
depend on which ten atoms come out on top - what the battery tracks for the
long-wavelength standard-protein sets. A new key only: no report version bump.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
In a cubic Laue class AnalyzeAnisotropy returns before any fit with the eigenvalues set to
zero but the eigenvectors left at their NaN defaults. AnisotropyTensorHKL multiplies them
(0 * NaN), so the anisotropic French-Wilson prior of 4979f8aa6 was NaN for every cubic
data set: every shell mean became NaN and the second French-Wilson pass rewrote all
weak amplitudes from a NaN prior (on 5epe the weak-reflection F rose by 2% at low and
14% at high resolution against the merge-time pass, IMEAN unchanged). The cubic result
now carries the crystal axes as its (arbitrary) principal frame, so the tensor is zero
and the prior isotropic, as the commit intended.
The second pass otherwise sees exactly the merge-time pass's input (same reflections,
d range, shells and space group; checked on 5epe), so in non-cubic groups the
difference between the two passes is the tensor alone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Non-Sohncke candidates were enumerated only when their PROPER rotations equalled the measured
point group's, which only centrosymmetric groups satisfy. Groups without a centre (Pc, Pna2_1,
I-42d, I4_1md, Fdd2, P-42_1c, I-43d) were never candidates: where a centrosymmetric group shares
their absences rugnux wrote it silently (Pc -> P2/c), and where none does the dead glide zone was
dropped and a Sohncke subgroup written (KDP-type I-42d data -> I4_122).
- Enumerate non-Sohncke groups by Laue class. The Sohncke-signature dedup, CellHostsRotations and
the glide-evidence bar are unchanged. Groups with identical absences in the same Laue class are
scored once and carried as `same_absences`; a selected candidate brings them into
`alternatives`.
- Convention for which twin is written: the centrosymmetric one where it exists (missed centres are
the common error, Baur & Kassner 1992), otherwise the lowest-numbered (I4_1md over I-42d).
- The centrosymmetric default gives way only when, on the general reflections of the Laue class of
the final merge, <|L|>, <|E^2-1|> and N(0.1), each calibrated against acentric/centric intensities
simulated with the reflections' own sigmas (fixed-seed mt19937_64, own deviate transforms), read
acentric (L f <= 0.3, others <= 0.5), every f lies in [-0.3, 1.3], the always-centric control reads
centric (f >= 0.5), and the lattice excludes twinning (gemmi Le Page metric admits no rotation
beyond the Laue class, no TWIN_DOMAIN leftover lattice). Then the group is switched and re-merged.
- Report: SPACE_GROUP_CENTRE (IMPLIED_BY_ABSENCES / ABSENT_BY_ABSENCES / NOT_DETERMINED /
ABSENT_BY_STATISTICS), CENTRE_TWINNING_EXCLUDED, CENTRE_STATISTICS_* on every searched run, and
prose for the absence-equivalent alternatives.
Battery (targeted): all 14 small-molecule sets vs 58a4bf (smt-all, kdp-all): every group
unchanged except kdp_x10sa_20keV I 41 2 2 -> I 41 m d with I -4 2 d as the alternative; dnba stays
C 1 2/c 1, now NOT_DETERMINED with C 1 c 1 listed (statistics read centric, f 0.96-1.24). The
P2_1/c, Pbca and Ia-3d sets read IMPLIED_BY_ABSENCES with no alternative.
Protein panel (19 sets, twinned panel + controls, run glide-prot): every space group equals the
rc174-all base (3r6o re-run with the base binary: I 41 2 2 both); no SPACE_GROUP_CENTRE on any.
Twinned proteins read the centre statistics NOT_APPLICABLE (L f -0.3 to -1.7) or ACENTRIC.
Private subset (8 sets): 8/8 groups unchanged vs 58a4bf.
The glide-zone evidence scan over battery 3 (every non-Sohncke group of each run's Laue class,
centrosymmetric or not) peaks at 0.69 nats/reflection on proteins against the 2.0 bar.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A crystal of slightly misaligned domains (a ferroelastic domain twin below a phase transition, a
split crystal) records each reflection as two or more compact spots around the averaged lattice's
prediction, moving apart with resolution. The pre-scan footprint is measured about each spot, so it
saw compact spots; the r1 disk held the gap between them and the background ring sat on them.
The geometry pre-pass now compares every indexed spot with the predicted position of its own
reflection on the same frame and adds the mean square offset (radial and tangential, by distance
from the beam) to the pre-scan widths; the canonical pass integrates with that table. Where spots sit
on their predictions this moves the widths by the prediction error alone (lysozyme: 0.2-1.0 px, no
reflection outgrows r1); on a 100 K KDP domain twin the offsets reach 8-16 px at the edge.
KDP (kdp_x10sa_20keV), battery SHELXL recipe on the COD model: R1 0.272 -> 0.048, wR2 0.685 ->
0.129, EXTI 26.9 -> 0.025, GooF 3.2 -> 1.29 (XDS: 0.112 / 0.333 / 0.054 / 1.25); R_meas 21.6% -> 5.3%.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The multiplicative outlier band (c8de5f8d6) took its width from the error
model's b. b is a Gaussian width in I and grows with whatever tail the fulls
have: on a low-ISa hexagonal small-molecule set with a population of fulls
lost to near zero, b = 0.54 against a core ln-spread of 0.25, and exp(6 b) = 25
let the high outliers of the weak reflections through (SHELXL R1 0.191 ->
0.333 there, 0.111 -> 0.124 on its 25 keV sweep).
The spread is now measured on the merge's own fulls: the weighted median of
|ln(I / median)|, weighted by (<I>/sigma_counting)^2 so the fulls whose ratio
counting noise does not blur carry it; b remains the fallback where nothing
can be measured.
Battery (29 sets: 13 small-molecule, 16 protein), SHELXL R1 against the
median fix alone: the strongly absorbing cubic set 0.116 -> 0.0998 (b-band
0.103); the low-ISa hexagonal set 0.191 -> 0.174 at 20 keV, 0.111 -> 0.101 at
25 keV; organics within +-0.0003 or better (cytidine 0.0617 -> 0.0613,
lalanine 0.0541 -> 0.0533). Proteins unchanged except two low-ISa sets whose
CC1/2 cutoff moves: 6yqf 3.32 -> 2.89 A (placement R-free at a fixed 3.32 A
0.4329 -> 0.4326, at 2.89 A 0.4642 -> 0.4630), and the split myoglobin set
1.77 -> 1.94 A (low-resolution R_meas 33.0% -> 27.3%). Private subset: two
sets' R_meas lower, the rest unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The 6-sigma outlier band was symmetric in I about the reflection's median,
while the error model's systematic term b*<I> is a multiplicative error
(absorption, illuminated volume, a scale that is off), as likely a factor 1/f
as f. With b of a few percent the two agree; with a large b the linear band
reaches far below the median and only a little above it. On a strongly
absorbing cubic crystal (internal b = 0.34) 96% of the 828 rejected fulls read
2-7x their median in well-illuminated frames - the least absorbed, nearest the
true intensity - and almost none read low.
The b part of the band is now exp(+-n*b) about the expected intensity, the
counting part stays linear (OutlierBand.h, one formula for the host loop and
the device kernel; the two exp constants are taken once on the host). To
first order in n*b it is the old band.
That set's SHELXL R1 0.116 -> 0.103 (--reject-outliers 12 / 20 on the old
band: 0.107 / 0.101; no rejection at all: 0.31).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The per-frame scale was fitted on the partials from 50 rocking events per
frame and taken from the fulls alone below. The counts of small-molecule
sweeps (3-43) and proteins (5-900) overlap, and a weak protein at 36
events per frame scaled from its fulls had no resolved error model at all
(ISa undetermined, d_min 5.10 A) where its partials gave ISa 32 and 4.88 A.
The first merge of a pass that is not a space-group search is now made
both ways, and the count stands as the prior unless its merge has no
resolved error model while the other merge has one. Search merges (P1 /
subgroups) take the prior.
The two ISa values are deliberately not compared beyond that. Tried as
the arbiter, "higher ISa wins" (and weighted R_meas as fallback) agreed
with the external yardsticks on 10 of 11 crystals but chose the partials
on a 6-events-per-frame small-molecule sweep: ISa 10.5 vs 8.7, weighted
R_meas 0.102 vs 0.115, SHELXL R1 0.105 vs 0.062 - the partiality error
the partial scale absorbs is shared by symmetry mates at the same
rocking geometry, so their agreement cannot see it. Statistics of the
difference between the two arms' frame scales did not separate that
sweep from the proteins that want the partials either.
Effect: only crystals whose prior arm fails change; every small-molecule
set and every protein control keeps its arm. Private weak protein: ISa
undetermined -> 32.3, d_min 5.10 -> 4.88 A, weighted R_meas .343 -> .337.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A saturated pixel in a spot means the brightest part of the reflection was
not measured. The integration used to drop the peak frame's partial (its
peak pixel is unreadable) and keep the flanks, so the combine extrapolated
the event from its tails by the partiality model: on a strongly
diffracting small-molecule crystal the strongest low-order reflections
read 2-3x low and were the largest SHELXL misfits. XDS drops such a
reflection (OVERLOAD); so does rugnux now.
- Integration (CPU + GPU engines): a reflection is `overloaded` when a
signal-disk pixel is saturated, or unreadable on this frame but not in
the run's pixel mask - EIGER/PILATUS write their error value for a
pixel they could not count, which the preprocessor turns into a masked
pixel like a gap's. The engines now receive the PixelMask to tell the
two apart (an earlier attempt that re-classified the marker as
saturation in the preprocessor broke a dataset whose gaps are not in
the file's mask). An overloaded reflection is kept with its box sum,
unfitted, only so its event can be recognised.
- Rotation combine (CPU + GPU): an event with any overloaded partial is
dropped whole; counted in the log and the report
(OBSERVATIONS_REJECTED_OVERLOAD=). The unmerged MTZ export drops it too.
- Everything else that reads reflections leaves an overloaded one out:
AcceptReflection (stills merge, per-image scaling), the post-refinement
gather, the axial-row sums.
- Capture uncertainty: the merge rebuilds each full's variance at the
reflection's mean (counting_variance / ModelSigma) and dropped the
capture term the combine had put into sigma, so a full extrapolated
from part of its rocking curve merged at the weight of a whole one.
Fulls now carry it (Obs::capture) and the rebuilt variance adds
(capture * <I>)^2, host and device.
SHELXL R1 on rugnux's own integration (harness), median fix -> this:
citric acid .0648 -> .0420 (XDS .051; 221 events dropped, EXTI 1.02 -> 0.29),
HEPES .0396 -> .0381 (184), aspirin 20 keV .0387 -> .0385 (6),
aspirin 25 keV .0376 -> .0375 (5); metformin/nidppe/dnba/lalanine/cytidine
no overloads, unchanged. YAG .116 -> .128 (87 dropped; its scale loop does
not settle either way). Proteins and private subset: see the branch report.
Tests: BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped (new),
BraggIntegrationEngineGPU_MatchesCPU (overloaded flag compared),
AcceptReflection_ResolutionLimits, [write_reflections], [large].
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Two defects that together turned a tetragonal small-molecule crystal (open-arm cuhf2, published
P4/nmm) into P222.
1. HotPixelFinder masked a persistent strong pixel only when it stood alone or in a pair ("a
larger patch is a feature of the scattering"). On cuhf2 a ring of ~20 pixels reads 1e5-3e5
counts on EVERY frame (dead centre) - stationary in the lab, so no reflection of the rotating
crystal. Unmasked, the reflections crossing it ((2,6,+-9)) merged to 3.5e6 / 6.7e5 against 4e4
for the strongest real reflection. The final merge's outlier test removes them (12
equivalents), but the space-group search's P1-like merge has no equivalents to judge by, and
the h<->k operator read CC 0.18 / R 0.28 instead of ~0.99 / 0.013. The persistence and strength
tests are unchanged; only the isolation requirement is dropped.
2. The sub-lattice ask kept every metric two-fold that passes min_operator_cc. On a pseudo-cubic
cell two false cubic two-folds still correlate at 0.31-0.33, LePage closes them with the
genuine ones into the full cubic metric, and the tetragonal class in between is never asked.
An operator now also has to pass the R contrast every promotion's added operators are held to
(min_operator_r_contrast against random_pairing_r / global_best_operator_r, read only where the
search reads it): the false ones sit at 0.2, the genuine ones at 1.0.
cuhf2: P222 -> P422, R_meas 2.9% -> 2.6%, ISa 45.5 -> 52.3 (overnight-cint had found P422 by the
sub-lattice path; rc174-all lost it). Prescan survey, masked pixels base -> new: unchanged on
lyso_x06da_ref/5keV/atten_wedge, thau, insu, cytc, myob_split, 9qw8, aspirin20, HEPES, YAG,
metformin, nidppe; cuhf2 3 -> 42, 5reo 2 -> 6, lcystine25 0 -> 9 (two clusters of noisy pixels at
3-7 counts/frame on a zero background). Targeted battery (sm2rest-fix vs rc174all-ctl / smt-all):
the 15 protein / sub-lattice sets (lyso x3, thau, insu, cytc, myob_split, 5reo, 9qw8, 6iu6, 6iu8,
6iu9, 6z8o, 8t7r, 9ea5) identical to the 4th digit; SM sets identical except cuhf2 (above) and
lcystine25, SHELXL R1 .112 -> .151: the nine masked pixels flip the fulls-only per-frame scale
smoother on that polycrystalline sweep from "settled after 29 iterations" to "stopped settling"
(the base run reproduces bit-identically) - a sensitivity of that smoother, reported to the
scaling work, not of the mask.
Test: HotPixelFinder_PersistentPixelNotBragg gains a 3x3 patch that must be masked. The
device-vs-host test now waits for each queued frame before overwriting the shared device buffer;
it remains flaky at ~4/25 on the base binary as well (a borderline pixel or two differ), which is
pre-existing and not addressed here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The per-reflection median the 6-sigma outlier test is centred on was weighted
by each observation's own counting variance. That variance falls with the
observation's own intensity, so the median went to whichever equivalents came
out low. On a strongly absorbing cubic crystal, where frames near the edge-on
orientation read some equivalents 10-100x down by their diffracted direction,
the median sat on the absorbed ones and the test removed ~10% of the
observations, all of them correct measurements 3-20x above it.
The median is now weighted by the counting variance at the reflection's
expected intensity - the rule the merge weights already use - which keeps the
down-weighting of frames scaled up by 1/G and drops the dependence on the
observation's own fluctuation. In-house garnet (Ia-3d) SHELXL R1 0.139 -> 0.116
(--mode scale on the same integration).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A spot that outgrows the r1 disk is summed over its measured footprint ellipse and its background
ring starts beyond it. That reach was the same 3 sigma that decides whether the spot outgrew the
disk at all. Wide spots are not Gaussian - mosaic streaks and diffuse halos carry flux past 3 sigma -
so the ring started on the spot's own tails and read them as background.
The install test stays at 3 sigma (BRAGG_FOOTPRINT_NSIGMA); the new BRAGG_FOOTPRINT_REACH = 4 sets
how far the summation ellipse, the ring start and the profile grid go. Compact protein spots never
install the footprint, so they are unchanged bit for bit.
Evidence (rugnux's own combined fulls put through XDS's own per-observation corrections, so only
the integration differs; SHELXL R1(>4sig) on ~93% of observations matched to XDS by hkl):
citric acid: 3 sigma .0521, 4 sigma .0519, 5 sigma .0526 (XDS 3D summation .0484)
HEPES: 3 sigma .0348, 4 sigma .0338, 5 sigma .0344 (XDS .0311)
Full pipeline SHELXL R1: citric .0697 -> .0685, HEPES .0405 -> .0370.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The open-arm L-alanine set's COD reference (2104782) is a 0.2 GPa room-temperature
diamond-anvil-cell structure (372 reflections, 43% complete), not the ambient one the
docs called it. The set's own cell (5.787 5.940 12.263) is the 100 K ambient form, so
the reference is now COD 2311261 (100 K, 5.791 5.944 12.269; Parsons, Flack & Wagner,
Acta Cryst. B69 (2013) 249-259). SHELXL's refined R1 does not depend on the starting
model (0.0546 vs 0.0524), but the fixed-model R1(F) the battery reports goes from 0.168
to 0.087, so that number now measures the data rather than the pressure difference;
cell_dev_pct likewise compares against the right cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Merged dq-sm-absorb, whose ingest fix (corr_ingested carries the recomputed partiality, not the
predictor's) repairs the same defect as this branch's unit partial scale on the path without
partial scaling. The two did not double-apply - the unit scale rebuilt corr from scratch - but
they differ by the incident flux, and measured head to head the flux costs the sparse sweeps:
SHELXL R1 on four in-house organic sweeps 0.0459/0.0396/0.0709/0.0413 with it, 0.0421/0.0393/
0.0697/0.0405 without; on five open small-molecule sets equal or better without (0.0707 ->
0.0660 on one). Only the cubic absorbing sweep prefers it (0.119 vs 0.139): there the
background really does fall with the absorbed beam. So the unit partial scale and its device
kernel are removed and the ingest fix stands alone; the flux meter's smoothing, which only
mattered on that path, is reverted, so the partial-scaling path sees the flux exactly as
before. The penalised per-frame scale of the fulls stays.
Also restores two characters of CPU_DATA_ANALYSIS_INTEGRATION.md that the previous commit's
rewrite had turned from a stray carriage return into a line break.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Stage A's gates all compare the operators' disagreement with a reference - a parent group, the best
operator, the noise floor - and a pseudo-symmetric structure or a twin defeats every one of them: the
added operators correlate nearly as well as real ones. On the open arm that over-called 2wnz and 2xfw
(deposited P2_1, pseudo-merohedral C-orthorhombic metric) as C222_1, and 7bgu (deposited P1 with
b = c to 2.7%) as C2.
New gate, for a candidate that holds every rotation of its lattice (gemmi find_lattice_symmetry at
3 deg obliquity, with the lattice centring now passed in SearchSpaceGroupOptions::lattice_centring):
no twin law exists for such a group, so intensities merged under it that read like a twin's are not a
twin. AnalyzeLTestUnderMerge reads the Padilla-Yeates L-test twice on the same pairs of the P1 merge -
as measured, and with every intensity replaced by its plain orbit mean under the candidate - so the
narrowing is the merge's own doing and not a property of the data. Refused when the merged <|L|>
reads twin-like (0.375 <= <|L|> < 0.44) and the averaging closed a third or more of the gap from the
unmerged <|L|> to 0.375.
Calibration on the C++ search merges (targeted battery sympg-pg1): genuine lattice-holohedral groups
close 0.01-0.25 of the gap (highest 6z9g, indexed on half its deposited cell, 0.23-0.25; 5vml 0.16-
0.17; 7kcn 0.06-0.09); the over-calls 2wnz 0.47, 2xfw 0.46-0.47, 7bgu 0.79. A merge reading below
0.375 is not read - nothing a twin or a false operator does reaches it, so something else compresses
the intensities: 8c3e (0.353) and the genuine 9zmu (0.351, 494 A axis) look alike there. Near-
perfect twins (4bwl 0.14-0.16, 2wnq 0.13-0.20) are out of reach by construction, and 6p8j (0.19-0.30,
deposited as a twin of P2_1) is left alone. Cost: 170 ms per test on a 730k-reflection merge, only for
candidates that pass every other gate and hold the lattice's full symmetry.
Targeted battery sympg-final (44 open/in-house sets: the over-calls and a control panel of genuine
lattice-holohedral, twinned and twin-like sets) and the private arm: 2wnz C2221 -> P1211 (R_free with
the deposited model 0.227), 2xfw C2221 -> P1211 (0.215), 7bgu C121 -> P1 (R_meas 0.28 -> 0.14), all
the deposited groups. Every other set keeps its space group; against the rc174-cand runs of the same
sets, open and private, nothing else moves beyond run time. Still over-called: 8c3e, 4bwl, 2wnq,
6p8j, 3r6o (reasons above; 3r6o and 8c3e read below 0.375).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Every partial's corr is formed at integration as image_scale_corr = LP*QE*flight / partiality with
the predictor's per-frame partiality. Ingest then recomputes every partiality from the
frame-order-smoothed mosaicity (and the one exact-Bragg angle per rocking event), but corr_ingested
kept the predictor's 1/p. Where partials are scaled every fitted frame's corr is rebuilt from the
recomputed partiality, so that path only saw it in the first reference. Where they are not - fewer
than 50 rocking events per frame: small molecules, and long-wavelength protein sweeps - corr is what
the 3D combine divides each partial by, and the full became a weighted mean of I/p_pred while the
event's recomputed fractions summed to 1.00: the combined fulls tracked 1/sum(p_pred) (r = -0.96)
and scattered 13% rms about the plain sum of the same partials.
corr_ingested is now rescaled by p_pred/p_recomputed once at ingest.
Targeted battery against rc174-cand: every set on the partial-scaling path is bit-identical
(5reo, 9qw8, lyso_x06da_ref, thau_x10sa_0p1deg, lyso_x06da_atten_wedge, myob_x06da_split,
cytc_x06da_1, insu_I_x06da_5keV/6keV). Small molecules, SHELXL R1 against the COD model:
aspirin 20 keV 0.0515 -> 0.0432 (ISa 10.3 -> 30.9), aspirin 25 keV 0.0451 -> 0.0396, HEPES
0.0452 -> 0.0420, citric acid 0.0747 -> 0.0726, dnba 0.0472 -> 0.0273, metformin 0.0405 -> 0.0326,
nidppe 0.0498 -> 0.0426, lalanine 0.081 -> 0.057, cytidine 0.077 -> 0.071. Long-wavelength
proteins on the fulls-alone path: lyso_x06da_5keV ISa 17.9 -> 27.5, thau_bl1a_3p8keV 21.7 -> 28.4,
thau_bl1a_4p6keV 26.4 -> 33.5. One regression: lcystine_x10sa_25keV (a polycrystalline aggregate)
now refuses 622 and reports P31 instead of P6122.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The French-Wilson prior of each reflection is now epsilon * K_shell * a(h), with
a(h) = exp(-1/2 s^T B s) from the deviatoric tensor AnalyzeAnisotropy already fits
(the form it is fitted in) and K_shell = sum(I/eps) / sum(a), so a shell's priors still
average to its measured mean. The amplitudes are made isotropically at the merge as
before and made again once the tensor exists (full pipeline and --mode scale). Only
F/SIGF and F(+)/F(-) change; IMEAN/I(+)/I(-) are bit-identical. Applied whenever a
tensor was fitted, with no detection gate: a near-isotropic tensor gives a(h) ~ 1 and
the isotropic prior back, and the prior wants the best estimate of <I> along h whatever
its cause. Follows ctruncate's anisotropic prior (Ballard & Stein, CCP4); credit in
ACKNOWLEDGEMENT.md, CPU_DATA_ANALYSIS.md and at the algorithm.
--model scaling (ModelScaling.cpp) was checked: k_overall + symmetry-constrained
anisotropic B + flat bulk solvent fitted on the working set, against the same FW F
written to the MTZ - as REFMAC/phenix.refine do. No change needed.
Evidence (REFMAC 10-cycle restrained refinement of the deposited model, R-free on
the depositor's free reflections shared by both data sets; base = rc174 processing,
same IMEAN):
set base new d set base new d
9rcs 0.3475 0.3494 +0.0019 8qq7 0.4452 0.4503 +0.0051
9yzk 0.3192 0.3171 -0.0021 9hs7 0.2898 0.2465 -0.0433
6yqf 0.4642 0.4543 -0.0099 5nw5 0.3256 0.3206 -0.0050
7n2s 0.3126 0.2998 -0.0128 6z8o 0.2927 0.2892 -0.0035
6qaj 0.3390 0.3038 -0.0352 6moj 0.2756 0.2651 -0.0105
6r72 0.3826 0.3797 -0.0029 7qij 0.3239 0.3140 -0.0099
anisotropic sets: median -0.0075, mean -0.0107, 10/12 better
isotropic controls: 5reo -0.0014, 7kcn +0.0003, 6fid +0.0002, 11if 0.0000
rugnux's own --model R-free moves the same way (median about -0.019; 5nw5 +0.006),
R_model shell-scaled too; dep_cc_delta unchanged (intensity based). The adoption rule
(median gain >= 0.005 on the anisotropic sets, no control worse than +0.002) is met.
The two sets that lose are the one with a FLAT resolution signature (8qq7) and 9rcs,
where the exp form drives the dead direction's prior to ~0 beyond 3.7 A.
For scale: ctruncate's own anisotropic prior on the same merges moved the same
REFMAC R-free by a median of only -0.0008 (9hs7 +0.026).
Inhouse lyso_x06da_ref, thau_x10sa_0p1deg: every battery metric unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Three defects in the per-frame scaling of sparse (small-molecule, weak) rotation sweeps:
1. The fulls' per-frame scale pooled sparse frames by their RAW full count, but in a
high-symmetry group with many systematic absences most usable counts stayed under the
minimum, so most frames were never fitted and kept corr = 1 beside pinned, fitted frames
(two gauges in one reference). The release step then read the gauge offset (a constant
134x on a cubic Ia-3d small-molecule sweep) as signal and gave each frame exp(kept_f * 4.9) - the e^-5
errors on a quarter of the frames, R1 0.62. A fixed box window also cannot follow a
100x absorption ramp over a few degrees, and frames under the credible floor, exempt
from the window, ran away to 1e-7.
Now each round fits every frame on its own fulls (no pooling, no minimum), and the scale
is a penalised second-difference smoother of log G (Whittaker/Eilers), each frame at
the information of its fit, lambda by cross-validation over blocks one rocking curve
wide (interleaved single frames leak through shared rocking curves and chose to follow
every frame). After convergence the existing ShrinkToRestrained hands back the per-frame
deviation its neighbour shares. Pooling and the box window are gone from the fulls loop;
the partials loop is unchanged.
2. With partial scaling off (< 50 rocking events per frame) the partials kept the
integration-time corr: no incident-flux correction and not the partiality of the
ingest-smoothed geometry, because only the partial scaling loop rewrote corr. They now
get corr = prescaling_corr / partiality at G = 1 (host, and a device kernel).
3. The flux meter (per-frame mean background) jumped 30x between neighbouring frames of a
sparse sweep - on a few reflections it measures which reflections the frame holds. It
is read through the same smoother at the precision of each frame's mean.
SHELXL R1(>4sig) against the published structures, rc174-cand -> this, on the in-house
small-molecule sweeps: cubic Ia-3d 0.615 -> 0.119 (XDS 0.088); four organic sweeps
(monoclinic / orthorhombic) 0.0515 -> 0.0459, 0.0451 -> 0.0396, 0.0747 -> 0.0709,
0.0452 -> 0.0413; ISa up to 11.6 -> 31. Raw flux instead of smoothed costs 0.002-0.003 R1 on
the first two. A weak, decaying protein sweep on the fulls-only path: ISa 6.7 -> 27.1.
CPU and GPU paths agree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The SHELX .hkl test removed its file while its own ifstream still held it
open, which Windows refuses. The GPU-vs-gemmi model-scale check held b_star
to 1e-4 of its largest element alone, which on an F4132 case fitted at
B 0.6 A^2 failed on a 1e-4 A^2 difference; it now also allows 0.001 A^2
of B.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Where the file's centre and the background-measured one index the same lattice in the same class,
the check kept the file's centre and carried nothing forward. But a centre off by about one
reflection spacing indexes the same cell just as well, with reflections given an index one out over
part of the sweep, and only the merge shows it. On 9hnc (pseudo-tetragonal P2, c* spacing 11.3 px on
the detector) the post-refinement walked the file's centre to 1219.4 px while the background measured
1229.95 +- 0.34 px: the run merged at R_meas 0.58, ISa 1.3, CC1/2 0.84 and adopted P121.
Now the measured centre is kept as a hypothesis when it is further from the file's than this
geometry absorbs (and than 3 sigma of its own fit), and RunAllPasses runs the first pass there only
when the file's centre merged inconsistently: CC1/2 below 0.9 in the lowest-resolution shell of the
search merge, where every reflection is strong. Over the corpus runs read that shell at 0.967-0.998;
9hnc's file centre reads 0.610. Running it unconditionally would cost a second first pass on 107 of
227 battery sets.
MeasuredCentreWins on the same lattice compared only the strong-reflection count, which the
misindexed arm won by 2% (335800 vs 329330). It now also lets the measured centre win when its
inner-shell CC1/2 beats the file's by MAX_CC_HALF_LOSS - 0.997 against 0.610 on 9hnc.
9hnc: P121 R_meas 0.578 ISa 1.33 CC1/2 0.844 -> P1211 (= deposited) R_meas 0.129 ISa 13.5 CC1/2 0.998.
Its screw now reads 2_1 (32 absent, 0 violations) beside a detected (1/2,1/2,0) pseudo-translation,
against the deposited P121, so the battery verdict moves pass -> fail on the screw alone; R_free with
the deposited model 0.443 -> 0.255.
Battery sympg-bc1 (22 targeted sets, 13 of them in the same-lattice branch with a centre further
off than the geometry absorbs): only 9hnc tries the measured centre; the others read 0.984-0.999 in
the inner shell and are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The re-selection among tied higher-group candidates left
sohncke_space_group at the old lowest-numbered pick, so the report read
SPACE_GROUP_NAME P 42 21 2 beside SOHNCKE_SPACE_GROUP P 4 2 2.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
refined_shared() recomputed R-free as sum|F - FC_ALL_LS| / sum F from REFMAC's output MTZ. That
MTZ's F is not the input F: REFMAC writes it with its overall anisotropic scale applied. On
strongly anisotropic data the scale multiplies the noise of the weak direction, and the recomputed
R comes out far above REFMAC's own R-free over the very same reflections.
9RCS (deltaB ~250 A^2, deposit STARANISO-corrected): the recomputed shared-set R-free was 0.503
against REFMAC's own 0.368 on the same 378 free reflections (deposit 0.366/0.365), which is the
whole of the +0.137 the open arm reported for this set. Across the 152 untwinned rc173 sets the
two agree to a median 0.0001 and to within 0.03 everywhere else.
Both refinements' free sets are the shared set by construction, so REFMAC's own R-free is the
shared-set number already, as the twin branch used it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Two first-pass decisions taken on frame counts at their noise floor, both found on a hexagonal
small-molecule crystal embedded in a powder of its own microcrystals (the non-ice powder rings
index on the crystal's own cell; 13-16% of the pooled validation spots lie on the lattice):
- Sparse lattice. A lattice whose pooled spots fall under the per-frame floor (20%) is integrated
on every frame only if fewer than 1/6 of the validation frames clear the floor. At 12/60 the floor
was kept, so only the frames in the upper tail of the same population were integrated: 267 of
1800 frames merged, 22.7k rocking events -> 4.2k fulls, completeness 74%. The bar is now the
majority (score < N/2). 25 keV sweep: completeness 74% -> 99% to 0.56 A, frames rejected
1533 -> 137, SHELXL R1 (COD model) 0.150 -> 0.156 / 0.150 -> 0.139 depending on the COD entry.
No other set of the overnight battery (open, in-house, private) is in the changed band
(pooled < 20% with 10-29 of 60 frames).
- Axis sign. The opposite spindle sign was adopted on 1/60 frames against 0/60; every rescue after
that ran on the wrong sign and the 20 keV sweep ended in "No crystal lattice found" (with some
compiler flags the file's sign scored 9/60 and the run indexed). The flipped lattice must now
also beat chance on the pooled spots, the test the run puts its final lattice to; a sparse
pattern at the right sign still passes (2/60 frames, 89% of the validation spots). 20 keV
sweep: fails -> P 61 2 2, 5.41/56.0 A, SHELXL R1 0.137 (was 0.76 in the runs that indexed).
tools/battery/shelx_check.py: the fixed-model R1(F) now fits SHELXL's extinction coefficient
together with the scale (scan over x), since extinction belongs to the crystal and otherwise
dominates the strongest reflections (HEPES 0.099 -> 0.059).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Mapped every single-axis screw candidate of every primitive-lattice set in
battery 3 (open + in-house) and the private arm against the deposited group.
False zones with no violation read at most +4 nats (plus one pseudo-
translation row at +30.5 that no bound separates); true zones go down to
+10. The true zones under 20 are all short monoclinic rows (b ~ 25-30 A:
four to six 0k0-odd reflections inside the search's resolution range, on
weak data at a few percent of their row), refused at 20 on five crystals
that are P2_1: four myoglobin sweeps (10.0-16.3 nats) and 6cs9 (10.8).
The rc174-cand myob_x06da_split call sat at 21.1, one refit away from P2.
Re-scoring the stored candidate tables at the new bound changes only those
five plus 3r6o (I4_1 2 2 newly eligible, toward the deposited I4_1 screw);
no adopted group elsewhere. New test section pins both sides: four 0k0-odd
at 2% of their row are claimed (11.3 nats), at 20% they are not.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
When a refused point group is re-asked on the adopted group's merge (twin-immune
zone / merge comparison), that merge no longer holds the adopted group's
screw-absent axial reflections. The higher search then had nothing to judge on
those rows, every candidate of the point group tied, and the lowest-numbered -
the one claiming no screw - was written. Measured on an open-arm P4_2 2_1 2
crystal (8v2t): adopted P2_1 2_1 2_1, promoted by the twin-immune zone, all
three axial rows "no control class", P 4 2 2 written.
Among the candidates the higher search cannot separate, take the first that
keeps every axial absence the adopted group claims (as the centring already
is taken), drop from the alternatives the ones that do not, and stop reporting
those rows as undetermined. Rows the adopted group left present are still in
the merge and still decided by the higher search.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Its deposited sweep lacks 40 deg (frames 561-720 are not in the
archive; the depositors processed it as two sweeps), so it tests
gap handling rather than processing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
SRS Daresbury PX10.1 (6RYM), EMBL Hamburg DORIS X13 (7BGU), NSLS X29A and X25
(8V4J, 8V2T), ESRF ID14-2 (5JK4), Australian Synchrotron MX2 (6CS9), CLSI 08B1-1
(7UDI), SSRF BL17UM (9LXL), MAX IV BioMAX (9S02), ALBA XALOC (6GVK) - marCCD,
ADSC (Quantum 4/210r/315), PILATUS and EIGER2 files. Each read and processed with
the current rugnux; 7BGU (C2 imposed on a P1 lattice) and 7UDI (P41 under-called
to P2) disagree with the deposition. Multi-sweep archives pinned to one sweep.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Pseudo-merohedral and merohedral twins, pseudo-symmetric cells and their
negative controls, chosen to test the space-group search's promotions:
- 8c3e (P3121, merohedral twin ~0.23, home-source HyPix CBF)
- 4bwl, 2wnq, 2xfw, 2wnn (P21 NAL crystals with the pseudo-merohedral
twin law -h,-k,h+l at fractions 0.49/0.46/0.10/0.33; ADSC SMV) and 2wnz,
the same crystal form deposited untwinned
- 6oww, 6p8j, 9qw2 (P21 with near-90 beta or a~c), 9qvv (I222, b=c),
5ojv (P21212, a~c), 7q6j (P212121, b~c)
Data staged in /data/scout_twin, linked from the open data root.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The XDS references of aspirin (20, 25 keV), citric acid, HEPES and YAG were made with
OVERLOAD=65534, half of the EIGER2's saturation value (133202). CORRECT ignored 13-313
reflections per set whose peak pixel lay between the two - the strongest, most
extinguished reflections - so the references described a data set without them.
Re-ran only the CORRECT step on the unchanged INTEGRATE.HKL with every input parameter
reconstructed from the old CORRECT.LP and OVERLOAD=133201 (saturation - 1, as the other
in-house sets already use); a rerun at 65534 reproduced the old CORRECT.LP. The old outputs
are kept in xds_overload_half/ beside each data set. `refs --write` picked up the new values.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
BraggIntegrationEngineGPU_MatchesCPU gains three footprint sections (spaced, crowded under overlap
exclude, with the radial background correction): both engines classify the summation ellipse, the
grown ring and the footprint Gaussian alike. Integration chapter and changelog describe the measured
footprint.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The integrator's r1 disk and r2..r3 background ring are fixed in pixels and chosen from spots near
the beam. On small-molecule data at 20-25 keV a spot's standard deviation grows from ~1 px near the
beam to ~5 px at the edge (radially from parallax/obliquity, tangentially from the crystal's
azimuthal spread), so the r1 = 4 disk holds a quarter of the flux there, the background ring a third
of it, and the in-disk second moments the Gaussian is built from saturate near r1^2/4. On top of
that, the profile/summation runaway guard sent 20-30% of these reflections - the strong, wide ones -
back to the truncated r1 box sum.
- SpotFootprint: every pre-scan spot (width frames) is measured with a window that follows it
(3 sigma, iterated, re-centred), radially and tangentially; the medians per distance-from-beam bin
become BraggIntegrationSettings::Footprint. Installed only where some bin outgrows r1, and on the
adaptive side like the radius (pre-pass without; the starvation guard falls back to the settings
without it).
- BraggStencil: where 3 sigma > r1 the background ring starts at 3 sigma along and across the radius,
the summation region is the r1 disk plus the 3-sigma footprint ellipse (so the guard's fallback is a
complete intensity), and the per-reflection Gaussian takes the footprint widths. Compact spots keep
the stencil bit for bit. Both engines build it from the same header.
SHELXL against COD (R1 / fixed-XDS-model R1(F)): citric acid .101/.230 -> .077/.055, HEPES
.070/.179 -> .048/.050, aspirin 20 keV .059/.070 -> .052/.061, aspirin 25 keV unchanged, L-cystine
25 keV unchanged (.145 -> .144).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The SHELX HKLF 4 file now has one record per full reflection - its partials summed,
the per-frame scale and every correction applied, sigma(I) as the merge weighted it -
at the index it was measured at, not averaged with its equivalents: the chemical
crystallographer's convention, so SHELXL computes Rint and Rsigma itself. Outliers
the merge rejected and fulls beyond its resolution cut are left out; no batch column
(it would select a BASF scale in SHELXL). The engine hands the fulls back only for a
merge that may be written (RotationScaleMerge::SetExportScaledFulls), so the search
merges, the pre-pass and the P1 cross-check carry no copy; the fulls follow the same
relabelling as the merged reflections (merge_to_written). Stills keep the merged
file. --mode scale writes the unmerged form too.
Validation: p.mtz md5 unchanged on myob/cytc/thau (GPU). SHELXL on the same runs,
merged-old vs unmerged-new (COD models, harness /data/tmp/sm_shared):
aspirin 20 keV Rint 0 -> 0.071, Rsigma 0.036 -> 0.043, R1 0.0964 -> 0.0969,
wR2 0.312 -> 0.310, GooF 1.53 -> 1.47; HEPES 20 keV Rint 0 -> 0.163, Rsigma 0.057
-> 0.068, R1 0.0903 -> 0.0897, wR2 0.318 -> 0.263, GooF 1.57 -> 1.10; the "input
data appear to be merged" warning is gone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
shelx_check.py refines the row's published structure (manifest key "cod", cached from the
Crystallography Open Database into the site's cod_cache) against rugnux's p.hkl with one fixed
recipe: data reindexed into the COD setting (lowest-R1 integer matrix), non-H anisotropic, H fixed,
EXTI, MERG 2, three rounds of SHELXL's suggested weights. It records R1/wR2/GooF/EXTI/WGHT/residual
density/R(int)/R(sigma)/K of the strongest bin and a fixed-model R1(F) (|Fc| of the COD model as
published, gemmi). Reported, never scored. The report gets a small-molecule table; compare lists
SHELXL R1/wR2/GooF/EXTI deltas; report/compare fill the check in for older runs.
COD entries matched by Niggli-reduced cell and space group: aspirin 7050897, citric acid 5000063,
HEPES 2224210, YAG 2003066, L-cystine 1513328 (2005, replaces the 1959 model for refinement),
cytidine 2001311, 3,5-dinitrobenzoic acid 4510615, L-alanine 2104782, metformin HCl 2108029,
NiCl2(dppe) 2012031. cuhf2 has no reference cell and no match: left without one.
SHELXL is called, not shipped (site key "shelxl" or PATH; it comes with CCP4).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The error model was fitted once, on deviations from a mean that weighs every full by its COUNTING
variance, and the outlier test's median took the same weights. Where equivalents disagree beyond
counting statistics, the low-count observations dominate that centre: on a strongly absorbing crystal
(YAG, Ia-3d, equivalents spread over a factor 100 after scaling) the mean of (0 4 0) sat at 21k among
observations from 11k to 1.9M, a ran into its bound (100), b came out at 800% internally, and the
six-sigma test about the biased median removed 63% of the observations - the strong ones.
Now, after the first fit, the model is refitted about the model-weighted mean (each full weighted by
the variance the fitted model gives it at the reflection's mean) until a and b settle, and the
rejection median takes the same weights. Where counting statistics are right nothing moves. Following
Blessing (1997) J. Appl. Cryst. 30, 421-426. GPU path: the refitted means are uploaded (SetEmMean).
Measured (SHELXL R1(>4sigma) against the COD model, sm-a's harness; rc174 scaling):
YAG 0.556 -> 0.127 (XDS 0.083 merged), rejected 8055 -> 53, normalised deviations calibrated
(median |z| 0.62-0.69 in every intensity decile); aspirin 20 keV 0.0964 -> 0.0958; citric acid
0.161 -> 0.159; HEPES 0.0903 -> 0.0899; L-cystine 25 keV 0.1425 -> 0.1456;
aspirin 25 keV 0.094 -> 0.106 (fixed-model R1 0.107 -> 0.173): its strong equivalents split into two
frame-dependent populations from the per-frame partial scaling (sm-a's dq-smallmol), which the old
under-sized sigmas happened to cut; with that scaling fixed (f69339ce6 + pooling, --no-scale-partials)
this change is neutral to better on every small molecule (aspirin 20 .0618 -> .0586, aspirin 25
.0456 -> .0454, citric .1093 -> .1006, HEPES .0703 -> .0697, YAG .649 -> .222; SHELXL GooF ~1.1).
=> ship together with the scaling fix.
Proteins (GPU full runs): CC1/2 and R_meas unchanged to 0.002; ISa myob 9.06 -> 8.20, thau 52.5 -> 47.5,
cytc 25.8 -> 25.4, lyso 29.4 -> 29.4 (still above XDS's 5.2 / 44.5 / 31.8 / 28.3 except cytc).
CPU build gives the same statistics as the GPU build on aspirin 20 keV and myob.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Replaces the prototype --no-scale-partials switch with a rule read off the data: a merge whose
sweep holds fewer than 50 rocking events per frame takes its per-frame scale from the fulls alone
(scale-fulls, a sparse frame fitted over its neighbours - PoolHalfWidth), and logs that it did.
The rocking-event walk already run for the smoothing window now also returns its event count.
Within one rocking curve a partial's scale and an error of the partiality model are the same
thing; on a sparse fine-sliced sweep the partial fit takes the one for the other and imprints an
hkl-dependent bias common to all equivalents. Measured populations: small-molecule sweeps 2.6-23
events per frame, protein sets 84-900; on the proteins the fulls-only scale leaves model R-free
unchanged (+-0.002 over the smoke tier's open-arm sets) but lowers ISa, so they keep the partial
scale. p.mtz md5 unchanged on the three profiling sets, GPU and CPU builds.
SHELXL against the COD structures, R1(>4sig) rc174 -> this: aspirin 20 keV 0.096 -> 0.062,
aspirin 25 keV 0.094 -> 0.046, citric acid 0.161 -> 0.109, HEPES 0.090 -> 0.070 (XDS 0.030-0.038);
SHELXL's weight a comes off its 0.2 cap on all four. CPU and GPU paths agree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The same source built with and without -march=x86-64-v3 gave different results on battery sets
(9zmu axis-harmonic supercell arbiter fired in one build only; 8rud resolution cut 1.69 vs 1.70 A;
myob_x06da_powder_1 cut 1.07 vs 1.42 A; 7mzt short-axis first pass; lcystine_x10sa_20keV indexed
vs no lattice), against the rule that no decision may depend on compiler flags.
Two mechanisms, found by building the merged tree four ways (baseline, x86-64-v2, x86-64-v3,
x86-64-v3 -mno-fma) with and without -ffp-contract=off and comparing p.mtz:
- FMA contraction. GCC contracts a*b+c whenever the target has FMA. With -ffp-contract=off the
x86-64-v3 build gives a p.mtz byte-identical to the baseline build on 8 of 9 sets (7mzt, 8rud,
9zmu, insu_I_x06da_5keV_2, myob_x06da_powder_1, myob_x10sa, cytc_x10sa, thau_x10sa_16keV), on
both the GPU and the CPU build. Cost: none measurable (user core-s, CPU build, x86-64-v3 vs the
same with -ffp-contract=off: 1877/1874, 3285/3253, 2212/2195 on myob/cytc/thau; GPU likewise
within noise). Set project-wide for C, C++ and CUDA host code; MSVC does not contract under
/fp:precise.
- SIMD width. lcystine still differed: baseline and x86-64-v2 (128-bit) agreed, x86-64-v3 with or
without FMA (256-bit) disagreed - Eigen's HouseholderQR in the FFT indexer's candidate refinement
(PostIndexingRefinement.cpp) reduces column norms over all spots in packets of the target width.
Replaced by the 3x3 normal equations summed in spot order in double. All four builds now agree on
all nine sets.
Changes results of the default x86-64-v3 build (contraction off); lcystine_x10sa_20keV now gives no
lattice in every build (its first pass is a knife-edge: 0/60 vs 9/60 validation frames before).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Prototype switch --no-scale-partials: skip the per-frame scaling of the partials and let the
fulls (scale-fulls) carry the per-frame scale, as XDS does. Without partial scaling a frame whose
fulls number fewer than MIN_REFLECTIONS is fitted over the nearest frames on either side that
together hold enough (PoolHalfWidth), on the host and in the GPU kernel. Default path unchanged
(p.mtz md5 identical on the three profiling sets).
Why: on fine-sliced small-molecule sweeps the per-frame partial scale and the partiality model are
degenerate within a rocking curve, and the fit swings G 0.23..1.2 with a 180 deg period (XDS's own
frame scale: 0.79..0.99). That imprints an hkl-dependent bias common to all equivalents, which
R_meas/CC1/2/ISa cannot see but a refinement against the known structure does. And scale-fulls
never fitted a frame on such data: a full is filed under one frame, about 8 per frame, below
MIN_REFLECTIONS, so every frame kept G = 1.
Measured with SHELXL refining the COD structures (R1 >4sig), default -> switch:
aspirin 20 keV 0.096 -> 0.062, aspirin 25 keV 0.094 -> 0.046, citric acid 0.161 -> 0.109,
HEPES 0.090 -> 0.070 (XDS 0.030-0.038). Proteins lose ISa with the switch (myob 9.1 -> 7.6,
cytc 25.8 -> 13.7), so it is not a default; the choice is to be made from the data.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The first version judged "the same lattice" by class and centring, so an
F-centred cubic re-index (57/60) and pass 1's triclinic primitive of the very
same lattice (60/60) counted as different, and the unconstrained primitive won
the frame count - 6oel lost its cubic setting (R_meas 36.6 -> 42.1 %). The
two are now compared on their Niggli-reduced primitive edges (within 2 %, the
battery's lattice-identity test), so a symmetric setting never loses to its own
primitive, while 9qw8's C-centred re-index (reduced edges 5.7 % off pass 1's)
is still judged against pass 1's lattice on the frames.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A promoted point group whose <|L|> reads below 0.375 (NOT_READABLE) previously
raised no TWINNING warning at all; a twinned subgroup whose law the promotion
absorbed predicts the same data, so the adopted group is not confirmed. Warn.
The twin-immune zone control reading more compressed than a perfect twin's
acentric population (0.541) is something no twin fraction produces (overlap or
neighbour correlation); genuine symmetry then reads acentric in its zones too
(measured: a genuine 622 with its control at 0.528 read its 2-folds at -950 to
-2640 nats). Such zones are now marked ambiguous in the text and the zone
decision line. Report-only: no decision and no output file but the report changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The non-reference settings of the adopted point group were offered only when the cell hosted their
rotations to within ~0.1 deg, while the reference settings - which hold the very same rotations -
were offered without asking. A cell refined free after integration a few tenths of a degree off 90
therefore lost every setting but the reference one. On an orthorhombic set whose measured screws
lie on a and c and whose b row was never recorded, that left P2(1)2(1)2(1) as the only candidate
covering both screws, and it was reported as determined. With P 21 2 21 offered, the two tie, the
b-axis screw is reported as undetermined, and the model check uses the setting the data describe.
The point-group stage still asks the cell whether a rotation set it adds is hosted; only the
setting enumeration within an already chosen set stops asking.
Validation: myob/cytc/thau x10sa p.mtz byte-identical (GPU); 7mzt fail -> unscored (b screw
undetermined, P 21 21 21 or P 21 2 21); 5cc8 unchanged; [SearchSpaceGroup] 23 cases pass incl. a
new section with a cell 0.2-0.3 deg off 90.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The space-group hero card (and the "indistinguishable" line) render the short
Hermann-Mauguin symbol as rich text: screw axes subscripted, rotoinversions
overlined (P2_1/c, P4_12_12, Fd-3m). It is built from gemmi's spaced symbol
with short_name()'s monoclinic shortening applied; checked against all 564
gemmi settings to reduce to short_name() once the markup is stripped.
The report's SUMMARY now also records its pathology rows as typed checks
(ReportDocument::checks): PRESENT exactly when one of the row's codes is in
PATHOLOGY_FLAGS, ABSENT when it was measured and did not fire, UNDETERMINED
when it could not be measured. A flag no row stands for gets a check named
after its code, so the list never shows less than PATHOLOGY_FLAGS. The text
report is unchanged. The viewer builds the document once, renders the text
from it and shows the checks phenix.xtriage-style: a green/red/grey light per
pathology, the summary words and fired warnings in the tooltip.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two knife-edges let a sub-0.05 px change of the measured beam centre turn a
P1 crystal (9qw8) from a pass into a merge with R_meas 60000% / ISa 0:
- The refined-geometry pass re-indexes de novo and fell back to pass 1's
lattice only when the re-index scored under the 1/6 validation floor. The
long-axis rescue lifted a related C-centred 303 A cell to 41/60 - over the
floor - while pass 1's lattice indexes 55/60 at the same geometry. Pass 1's
lattice is now scored as a hypothesis of its own whenever the re-index
found a DIFFERENT lattice (class + primitive volume within 2 %), and is
integrated when it indexes more validation frames. The same lattice found
again is kept as the re-index refined it, so ordinary crystals are
untouched.
- The two-pass quality guard's "going back to the header geometry" re-ran
pass 1 de novo, a hypothesis nobody had judged: at the centre where pass 1
indexed 56/60 it found 5/60, flipped the axis sign and shipped garbage. It
now forces pass 1's whole indexing result, which is what the guard preferred.
9qw8: P1 1.71 A on -march=x86-64-v3 and on no-march GPU builds (both failed
before, one catastrophically). myob/cytc/thau p.mtz md5 unchanged (GPU).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Battery: 20261003-1424_581e1c_perf-merge-full (+_private) vs rc174 built with the same flags:
no verdict change except the second scaling engine's GPU OOM on the three largest sets,
removed in 0f728ecab (8a1a/8qaw/8tyy pass again: 20261003-2032_581e1c_perf-oom-fix).
One-time result change from the GPU/CPU-parity beam-centre walk (b2).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The search merge was cut at the first thin shell whose mean <I/sigma> fell under 1. On a merge
whose <I/sigma> is flat near 1 - its ISa has collapsed, as on a strongly absorbing garnet - the first
single shell to dip is decided by noise: the same crystal cut at 1.19 A without -march and 0.85 A with
-march=x86-64-v3, and the coarser cut left each glide zone fewer than 20 absences, so Ia-3d became
unjudgeable and I4(1)32 was adopted. The cut now comes from a non-increasing (pool-adjacent-violators)
fit of the shell means, which moves only as much as its input does; that garnet's profile never falls
under 1, so its search sees the full range and adopts Ia-3d under both builds with identical
candidate tables. On monotone profiles nothing changes: myob/cytc/thau x10sa keep their cuts and
p.mtz byte for byte.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The all-observation arm of the space-group search and the P1 cross-check
were made on a second RotationScaleMerge engine beside the run's own
(871347b7a). That engine holds a second device copy of every observation,
and on the largest sets the two no longer fit a 16 GB card: 8a1a, 8qaw and
8tyy (55-125 M partial observations) stopped with an out-of-memory error
in scaling. Measured on a quiet box the engine bought 0.7 s (cytc) and
1.0 s (thau) of tail and nothing on myob, which does not justify a memory
budget, so it is removed and both merges run on rsm in sequence, as
before 871347b7a.
Kept from 871347b7a: the GPU scaling's own non-blocking stream, the
cross-check not writing per-frame G/CC/mosaicity back (the per-image table
still describes the merge that was written), the restored scaling
iteration counts, and the anisotropy analysis beside the other analyses.
p.mtz byte-identical to before on myob/cytc/thau, GPU and CPU builds;
p_plot.txt unchanged apart from the GPU bkg column.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
cpuinfo_max_freq is the highest boost state, not the base clock -
mislabeling it "nominal" understated the gap between the two on an AMD
box using acpi-cpufreq (this machine: 5.08 GHz boost vs 3.40 GHz base).
Prefer, in order: the model-name string, cpufreq's own base_frequency
(intel_pstate, kHz), ACPI CPPC's nominal_freq (MHz; what acpi-cpufreq
exposes instead), else "unknown". cpuinfo_max_freq is now reported
alongside, separately, as "max boost", never folded into "nominal".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Collect CPU model, nominal/base frequency (with its source), physical
core/thread/socket counts, RAM and GPU name(s)/memory at run time
(Linux via /proc, /sys/.../cpufreq, nvidia-smi; a short best-effort on
macOS via sysctl), store it under manifest.json's new "hardware" key,
and render it as a line in the report header. Runs from before this
change have no such key; the report prints "not recorded" for them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ApplyCellSurface (detector modulation, time x detector, crystal-frame SH and
goniometer-frame absorption) spends most of its time in two passes per round:
the per-group reference sums and the per-(block, cell) fit sums. With a GPU
both now run on the device (RotationScaleMergeGPU::Surface*) over the same
terms in the same order:
- reference: one thread per ASU group, walking a group-order permutation of
the terms in fulls order;
- fit: each subset cut into the host's reduction blocks with every block's
terms ordered by cell (stable counting sort, host); a per-term kernel forms
w*Is, w*Iref, Iref and one thread per (block, cell) runs the two fma chains;
the block slots are added per cell in block order.
Every rounding is spelled out (__dmul_rn/__dadd_rn/fma) to be the one the host
build makes: GCC at -march=x86-64-v3 fuses swI's multiply-add only in the
parity-filtered copy of the reference loop, and both fit sums.
Host side, exact on both paths: the 19 serial nth_element selections of the
shell edges become one parallel sort (same order statistics), the per-term shell
lookup runs on all threads, and the gate's per-shell CC is one walk over the
groups instead of one per shell.
Exact: p.mtz md5 identical to the oracle on myob/cytc/thau x10sa, GPU build
(all CUDA architectures) and CPU build. CorrectionSurfaceGPU test checks the
device sums bit for bit against an explicitly rounded host loop.
Measured (cytc/thau, two interleaved A/B pairs, box at load 13-25 from sibling
work): ApplyCellSurface host core-seconds -83% on cytc; SG adoption -> writing
reflections 4.58->3.75 and 3.85->2.54 s (cytc), 3.15->2.28 and 2.29->2.03 s
(thau); RSM final merge -0.7..-0.9 s and P1 cross-check -1.5..-1.8 s on cytc.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The hot-pixel step of the pre-scan (MaskDefectivePixels) on a GPU build:
- The device half is built once, before the workers start (HotPixelFinder::PrepareDevice),
instead of by the first worker's frame while the others waited. The unmasked pixels
grouped by key are sorted on the device (stable radix sort: the same order the host
fill gave) instead of scattered on the host and uploaded.
- No per-frame host round trip: the ring-sector levels and lit thresholds are made on
the device from the order statistics, and the per-key frame and level sums are kept
there too; frames queue on their workers' streams and the per-pixel accumulation is
ordered by an event instead of a host synchronisation.
- The mask: the chance rate's per-ring counts are summed on the device, and only the
pixels the tests can pass (error value on most frames, or lit on at least
min(max(2, k_chance), valid frames)) come back with their sums - not the five
per-pixel arrays (470 MB pageable on a 16 Mpx detector). The host tests run on them
unchanged.
The threshold is written as fma(nsigma, noise, level) + offset on the host - what GCC
already contracted it to - and the device takes the same two roundings, so the levels
are bit-identical (and no longer depend on whether a compiler contracts).
Exact: hot-pixel mask and p.mtz byte-identical to f849e2d1b on myoglobin, cytochrome C
and thaumatin, GPU and CPU builds. Hot-pixel step (GPU, box at load 18-23, interleaved
A/B, two pairs each): 0.94-1.32 s -> 0.65-0.81 s; frames 0.37-0.46 -> 0.18-0.23 s,
mask 0.21-0.34 -> 0.08-0.18 s. Device memory of the finder: 543 MB as before, plus a
~0.36 GB transient for the sort while it is built.
Tests: [HotPixelFinder] (HotPixelFinder_DeviceMatchesHost bit-exact), [ShadowFinder],
[BeamCenter].
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The pre-scan's spot measurement (spot width and bandwidth, powder rings,
spot-symmetry beam centre) ran the CPU preprocessor and adaptive spot finder
on every sampled frame, ~13-19 core-seconds per 16M-pixel run. In a GPU build
it now uses the same engines as the image loops: the frame is decoded and
preprocessed on the device (with the host decoder as fallback), the spots
are found and extracted there, and the preprocessed image comes back only
when the spot width needs it.
This is a choice of where, not of what: the device finders reproduce the
host's spot list exactly (integer ring sums, the same connected components
in the same order). Checked frame by frame with both engines side by side on
myob/cytc/thau x10sa and all 21 smoke-tier sets (~540k spots, HDF5, CBF,
marCCD, SMV, pink beam, 3.8 keV): identical spot lists in identical order on
every frame, and an identical preprocessed image on every width frame.
p.mtz md5 identical to the oracle on myob/cytc/thau, GPU and CPU builds.
Measured on a loaded box (load 20-30): the spot measurement takes 0.8-0.95 s
instead of 2.5-3.6 s and no longer competes for the cores the beam-stop mask
and the background beam-centre fit run on; the pre-scan falls from 3.0-4.9 s
to 1.7-2.4 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The two pre-scan steps that were still CPU-bound in a GPU build now run where the
projection already is.
- FindBeamCenterFromBackground: the per-iteration binning pass and the two clip rounds
run on the device (BeamCenterBackgroundGPU); the fit itself stays on the host. Each
cell is summed in the host's order (pixel order within the host's row blocks, blocks
in order), and the per-pixel cell/derivative formula is shared (BackgroundBand.h).
The angles come from BackgroundAtan2 (IEEE ops only) instead of atan2f, and both
translation units are compiled without FMA contraction, so host and device give the
same bits: 0 of 6.5 M pixels in a different cell, identical walks on the three
in-house rotation sets. With glibc/CUDA atan2f and default contraction ~30 pixels per
16 Mpx sweep changed cell and the fitted centre moved by up to 0.05 px.
- ShadowFinder::GetMask: the whole mask (pooling, ring medians, components, morphology,
hole fill, arm search) runs on the device from ShadowAccumulatorGPU's projection
(ShadowMaskGPU), so the 360 MB projection no longer comes back; the mean projection is
divided on the device too (same bits). The two small fits over rings and sectors
(BlockedOutTo, HarmonicFit) are shared with the host path in ShadowFinderInternal.h.
Integers, comparisons, sorts and components are exact; the polarization trig, the
Poisson log and the arm-search azimuth are not, so a pixel at a threshold can differ.
The one-time change against the previous CPU arithmetic (BackgroundAtan2, no
contraction), measured on the myoglobin, cytochrome C and thaumatin rotation sets:
ring centre moves 0.002-0.045 px (fit sigma 0.75-1.2 px), beam-centre capture
0.01-0.04 px; beam-stop mask differs on 31 / 144 / 53 pixels of 259k / 144k / 198k
(25 of the myoglobin ones are GPU-vs-CPU arithmetic in the mask, the rest follow the
centre); hot-pixel mask identical. Spot width, integration radii, bandwidth, beam-centre
arbitration, indexing, space group, cell, resolution and the merged statistics table
are identical; only the error model moves in its 4th digit. CPU build: the same
centres and decisions.
Timing (GPU, box at load 30-38): ring walk 0.54 -> 0.23-0.27 s, mask 1.24-1.44 ->
0.18-0.22 s, beam-centre capture walk 1.1-1.3 -> 0.31-0.35 s.
Tests: ShadowFinder_DeviceMaskMatchesHost, BeamCenterFromBackground_DeviceMatchesHost
(bit-exact), plus [ShadowFinder], [BeamCenter], [HotPixelFinder].
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The standalone GPU azimuthal integration did three shared-memory atomics per
pixel on the same few ring addresses, which is what it was limited by. It now
reads four pixels per thread as vector loads and keeps a running total per ring,
flushed when the ring changes - the scheme the adaptive finder's ring pass
(reduce_rings_shared) already uses. The npix % 4 leftovers are done one at a time.
Used wherever the fused adaptive engine is not (fixed-threshold spot finding, the
broker's non-adaptive path). Measured on a 16 Mpx sweep (1800 frames,
--no-adaptive-spots, RTX 5080): 843 -> 295 us per call (min 621 -> 196 us).
Not bit-identical, and the old kernel was not either: float atomics arrive in any
order, so two runs of the OLD kernel already differ by up to 1.7e-6 relative in
the per-frame profile; new vs old differs by up to 1.9e-6, the same order. Per-ring
pixel counts are identical. Default rugnux runs do not reach this kernel (p.mtz
md5 unchanged on three sets); on the fixed-threshold path p_unmerged.mtz is
md5-identical to the old kernel's. New test: GPU vs CPU engine on a pixel count
that is not a multiple of four, with masked and saturated pixels.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Exact: p.mtz and the pre-scan products (shadow mask, mean projection,
defective-pixel mask, ring and capture centres, compared as hashes and
hex floats) are bit-identical to rc174 on three in-house rotation sets,
GPU and CPU builds.
- ShadowFinder::GetMask: the serial parts run in parallel - connected
components by row band joined with union-find (both the shadow and
the transmitting-arm searches, and the hole fill), ring binning and
the harmonic sector gather by blocks, gap bridging by line; ring pixel
counts read off the ring offsets. Mean projection filled in parallel.
- ShadowFinder host accumulation: one band-locked projection instead of
a 20 B/px shard per pre-scan worker (2.7 GB zeroed and folded on a
16M detector); SetShardCount and the shard argument are gone.
- FindBeamCenterFromBackground: the usable-pixel test is made once, the
in-band pixels are kept in pixel order so the clipping rounds no
longer sweep the whole detector, the 67 MB cell map is gone and the
per-iteration block fold runs in parallel - same sums, same order.
- HotPixelFinder::GetMask: the chance-rate counts in parallel (integers).
Measured on a loaded box (load ~25 from other jobs), pre-scan window:
GPU 5.9-6.5 s -> 3.2-3.4 s, CPU 8.4-9.0 s -> 6.1-7.4 s. The GPU-build
pre-scan now ends with its background spot measurement (CPU spot finder
on ~120 frames, ~13 core-s on 8 workers).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The tail of the canonical pass made five merges one after another. Two of
them read nothing the space-group search or the in-symmetry merge decides:
the all-observation arm of the search and the P1 cross-check. They are now
made on a second RotationScaleMerge engine, ingested beside the run's own
before any merge writes per-frame values back, and taken where they were
made before; where the run re-ingests (a reindex, a cell change) they are
made on the run's engine as before.
- RotationScaleMerge::SetWriteBackPerFrameScale(false) keeps a merge that
is not the run's answer from writing G/CC/mosaicity onto the outcomes.
The P1 cross-check no longer overwrites them, so _plot.txt's scale_G,
cc_to_merge and cc_n now describe the merge that was written (in the
determined group) instead of the P1 cross-check; the cross-check also
no longer leaks its scaling iteration count into the report.
- RotationScaleMergeGPU runs on its own non-blocking stream instead of
the legacy NULL stream, so the two engines (and a probe pass beside
them) do not serialise at every launch and synchronisation.
- The anisotropy analysis (mostly ScaledObservations) runs beside tNCS,
twinning and the other report-only analyses.
p.mtz, p_P1.mtz, p.cif, p.hkl and p_unmerged.mtz are byte-identical on
myob/cytc/thau (GPU and CPU builds). Tail on cytc GPU 8.1 -> ~6.8-7.7 s
under a loaded box.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
AdaptiveSpotFinderCPU::AccumulateRingsBlock reads each pixel once for both the
ring histogram and the fused azimuthal profile (was two loops), and no longer
keeps the per-ring integer sums: they are taken from the histogram, as the two
sigma-clip passes already were (ClipRings(INFINITY)). Integer sums, so the same
totals; the profile's float sums keep their pixel order.
FlagRow is branch-free and works a 32-pixel word at a time, so it vectorises;
pixels outside every ring meet a +inf threshold in an extra ring_thr entry.
BraggPredictionRot::Calc takes A*h, A*h + B*k, C*l and 4*S0*S0 out of the inner
loops; p0 is the same ((A*h) + (B*k)) + (C*l) as before.
Measured on cytc (CPU build, both binaries run concurrently on a loaded box):
AccumulateRingsBlock -35%, FlagRow -47%, Calc + Coord ops -30% cycles; whole run
-5% cycles. p.mtz md5 unchanged on myob/cytc/thau, CPU and GPU builds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The crystal refinement (XtalOptimizer, both the seven-block and the reduced
beam+orientation form, and XtalOptimizerRotationOnly) no longer builds a
ceres::Problem. XtalRefine holds the problem as data and solves it with
LMSolver, which follows Ceres' trust-region LM step for step - Jacobi scaling,
damping and radius updates, stopping rules, box projection, the projected
Armijo line search with cubic interpolation on bounded problems, the
SphereManifold for the spindle - but takes J^T J and J^T r directly instead of
a Jacobian. The residual is the same XtalResidual code, now Ceres-free and
evaluated on a forward-mode Dual (Dual.h); everything that depends on
parameters alone (detector-angle trig, per-frame back-rotation, reciprocal
basis, orientation rotation) is worked out once per evaluation, and the
observed and predicted halves carry 6 and 9 derivative lanes rather than 16.
The sums are cut into blocks that depend on the residual count alone, so the
answer does not depend on the thread count. Because the line-search trial point
is the candidate point, a bounded iteration costs one evaluation instead of
Ceres' three.
Validation (rc174 + this, -march=x86-64-v3):
- p.mtz md5 identical to the Ceres build on myob/cytc/thau x10sa, GPU and CPU
builds, and on the lyso8 stills reference.
- Solve corpus (every 16-parameter solve and every 10th per-image solve of the
three sets, 8.3k problems, inputs and Ceres results dumped from a run that
reproduced the md5s): usable/failed agree on all, iteration counts identical
on all, parameters agree to <2e-11 (in px / rad / 0.01 A units), costs to
1e-13.
- Same process, same threads: 7-9x faster per solve than Ceres.
- In-run (GPU, loaded box): xtal 16-parameter solves myob 22.2 -> 6.4 core-s,
cytc 88 -> 26 core-s; per-image solves 5.4 -> 1.2 core-s (myob); cytc first
pass indexing windows 1.9 -> 0.85 s, myob 1.1 -> 0.45 s; solver share of the
whole cytc run 17% -> 4% of CPU samples.
New tests compare the solver with Ceres on synthetic rotation problems
(full/weighted/reduced) and check thread-count independence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
lcystine_x10sa_20keV and lcystine_x10sa_25keV are scored against the
literature structure only (hexagonal L-cystine, P 61 2 2, COD 5000005,
Oughton & Harrison 1959): XDS indexes under 10% of the spots of either
sweep, with the cell supplied and with the spot range cut to 150 frames,
so there is no CORRECT.LP to take a reference from. On the current build
the 25 keV sweep passes (P 61 2 2, cell within 0.5%) and the 20 keV sweep
finds no lattice.
EXTERNAL_TEST_DATA.md: none of the open-arm datasets is pink beam; every
PDB record is single-wavelength (9Q41 used a multilayer monochromator).
The pink-beam sets are the in-house ones.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every open and in-house manifest row now carries `wavelength` (A), taken
from WAVELENGTH in rugnux's own p_report.txt of the latest run of that set
(the full rc173 run 20260929-2003 plus the two 10-02 runs for the sets
added since). Descriptive only; the scorer ignores it.
docs/BATTERY_REPORT.md is report.md of the full rc173 open + in-house run
(210 sets), verbatim, to be polished. The private arm runs separately and
has its own report, so nothing from it appears here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Open arm, all IRRMC, single Cu K-alpha sweeps on Rigaku Saturn CCDs read
through the d*TREK SMV path: 3mc4 (H 3), 3meb (P 1 21 1, Saturn 944,
overflow ratio 32), 3p85 (P 63 2 2), 3r6o (I 41), 5uth (P 31 2 1), 5vml
(P 42 21 2), 6cee and 6v2r (P 21 21 21 / P 41 21 2, Saturn A200, unbinned
2048x2048, 0.1 mm pixels). Detector 2theta spans -10 to +10 deg. Run
20261002-1326_84228b_home-source-12: all pass except 3r6o, called I 4 2 2
against the deposited I 41 with twinning suspected (<|L|> 0.32). Two
further candidates (3r6h, 3sgw) were dropped: what the repository lists as
one dataset is two sweeps under one file template, split by a phi change,
which the reader refuses.
In-house arm: aspirin (20 and 25 keV), anhydrous citric acid, HEPES and YAG,
measured at X10SA. The reference is XDS's CORRECT.LP (refs --write) with
the space group taken from the literature as ref_override, because XDS
reports only Sohncke groups; each override cites its COD entry and paper.
Run 20261002-1354_84228b_small-molecules-inhouse: four pass, including
HEPES as P b c a where XDS has P 21 21 21; YAG is called I 41 3 2 against
I a -3 d (both programs merge it poorly, rugnux CC1/2 0.40, XDS ISa 3.2).
EXTERNAL_TEST_DATA.md gains the eight depositions (DOIs checked on DataCite)
and updated counts; the battery README lists the small-molecule standards
and has the current count of symlinked open-arm directories (42).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The SMV reader only knew the ADSC vocabulary, so a Rigaku Saturn frame was
recognised as SMV and then refused for want of PIXEL_SIZE. A header naming
DETECTOR_NAMES is now read as d*TREK:
- pixel size and the point of normal incidence from <det>SPATIAL_DISTORTION_INFO;
- image directions from <det>DETECTOR_VECTORS combined by
<det>SPATIAL_DISTORTION_VECTORS, matched to a DetectorOrientation (a Saturn
944+ image is mirrored and turned a quarter, so the hand is preserved);
- distance and the detector circles (2theta etc.) from <det>GONIO_*, composed
into PONI angles; spindle from ROTATION_VECTOR, angles from ROTATION;
- wavelength from SCAN_WAVELENGTH, or the second number of SOURCE_WAVELENGTH
(the first is a count - the old NumAny fallback would have read 1.0 A);
- pixels above 32767 decoded as (v - 32768) * RAXIS_COMPRESSION_RATIO.
dxtbx's FormatSMVRigakuSaturnNoTS, which claims headers without
DTREK_DATE_TIME, ignores the distortion vectors; on a Saturn 944+ sweep that
puts the spindle 90 degrees off (DIALS: 12% indexed vs 98% with the vectors
applied, at the deposited cell). The arm geometry is confirmed by the data:
the refined direct beam of a 2theta = 10 deg sweep lands at x = 621.1 against
621.6 predicted. The anomalous map peaks at 11.5 sigma on the two K+ and
6-7 sigma on S and P, so the hand is right.
Battery, open arm: 5cc8 (rotating anode, Saturn 944+ SMV, 2theta = 10 deg
sweep) and 9jq9 (Ga K-alpha liquid-metal jet, PILATUS3 1M miniCBF, single
450 deg sweep), both IRRMC. Run 20261001-2132_84228b_dtrek-home-source:
9jq9 passes (P 21 21 21, 1.65 A); 5cc8 merges to 1.53 A at the deposited
cell but is called P 21 21 21 against the deposited P 21 21 2 (screw
evidence on 00l 68-78 nats, tNCS at (1/2,1/2,0.11)) - left failing, not
yet adjudicated. The Saturn gain (~5 ADU/photon) is not in the header and
is not modelled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
