CPU_DATA_ANALYSIS_IMAGE §1.4 now follows the decision flow in Rugnux.cpp:
header provenance lines, the background measurement on every run (capture
+ walk, what the check lines mean), the second first pass and its five
outcomes, the merge-judged arms (metric-symmetry, lean-depth and the
same-lattice inner-shell CC1/2 < 0.9 retry), the search after a failed
pass, the post-refinement bound, --estimate-beam-center's acceptance rule,
stills, report keys and options. Removes the stale "where both work and
disagree, neither is chosen" and the "centroid of the radial background
profile" fall-through.
RUGNUX_ADVANCED: --beam-center-check lists every adoption route and that it
still runs with --beam-x/--beam-y; --beam-center-search re-finds spots per
rung (the table said spot finding was not repeated); --estimate-beam-center
states its acceptance rule. OVERVIEW and TUTORIAL brought in line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The root README described only data acquisition. The install page now
lists the pre-scan and the correction surfaces among what runs on the GPU,
and says that -march changes speed and not results (floating-point
contraction is off project-wide).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Both DOIs were resolved against doi.org/Crossref (title, volume and pages
match), so the "DOI not verified" notes go. The analysis reference's
References list gains Wilson / Howells-Phillips-Rogers and Baur & Kassner,
whose methods 13.1 now describes (CLAUDE.md: a method described there is
listed there as well as in ACKNOWLEDGEMENT.md).
Every other DOI in ACKNOWLEDGEMENT.md, the CPU_DATA_ANALYSIS* pages, the
RUGNUX* pages, README.md and the battery README resolves.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Space group (13.1): every setting of a chosen rotation set is offered;
screws are claimed at 8 nats, with how that bound was set; the P1 search
merge is cut where a monotone fit of <I/sigma> drops under 1; the
operator-homogeneity gate reads the R gap between a subgroup and the rest;
a lattice-holohedral group is refused when merging under it narrows the
L-test like a twin.
Outliers (13.3): the rotation median is weighted at the expected
intensity, and the band's systematic part is multiplicative, with its
spread measured on the merge's own fulls.
Indexing: a sparse lattice is integrated on every frame; pass 1's lattice
is a hypothesis against a different re-index, and a pass sent back reuses
pass 1's indexing. Image: the flipped axis sign must beat chance; the
same-lattice beam-centre retry on an inconsistent merge.
Integration: five correction surfaces, not four; the .hkl carries no
amplitudes; the fulls' scale is smoothed differently from the partials'.
Tutorial: the correction surfaces are cross-validated on the half-set
correlation (as the reference says), not on an R_meas-like deviation.
Overview: the footprint, the 5 A indexing hypothesis, the centre of
symmetry, and the GPU pre-scan.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The first non-search merge on an ingest was made both ways (partials scaled, and
the scale from the fulls alone) and PartialScalingWins picked one. When the prior
is to scale the partials and that merge resolves its error model, the decision is
the prior whatever the other arm says, so the fulls-alone merge was a whole final
merge thrown away - 16% of all scaling time over the battery (238 of 258
decisions), e.g. ~1.5 s of a 22 s GPU run on an in-house cytochrome C set, 8-13 s on
the largest open-arm sets.
That arm is now made only when the partials' merge fails to resolve its error
model (then the old sequence runs unchanged) or when the prior is the fulls
(small molecules; unchanged). The partials' arm starts from g_partial = 1, the
only state the fulls-alone arm left for the arm after it (corr is restored,
g_full reset by the combine, device g/scaled cleared per call, the sweep /
rejection / damage diagnostics reset where they are measured), so the merge is
bit-identical to the one the pair returned.
The log's decision line no longer quotes the fulls-alone ISa when that merge is
not made.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
- OBSERVATIONS_REJECTED / _WILSON / _OVERLOAD: what each counts and how
they relate (the overload count is not part of OBSERVATIONS_REJECTED).
- CENTRE_STATISTICS_REFLECTIONS and the three per-statistic _F keys are
named; NOT_APPLICABLE is stated as the code decides it (any of the three
outside -0.3..1.3, or the control below 0.5).
- BUILD_CXX_FLAGS no longer says that -march moves the results: the build
turns floating-point contraction off and the indexer's reductions no
longer depend on SIMD width, so CPU levels agree. The RUGNUX_GIT and
BUILD_CXX_FLAGS keys were also listed twice in the header paragraph.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Running Rugnux gains a "Small-molecule data" section: what the default run
does about short axes, wide and split spots, sparse patterns, overloads and
glide / centre-of-symmetry space groups, the two settings worth checking
(polarization on a laboratory source, the resolution cut), and what the
unmerged HKLF 4 file holds. Rugnux with other programs gains a worked
SHELXT -> SHELXL route (instruction file built from the report keys, what
SHELXL does with the unmerged file). The SHELXT/SHELXL commands were run on
a battery set's output to check they work as written.
The quick start's file list now includes <prefix>_detector.jpg, which every
default rotation run writes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
What Rugnux reads now describes the d*TREK vocabulary the SMV reader
learned in this release (pixel size, image directions, 2theta arm,
wavelength, encoded overflows), and no longer says that SMV never states a
saturation value: CCD_IMAGE_SATURATION / SATURATED_VALUE are read where
present.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
With the static Qt of the Linux viewer, QJpegPlugin brings Qt's bundled libjpeg-turbo
(3.1.4, JPEG_LIB_VERSION 80) next to ours from preview/ (3.2.0, ABI 62). Both export the
same jpeg_* symbols and the linker silently kept Qt's, so WriteJPEGToMem - compiled against
our headers - failed with "Wrong JPEG library version". In the viewer that aborted an
in-process rugnux run at the end, where _detector.jpg is collected. The rc173 artifact shows
it directly: jpeg_CreateCompress checks for 80 and the only libjpeg version string is 3.1.4.
The viewer now excludes Qt::QJpegPlugin and the unused Qt::QTiffPlugin (which bound to
whichever libtiff the link offered), and its clipboard copy and "Save image as JPEG" encode
through WriteJPEGToMem instead of QImage::save. The Linux viewer CI job checks the binary
holds no Qt JPEG/TIFF handler and exactly one libjpeg-turbo.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
6iu9 (deposited P3_1, merohedrally twinned at 0.4 or more by its 321 law: H 0.097, CC 0.79) went
from P3_1 to P3_121 when the penalised per-frame scale smoother (495f6a97b) landed; the ingest
partiality fix alone (7d6c201e9) keeps P3_1. Bisected on the two binaries: smoother-only 6iu9 P3_121,
partiality-only P3_1. The smoother only removed the luck: the P3_1 answer rested on the
Lorentz-filtered arm's added-operator R contrast reading 0.71 against a bound of 0.72 (the
all-observation arm already passed 32 at 0.82); better scaling moved it to 0.87, no gate refused, and
the twin-immune zone - which reads -401 nats, acentric - is only consulted where a gate fired.
A near-perfect twin passes every agreement gate by construction (H ratio 0.97 here); only the zone
can refuse it. It was confined to refusals because a genuine trigonal crystal read -46 nats with no
gate firing; on the battery 5uth (genuine P3_121, twinned by a 622 law, other-law CC 0.14) reads
-206. The reason: the centric density was untwinned, but a twin by a law OTHER than the zone's
operators reaches the zone as it reaches every reflection, so a genuine zone reads 0.81 at a = 0.2.
The operators' own twin law cannot reach their zone, so only the centric side needs it.
- TwinningAnalysis: the zone's centric density is the twinned one at a caller-given fraction
(weighted sum of two chi^2_1, via exp(-y) I0(y)); the control's calibration expectation is taken
at the same fraction (-KL(acentric || centric_a), -0.130 at a = 0 as before). At a = 0 nothing
changes.
- SearchSpaceGroup: twin_fraction_outside = the fraction implied by the strongest CC of a lattice
rotation outside the group, relative to the group's own mean CC, through rho = 2a(1-a)/((1-a)^2+a^2);
exported for the adopted group and used by Rugnux's TwinZoneVerdict and the zone report.
- New Stage A test: a candidate that does not hold every lattice rotation, on a P1 merge whose <|L|>
is in the partial-twin band [0.375, 0.44), is refused when the zone over one of its index-2
subgroups reads acentric by 20 nats (the TwinZoneVerdict bound), unless zones_ambiguous. The band's
lower end is the L-test gate's: below it something else compresses the zones too (a pseudo-cubic
small-molecule set, cuhf2, read its 422 zone at 0.64 beside a control at 0.58 and was turned to
P222 without it).
Zone evidence (calibrated, at the other-law fraction): 6iu9 32 -230 / -143 (refused, P3_1);
5uth +209 / +33 (P3_121 kept); 8xtg +324 / +65; 6vww P6 +20 / +8.5; 7k1l P6 +8 / +7.5.
Battery (open+inhouse, targeted, against 20261004-2350 all2-full; only beyond-noise change listed):
- 6iu9: fail -> pass, P3_121 -> P3_1, R_meas 17.4% -> 14.4%, ISa 4.6 -> 5.4, R-free 0.318 -> 0.327.
- unchanged: 6iu5, 6iu6, 6iu8 (P3_1), 5uth, 5j23, insu_H_x06da_twin/notwin, 6vww, 7k1l, 8xte, 8xtg,
9i80, 4bwl, 2wnq, 8c3e, 2wnn, 2xfw, 5ebi, 6p8j, 6z9g, 6rlr, 6toc, 3mc4; 321/312 controls 5lzl,
6w4h, 9gqg, 6pxb, 9z72; all 14 small-molecule sets (cuhf2 checked again after the band).
- private subset (8 sets): no change beyond noise.
Tests: [twinning] (new: zones read at the other twin laws' fraction), SearchSpaceGroup*.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The gate that refuses a point group whose own operators "disagree among
themselves" read worst-agreeing over best-agreeing operator R. That divides
by an extreme order statistic - the pole the contrast gate was introduced to
remove - so a genuine group whose operators merely spread in a continuum is
refused as soon as one operator happens to agree unusually well.
That is what refused 432 on a private cubic F432 crystal after the overload
drop (37a8c8e24; bisected: that commit alone gives F23, b126b9fc1 does not
change the search). The crystal's intensity-weighted R is dominated by a few
dozen strong low-order reflections. Dropping the overloaded ones (whose
flank-extrapolated values disagreed ~2x with their mates) raised every
operator CC and made the cleanest 432 operator cleaner still, while
anisotropic systematic error keeps the genuine 23 operators spread ~2x
among themselves: worst-over-best went 2.68 -> 4.0 against the 3.2 bound.
H ratio (1.25) and contrast (0.78) passed throughout.
A mixture of real and false operators is specific: the real ones form a
subgroup (the true group's intersection with the candidate) and everything
outside it is false. The statistic is now the largest factor by which every
operator outside some subgroup reads worse than every operator inside it -
a gap, found by walking the operators sorted by R and reading each prefix
that closes into a group. It is never above worst-over-best, so nothing
admitted before can be refused; the bound stays 3.2. Read off the operator
tables of the all2 full battery and run 3: genuine adopted groups reach at
most 2.5 (the cubic crystal 1.8-2.0), the 5ebi 622 mixture reads 3.6
(was 6.0), 9hnc's false 422 3.4, and 2wnn's false 222 3.8 (unchanged, still
refused by this gate).
Targeted runs vs all2-full (f64f76): the private cubic set F23 -> F432
(pass); every other set identical to the last digit - five other private sets; open 5epe 8sqt 3inp 9fhc 6oel 7orr
8tyy 3ky7 6h5t 6moj 2wnn 5ebi 9hnc; lyso_x06da_ref, lyso_x06da_5keV,
myob_x06da_split; all small-molecule sets (citric R1 .0374, aspirin
.0363/.0358, hepes .0310, yag .1021, kdp, lcystine, cytidine, dnba,
lalanine, metformin, nidppe, cuhf2); and the high-spread trigonal/
tetragonal/hexagonal panel (insu_H twin/notwin, insu_I weak/low_isa, 6v2r,
8xtg, 5j23, 6iu6, 6iu8, 3mc4, 6vww, 7k1l, 9i80, 6pxb, 9hs7).
SearchSpaceGroup* and [SearchSpaceGroup] test cases pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Open set 5ebi (deposited P2_1, a = c and beta = 120: a monoclinic crystal on a hexagonal metric,
pseudo-merohedrally twinned, <|L|> 0.433) regressed from P2_1 to C222_1 with dq-sm2-rest baf2017c9.
Of that commit's two changes it is the sub-lattice ask's R-contrast operator test, not the hot-patch
mask: 5ebi is a CCD (.img) sweep, so HotPixelFinder never runs on it. Without the R contrast every
metric two-fold that passes min_operator_cc was kept, LePage closed them into the whole hexagonal
metric and no sub-lattice ask was made. With it the false hexagonal two-folds drop out, the two
in-plane two-folds of the C-centred orthohexagonal cell survive (CC 0.97, R 0.061 against 0.045 for
the genuine screw: contrast 0.96), and the ask on that sub-lattice adopted C222_1. The main search
had refused 622 on the added-operator R (contrast 0.70 against 0.72), but the ask's own search
cannot see that: the gates there are relative and the twin law passes them on its own. Stage A's
L-test-under-merge gate does not reach it either - C222_1 is not holohedral on a hexagonal metric -
and read per operator it does not separate it: merging under the in-plane two-fold moves <|L|> only
0.425 -> 0.418 (14% of the gap to 0.375).
Fix: where a twin gate of the search in hand refused a higher group, an ask that would promote
re-opens what that gate answered, so it is arbitrated as every other promotion over a refusal is -
by TwinZoneVerdict on the operators it adds over the group in hand (that group carried into the
ask's setting). Acentric with margin and the group in hand stands. 5ebi: zone 222 over 2,
n 1865, calibrated -23.9 nats -> twin law, P 1 21 1 kept. cuhf2 (the case baf2017c9 fixed) has no
twin-gate refusal in hand, so its zone is not read and P422 stands.
Targeted battery vs the rc174-all2 full run (20261004-2350_f64f76_all2-full), same base:
reg4-v1 (32 sets: 5ebi, cuhf2, sub-lattice sets 6iu6 6iu8 6iu9 6z8o 8t7r 9ea5, twinned/pseudo-
merohedral 9hnc 3meb 2wnn 2xfw 2wnz 4bwl 2wnq insu_H_x06da_twin, controls lyso_x06da_ref
thau_x10sa_0p1deg 5reo, all 14 small-molecule sets): only 5ebi changes - C2221 -> P1211 (pass),
ISa 10.9 -> 14.8, R_meas 12.3% -> 10.2%, R_free (deposited model) 0.571 -> 0.537, cell dev 40% ->
0.05%. Every other set identical in space group, ISa, R_meas, R_free, d_min and SHELXL R1.
reg4-v1b (the 12 other open/in-house sets whose run adopts a metric or sub-lattice ask): identical.
Private arm (the 7 sets whose mask changed with baf2017c9 and the 4 that take an ask): identical.
Still failing as before: 4bwl, 2wnq (near-perfect twins, C222_1 via the metric ask with no refusal
in hand), 6iu9, 9hnc, kdp.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
ANOMALOUS_SCATTERER_MEAN_SIGMA= is the anomalous difference map, read at every
model atom from phosphorus up (S of Met/Cys, metals, Cl, I) with the same cubic
interpolation as ANOMALOUS_SITE_*, averaged in map sigma; ANOMALOUS_SCATTERERS=
is their count. One number for the anomalous signal of a merge that does not
depend on which ten atoms come out on top - what the battery tracks for the
long-wavelength standard-protein sets. A new key only: no report version bump.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Non-Sohncke candidates were enumerated only when their PROPER rotations equalled the measured
point group's, which only centrosymmetric groups satisfy. Groups without a centre (Pc, Pna2_1,
I-42d, I4_1md, Fdd2, P-42_1c, I-43d) were never candidates: where a centrosymmetric group shares
their absences rugnux wrote it silently (Pc -> P2/c), and where none does the dead glide zone was
dropped and a Sohncke subgroup written (KDP-type I-42d data -> I4_122).
- Enumerate non-Sohncke groups by Laue class. The Sohncke-signature dedup, CellHostsRotations and
the glide-evidence bar are unchanged. Groups with identical absences in the same Laue class are
scored once and carried as `same_absences`; a selected candidate brings them into
`alternatives`.
- Convention for which twin is written: the centrosymmetric one where it exists (missed centres are
the common error, Baur & Kassner 1992), otherwise the lowest-numbered (I4_1md over I-42d).
- The centrosymmetric default gives way only when, on the general reflections of the Laue class of
the final merge, <|L|>, <|E^2-1|> and N(0.1), each calibrated against acentric/centric intensities
simulated with the reflections' own sigmas (fixed-seed mt19937_64, own deviate transforms), read
acentric (L f <= 0.3, others <= 0.5), every f lies in [-0.3, 1.3], the always-centric control reads
centric (f >= 0.5), and the lattice excludes twinning (gemmi Le Page metric admits no rotation
beyond the Laue class, no TWIN_DOMAIN leftover lattice). Then the group is switched and re-merged.
- Report: SPACE_GROUP_CENTRE (IMPLIED_BY_ABSENCES / ABSENT_BY_ABSENCES / NOT_DETERMINED /
ABSENT_BY_STATISTICS), CENTRE_TWINNING_EXCLUDED, CENTRE_STATISTICS_* on every searched run, and
prose for the absence-equivalent alternatives.
Battery (targeted): all 14 small-molecule sets vs 58a4bf (smt-all, kdp-all): every group
unchanged except kdp_x10sa_20keV I 41 2 2 -> I 41 m d with I -4 2 d as the alternative; dnba stays
C 1 2/c 1, now NOT_DETERMINED with C 1 c 1 listed (statistics read centric, f 0.96-1.24). The
P2_1/c, Pbca and Ia-3d sets read IMPLIED_BY_ABSENCES with no alternative.
Protein panel (19 sets, twinned panel + controls, run glide-prot): every space group equals the
rc174-all base (3r6o re-run with the base binary: I 41 2 2 both); no SPACE_GROUP_CENTRE on any.
Twinned proteins read the centre statistics NOT_APPLICABLE (L f -0.3 to -1.7) or ACENTRIC.
Private subset (8 sets): 8/8 groups unchanged vs 58a4bf.
The glide-zone evidence scan over battery 3 (every non-Sohncke group of each run's Laue class,
centrosymmetric or not) peaks at 0.69 nats/reflection on proteins against the 2.0 bar.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The per-frame scale was fitted on the partials from 50 rocking events per
frame and taken from the fulls alone below. The counts of small-molecule
sweeps (3-43) and proteins (5-900) overlap, and a weak protein at 36
events per frame scaled from its fulls had no resolved error model at all
(ISa undetermined, d_min 5.10 A) where its partials gave ISa 32 and 4.88 A.
The first merge of a pass that is not a space-group search is now made
both ways, and the count stands as the prior unless its merge has no
resolved error model while the other merge has one. Search merges (P1 /
subgroups) take the prior.
The two ISa values are deliberately not compared beyond that. Tried as
the arbiter, "higher ISa wins" (and weighted R_meas as fallback) agreed
with the external yardsticks on 10 of 11 crystals but chose the partials
on a 6-events-per-frame small-molecule sweep: ISa 10.5 vs 8.7, weighted
R_meas 0.102 vs 0.115, SHELXL R1 0.105 vs 0.062 - the partiality error
the partial scale absorbs is shared by symmetry mates at the same
rocking geometry, so their agreement cannot see it. Statistics of the
difference between the two arms' frame scales did not separate that
sweep from the proteins that want the partials either.
Effect: only crystals whose prior arm fails change; every small-molecule
set and every protein control keeps its arm. Private weak protein: ISa
undetermined -> 32.3, d_min 5.10 -> 4.88 A, weighted R_meas .343 -> .337.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A saturated pixel in a spot means the brightest part of the reflection was
not measured. The integration used to drop the peak frame's partial (its
peak pixel is unreadable) and keep the flanks, so the combine extrapolated
the event from its tails by the partiality model: on a strongly
diffracting small-molecule crystal the strongest low-order reflections
read 2-3x low and were the largest SHELXL misfits. XDS drops such a
reflection (OVERLOAD); so does rugnux now.
- Integration (CPU + GPU engines): a reflection is `overloaded` when a
signal-disk pixel is saturated, or unreadable on this frame but not in
the run's pixel mask - EIGER/PILATUS write their error value for a
pixel they could not count, which the preprocessor turns into a masked
pixel like a gap's. The engines now receive the PixelMask to tell the
two apart (an earlier attempt that re-classified the marker as
saturation in the preprocessor broke a dataset whose gaps are not in
the file's mask). An overloaded reflection is kept with its box sum,
unfitted, only so its event can be recognised.
- Rotation combine (CPU + GPU): an event with any overloaded partial is
dropped whole; counted in the log and the report
(OBSERVATIONS_REJECTED_OVERLOAD=). The unmerged MTZ export drops it too.
- Everything else that reads reflections leaves an overloaded one out:
AcceptReflection (stills merge, per-image scaling), the post-refinement
gather, the axial-row sums.
- Capture uncertainty: the merge rebuilds each full's variance at the
reflection's mean (counting_variance / ModelSigma) and dropped the
capture term the combine had put into sigma, so a full extrapolated
from part of its rocking curve merged at the weight of a whole one.
Fulls now carry it (Obs::capture) and the rebuilt variance adds
(capture * <I>)^2, host and device.
SHELXL R1 on rugnux's own integration (harness), median fix -> this:
citric acid .0648 -> .0420 (XDS .051; 221 events dropped, EXTI 1.02 -> 0.29),
HEPES .0396 -> .0381 (184), aspirin 20 keV .0387 -> .0385 (6),
aspirin 25 keV .0376 -> .0375 (5); metformin/nidppe/dnba/lalanine/cytidine
no overloads, unchanged. YAG .116 -> .128 (87 dropped; its scale loop does
not settle either way). Proteins and private subset: see the branch report.
Tests: BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped (new),
BraggIntegrationEngineGPU_MatchesCPU (overloaded flag compared),
AcceptReflection_ResolutionLimits, [write_reflections], [large].
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The open-arm L-alanine set's COD reference (2104782) is a 0.2 GPa room-temperature
diamond-anvil-cell structure (372 reflections, 43% complete), not the ambient one the
docs called it. The set's own cell (5.787 5.940 12.263) is the 100 K ambient form, so
the reference is now COD 2311261 (100 K, 5.791 5.944 12.269; Parsons, Flack & Wagner,
Acta Cryst. B69 (2013) 249-259). SHELXL's refined R1 does not depend on the starting
model (0.0546 vs 0.0524), but the fixed-model R1(F) the battery reports goes from 0.168
to 0.087, so that number now measures the data rather than the pressure difference;
cell_dev_pct likewise compares against the right cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Merged dq-sm-absorb, whose ingest fix (corr_ingested carries the recomputed partiality, not the
predictor's) repairs the same defect as this branch's unit partial scale on the path without
partial scaling. The two did not double-apply - the unit scale rebuilt corr from scratch - but
they differ by the incident flux, and measured head to head the flux costs the sparse sweeps:
SHELXL R1 on four in-house organic sweeps 0.0459/0.0396/0.0709/0.0413 with it, 0.0421/0.0393/
0.0697/0.0405 without; on five open small-molecule sets equal or better without (0.0707 ->
0.0660 on one). Only the cubic absorbing sweep prefers it (0.119 vs 0.139): there the
background really does fall with the absorbed beam. So the unit partial scale and its device
kernel are removed and the ingest fix stands alone; the flux meter's smoothing, which only
mattered on that path, is reverted, so the partial-scaling path sees the flux exactly as
before. The penalised per-frame scale of the fulls stays.
Also restores two characters of CPU_DATA_ANALYSIS_INTEGRATION.md that the previous commit's
rewrite had turned from a stray carriage return into a line break.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The French-Wilson prior of each reflection is now epsilon * K_shell * a(h), with
a(h) = exp(-1/2 s^T B s) from the deviatoric tensor AnalyzeAnisotropy already fits
(the form it is fitted in) and K_shell = sum(I/eps) / sum(a), so a shell's priors still
average to its measured mean. The amplitudes are made isotropically at the merge as
before and made again once the tensor exists (full pipeline and --mode scale). Only
F/SIGF and F(+)/F(-) change; IMEAN/I(+)/I(-) are bit-identical. Applied whenever a
tensor was fitted, with no detection gate: a near-isotropic tensor gives a(h) ~ 1 and
the isotropic prior back, and the prior wants the best estimate of <I> along h whatever
its cause. Follows ctruncate's anisotropic prior (Ballard & Stein, CCP4); credit in
ACKNOWLEDGEMENT.md, CPU_DATA_ANALYSIS.md and at the algorithm.
--model scaling (ModelScaling.cpp) was checked: k_overall + symmetry-constrained
anisotropic B + flat bulk solvent fitted on the working set, against the same FW F
written to the MTZ - as REFMAC/phenix.refine do. No change needed.
Evidence (REFMAC 10-cycle restrained refinement of the deposited model, R-free on
the depositor's free reflections shared by both data sets; base = rc174 processing,
same IMEAN):
set base new d set base new d
9rcs 0.3475 0.3494 +0.0019 8qq7 0.4452 0.4503 +0.0051
9yzk 0.3192 0.3171 -0.0021 9hs7 0.2898 0.2465 -0.0433
6yqf 0.4642 0.4543 -0.0099 5nw5 0.3256 0.3206 -0.0050
7n2s 0.3126 0.2998 -0.0128 6z8o 0.2927 0.2892 -0.0035
6qaj 0.3390 0.3038 -0.0352 6moj 0.2756 0.2651 -0.0105
6r72 0.3826 0.3797 -0.0029 7qij 0.3239 0.3140 -0.0099
anisotropic sets: median -0.0075, mean -0.0107, 10/12 better
isotropic controls: 5reo -0.0014, 7kcn +0.0003, 6fid +0.0002, 11if 0.0000
rugnux's own --model R-free moves the same way (median about -0.019; 5nw5 +0.006),
R_model shell-scaled too; dep_cc_delta unchanged (intensity based). The adoption rule
(median gain >= 0.005 on the anisotropic sets, no control worse than +0.002) is met.
The two sets that lose are the one with a FLAT resolution signature (8qq7) and 9rcs,
where the exp form drives the dead direction's prior to ~0 beyond 3.7 A.
For scale: ctruncate's own anisotropic prior on the same merges moved the same
REFMAC R-free by a median of only -0.0008 (9hs7 +0.026).
Inhouse lyso_x06da_ref, thau_x10sa_0p1deg: every battery metric unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Three defects in the per-frame scaling of sparse (small-molecule, weak) rotation sweeps:
1. The fulls' per-frame scale pooled sparse frames by their RAW full count, but in a
high-symmetry group with many systematic absences most usable counts stayed under the
minimum, so most frames were never fitted and kept corr = 1 beside pinned, fitted frames
(two gauges in one reference). The release step then read the gauge offset (a constant
134x on a cubic Ia-3d small-molecule sweep) as signal and gave each frame exp(kept_f * 4.9) - the e^-5
errors on a quarter of the frames, R1 0.62. A fixed box window also cannot follow a
100x absorption ramp over a few degrees, and frames under the credible floor, exempt
from the window, ran away to 1e-7.
Now each round fits every frame on its own fulls (no pooling, no minimum), and the scale
is a penalised second-difference smoother of log G (Whittaker/Eilers), each frame at
the information of its fit, lambda by cross-validation over blocks one rocking curve
wide (interleaved single frames leak through shared rocking curves and chose to follow
every frame). After convergence the existing ShrinkToRestrained hands back the per-frame
deviation its neighbour shares. Pooling and the box window are gone from the fulls loop;
the partials loop is unchanged.
2. With partial scaling off (< 50 rocking events per frame) the partials kept the
integration-time corr: no incident-flux correction and not the partiality of the
ingest-smoothed geometry, because only the partial scaling loop rewrote corr. They now
get corr = prescaling_corr / partiality at G = 1 (host, and a device kernel).
3. The flux meter (per-frame mean background) jumped 30x between neighbouring frames of a
sparse sweep - on a few reflections it measures which reflections the frame holds. It
is read through the same smoother at the precision of each frame's mean.
SHELXL R1(>4sig) against the published structures, rc174-cand -> this, on the in-house
small-molecule sweeps: cubic Ia-3d 0.615 -> 0.119 (XDS 0.088); four organic sweeps
(monoclinic / orthorhombic) 0.0515 -> 0.0459, 0.0451 -> 0.0396, 0.0747 -> 0.0709,
0.0452 -> 0.0413; ISa up to 11.6 -> 31. Raw flux instead of smoothed costs 0.002-0.003 R1 on
the first two. A weak, decaying protein sweep on the fulls-only path: ISa 6.7 -> 27.1.
CPU and GPU paths agree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Its deposited sweep lacks 40 deg (frames 561-720 are not in the
archive; the depositors processed it as two sweeps), so it tests
gap handling rather than processing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
SRS Daresbury PX10.1 (6RYM), EMBL Hamburg DORIS X13 (7BGU), NSLS X29A and X25
(8V4J, 8V2T), ESRF ID14-2 (5JK4), Australian Synchrotron MX2 (6CS9), CLSI 08B1-1
(7UDI), SSRF BL17UM (9LXL), MAX IV BioMAX (9S02), ALBA XALOC (6GVK) - marCCD,
ADSC (Quantum 4/210r/315), PILATUS and EIGER2 files. Each read and processed with
the current rugnux; 7BGU (C2 imposed on a P1 lattice) and 7UDI (P41 under-called
to P2) disagree with the deposition. Multi-sweep archives pinned to one sweep.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
BraggIntegrationEngineGPU_MatchesCPU gains three footprint sections (spaced, crowded under overlap
exclude, with the radial background correction): both engines classify the summation ellipse, the
grown ring and the footprint Gaussian alike. Integration chapter and changelog describe the measured
footprint.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The SHELX HKLF 4 file now has one record per full reflection - its partials summed,
the per-frame scale and every correction applied, sigma(I) as the merge weighted it -
at the index it was measured at, not averaged with its equivalents: the chemical
crystallographer's convention, so SHELXL computes Rint and Rsigma itself. Outliers
the merge rejected and fulls beyond its resolution cut are left out; no batch column
(it would select a BASF scale in SHELXL). The engine hands the fulls back only for a
merge that may be written (RotationScaleMerge::SetExportScaledFulls), so the search
merges, the pre-pass and the P1 cross-check carry no copy; the fulls follow the same
relabelling as the merged reflections (merge_to_written). Stills keep the merged
file. --mode scale writes the unmerged form too.
Validation: p.mtz md5 unchanged on myob/cytc/thau (GPU). SHELXL on the same runs,
merged-old vs unmerged-new (COD models, harness /data/tmp/sm_shared):
aspirin 20 keV Rint 0 -> 0.071, Rsigma 0.036 -> 0.043, R1 0.0964 -> 0.0969,
wR2 0.312 -> 0.310, GooF 1.53 -> 1.47; HEPES 20 keV Rint 0 -> 0.163, Rsigma 0.057
-> 0.068, R1 0.0903 -> 0.0897, wR2 0.318 -> 0.263, GooF 1.57 -> 1.10; the "input
data appear to be merged" warning is gone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
