ANOMALOUS_SCATTERER_MEAN_SIGMA= is the anomalous difference map, read at every
model atom from phosphorus up (S of Met/Cys, metals, Cl, I) with the same cubic
interpolation as ANOMALOUS_SITE_*, averaged in map sigma; ANOMALOUS_SCATTERERS=
is their count. One number for the anomalous signal of a merge that does not
depend on which ten atoms come out on top - what the battery tracks for the
long-wavelength standard-protein sets. A new key only: no report version bump.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Non-Sohncke candidates were enumerated only when their PROPER rotations equalled the measured
point group's, which only centrosymmetric groups satisfy. Groups without a centre (Pc, Pna2_1,
I-42d, I4_1md, Fdd2, P-42_1c, I-43d) were never candidates: where a centrosymmetric group shares
their absences rugnux wrote it silently (Pc -> P2/c), and where none does the dead glide zone was
dropped and a Sohncke subgroup written (KDP-type I-42d data -> I4_122).
- Enumerate non-Sohncke groups by Laue class. The Sohncke-signature dedup, CellHostsRotations and
the glide-evidence bar are unchanged. Groups with identical absences in the same Laue class are
scored once and carried as `same_absences`; a selected candidate brings them into
`alternatives`.
- Convention for which twin is written: the centrosymmetric one where it exists (missed centres are
the common error, Baur & Kassner 1992), otherwise the lowest-numbered (I4_1md over I-42d).
- The centrosymmetric default gives way only when, on the general reflections of the Laue class of
the final merge, <|L|>, <|E^2-1|> and N(0.1), each calibrated against acentric/centric intensities
simulated with the reflections' own sigmas (fixed-seed mt19937_64, own deviate transforms), read
acentric (L f <= 0.3, others <= 0.5), every f lies in [-0.3, 1.3], the always-centric control reads
centric (f >= 0.5), and the lattice excludes twinning (gemmi Le Page metric admits no rotation
beyond the Laue class, no TWIN_DOMAIN leftover lattice). Then the group is switched and re-merged.
- Report: SPACE_GROUP_CENTRE (IMPLIED_BY_ABSENCES / ABSENT_BY_ABSENCES / NOT_DETERMINED /
ABSENT_BY_STATISTICS), CENTRE_TWINNING_EXCLUDED, CENTRE_STATISTICS_* on every searched run, and
prose for the absence-equivalent alternatives.
Battery (targeted): all 14 small-molecule sets vs 58a4bf (smt-all, kdp-all): every group
unchanged except kdp_x10sa_20keV I 41 2 2 -> I 41 m d with I -4 2 d as the alternative; dnba stays
C 1 2/c 1, now NOT_DETERMINED with C 1 c 1 listed (statistics read centric, f 0.96-1.24). The
P2_1/c, Pbca and Ia-3d sets read IMPLIED_BY_ABSENCES with no alternative.
Protein panel (19 sets, twinned panel + controls, run glide-prot): every space group equals the
rc174-all base (3r6o re-run with the base binary: I 41 2 2 both); no SPACE_GROUP_CENTRE on any.
Twinned proteins read the centre statistics NOT_APPLICABLE (L f -0.3 to -1.7) or ACENTRIC.
Private subset (8 sets): 8/8 groups unchanged vs 58a4bf.
The glide-zone evidence scan over battery 3 (every non-Sohncke group of each run's Laue class,
centrosymmetric or not) peaks at 0.69 nats/reflection on proteins against the 2.0 bar.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A crystal of slightly misaligned domains (a ferroelastic domain twin below a phase transition, a
split crystal) records each reflection as two or more compact spots around the averaged lattice's
prediction, moving apart with resolution. The pre-scan footprint is measured about each spot, so it
saw compact spots; the r1 disk held the gap between them and the background ring sat on them.
The geometry pre-pass now compares every indexed spot with the predicted position of its own
reflection on the same frame and adds the mean square offset (radial and tangential, by distance
from the beam) to the pre-scan widths; the canonical pass integrates with that table. Where spots sit
on their predictions this moves the widths by the prediction error alone (lysozyme: 0.2-1.0 px, no
reflection outgrows r1); on a 100 K KDP domain twin the offsets reach 8-16 px at the edge.
KDP (kdp_x10sa_20keV), battery SHELXL recipe on the COD model: R1 0.272 -> 0.048, wR2 0.685 ->
0.129, EXTI 26.9 -> 0.025, GooF 3.2 -> 1.29 (XDS: 0.112 / 0.333 / 0.054 / 1.25); R_meas 21.6% -> 5.3%.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The multiplicative outlier band (c8de5f8d6) took its width from the error
model's b. b is a Gaussian width in I and grows with whatever tail the fulls
have: on a low-ISa hexagonal small-molecule set with a population of fulls
lost to near zero, b = 0.54 against a core ln-spread of 0.25, and exp(6 b) = 25
let the high outliers of the weak reflections through (SHELXL R1 0.191 ->
0.333 there, 0.111 -> 0.124 on its 25 keV sweep).
The spread is now measured on the merge's own fulls: the weighted median of
|ln(I / median)|, weighted by (<I>/sigma_counting)^2 so the fulls whose ratio
counting noise does not blur carry it; b remains the fallback where nothing
can be measured.
Battery (29 sets: 13 small-molecule, 16 protein), SHELXL R1 against the
median fix alone: the strongly absorbing cubic set 0.116 -> 0.0998 (b-band
0.103); the low-ISa hexagonal set 0.191 -> 0.174 at 20 keV, 0.111 -> 0.101 at
25 keV; organics within +-0.0003 or better (cytidine 0.0617 -> 0.0613,
lalanine 0.0541 -> 0.0533). Proteins unchanged except two low-ISa sets whose
CC1/2 cutoff moves: 6yqf 3.32 -> 2.89 A (placement R-free at a fixed 3.32 A
0.4329 -> 0.4326, at 2.89 A 0.4642 -> 0.4630), and the split myoglobin set
1.77 -> 1.94 A (low-resolution R_meas 33.0% -> 27.3%). Private subset: two
sets' R_meas lower, the rest unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The 6-sigma outlier band was symmetric in I about the reflection's median,
while the error model's systematic term b*<I> is a multiplicative error
(absorption, illuminated volume, a scale that is off), as likely a factor 1/f
as f. With b of a few percent the two agree; with a large b the linear band
reaches far below the median and only a little above it. On a strongly
absorbing cubic crystal (internal b = 0.34) 96% of the 828 rejected fulls read
2-7x their median in well-illuminated frames - the least absorbed, nearest the
true intensity - and almost none read low.
The b part of the band is now exp(+-n*b) about the expected intensity, the
counting part stays linear (OutlierBand.h, one formula for the host loop and
the device kernel; the two exp constants are taken once on the host). To
first order in n*b it is the old band.
That set's SHELXL R1 0.116 -> 0.103 (--reject-outliers 12 / 20 on the old
band: 0.107 / 0.101; no rejection at all: 0.31).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The per-frame scale was fitted on the partials from 50 rocking events per
frame and taken from the fulls alone below. The counts of small-molecule
sweeps (3-43) and proteins (5-900) overlap, and a weak protein at 36
events per frame scaled from its fulls had no resolved error model at all
(ISa undetermined, d_min 5.10 A) where its partials gave ISa 32 and 4.88 A.
The first merge of a pass that is not a space-group search is now made
both ways, and the count stands as the prior unless its merge has no
resolved error model while the other merge has one. Search merges (P1 /
subgroups) take the prior.
The two ISa values are deliberately not compared beyond that. Tried as
the arbiter, "higher ISa wins" (and weighted R_meas as fallback) agreed
with the external yardsticks on 10 of 11 crystals but chose the partials
on a 6-events-per-frame small-molecule sweep: ISa 10.5 vs 8.7, weighted
R_meas 0.102 vs 0.115, SHELXL R1 0.105 vs 0.062 - the partiality error
the partial scale absorbs is shared by symmetry mates at the same
rocking geometry, so their agreement cannot see it. Statistics of the
difference between the two arms' frame scales did not separate that
sweep from the proteins that want the partials either.
Effect: only crystals whose prior arm fails change; every small-molecule
set and every protein control keeps its arm. Private weak protein: ISa
undetermined -> 32.3, d_min 5.10 -> 4.88 A, weighted R_meas .343 -> .337.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A saturated pixel in a spot means the brightest part of the reflection was
not measured. The integration used to drop the peak frame's partial (its
peak pixel is unreadable) and keep the flanks, so the combine extrapolated
the event from its tails by the partiality model: on a strongly
diffracting small-molecule crystal the strongest low-order reflections
read 2-3x low and were the largest SHELXL misfits. XDS drops such a
reflection (OVERLOAD); so does rugnux now.
- Integration (CPU + GPU engines): a reflection is `overloaded` when a
signal-disk pixel is saturated, or unreadable on this frame but not in
the run's pixel mask - EIGER/PILATUS write their error value for a
pixel they could not count, which the preprocessor turns into a masked
pixel like a gap's. The engines now receive the PixelMask to tell the
two apart (an earlier attempt that re-classified the marker as
saturation in the preprocessor broke a dataset whose gaps are not in
the file's mask). An overloaded reflection is kept with its box sum,
unfitted, only so its event can be recognised.
- Rotation combine (CPU + GPU): an event with any overloaded partial is
dropped whole; counted in the log and the report
(OBSERVATIONS_REJECTED_OVERLOAD=). The unmerged MTZ export drops it too.
- Everything else that reads reflections leaves an overloaded one out:
AcceptReflection (stills merge, per-image scaling), the post-refinement
gather, the axial-row sums.
- Capture uncertainty: the merge rebuilds each full's variance at the
reflection's mean (counting_variance / ModelSigma) and dropped the
capture term the combine had put into sigma, so a full extrapolated
from part of its rocking curve merged at the weight of a whole one.
Fulls now carry it (Obs::capture) and the rebuilt variance adds
(capture * <I>)^2, host and device.
SHELXL R1 on rugnux's own integration (harness), median fix -> this:
citric acid .0648 -> .0420 (XDS .051; 221 events dropped, EXTI 1.02 -> 0.29),
HEPES .0396 -> .0381 (184), aspirin 20 keV .0387 -> .0385 (6),
aspirin 25 keV .0376 -> .0375 (5); metformin/nidppe/dnba/lalanine/cytidine
no overloads, unchanged. YAG .116 -> .128 (87 dropped; its scale loop does
not settle either way). Proteins and private subset: see the branch report.
Tests: BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped (new),
BraggIntegrationEngineGPU_MatchesCPU (overloaded flag compared),
AcceptReflection_ResolutionLimits, [write_reflections], [large].
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Two defects that together turned a tetragonal small-molecule crystal (open-arm cuhf2, published
P4/nmm) into P222.
1. HotPixelFinder masked a persistent strong pixel only when it stood alone or in a pair ("a
larger patch is a feature of the scattering"). On cuhf2 a ring of ~20 pixels reads 1e5-3e5
counts on EVERY frame (dead centre) - stationary in the lab, so no reflection of the rotating
crystal. Unmasked, the reflections crossing it ((2,6,+-9)) merged to 3.5e6 / 6.7e5 against 4e4
for the strongest real reflection. The final merge's outlier test removes them (12
equivalents), but the space-group search's P1-like merge has no equivalents to judge by, and
the h<->k operator read CC 0.18 / R 0.28 instead of ~0.99 / 0.013. The persistence and strength
tests are unchanged; only the isolation requirement is dropped.
2. The sub-lattice ask kept every metric two-fold that passes min_operator_cc. On a pseudo-cubic
cell two false cubic two-folds still correlate at 0.31-0.33, LePage closes them with the
genuine ones into the full cubic metric, and the tetragonal class in between is never asked.
An operator now also has to pass the R contrast every promotion's added operators are held to
(min_operator_r_contrast against random_pairing_r / global_best_operator_r, read only where the
search reads it): the false ones sit at 0.2, the genuine ones at 1.0.
cuhf2: P222 -> P422, R_meas 2.9% -> 2.6%, ISa 45.5 -> 52.3 (overnight-cint had found P422 by the
sub-lattice path; rc174-all lost it). Prescan survey, masked pixels base -> new: unchanged on
lyso_x06da_ref/5keV/atten_wedge, thau, insu, cytc, myob_split, 9qw8, aspirin20, HEPES, YAG,
metformin, nidppe; cuhf2 3 -> 42, 5reo 2 -> 6, lcystine25 0 -> 9 (two clusters of noisy pixels at
3-7 counts/frame on a zero background). Targeted battery (sm2rest-fix vs rc174all-ctl / smt-all):
the 15 protein / sub-lattice sets (lyso x3, thau, insu, cytc, myob_split, 5reo, 9qw8, 6iu6, 6iu8,
6iu9, 6z8o, 8t7r, 9ea5) identical to the 4th digit; SM sets identical except cuhf2 (above) and
lcystine25, SHELXL R1 .112 -> .151: the nine masked pixels flip the fulls-only per-frame scale
smoother on that polycrystalline sweep from "settled after 29 iterations" to "stopped settling"
(the base run reproduces bit-identically) - a sensitivity of that smoother, reported to the
scaling work, not of the mask.
Test: HotPixelFinder_PersistentPixelNotBragg gains a 3x3 patch that must be masked. The
device-vs-host test now waits for each queued frame before overwriting the shared device buffer;
it remains flaky at ~4/25 on the base binary as well (a borderline pixel or two differ), which is
pre-existing and not addressed here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The per-reflection median the 6-sigma outlier test is centred on was weighted
by each observation's own counting variance. That variance falls with the
observation's own intensity, so the median went to whichever equivalents came
out low. On a strongly absorbing cubic crystal, where frames near the edge-on
orientation read some equivalents 10-100x down by their diffracted direction,
the median sat on the absorbed ones and the test removed ~10% of the
observations, all of them correct measurements 3-20x above it.
The median is now weighted by the counting variance at the reflection's
expected intensity - the rule the merge weights already use - which keeps the
down-weighting of frames scaled up by 1/G and drops the dependence on the
observation's own fluctuation. In-house garnet (Ia-3d) SHELXL R1 0.139 -> 0.116
(--mode scale on the same integration).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
A spot that outgrows the r1 disk is summed over its measured footprint ellipse and its background
ring starts beyond it. That reach was the same 3 sigma that decides whether the spot outgrew the
disk at all. Wide spots are not Gaussian - mosaic streaks and diffuse halos carry flux past 3 sigma -
so the ring started on the spot's own tails and read them as background.
The install test stays at 3 sigma (BRAGG_FOOTPRINT_NSIGMA); the new BRAGG_FOOTPRINT_REACH = 4 sets
how far the summation ellipse, the ring start and the profile grid go. Compact protein spots never
install the footprint, so they are unchanged bit for bit.
Evidence (rugnux's own combined fulls put through XDS's own per-observation corrections, so only
the integration differs; SHELXL R1(>4sig) on ~93% of observations matched to XDS by hkl):
citric acid: 3 sigma .0521, 4 sigma .0519, 5 sigma .0526 (XDS 3D summation .0484)
HEPES: 3 sigma .0348, 4 sigma .0338, 5 sigma .0344 (XDS .0311)
Full pipeline SHELXL R1: citric .0697 -> .0685, HEPES .0405 -> .0370.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The open-arm L-alanine set's COD reference (2104782) is a 0.2 GPa room-temperature
diamond-anvil-cell structure (372 reflections, 43% complete), not the ambient one the
docs called it. The set's own cell (5.787 5.940 12.263) is the 100 K ambient form, so
the reference is now COD 2311261 (100 K, 5.791 5.944 12.269; Parsons, Flack & Wagner,
Acta Cryst. B69 (2013) 249-259). SHELXL's refined R1 does not depend on the starting
model (0.0546 vs 0.0524), but the fixed-model R1(F) the battery reports goes from 0.168
to 0.087, so that number now measures the data rather than the pressure difference;
cell_dev_pct likewise compares against the right cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Merged dq-sm-absorb, whose ingest fix (corr_ingested carries the recomputed partiality, not the
predictor's) repairs the same defect as this branch's unit partial scale on the path without
partial scaling. The two did not double-apply - the unit scale rebuilt corr from scratch - but
they differ by the incident flux, and measured head to head the flux costs the sparse sweeps:
SHELXL R1 on four in-house organic sweeps 0.0459/0.0396/0.0709/0.0413 with it, 0.0421/0.0393/
0.0697/0.0405 without; on five open small-molecule sets equal or better without (0.0707 ->
0.0660 on one). Only the cubic absorbing sweep prefers it (0.119 vs 0.139): there the
background really does fall with the absorbed beam. So the unit partial scale and its device
kernel are removed and the ingest fix stands alone; the flux meter's smoothing, which only
mattered on that path, is reverted, so the partial-scaling path sees the flux exactly as
before. The penalised per-frame scale of the fulls stays.
Also restores two characters of CPU_DATA_ANALYSIS_INTEGRATION.md that the previous commit's
rewrite had turned from a stray carriage return into a line break.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Stage A's gates all compare the operators' disagreement with a reference - a parent group, the best
operator, the noise floor - and a pseudo-symmetric structure or a twin defeats every one of them: the
added operators correlate nearly as well as real ones. On the open arm that over-called 2wnz and 2xfw
(deposited P2_1, pseudo-merohedral C-orthorhombic metric) as C222_1, and 7bgu (deposited P1 with
b = c to 2.7%) as C2.
New gate, for a candidate that holds every rotation of its lattice (gemmi find_lattice_symmetry at
3 deg obliquity, with the lattice centring now passed in SearchSpaceGroupOptions::lattice_centring):
no twin law exists for such a group, so intensities merged under it that read like a twin's are not a
twin. AnalyzeLTestUnderMerge reads the Padilla-Yeates L-test twice on the same pairs of the P1 merge -
as measured, and with every intensity replaced by its plain orbit mean under the candidate - so the
narrowing is the merge's own doing and not a property of the data. Refused when the merged <|L|>
reads twin-like (0.375 <= <|L|> < 0.44) and the averaging closed a third or more of the gap from the
unmerged <|L|> to 0.375.
Calibration on the C++ search merges (targeted battery sympg-pg1): genuine lattice-holohedral groups
close 0.01-0.25 of the gap (highest 6z9g, indexed on half its deposited cell, 0.23-0.25; 5vml 0.16-
0.17; 7kcn 0.06-0.09); the over-calls 2wnz 0.47, 2xfw 0.46-0.47, 7bgu 0.79. A merge reading below
0.375 is not read - nothing a twin or a false operator does reaches it, so something else compresses
the intensities: 8c3e (0.353) and the genuine 9zmu (0.351, 494 A axis) look alike there. Near-
perfect twins (4bwl 0.14-0.16, 2wnq 0.13-0.20) are out of reach by construction, and 6p8j (0.19-0.30,
deposited as a twin of P2_1) is left alone. Cost: 170 ms per test on a 730k-reflection merge, only for
candidates that pass every other gate and hold the lattice's full symmetry.
Targeted battery sympg-final (44 open/in-house sets: the over-calls and a control panel of genuine
lattice-holohedral, twinned and twin-like sets) and the private arm: 2wnz C2221 -> P1211 (R_free with
the deposited model 0.227), 2xfw C2221 -> P1211 (0.215), 7bgu C121 -> P1 (R_meas 0.28 -> 0.14), all
the deposited groups. Every other set keeps its space group; against the rc174-cand runs of the same
sets, open and private, nothing else moves beyond run time. Still over-called: 8c3e, 4bwl, 2wnq,
6p8j, 3r6o (reasons above; 3r6o and 8c3e read below 0.375).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Every partial's corr is formed at integration as image_scale_corr = LP*QE*flight / partiality with
the predictor's per-frame partiality. Ingest then recomputes every partiality from the
frame-order-smoothed mosaicity (and the one exact-Bragg angle per rocking event), but corr_ingested
kept the predictor's 1/p. Where partials are scaled every fitted frame's corr is rebuilt from the
recomputed partiality, so that path only saw it in the first reference. Where they are not - fewer
than 50 rocking events per frame: small molecules, and long-wavelength protein sweeps - corr is what
the 3D combine divides each partial by, and the full became a weighted mean of I/p_pred while the
event's recomputed fractions summed to 1.00: the combined fulls tracked 1/sum(p_pred) (r = -0.96)
and scattered 13% rms about the plain sum of the same partials.
corr_ingested is now rescaled by p_pred/p_recomputed once at ingest.
Targeted battery against rc174-cand: every set on the partial-scaling path is bit-identical
(5reo, 9qw8, lyso_x06da_ref, thau_x10sa_0p1deg, lyso_x06da_atten_wedge, myob_x06da_split,
cytc_x06da_1, insu_I_x06da_5keV/6keV). Small molecules, SHELXL R1 against the COD model:
aspirin 20 keV 0.0515 -> 0.0432 (ISa 10.3 -> 30.9), aspirin 25 keV 0.0451 -> 0.0396, HEPES
0.0452 -> 0.0420, citric acid 0.0747 -> 0.0726, dnba 0.0472 -> 0.0273, metformin 0.0405 -> 0.0326,
nidppe 0.0498 -> 0.0426, lalanine 0.081 -> 0.057, cytidine 0.077 -> 0.071. Long-wavelength
proteins on the fulls-alone path: lyso_x06da_5keV ISa 17.9 -> 27.5, thau_bl1a_3p8keV 21.7 -> 28.4,
thau_bl1a_4p6keV 26.4 -> 33.5. One regression: lcystine_x10sa_25keV (a polycrystalline aggregate)
now refuses 622 and reports P31 instead of P6122.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The French-Wilson prior of each reflection is now epsilon * K_shell * a(h), with
a(h) = exp(-1/2 s^T B s) from the deviatoric tensor AnalyzeAnisotropy already fits
(the form it is fitted in) and K_shell = sum(I/eps) / sum(a), so a shell's priors still
average to its measured mean. The amplitudes are made isotropically at the merge as
before and made again once the tensor exists (full pipeline and --mode scale). Only
F/SIGF and F(+)/F(-) change; IMEAN/I(+)/I(-) are bit-identical. Applied whenever a
tensor was fitted, with no detection gate: a near-isotropic tensor gives a(h) ~ 1 and
the isotropic prior back, and the prior wants the best estimate of <I> along h whatever
its cause. Follows ctruncate's anisotropic prior (Ballard & Stein, CCP4); credit in
ACKNOWLEDGEMENT.md, CPU_DATA_ANALYSIS.md and at the algorithm.
--model scaling (ModelScaling.cpp) was checked: k_overall + symmetry-constrained
anisotropic B + flat bulk solvent fitted on the working set, against the same FW F
written to the MTZ - as REFMAC/phenix.refine do. No change needed.
Evidence (REFMAC 10-cycle restrained refinement of the deposited model, R-free on
the depositor's free reflections shared by both data sets; base = rc174 processing,
same IMEAN):
set base new d set base new d
9rcs 0.3475 0.3494 +0.0019 8qq7 0.4452 0.4503 +0.0051
9yzk 0.3192 0.3171 -0.0021 9hs7 0.2898 0.2465 -0.0433
6yqf 0.4642 0.4543 -0.0099 5nw5 0.3256 0.3206 -0.0050
7n2s 0.3126 0.2998 -0.0128 6z8o 0.2927 0.2892 -0.0035
6qaj 0.3390 0.3038 -0.0352 6moj 0.2756 0.2651 -0.0105
6r72 0.3826 0.3797 -0.0029 7qij 0.3239 0.3140 -0.0099
anisotropic sets: median -0.0075, mean -0.0107, 10/12 better
isotropic controls: 5reo -0.0014, 7kcn +0.0003, 6fid +0.0002, 11if 0.0000
rugnux's own --model R-free moves the same way (median about -0.019; 5nw5 +0.006),
R_model shell-scaled too; dep_cc_delta unchanged (intensity based). The adoption rule
(median gain >= 0.005 on the anisotropic sets, no control worse than +0.002) is met.
The two sets that lose are the one with a FLAT resolution signature (8qq7) and 9rcs,
where the exp form drives the dead direction's prior to ~0 beyond 3.7 A.
For scale: ctruncate's own anisotropic prior on the same merges moved the same
REFMAC R-free by a median of only -0.0008 (9hs7 +0.026).
Inhouse lyso_x06da_ref, thau_x10sa_0p1deg: every battery metric unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Three defects in the per-frame scaling of sparse (small-molecule, weak) rotation sweeps:
1. The fulls' per-frame scale pooled sparse frames by their RAW full count, but in a
high-symmetry group with many systematic absences most usable counts stayed under the
minimum, so most frames were never fitted and kept corr = 1 beside pinned, fitted frames
(two gauges in one reference). The release step then read the gauge offset (a constant
134x on a cubic Ia-3d small-molecule sweep) as signal and gave each frame exp(kept_f * 4.9) - the e^-5
errors on a quarter of the frames, R1 0.62. A fixed box window also cannot follow a
100x absorption ramp over a few degrees, and frames under the credible floor, exempt
from the window, ran away to 1e-7.
Now each round fits every frame on its own fulls (no pooling, no minimum), and the scale
is a penalised second-difference smoother of log G (Whittaker/Eilers), each frame at
the information of its fit, lambda by cross-validation over blocks one rocking curve
wide (interleaved single frames leak through shared rocking curves and chose to follow
every frame). After convergence the existing ShrinkToRestrained hands back the per-frame
deviation its neighbour shares. Pooling and the box window are gone from the fulls loop;
the partials loop is unchanged.
2. With partial scaling off (< 50 rocking events per frame) the partials kept the
integration-time corr: no incident-flux correction and not the partiality of the
ingest-smoothed geometry, because only the partial scaling loop rewrote corr. They now
get corr = prescaling_corr / partiality at G = 1 (host, and a device kernel).
3. The flux meter (per-frame mean background) jumped 30x between neighbouring frames of a
sparse sweep - on a few reflections it measures which reflections the frame holds. It
is read through the same smoother at the precision of each frame's mean.
SHELXL R1(>4sig) against the published structures, rc174-cand -> this, on the in-house
small-molecule sweeps: cubic Ia-3d 0.615 -> 0.119 (XDS 0.088); four organic sweeps
(monoclinic / orthorhombic) 0.0515 -> 0.0459, 0.0451 -> 0.0396, 0.0747 -> 0.0709,
0.0452 -> 0.0413; ISa up to 11.6 -> 31. Raw flux instead of smoothed costs 0.002-0.003 R1 on
the first two. A weak, decaying protein sweep on the fulls-only path: ISa 6.7 -> 27.1.
CPU and GPU paths agree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The SHELX .hkl test removed its file while its own ifstream still held it
open, which Windows refuses. The GPU-vs-gemmi model-scale check held b_star
to 1e-4 of its largest element alone, which on an F4132 case fitted at
B 0.6 A^2 failed on a 1e-4 A^2 difference; it now also allows 0.001 A^2
of B.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Where the file's centre and the background-measured one index the same lattice in the same class,
the check kept the file's centre and carried nothing forward. But a centre off by about one
reflection spacing indexes the same cell just as well, with reflections given an index one out over
part of the sweep, and only the merge shows it. On 9hnc (pseudo-tetragonal P2, c* spacing 11.3 px on
the detector) the post-refinement walked the file's centre to 1219.4 px while the background measured
1229.95 +- 0.34 px: the run merged at R_meas 0.58, ISa 1.3, CC1/2 0.84 and adopted P121.
Now the measured centre is kept as a hypothesis when it is further from the file's than this
geometry absorbs (and than 3 sigma of its own fit), and RunAllPasses runs the first pass there only
when the file's centre merged inconsistently: CC1/2 below 0.9 in the lowest-resolution shell of the
search merge, where every reflection is strong. Over the corpus runs read that shell at 0.967-0.998;
9hnc's file centre reads 0.610. Running it unconditionally would cost a second first pass on 107 of
227 battery sets.
MeasuredCentreWins on the same lattice compared only the strong-reflection count, which the
misindexed arm won by 2% (335800 vs 329330). It now also lets the measured centre win when its
inner-shell CC1/2 beats the file's by MAX_CC_HALF_LOSS - 0.997 against 0.610 on 9hnc.
9hnc: P121 R_meas 0.578 ISa 1.33 CC1/2 0.844 -> P1211 (= deposited) R_meas 0.129 ISa 13.5 CC1/2 0.998.
Its screw now reads 2_1 (32 absent, 0 violations) beside a detected (1/2,1/2,0) pseudo-translation,
against the deposited P121, so the battery verdict moves pass -> fail on the screw alone; R_free with
the deposited model 0.443 -> 0.255.
Battery sympg-bc1 (22 targeted sets, 13 of them in the same-lattice branch with a centre further
off than the geometry absorbs): only 9hnc tries the measured centre; the others read 0.984-0.999 in
the inner shell and are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The re-selection among tied higher-group candidates left
sohncke_space_group at the old lowest-numbered pick, so the report read
SPACE_GROUP_NAME P 42 21 2 beside SOHNCKE_SPACE_GROUP P 4 2 2.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
refined_shared() recomputed R-free as sum|F - FC_ALL_LS| / sum F from REFMAC's output MTZ. That
MTZ's F is not the input F: REFMAC writes it with its overall anisotropic scale applied. On
strongly anisotropic data the scale multiplies the noise of the weak direction, and the recomputed
R comes out far above REFMAC's own R-free over the very same reflections.
9RCS (deltaB ~250 A^2, deposit STARANISO-corrected): the recomputed shared-set R-free was 0.503
against REFMAC's own 0.368 on the same 378 free reflections (deposit 0.366/0.365), which is the
whole of the +0.137 the open arm reported for this set. Across the 152 untwinned rc173 sets the
two agree to a median 0.0001 and to within 0.03 everywhere else.
Both refinements' free sets are the shared set by construction, so REFMAC's own R-free is the
shared-set number already, as the twin branch used it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Two first-pass decisions taken on frame counts at their noise floor, both found on a hexagonal
small-molecule crystal embedded in a powder of its own microcrystals (the non-ice powder rings
index on the crystal's own cell; 13-16% of the pooled validation spots lie on the lattice):
- Sparse lattice. A lattice whose pooled spots fall under the per-frame floor (20%) is integrated
on every frame only if fewer than 1/6 of the validation frames clear the floor. At 12/60 the floor
was kept, so only the frames in the upper tail of the same population were integrated: 267 of
1800 frames merged, 22.7k rocking events -> 4.2k fulls, completeness 74%. The bar is now the
majority (score < N/2). 25 keV sweep: completeness 74% -> 99% to 0.56 A, frames rejected
1533 -> 137, SHELXL R1 (COD model) 0.150 -> 0.156 / 0.150 -> 0.139 depending on the COD entry.
No other set of the overnight battery (open, in-house, private) is in the changed band
(pooled < 20% with 10-29 of 60 frames).
- Axis sign. The opposite spindle sign was adopted on 1/60 frames against 0/60; every rescue after
that ran on the wrong sign and the 20 keV sweep ended in "No crystal lattice found" (with some
compiler flags the file's sign scored 9/60 and the run indexed). The flipped lattice must now
also beat chance on the pooled spots, the test the run puts its final lattice to; a sparse
pattern at the right sign still passes (2/60 frames, 89% of the validation spots). 20 keV
sweep: fails -> P 61 2 2, 5.41/56.0 A, SHELXL R1 0.137 (was 0.76 in the runs that indexed).
tools/battery/shelx_check.py: the fixed-model R1(F) now fits SHELXL's extinction coefficient
together with the scale (scan over x), since extinction belongs to the crystal and otherwise
dominates the strongest reflections (HEPES 0.099 -> 0.059).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Mapped every single-axis screw candidate of every primitive-lattice set in
battery 3 (open + in-house) and the private arm against the deposited group.
False zones with no violation read at most +4 nats (plus one pseudo-
translation row at +30.5 that no bound separates); true zones go down to
+10. The true zones under 20 are all short monoclinic rows (b ~ 25-30 A:
four to six 0k0-odd reflections inside the search's resolution range, on
weak data at a few percent of their row), refused at 20 on five crystals
that are P2_1: four myoglobin sweeps (10.0-16.3 nats) and 6cs9 (10.8).
The rc174-cand myob_x06da_split call sat at 21.1, one refit away from P2.
Re-scoring the stored candidate tables at the new bound changes only those
five plus 3r6o (I4_1 2 2 newly eligible, toward the deposited I4_1 screw);
no adopted group elsewhere. New test section pins both sides: four 0k0-odd
at 2% of their row are claimed (11.3 nats), at 20% they are not.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
When a refused point group is re-asked on the adopted group's merge (twin-immune
zone / merge comparison), that merge no longer holds the adopted group's
screw-absent axial reflections. The higher search then had nothing to judge on
those rows, every candidate of the point group tied, and the lowest-numbered -
the one claiming no screw - was written. Measured on an open-arm P4_2 2_1 2
crystal (8v2t): adopted P2_1 2_1 2_1, promoted by the twin-immune zone, all
three axial rows "no control class", P 4 2 2 written.
Among the candidates the higher search cannot separate, take the first that
keeps every axial absence the adopted group claims (as the centring already
is taken), drop from the alternatives the ones that do not, and stop reporting
those rows as undetermined. Rows the adopted group left present are still in
the merge and still decided by the higher search.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
