The open-arm L-alanine set's COD reference (2104782) is a 0.2 GPa room-temperature
diamond-anvil-cell structure (372 reflections, 43% complete), not the ambient one the
docs called it. The set's own cell (5.787 5.940 12.263) is the 100 K ambient form, so
the reference is now COD 2311261 (100 K, 5.791 5.944 12.269; Parsons, Flack & Wagner,
Acta Cryst. B69 (2013) 249-259). SHELXL's refined R1 does not depend on the starting
model (0.0546 vs 0.0524), but the fixed-model R1(F) the battery reports goes from 0.168
to 0.087, so that number now measures the data rather than the pressure difference;
cell_dev_pct likewise compares against the right cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
refined_shared() recomputed R-free as sum|F - FC_ALL_LS| / sum F from REFMAC's output MTZ. That
MTZ's F is not the input F: REFMAC writes it with its overall anisotropic scale applied. On
strongly anisotropic data the scale multiplies the noise of the weak direction, and the recomputed
R comes out far above REFMAC's own R-free over the very same reflections.
9RCS (deltaB ~250 A^2, deposit STARANISO-corrected): the recomputed shared-set R-free was 0.503
against REFMAC's own 0.368 on the same 378 free reflections (deposit 0.366/0.365), which is the
whole of the +0.137 the open arm reported for this set. Across the 152 untwinned rc173 sets the
two agree to a median 0.0001 and to within 0.03 everywhere else.
Both refinements' free sets are the shared set by construction, so REFMAC's own R-free is the
shared-set number already, as the twin branch used it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Two first-pass decisions taken on frame counts at their noise floor, both found on a hexagonal
small-molecule crystal embedded in a powder of its own microcrystals (the non-ice powder rings
index on the crystal's own cell; 13-16% of the pooled validation spots lie on the lattice):
- Sparse lattice. A lattice whose pooled spots fall under the per-frame floor (20%) is integrated
on every frame only if fewer than 1/6 of the validation frames clear the floor. At 12/60 the floor
was kept, so only the frames in the upper tail of the same population were integrated: 267 of
1800 frames merged, 22.7k rocking events -> 4.2k fulls, completeness 74%. The bar is now the
majority (score < N/2). 25 keV sweep: completeness 74% -> 99% to 0.56 A, frames rejected
1533 -> 137, SHELXL R1 (COD model) 0.150 -> 0.156 / 0.150 -> 0.139 depending on the COD entry.
No other set of the overnight battery (open, in-house, private) is in the changed band
(pooled < 20% with 10-29 of 60 frames).
- Axis sign. The opposite spindle sign was adopted on 1/60 frames against 0/60; every rescue after
that ran on the wrong sign and the 20 keV sweep ended in "No crystal lattice found" (with some
compiler flags the file's sign scored 9/60 and the run indexed). The flipped lattice must now
also beat chance on the pooled spots, the test the run puts its final lattice to; a sparse
pattern at the right sign still passes (2/60 frames, 89% of the validation spots). 20 keV
sweep: fails -> P 61 2 2, 5.41/56.0 A, SHELXL R1 0.137 (was 0.76 in the runs that indexed).
tools/battery/shelx_check.py: the fixed-model R1(F) now fits SHELXL's extinction coefficient
together with the scale (scan over x), since extinction belongs to the crystal and otherwise
dominates the strongest reflections (HEPES 0.099 -> 0.059).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Its deposited sweep lacks 40 deg (frames 561-720 are not in the
archive; the depositors processed it as two sweeps), so it tests
gap handling rather than processing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
SRS Daresbury PX10.1 (6RYM), EMBL Hamburg DORIS X13 (7BGU), NSLS X29A and X25
(8V4J, 8V2T), ESRF ID14-2 (5JK4), Australian Synchrotron MX2 (6CS9), CLSI 08B1-1
(7UDI), SSRF BL17UM (9LXL), MAX IV BioMAX (9S02), ALBA XALOC (6GVK) - marCCD,
ADSC (Quantum 4/210r/315), PILATUS and EIGER2 files. Each read and processed with
the current rugnux; 7BGU (C2 imposed on a P1 lattice) and 7UDI (P41 under-called
to P2) disagree with the deposition. Multi-sweep archives pinned to one sweep.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
Pseudo-merohedral and merohedral twins, pseudo-symmetric cells and their
negative controls, chosen to test the space-group search's promotions:
- 8c3e (P3121, merohedral twin ~0.23, home-source HyPix CBF)
- 4bwl, 2wnq, 2xfw, 2wnn (P21 NAL crystals with the pseudo-merohedral
twin law -h,-k,h+l at fractions 0.49/0.46/0.10/0.33; ADSC SMV) and 2wnz,
the same crystal form deposited untwinned
- 6oww, 6p8j, 9qw2 (P21 with near-90 beta or a~c), 9qvv (I222, b=c),
5ojv (P21212, a~c), 7q6j (P212121, b~c)
Data staged in /data/scout_twin, linked from the open data root.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
The XDS references of aspirin (20, 25 keV), citric acid, HEPES and YAG were made with
OVERLOAD=65534, half of the EIGER2's saturation value (133202). CORRECT ignored 13-313
reflections per set whose peak pixel lay between the two - the strongest, most
extinguished reflections - so the references described a data set without them.
Re-ran only the CORRECT step on the unchanged INTEGRATE.HKL with every input parameter
reconstructed from the old CORRECT.LP and OVERLOAD=133201 (saturation - 1, as the other
in-house sets already use); a rerun at 65534 reproduced the old CORRECT.LP. The old outputs
are kept in xds_overload_half/ beside each data set. `refs --write` picked up the new values.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
shelx_check.py refines the row's published structure (manifest key "cod", cached from the
Crystallography Open Database into the site's cod_cache) against rugnux's p.hkl with one fixed
recipe: data reindexed into the COD setting (lowest-R1 integer matrix), non-H anisotropic, H fixed,
EXTI, MERG 2, three rounds of SHELXL's suggested weights. It records R1/wR2/GooF/EXTI/WGHT/residual
density/R(int)/R(sigma)/K of the strongest bin and a fixed-model R1(F) (|Fc| of the COD model as
published, gemmi). Reported, never scored. The report gets a small-molecule table; compare lists
SHELXL R1/wR2/GooF/EXTI deltas; report/compare fill the check in for older runs.
COD entries matched by Niggli-reduced cell and space group: aspirin 7050897, citric acid 5000063,
HEPES 2224210, YAG 2003066, L-cystine 1513328 (2005, replaces the 1959 model for refinement),
cytidine 2001311, 3,5-dinitrobenzoic acid 4510615, L-alanine 2104782, metformin HCl 2108029,
NiCl2(dppe) 2012031. cuhf2 has no reference cell and no match: left without one.
SHELXL is called, not shipped (site key "shelxl" or PATH; it comes with CCP4).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
cpuinfo_max_freq is the highest boost state, not the base clock -
mislabeling it "nominal" understated the gap between the two on an AMD
box using acpi-cpufreq (this machine: 5.08 GHz boost vs 3.40 GHz base).
Prefer, in order: the model-name string, cpufreq's own base_frequency
(intel_pstate, kHz), ACPI CPPC's nominal_freq (MHz; what acpi-cpufreq
exposes instead), else "unknown". cpuinfo_max_freq is now reported
alongside, separately, as "max boost", never folded into "nominal".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Collect CPU model, nominal/base frequency (with its source), physical
core/thread/socket counts, RAM and GPU name(s)/memory at run time
(Linux via /proc, /sys/.../cpufreq, nvidia-smi; a short best-effort on
macOS via sysctl), store it under manifest.json's new "hardware" key,
and render it as a line in the report header. Runs from before this
change have no such key; the report prints "not recorded" for them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lcystine_x10sa_20keV and lcystine_x10sa_25keV are scored against the
literature structure only (hexagonal L-cystine, P 61 2 2, COD 5000005,
Oughton & Harrison 1959): XDS indexes under 10% of the spots of either
sweep, with the cell supplied and with the spot range cut to 150 frames,
so there is no CORRECT.LP to take a reference from. On the current build
the 25 keV sweep passes (P 61 2 2, cell within 0.5%) and the 20 keV sweep
finds no lattice.
EXTERNAL_TEST_DATA.md: none of the open-arm datasets is pink beam; every
PDB record is single-wavelength (9Q41 used a multilayer monochromator).
The pink-beam sets are the in-house ones.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every open and in-house manifest row now carries `wavelength` (A), taken
from WAVELENGTH in rugnux's own p_report.txt of the latest run of that set
(the full rc173 run 20260929-2003 plus the two 10-02 runs for the sets
added since). Descriptive only; the scorer ignores it.
docs/BATTERY_REPORT.md is report.md of the full rc173 open + in-house run
(210 sets), verbatim, to be polished. The private arm runs separately and
has its own report, so nothing from it appears here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Open arm, all IRRMC, single Cu K-alpha sweeps on Rigaku Saturn CCDs read
through the d*TREK SMV path: 3mc4 (H 3), 3meb (P 1 21 1, Saturn 944,
overflow ratio 32), 3p85 (P 63 2 2), 3r6o (I 41), 5uth (P 31 2 1), 5vml
(P 42 21 2), 6cee and 6v2r (P 21 21 21 / P 41 21 2, Saturn A200, unbinned
2048x2048, 0.1 mm pixels). Detector 2theta spans -10 to +10 deg. Run
20261002-1326_84228b_home-source-12: all pass except 3r6o, called I 4 2 2
against the deposited I 41 with twinning suspected (<|L|> 0.32). Two
further candidates (3r6h, 3sgw) were dropped: what the repository lists as
one dataset is two sweeps under one file template, split by a phi change,
which the reader refuses.
In-house arm: aspirin (20 and 25 keV), anhydrous citric acid, HEPES and YAG,
measured at X10SA. The reference is XDS's CORRECT.LP (refs --write) with
the space group taken from the literature as ref_override, because XDS
reports only Sohncke groups; each override cites its COD entry and paper.
Run 20261002-1354_84228b_small-molecules-inhouse: four pass, including
HEPES as P b c a where XDS has P 21 21 21; YAG is called I 41 3 2 against
I a -3 d (both programs merge it poorly, rugnux CC1/2 0.40, XDS ISa 3.2).
EXTERNAL_TEST_DATA.md gains the eight depositions (DOIs checked on DataCite)
and updated counts; the battery README lists the small-molecule standards
and has the current count of symlinked open-arm directories (42).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The SMV reader only knew the ADSC vocabulary, so a Rigaku Saturn frame was
recognised as SMV and then refused for want of PIXEL_SIZE. A header naming
DETECTOR_NAMES is now read as d*TREK:
- pixel size and the point of normal incidence from <det>SPATIAL_DISTORTION_INFO;
- image directions from <det>DETECTOR_VECTORS combined by
<det>SPATIAL_DISTORTION_VECTORS, matched to a DetectorOrientation (a Saturn
944+ image is mirrored and turned a quarter, so the hand is preserved);
- distance and the detector circles (2theta etc.) from <det>GONIO_*, composed
into PONI angles; spindle from ROTATION_VECTOR, angles from ROTATION;
- wavelength from SCAN_WAVELENGTH, or the second number of SOURCE_WAVELENGTH
(the first is a count - the old NumAny fallback would have read 1.0 A);
- pixels above 32767 decoded as (v - 32768) * RAXIS_COMPRESSION_RATIO.
dxtbx's FormatSMVRigakuSaturnNoTS, which claims headers without
DTREK_DATE_TIME, ignores the distortion vectors; on a Saturn 944+ sweep that
puts the spindle 90 degrees off (DIALS: 12% indexed vs 98% with the vectors
applied, at the deposited cell). The arm geometry is confirmed by the data:
the refined direct beam of a 2theta = 10 deg sweep lands at x = 621.1 against
621.6 predicted. The anomalous map peaks at 11.5 sigma on the two K+ and
6-7 sigma on S and P, so the hand is right.
Battery, open arm: 5cc8 (rotating anode, Saturn 944+ SMV, 2theta = 10 deg
sweep) and 9jq9 (Ga K-alpha liquid-metal jet, PILATUS3 1M miniCBF, single
450 deg sweep), both IRRMC. Run 20261001-2132_84228b_dtrek-home-source:
9jq9 passes (P 21 21 21, 1.65 A); 5cc8 merges to 1.53 A at the deposited
cell but is called P 21 21 21 against the deposited P 21 21 2 (screw
evidence on 00l 68-78 nats, tNCS at (1/2,1/2,0.11)) - left failing, not
yet adjudicated. The Saturn gain (~5 ADU/photon) is not in the header and
is not modelled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* jfjoch_broker: Optional per-dataset authentication - statistics, images and plots can require a bearer token, which jfjoch_viewer supports.
* jfjoch_viewer: Dark mode and a theme-matched colour scheme, a magnifier panel, and simpler contrast and background controls.
* Rugnux: Multiple performance improvements on GPU and CPU (CPU-only processing up to 40% faster, faster image decoding on ARM), with unchanged results.
* Rugnux: `--model` rigid-body refinement runs on the GPU, and the model-validation check is faster and more reliable.
* Rugnux: Improved scaling and merging - error model, outlier rejection, absorption correction and French-Wilson amplitudes now agree more closely with XDS and ctruncate.
* Rugnux: Improved integration - radial background on powder and ice rings, crowded rotation data keep their reflections, and CPU-only builds integrate large unit cells as GPU builds do.
* Rugnux: More robust detector geometry - measured beam centre, X-ray bandwidth and goniometer rate, and geometry refinement accepted only on significant evidence.
* Rugnux: Merged files are written in the standard setting, or in the setting of a reference MTZ, structure-factor mmCIF or model, with its free-R flags.
* Rugnux: Richer report - ice and powder rings, further lattices, superstructure candidates and mosaicity, with warnings worded as prompts to check.
* Rugnux: Clear error messages when a data set needs more GPU or host memory than is available.
Reviewed-on: #83
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
* Fixed `jfjoch_broker` cancelling every data collection with a CUDA "out of memory" error after long operation: GPU memory no longer leaks with each collection.
* Rugnux scales a rotation sweep until the per-frame scales settle instead of for a fixed three rounds, and says so when they did not - merged intensities, and the space group, resolution cut and frame rejection read off them, change accordingly; `--scaling-iterations` is now the cap on that loop (default 100).
* Rugnux places every frame of a marCCD, SMV or miniCBF series at the spindle angle its own header states, so a series with missing frames, or with angles written modulo 360, is no longer read at the wrong geometry or refused.
* Every rotation run writes two diagnostic files beside its reflections: `<prefix>_detector.jpg`, the detector projection with the pixel mask and the detected beam-stop shadow drawn on it, and `<prefix>_plot.txt`, one row per image.
Reviewed-on: #82
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
