The owner's scoring decision: no R_meas criterion, and merge quality
judged only by CC1/2 relative to XDS.
- The global rule (R_meas > 60% or CC1/2 < 0.5 fails) is gone.
- XDS arms: a set fails `merge` when (1/CC1/2 - 1) of the reference-range
table is more than twice XDS's, XDS's CC1/2 taken at the bottom of its
0.1% rounding. CC1/2 = S/(S+E), so 1/CC1/2 - 1 = E/S at any CC1/2, and
E goes as 1/observations: 2x is XDS's merge with half its observations.
Stored as cc_half_noise_ratio. No REFRES table or no XDS CC1/2: no
criterion.
- Open arm: no merge criterion; CC1/2 and R_meas stay reported numbers.
- Low-resolution R_meas is reported, not scored: the lowest shell of the
own table, the reference-range table and XDS's (lowres_*), with
lowres_r_meas_ratio in the like-for-like table and a ratio plot.
- Every schema-3 run is re-scored when it is read (report, compare,
baseline delta) with today's scorer and today's manifest rows, so both
sides of a comparison are scored alike. results.json keeps the verdicts
as scored at run time; rows without a lattice keep them.
Re-scoring the rc172 reference run: inhouse 29 -> 27 pass (three new
merge fails, one R_meas fail lifted), open 137 -> 138 (one R_meas fail
lifted).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
parse_correct_lp also reads the first row of CORRECT.LP's last resolution
table: its high-resolution limit (dmin_low) and R_meas (r_meas_low).
inhouse.json regenerated with `refs --arm inhouse --write`; the only
change is the two new keys per set (the private manifest was regenerated
the same way, outside the repository).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013nW6FNRP1bBJJ8pfHiByAT
At 16 keV a native crystal has no usable anomalous signal, so XDS's CORRECT
was rerun with FRIEDEL'S_LAW=TRUE and the set no longer runs with -A.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
--report-resolution now gets XDS's own d_min, the range its CORRECT.LP totals
cover, so the comparison is exact; shells past rugnux's cut print as not
merged. The derived reference d_min stays what rugnux's own cut is scored
against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
--model and --report-resolution are both processing-inert (merged reflections
bit-identical with and without them), so the bare/model and bare/xds variant
pairs collapse into one command per arm:
open: rugnux -o p --no-export-unmerged --model <deposited coords> <input>
(no --model where there is no deposited model)
inhouse/private: rugnux -o p --no-export-unmerged [-A] --report-resolution
<reference d_min>,<XDS d_max> <input>
Every row is rugnux's own result (own cut vs the reference, as `bare` was), and
on the XDS arms also carries the REFRES_* table, scored against CORRECT.LP's
completeness, multiplicity, R_meas, CC1/2 and ISa, with ISa and R_meas ratio
plots and a like-for-like table; REFRES_SHELLS_PAST_LIMIT > 0 marks the row as
coverage (rugnux's cut coarser than the reference). Nothing is forced on the
processing any more. On the open arm the space group is scored on the data's
own determination (SOHNCKE_SPACE_GROUP where SPACE_GROUP_ENANTIOMORPH is
ASSUMED_FROM_MODEL), the model's label kept as sg_label; R_FREE/R_WORK/CC_MODEL
/MODEL_FIT are trend fields (placement-only, own free set); REFMAC stays opt-in.
results.json schema 3: one row per set, no variant/first_read; older runs are
read through their bare rows (open-arm model R-factors folded in), so compare
and report keep working against them. Every time is now a first read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The crystal does have a single P2 lattice, holding ~6% of each frame's spots
on top of ice and protein powder; XDS indexes it with the ice rings excluded.
The reference is now that P2 run to 2.0 A instead of a failed P1 run, and the
set is renamed from aggregate to sparse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
XDS was rerun with the pin shadow masked and the frames where the crystal
leaves the beam excluded; the reference is now P43212 to 1.65 A instead of a
failed P1 run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The battery is bound by reading images from disk, so instead of one battery run per setting each
set now runs in all its variants in a row while its images are still in the page cache:
open arm bare, then model (rugnux --model <deposited coordinates>)
XDS arms bare, then xds (XDS's resolution range forced, -A where XDS was anomalous)
bare (plain rugnux <input>) runs first on every arm, so its time always carries the set's disk
read whichever variants are selected; each row records first_read and the report's timing table
says which variant's times are cold. --variants runs a subset; --unforced is gone (--variants
bare). Rows, work dirs (work/<arm>/<set>/<variant>/), compare and the report are per variant;
compare pairs (arm, set, variant) on the variants both runs have on an arm, and a schema-1 run is
read as one variant. The model variant takes the coordinates and published R-free from
model_check's RCSB cache (site key pdb_cache) and records rugnux's R-free/R-work and the ratio;
the REFMAC check is now opt-in (--model-check) and its keys moved to refmac_*. results_schema 2.
model_sweep.py is retired: the model variant replaces it (its --spot/--scaling-low-resolution 50
were rugnux's defaults, so the command is the same).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
On the XDS-referenced arms rugnux merges Friedel mates separately whenever
XDS ran with FRIEDEL'S_LAW=FALSE, in forced and unforced mode alike, so the
merging statistics compare like with like.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Our merge and the depositor's structure factors are both scored on the
depositor's free set by the same protocol, and the report plots that ratio.
R-free on our own free set reads low, because those reflections were mostly
work reflections in the deposited refinement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run after rugnux has written its MTZ, so processing never sees the model. Fetches the
entry (cached), matches the data to the model's setting by trying every Niggli-cell
change of basis and keeping the one that best fits the model's Fcalc on a mid-resolution
shell, re-expresses the data in the deposited space group and cell, and has REFMAC compute
R-work/R-free of the unmodified model (rigid-body mode read at its first cycle: no
dictionaries, no ligand stripped, no refinement). Baseline: the same protocol on the
depositor's own structure factors and free set, plus rugnux's data on the depositor's flags.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Directories follow <protein>_<beamline>_<feature> (e.g. lyso_x10sa_90deg_1,
thau_bl1a_6p5keV, thau_micromax_pink); previous ids stay as aliases so older
runs still pair up in compare. The ice-only control moved out of the in-house
arm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- rugnux_vs_xds.py: deleted. The inhouse and private arms of tools/battery run the
same comparison against CORRECT.LP, from a fixed manifest instead of a directory
scan, with persisted runs and a standard report.
- lyso_test/anomalous_scoreboard.sh: deleted. A one-dataset SHELXC/ANODE scoreboard
with hard-coded personal paths, superseded by rugnux_anomalous.py.
- rugnux_vs_pdb.py -> tools/battery/model_sweep.py. The rugnux --model sweep is
not superseded, but it now takes its inputs from the open arm's manifest and the
battery site config, so both tools process the same files.
- docs/TESTS.md and rugnux_stills_ab.py point at the battery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Site config: site.example.json is committed; the local site.json (git-ignored) or
$JFJOCH_BATTERY_SITE / --site names this machine's paths and the private manifest.
Data roots are the arms' own directories, so copying them and editing the site
config moves the battery.
- Run states: manifest records the runner's pid; a run is complete, running, aborted
(Ctrl-C/SIGTERM, or `abort RUN`) or unfinished. compare and report refuse a run
that did not finish unless --allow-incomplete, and the report says so.
- `list` shows runs and their state; `run` defaults its label to the rugnux version
and its baseline to the site's persisted one; a baseline must be complete and of
the same privacy as the run; a private report cannot be written into the repo.
- XDS reference d_min: when the finest shell of XDS's table has no significant
CC1/2, XDS merged past its own signal (no resolution cut, or a cut at the detector
edge) and its limit is the detector's reach. The reference is then where XDS's
CC1/2 falls through 0.30 (rugnux's own target), and that is also the range forced
on rugnux. The manifest keeps XDS's limit (dmin_xds) and the rule (dmin_rule).
Forcing the old 1.08 A on the weak insulin set turned rugnux's I23 into I222
(ISa 12.5); forced at the derived 1.81 A it is I23 again (ISa 18.0). Changed in
the inhouse arm: insu_I_weak 1.08 -> 1.81, lyso_half_image 0.80 -> 2.60,
cytc_eiger 2.04 -> 2.27, lyso_strong 1.18 -> 1.24; every other inhouse limit stands.
- inhouse.json follows the directory rename (ids and inputs, data root
/home/data/inhouse); `remap` applies such a rename and keeps the old ids as
aliases, so runs from before it still compare set by set.
- Optional open-arm model check (model_check.py, when present): R-free of the
deposited model against each merge, recorded and plotted.
- results.json has a fixed, documented row schema (results_schema 1); the HTML
report gains inline-SVG plots (verdicts per arm, d_min and R-free ratios per set).
- README.md: arms, data layout and provenance, prerequisites, running, run
directory, results schema, report, comparing, protocol, adding datasets.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The crystal was measured at two wavelengths; both sweeps are kept. The native
sweep carries the deposited resolution, the 1.89 A sweep is scored on symmetry
and cell only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One runner for the open (vs PDB deposition) and in-house (vs XDS) arms,
replacing the scattered scripts outside the repository. Each run writes a
new, read-only directory named <date>_<rugnux git>_<label> holding a copy
of the binary with its sha256 and build flags, manifest.json (inputs,
references, options, host, GPU sharing) and results.json in one schema
for all arms, plus per-set work directories and a Markdown and a
self-contained HTML report (per-arm and per-population pass rates,
resolution gain, ISa/R_meas distributions, timing, failures, delta vs a
baseline run).
- Dataset lists and references are data files: open.json (151 public
sets, deposited group/cell/resolution, pinned inputs) and inhouse.json
(standard test crystals and no-crystal controls, XDS references). The
private arm's manifest stays outside the repository; a private run gets
its own run directory and report.
- Scoring is setting-invariant: lattice by Niggli-reduced primitive cell
(beta vs 180-beta, C2 vs I2), symmetry via sgequiv on the open arm and
by point group against XDS, then merge usability. Re-scoring the rc169
and rc170 battery outputs reproduces their verdicts.
- Input discovery (`discover`) ranks sweeps by frame count, never by
file size or sort order; a run never searches, it runs what the
manifest names.
- `compare A B` flags per-set changes beyond noise; `--rerun-changed`
reruns them with A's saved binary to separate real changes from
run-to-run noise.
- `--tier smoke` selects a 22-set subset spanning the populations; a
full-arm run takes a lock; `--gpulock` queues on the shared GPU and
marks the timing as not a reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
