Commit Graph
2 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5 deb8572812 tools/battery: one canonical runner, derived XDS d_min, clean inhouse layout
- Site config: site.example.json is committed; the local site.json (git-ignored) or
  $JFJOCH_BATTERY_SITE / --site names this machine's paths and the private manifest.
  Data roots are the arms' own directories, so copying them and editing the site
  config moves the battery.
- Run states: manifest records the runner's pid; a run is complete, running, aborted
  (Ctrl-C/SIGTERM, or `abort RUN`) or unfinished. compare and report refuse a run
  that did not finish unless --allow-incomplete, and the report says so.
- `list` shows runs and their state; `run` defaults its label to the rugnux version
  and its baseline to the site's persisted one; a baseline must be complete and of
  the same privacy as the run; a private report cannot be written into the repo.
- XDS reference d_min: when the finest shell of XDS's table has no significant
  CC1/2, XDS merged past its own signal (no resolution cut, or a cut at the detector
  edge) and its limit is the detector's reach. The reference is then where XDS's
  CC1/2 falls through 0.30 (rugnux's own target), and that is also the range forced
  on rugnux. The manifest keeps XDS's limit (dmin_xds) and the rule (dmin_rule).
  Forcing the old 1.08 A on the weak insulin set turned rugnux's I23 into I222
  (ISa 12.5); forced at the derived 1.81 A it is I23 again (ISa 18.0). Changed in
  the inhouse arm: insu_I_weak 1.08 -> 1.81, lyso_half_image 0.80 -> 2.60,
  cytc_eiger 2.04 -> 2.27, lyso_strong 1.18 -> 1.24; every other inhouse limit stands.
- inhouse.json follows the directory rename (ids and inputs, data root
  /home/data/inhouse); `remap` applies such a rename and keeps the old ids as
  aliases, so runs from before it still compare set by set.
- Optional open-arm model check (model_check.py, when present): R-free of the
  deposited model against each merge, recorded and plotted.
- results.json has a fixed, documented row schema (results_schema 1); the HTML
  report gains inline-SVG plots (verdicts per arm, d_min and R-free ratios per set).
- README.md: arms, data layout and provenance, prerequisites, running, run
  directory, results schema, report, comparing, protocol, adding datasets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 18:45:02 +02:00
leonarski_fandClaude Opus 5 5e7b53dd7b tools/battery: committed, reproducible validation battery
One runner for the open (vs PDB deposition) and in-house (vs XDS) arms,
replacing the scattered scripts outside the repository. Each run writes a
new, read-only directory named <date>_<rugnux git>_<label> holding a copy
of the binary with its sha256 and build flags, manifest.json (inputs,
references, options, host, GPU sharing) and results.json in one schema
for all arms, plus per-set work directories and a Markdown and a
self-contained HTML report (per-arm and per-population pass rates,
resolution gain, ISa/R_meas distributions, timing, failures, delta vs a
baseline run).

- Dataset lists and references are data files: open.json (151 public
  sets, deposited group/cell/resolution, pinned inputs) and inhouse.json
  (standard test crystals and no-crystal controls, XDS references). The
  private arm's manifest stays outside the repository; a private run gets
  its own run directory and report.
- Scoring is setting-invariant: lattice by Niggli-reduced primitive cell
  (beta vs 180-beta, C2 vs I2), symmetry via sgequiv on the open arm and
  by point group against XDS, then merge usability. Re-scoring the rc169
  and rc170 battery outputs reproduces their verdicts.
- Input discovery (`discover`) ranks sweeps by frame count, never by
  file size or sort order; a run never searches, it runs what the
  manifest names.
- `compare A B` flags per-set changes beyond noise; `--rerun-changed`
  reruns them with A's saved binary to separate real changes from
  run-to-run noise.
- `--tier smoke` selects a 22-set subset spanning the populations; a
  full-arm run takes a lock; `--gpulock` queues on the shared GPU and
  marks the timing as not a reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 18:45:02 +02:00