battery: a "standard" timing benchmark on open data, with every model committed

battery.py timing --benchmark standard (now the default) runs 9zlo, 9s02 and
9gjx from their AlphaFold DB models (AF-P17090-F1, AF-P32322-F1, AF-Q65MG6-F1
v6, with their PAE, as the 2026-10-16 AlphaFold MR scan used them; all three
GREEN there) and 4nwv from its deposited model moved by the existing fixed
transform. --benchmark inhouse keeps the earlier myob_x10sa / cytc_x10sa /
4nwv list. Same checks; expected groups are the deposited ones, R-free <= 0.50
for the predictions (0.35 for 4nwv), placement CC >= 0.30.

So that a machine needs only the raw data:
- the deposited coordinates of 9zlo, 9s02, 9gjx, 4nwv and 4dc8 (the inhouse
  myoglobin reference) are committed in tools/battery/models/pdb/ as the
  compressed mmCIF RCSB serves (CC0; 50-660 kB each, 1.6 MB together), and
  model_check.deposition reads an entry there before the pdb_cache, taking
  its refinement record from the file itself - no cache JSON, no network;
- open/inhouse arms default their manifest and reference, so a site config
  is just runs_root plus the data roots (site.timing.example.json);
- baseline_timing is now per benchmark ({"standard": RUN, "inhouse": RUN}).

Verified on the workstation through the GPU lock with a site config whose
pdb_cache is an empty directory (still empty afterwards): standard 4/4 pass
(9zlo 19 s, 9s02 30 s, 9gjx 21 s, 4nwv 89 s), inhouse 3/3 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4

THIRD_PARTY_NOTICES.md

@@ -79,10 +79,11 @@ Not code and not linked into any binary: input files of the battery's timing ben
 
 | Component | Path | Copyright | License (SPDX) | License text |
 |---|---|---|---|---|
-| [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/) predictions AF-P68082-F1 and AF-P62894-F1 (v6), with their predicted aligned error | `tools/battery/models/` | DeepMind Technologies Limited; EMBL-EBI | CC-BY-4.0 | [alphafold-db.txt](licenses/alphafold-db.txt) |
+| [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/) predictions AF-P68082-F1, AF-P62894-F1, AF-P17090-F1, AF-P32322-F1 and AF-Q65MG6-F1 (v6), with their predicted aligned error | `tools/battery/models/AF-*` | DeepMind Technologies Limited; EMBL-EBI | CC-BY-4.0 | [alphafold-db.txt](licenses/alphafold-db.txt) |
+| [wwPDB](https://www.wwpdb.org/) entries 4DC8, 4NWV, 9GJX, 9S02 and 9ZLO, deposited coordinates (mmCIF as the [RCSB PDB](https://www.rcsb.org/) serves them) | `tools/battery/models/pdb/` | the depositors; the wwPDB | CC0-1.0 | [wwPDB usage policy](https://www.wwpdb.org/about/usage-policies) |
 
-The benchmark's third start model is made at run time from the deposited coordinates of PDB entry
-4NWV, downloaded from the RCSB PDB (PDB data are CC0 1.0); nothing of it is committed.
+The deposited entries are the reference models the benchmark's placements are checked against, and
+4NWV's is also moved by a fixed transform at run time to make one start model.
 
 ## Runtime libraries and SDKs (shipped in binaries, not in the source tree)
 

docs/RUGNUX_BENCHMARKING.md

@@ -21,10 +21,10 @@ the old ones were retired.
 - **The validation battery** (`battery.py run`) processes a fixed list of rotation datasets and
   scores each answer against an independent reference. It answers *is the result right, and as
   good as the reference*. A full run takes hours.
-- **The timing benchmark** (`battery.py timing`) processes three sets, each started from a model
+- **The timing benchmark** (`battery.py timing`) processes four sets, each started from a model
   that has to be placed by molecular replacement, so one short run exercises the whole pipeline
   and also checks that the answers are right. It answers *how fast, on this machine*. It takes
-  minutes.
+  minutes, and needs nothing on the machine but four open datasets.
 
 The two are kept apart on purpose. A battery run reads hundreds of gigabytes once, so its wall clock
 mostly measures the disk and the page cache (see [the page-cache trap](#the-page-cache)). Use the
@@ -52,8 +52,8 @@ the cache now is, not where its `<entry>.json` files say they were first fetched
 The data layout (`<data root>/<set>/...`) and where to get each arm's data are in the README's
 *Getting the data*. The open arm is public (each dataset's DOI is in
 [External test data](EXTERNAL_TEST_DATA.md)); the in-house data come from the maintainers. The
-timing benchmark needs only three dataset directories: `inhouse/myob_x10sa`, `inhouse/cytc_x10sa`
-and `open/4nwv`.
+timing benchmark needs only four open dataset directories and the smallest site config there is
+([below](#what-a-machine-needs)); models, references and expected answers are all in the repository.
 
 ## The validation battery
 
@@ -283,23 +283,66 @@ a release battery or for a paper.
 ## The timing benchmark
 
 ```
-$B timing --rugnux $R [--threads N] [--label L] [--baseline RUN|none] [--extra=<rugnux args>] [--gpulock]
+$B timing --rugnux $R [--benchmark standard|inhouse] [--threads N] [--label L] [--baseline RUN|none] [--extra=<rugnux args>] [--gpulock]
 ```
 
-The benchmark runs three sets. Each starts from a model that the run has to place by molecular
+The benchmark runs a few sets. Each starts from a model that the run has to place by molecular
 replacement, so a single run times every stage: reading, pre-scan, indexing, integration, scaling,
 the space-group search, MR and model validation. It also checks that every answer is right.
 
+`--benchmark standard`, the default, is the one to run on a new machine. It uses open data only:
+three EIGER 16M rotation sets of ordinary size and quality, and a virus capsid as the large case.
+
 | set | start model (`--model`) | what makes it hard |
 |---|---|---|
-| `myob_x10sa` | the AlphaFold DB prediction of horse myoglobin, committed in `tools/battery/models/` | a prediction, without heme |
-| `cytc_x10sa` | the AlphaFold DB prediction of bovine cytochrome c, also in `models/` | three copies related by a pseudo-translation; the data alone cannot tell P3₁21 from P3₂21, so the model must decide the hand |
+| `9zlo` | the AlphaFold DB prediction of UreE (*Proteus mirabilis*), committed in `tools/battery/models/` | two copies to place, P2₁2₁2₁, 2.0 Å |
+| `9s02` | the AlphaFold DB prediction of human PYCR1, also in `models/` | five copies, P2₁2₁2, 1.65 Å |
+| `9gjx` | the AlphaFold DB prediction of a *Bacillus licheniformis* nitroreductase, also in `models/` | six copies, P2₁, 2.4 Å, the largest of the three datasets (37 GB) |
 | `4nwv` | the deposited 4NWV (a virus capsid, I222, 45 chains once strict NCS is expanded), turned 30° and shifted 33 Å by a fixed transform | the largest: too far off for the rigid body, so MR places the whole asymmetric unit |
 
+`--benchmark inhouse` is the benchmark as first set up, kept so that earlier runs stay comparable:
+the predictions of horse myoglobin (`myob_x10sa`, a prediction without heme) and bovine cytochrome c
+(`cytc_x10sa`: three copies related by a pseudo-translation; the data alone cannot tell P3₁21 from
+P3₂21, so the model must decide the hand), and the same `4nwv`. Its two in-house datasets come from
+the maintainers.
+
 The predictions are committed next to their predicted-aligned-error files, which rugnux uses to cut
-them into rigid domains. The moved 4NWV is made at run time from the deposited coordinates in the
-site's `pdb_cache` (downloaded from the RCSB on first use), using the transform in
-`tools/battery/timing.json`.
+them into rigid domains. The deposited entries the placements are checked against are committed too
+(`tools/battery/models/pdb/`), and the moved 4NWV is made at run time from the committed 4NWV with
+the transform in `tools/battery/timing.json`. A timing run needs no `pdb_cache` and no network.
+
+### What a machine needs
+
+Copy these directories under one open data root (sizes as unpacked; please cite the DOIs):
+
+| directory | size | source |
+|---|---:|---|
+| `9zlo/` | 11 GB | Zenodo [10.5281/zenodo.18652652](https://doi.org/10.5281/zenodo.18652652) |
+| `9s02/` | 11 GB | MX Raw Data Repository [10.60884/NRNGS4](https://doi.org/10.60884/NRNGS4) |
+| `9gjx/` | 37 GB | IRRMC [10.18430/M39GJX](https://doi.org/10.18430/M39GJX) |
+| `4nwv/` | 16 GB | SBGrid Data Bank [10.15785/sbgrid/9](https://doi.org/10.15785/sbgrid/9) |
+
+The paths inside each directory are the `input`s of `tools/battery/open.json` (for example
+`9gjx/BlNTR_9gjx/data/SC6_1_master.h5`), and [External test data](EXTERNAL_TEST_DATA.md) says what
+each archive holds. The site config needs only the runs root and that data root
+(`tools/battery/site.timing.example.json`):
+
+```json
+{
+ "runs_root": "/path/to/runs",
+ "arms": {
+  "open": {"data_root": "/path/to/open"}
+ }
+}
+```
+
+and the run is
+
+```
+python3 tools/battery/battery.py --site /path/to/site.json timing --rugnux /path/to/rugnux
+```
+
+Python 3.9 or newer with `gemmi` and `numpy` is all the runner needs.
 
 ### The checks
 
@@ -309,8 +352,8 @@ A set passes only when every check holds. The thresholds are in `timing.json`.
 |---|---|---|
 | `space_group` | the battery's own verdict on lattice and space group passes (the hand excepted) | the data's own determination is right |
 | `placed` | `MODEL_PLACEMENT` is `MR` | molecular replacement placed the model, and the R-factors, maps and model written are of that placement |
-| `mr_group` | `MR_SPACE_GROUP` is P 1 21 1, P 32 2 1 and I 2 2 2 respectively, hand included | the model settles the hand, so this is where a wrong enantiomorph shows |
-| `r_free` | `R_FREE` ≤ 0.50 for the two predictions, ≤ 0.35 for 4NWV | a random structure gives 0.586 |
+| `mr_group` | `MR_SPACE_GROUP` is the deposited group, hand included (P 21 21 21, P 21 21 2, P 1 21 1 and I 2 2 2 for `standard`; P 1 21 1 and P 32 2 1 for the two in-house sets) | the model settles the hand, so this is where a wrong enantiomorph shows |
+| `r_free` | `R_FREE` ≤ 0.50 for the predictions, ≤ 0.35 for 4NWV | a random structure gives 0.586 |
 | `confirmed` | `CC_MODEL_CONFIRMED_TO_D_MIN` is not `NONE` | some shell's CC(model, data) clears rugnux's empirical null |
 | `placement_cc` | the placed `p_model.pdb` and the reference model have correlated intensities, ≥ 0.30 | see below |
 
@@ -342,8 +385,9 @@ per step and one column per set. Each `results.json` row carries `bench_verdict`
 `WALL_TIME`, so time spent waiting for a GPU queue is not in it. The run exits with status 1 when a
 set fails a check or crashes. A wrong answer is a failure even when the time is good.
 
-If the site config names a `baseline_timing` (an earlier timing run on this machine), the report is
-compared against it. Otherwise pass `--baseline RUN`. Compare timing runs only from the same machine.
+If the site config names a `baseline_timing` for the benchmark (`{"standard": RUN, "inhouse": RUN}`,
+earlier timing runs on this machine), the report is compared against it. Otherwise pass
+`--baseline RUN`. Compare timing runs only from the same machine and the same benchmark.
 
 ### CPU only
 

licenses/alphafold-db.txt

@@ -1,12 +1,20 @@
-AlphaFold Protein Structure Database predictions (tools/battery/models/)
+AlphaFold Protein Structure Database predictions (tools/battery/models/AF-*)
 
   AF-P68082-F1-model_v6.cif, AF-P68082-F1-predicted_aligned_error_v6.json
       myoglobin, Equus caballus (UniProt P68082), model version 6
   AF-P62894-F1-model_v6.cif, AF-P62894-F1-predicted_aligned_error_v6.json
       cytochrome c, Bos taurus (UniProt P62894), model version 6
+  AF-P17090-F1-model_v6.cif, AF-P17090-F1-predicted_aligned_error_v6.json
+      urease accessory protein UreE, Proteus mirabilis HI4320 (UniProt P17090), model version 6
+  AF-P32322-F1-model_v6.cif, AF-P32322-F1-predicted_aligned_error_v6.json
+      pyrroline-5-carboxylate reductase 1, Homo sapiens (UniProt P32322), model version 6
+  AF-Q65MG6-F1-model_v6.cif, AF-Q65MG6-F1-predicted_aligned_error_v6.json
+      NADH-dependent nitro/flavin oxidoreductase, Bacillus licheniformis ATCC 14580 (UniProt Q65MG6),
+      model version 6
 
 Source: AlphaFold Protein Structure Database, https://alphafold.ebi.ac.uk/ (EMBL-EBI and Google
-DeepMind), downloaded 2026-10-15 from https://alphafold.ebi.ac.uk/files/. The files are unmodified.
+DeepMind), downloaded from https://alphafold.ebi.ac.uk/files/ on 2026-10-15 (P68082, P62894) and
+2026-10-16 (P17090, P32322, Q65MG6). The files are unmodified.
 
 Licence: Creative Commons Attribution 4.0 International (CC BY 4.0),
 https://creativecommons.org/licenses/by/4.0/ - the full legal code follows below.

tools/battery/README.md

@@ -236,8 +236,11 @@ A relative `manifest` path is resolved against `tools/battery/`; every other rel
 lock file, caches, data roots) against the site config's own directory. `gpulock` is a command that
 `rugnux` is prefixed with under `--gpulock`. At PSI it is a wrapper script that queues for one of
 the shared GPU slots. Set it to `null` if you have no such wrapper. `pdb_cache` (optional) is where
-the deposited models and their metadata are cached for `--model` and the REFMAC check. `baseline_timing`
-(optional) is the timing run (`battery.py timing`, below) a new timing run is reported against.
+the deposited models and their metadata are cached for `--model` and the REFMAC check; an entry
+committed in `models/pdb/` (the timing benchmark's) is read from there instead. `baseline_timing`
+(optional) names, per benchmark, the timing run (`battery.py timing`, below) a new timing run is
+reported against: `{"standard": RUN, "inhouse": RUN}`. For the open and in-house arms `manifest` and
+`reference` may be left out (they default to the values above), so an arm can be just its `data_root`.
 `cod_cache` (optional, default `/data/battery/cod_cache`) caches the small molecules' COD structures,
 and `shelxl` (optional; else `shelxl` on `PATH`, e.g. from CCP4) is the SHELXL the small-molecule
 check calls - without one the check is skipped and says so.
@@ -356,30 +359,81 @@ adds 30 rugnux runs, about 6 minutes of rugnux on the RTX 5080 (measured: 6.1 mi
 ### The timing benchmark
 
 `battery.py timing` is the run to repeat on every machine rugnux is measured on (the workstation, the
-4x T4 server, a GB10, a Windows laptop, a Mac): three sets, each started from a model it has to place
+4x T4 server, a GB10, a Windows laptop, a Mac): a few sets, each started from a model it has to place
 by molecular replacement, so one run times the whole pipeline - reading, indexing, integration,
 scaling, the space-group search, MR and model validation - and says whether its answers are right.
 
 ```
-$B timing --rugnux $R [--gpulock] [--threads N] [--label L] [--baseline RUN|none]
+$B timing --rugnux $R [--benchmark standard|inhouse] [--gpulock] [--threads N] [--label L] [--baseline RUN|none]
 ```
 
-| set | arm | start model (`--model`) | reference model it is checked against |
-|---|---|---|---|
-| `myob_x10sa` | inhouse | AlphaFold DB AF-P68082-F1 v6 (horse myoglobin, UniProt P68082), `models/` | 4DC8 |
-| `cytc_x10sa` | inhouse | AlphaFold DB AF-P62894-F1 v6 (bovine cytochrome c, UniProt P62894), `models/` | `reference_model.cif` beside the data |
-| `4nwv` | open | the deposited 4NWV, strict NCS expanded (45 chains), turned 30 deg about (0.36, 0.48, 0.80) through its centroid and shifted (12, -18, 25) A | 4NWV as deposited |
+`timing.json` holds two benchmarks. **`standard`** (the default) needs only open data: three EIGER
+16M rotation sets of ordinary size and quality, the kind of crystallography rugnux is mostly run on,
+and the 4NWV virus capsid as the large case.
 
-The two predictions are committed in `models/` exactly as the AlphaFold DB serves them, each with its
-predicted aligned error under its own name, which rugnux finds beside the model and cuts the
-prediction into rigid domains by. They are CC BY 4.0 (`models/LICENSE.txt`, `THIRD_PARTY_NOTICES.md`).
-The moved 4NWV is not committed (10 MB): the runner makes it from the deposited coordinates in the
-site's `pdb_cache` (fetched from the RCSB on first use, like every open-arm model) into the set's work
-directory as `4nwv_moved.cif.gz`, with the transform in `timing.json`. That is far outside what the
-rigid body corrects, so the model is rejected as read and MR places it. Each set otherwise runs the
-battery's own command for its arm (above), so a timing run needs nothing beyond what the battery
-needs, and a machine needs only those three dataset directories under its data roots
-(`inhouse/myob_x10sa`, `inhouse/cytc_x10sa`, `open/4nwv`) and the usual site config.
+| set | start model (`--model`) | reference model it is checked against | expected MR group |
+|---|---|---|---|
+| `9zlo` | AlphaFold DB AF-P17090-F1 v6 (UreE, *Proteus mirabilis*, UniProt P17090), `models/`; 2 copies to place | 9ZLO as deposited, `models/pdb/` | P 21 21 21 |
+| `9s02` | AlphaFold DB AF-P32322-F1 v6 (PYCR1, human, UniProt P32322; the entry deposits isoform 3), `models/`; 5 copies | 9S02, `models/pdb/` | P 21 21 2 |
+| `9gjx` | AlphaFold DB AF-Q65MG6-F1 v6 (nitroreductase, *Bacillus licheniformis*, UniProt Q65MG6), `models/`; 6 copies | 9GJX, `models/pdb/` | P 1 21 1 |
+| `4nwv` | the deposited 4NWV (`models/pdb/`), strict NCS expanded (45 chains), turned 30 deg about (0.36, 0.48, 0.80) through its centroid and shifted (12, -18, 25) A | 4NWV, `models/pdb/` | I 2 2 2 |
+
+**`inhouse`** is the benchmark as it was first set up, kept so that runs made with it stay comparable:
+two in-house standard proteins and the same 4NWV.
+
+| set | start model (`--model`) | reference model it is checked against | expected MR group |
+|---|---|---|---|
+| `myob_x10sa` | AlphaFold DB AF-P68082-F1 v6 (horse myoglobin, UniProt P68082), `models/` | 4DC8, `models/pdb/` | P 1 21 1 |
+| `cytc_x10sa` | AlphaFold DB AF-P62894-F1 v6 (bovine cytochrome c, UniProt P62894), `models/` | `reference_model.cif` beside the data | P 32 2 1 |
+| `4nwv` | as above | as above | I 2 2 2 |
+
+**Everything but the raw data is in the repository.** The predictions are committed in `models/`
+exactly as the AlphaFold DB serves them, each with its predicted aligned error under its own name,
+which rugnux finds beside the model and cuts the prediction into rigid domains by (CC BY 4.0,
+`models/LICENSE.txt`). The deposited entries the sets are checked against, and 4NWV's start model, are
+committed in `models/pdb/` as the compressed mmCIF the RCSB serves (PDB data are CC0; both in
+`THIRD_PARTY_NOTICES.md`). `model_check.deposition` reads an entry from `models/pdb/` before it looks
+at the site's `pdb_cache`, and its refinement record (R-free, d_min) comes from that file itself, so a
+timing run needs no `pdb_cache` and no network. The moved 4NWV (10 MB) is not committed: the runner
+makes it from `models/pdb/4nwv.cif.gz` into the set's work directory as `4nwv_moved.cif.gz`, with the
+transform in `timing.json`. That is far outside what the rigid body corrects, so the model is rejected
+as read and MR places it. Each set otherwise runs the battery's own command for its arm (above).
+
+**On another machine** only the dataset directories have to be copied, under an open data root:
+
+| directory | size | source (please cite) | master file / first image |
+|---|---:|---|---|
+| `open/9zlo` | 11 GB | Zenodo [10.5281/zenodo.18652652](https://doi.org/10.5281/zenodo.18652652) | `9zlo/ASP0887_09_0114_master.h5` |
+| `open/9s02` | 11 GB | MX Raw Data Repository [10.60884/NRNGS4](https://doi.org/10.60884/NRNGS4) | `9s02/PYCR1-D11_1_master.h5` |
+| `open/9gjx` | 37 GB | IRRMC [10.18430/M39GJX](https://doi.org/10.18430/M39GJX) | `9gjx/BlNTR_9gjx/data/SC6_1_master.h5` |
+| `open/4nwv` | 16 GB | SBGrid Data Bank [10.15785/sbgrid/9](https://doi.org/10.15785/sbgrid/9) | `4nwv/sbgrid9/formal.001` |
+
+75 GB in all (`docs/EXTERNAL_TEST_DATA.md` says what each archive holds; the paths are `open.json`'s
+`input`s). For `--benchmark inhouse` add `inhouse/myob_x10sa` and `inhouse/cytc_x10sa` (from the
+maintainers) and keep `open/4nwv`. The site config then needs only where runs go and the data roots;
+`site.timing.example.json` is that config, and every other key is optional (the two committed arms
+default their `manifest` and `reference`):
+
+```json
+{
+ "runs_root": "/path/to/runs",
+ "arms": {
+  "open": {"data_root": "/path/to/open"}
+ }
+}
+```
+
+```
+python3 tools/battery/battery.py --site /path/to/site.json timing --rugnux /path/to/rugnux
+```
+
+Add `"gpulock"` (a command rugnux is prefixed with) only to use `--gpulock`, and `"inhouse":
+{"data_root": ...}` for the in-house benchmark. Python 3.9+ with `gemmi` and `numpy` (`pip install
+gemmi numpy`) is all the runner needs. macOS runs it as Linux does. Windows has no process groups and
+no `flock`, which a timing run does not need; run it as `python -X utf8 tools\battery\battery.py ...`
+(the report and the console table are UTF-8), with `--rugnux` naming `rugnux.exe` (the run directory
+keeps its own copy, so any DLL it needs must be found on `PATH`). Relative paths in the site config are
+taken from the config's own directory ([Measuring Rugnux](../../docs/RUGNUX_BENCHMARKING.md#setting-up-a-machine)).
 
 A set passes when every check holds (`timing.py`; thresholds in `timing.json`):
 
@@ -387,41 +441,36 @@ A set passes when every check holds (`timing.py`; thresholds in `timing.json`):
 |---|---|
 | `space_group` | the battery's own verdict passes: lattice and space group against the reference, by the scorer's rules (the hand excepted) |
 | `placed` | `MODEL_PLACEMENT` is `MR`: molecular replacement placed the model, and the R-factors, maps and model written are of that placement |
-| `mr_group` | `MR_SPACE_GROUP` is the expected group, hand included (P 1 21 1, P 32 2 1, I 2 2 2): a model decides the hand |
-| `r_free` | `R_FREE` at most 0.50 for the predictions and 0.35 for 4NWV. Measured: 0.44 (myoglobin, AlphaFold), 0.315 (4NWV after MR); the reference models placed by the rigid body give 0.246, 0.356 and 0.297; a random structure 0.586 |
+| `mr_group` | `MR_SPACE_GROUP` is the expected group (tables above), hand included: a model decides the hand |
+| `r_free` | `R_FREE` at most 0.50 for the predictions and 0.35 for 4NWV. Measured: 0.468, 0.371 and 0.384 (9zlo, 9s02, 9gjx), 0.44 (myoglobin), 0.315 (4NWV after MR); a random structure 0.586 |
 | `confirmed` | `CC_MODEL_CONFIRMED_TO_D_MIN` is not `NONE`: some shell's CC(model, data) clears rugnux's empirical null |
-| `placement_cc` | the placed model `p_model.pdb` has the reference model's intensities: the correlation of the two models' \|F\|^2, each shell normalised to its mean, 10-4 A, the best over the changes of basis the two lattices allow, is at least 0.30. \|F\| is the same for every origin and every symmetry copy, so this needs no superposition. Measured: 0.55 (myoglobin, AlphaFold against 4DC8: 0.9 A apart as models), 0.90 (cytochrome c), 0.96 (4NWV); a wrong placement 0.01-0.04, the cytochrome c placement relabelled to the other hand 0.18 |
+| `placement_cc` | the placed model `p_model.pdb` has the reference model's intensities: the correlation of the two models' \|F\|^2, each shell normalised to its mean, 10-4 A, the best over the changes of basis the two lattices allow, is at least 0.30. \|F\| is the same for every origin and every symmetry copy, so this needs no superposition. Measured: 0.61, 0.78 and 0.67 (9zlo, 9s02, 9gjx), 0.55 (myoglobin, AlphaFold against 4DC8: 0.9 A apart as models), 0.90 (cytochrome c), 0.96 (4NWV); a wrong placement 0.01-0.04, the cytochrome c placement relabelled to the other hand 0.18 |
 
 `placement_cc` is independent of rugnux's own scores and is what catches a placement rugnux reports
 as right: the myoglobin placement turned by 30 deg in a copy of its work directory, with rugnux's
 report unchanged, fails it at 0.035, and the moved 4NWV start model at 0.011. A wrong model
 (the cytochrome c prediction on the myoglobin data) is not placed by rugnux and fails `placed`.
+The three standard sets were chosen from the 2026-10-16 AlphaFold MR scan of the open arm, which placed
+all three with every one of these checks passing.
 
 The run is an ordinary run directory (`manifest.json`, `results.json`, `report.md`), so `list`,
-`table`, `compare` and `keep` work on it. Each row carries `bench_verdict`, `bench_reason`,
-`bench_checks` (`{check: [ok, value, limit]}`), `mr_seconds`, `mr_decision` and `steps` (seconds per
-rugnux step, from the `TIMING step` lines of `p_log.txt`); the report opens with a **Timing benchmark**
-section (the checks per set, then the step times), and the console prints both. `wall_s` is rugnux's
-own WALL_TIME. The run exits with status 1 when a set failed a check or crashed. Its report is compared
-against the site's `baseline_timing` (a timing run; none by default). As everywhere, `--gpulock`
-marks the timing "NOT a reference": run it with the machine to itself when the numbers are the point.
+`table`, `compare` and `keep` work on it; its default label is `timing-<benchmark>`. Each row carries
+`bench_verdict`, `bench_reason`, `bench_checks` (`{check: [ok, value, limit]}`), `mr_seconds`,
+`mr_decision` and `steps` (seconds per rugnux step, from the `TIMING step` lines of `p_log.txt`); the
+report opens with a **Timing benchmark** section (the checks per set, then the step times), and the
+console prints both. `wall_s` is rugnux's own WALL_TIME. The run exits with status 1 when a set failed
+a check or crashed. Its report is compared against the site's `baseline_timing` for that benchmark
+(`{"standard": RUN, "inhouse": RUN}`; none by default). As everywhere, `--gpulock` marks the timing
+"NOT a reference": run it with the machine to itself when the numbers are the point.
 
-On another machine: Python 3.9+ with `gemmi` and `numpy` (`pip install gemmi numpy`), a site config
-whose data roots hold the three directories and whose `gpulock` is `null`, and the command above.
-macOS runs it as Linux does. Windows has no process groups and no `flock`, which a timing run does not
-need; run it as `python -X utf8 tools\battery\battery.py ...` (the report and the console table are
-UTF-8), with `--rugnux` naming `rugnux.exe` (the run directory keeps its own copy, so any DLL it needs
-must be found on `PATH`). Both have been run (2026-10-16). Relative paths in the site config are
-taken from the config's own directory, and a `pdb_cache` copied from another machine works as it is
-([Measuring Rugnux](../../docs/RUGNUX_BENCHMARKING.md#setting-up-a-machine)).
-
-Measured on the workstation (RTX 5080, rc175): myoglobin 10 s, cytochrome c 13 s, 4NWV 106 s
-(of which MR 34 s and model validation 8 s; about 16 GB peak resident memory, the most of the
-three by far). Cytochrome c is the hard one: the prediction is a monomer and the crystal holds
-three copies related by a pseudo-translation, and MR places three copies in both P 32 2 1 (R-free
-0.427) and P 31 2 1 (0.437), under the 0.03 R-free margin. It was `MR_DECISION= UNDECIDED` when the
-benchmark was added; with as many copies in both groups, MR now compares them on every reflection
-(paired residuals, [RUGNUX_REPORT ▸ Molecular replacement](../../docs/RUGNUX_REPORT.md#molecular-replacement))
+Measured on the workstation (RTX 5080, rc175): for `standard` 9zlo 19 s, 9s02 30 s, 9gjx 21 s and
+4NWV 89 s (of which MR 17 s; 2026-10-16, under `--gpulock`); for `inhouse` myoglobin 10 s, cytochrome
+c 13 s, 4NWV 106 s (of which MR 34 s and model validation 8 s; about 16 GB peak resident memory, the
+most of the sets by far). Cytochrome c is the hard one: the prediction is a monomer and the
+crystal holds three copies related by a pseudo-translation, and MR places three copies in both P 32 2 1
+(R-free 0.427) and P 31 2 1 (0.437), under the 0.03 R-free margin. It was `MR_DECISION= UNDECIDED`
+when the benchmark was added; with as many copies in both groups, MR now compares them on every
+reflection (paired residuals, [RUGNUX_REPORT ▸ Molecular replacement](../../docs/RUGNUX_REPORT.md#molecular-replacement))
 and decides P 32 2 1.
 
 ### Stress: hunting crashes that do not reproduce

tools/battery/battery.py

@@ -19,11 +19,14 @@ Usage (README.md in this directory is the full guide):
                                                   the wrong-model null on its own, in a run directory of
                                                   its own: exit 1 if a pair failed (CI runs it as its own
                                                   step, after run --tier ci --no-nullmodel)
-  battery.py timing --rugnux BIN [--label L] [--build-dir DIR] [--gpulock] [--threads N] [--baseline RUN|none]
-                                                  the timing benchmark (timing.json): myob_x10sa and cytc_x10sa
-                                                  from their AlphaFold DB models, 4nwv from its deposited model
-                                                  moved away, all placed by molecular replacement; checks each
-                                                  answer and prints wall and step times; exit 1 if one is wrong
+  battery.py timing --rugnux BIN [--benchmark standard|inhouse] [--label L] [--build-dir DIR] [--gpulock]
+                    [--threads N] [--baseline RUN|none]
+                                                  the timing benchmark (timing.json). standard (the default,
+                                                  open data only): 9zlo, 9s02 and 9gjx from their AlphaFold DB
+                                                  models, 4nwv from its deposited model moved away; inhouse:
+                                                  myob_x10sa and cytc_x10sa from AlphaFold DB models, and 4nwv.
+                                                  All placed by molecular replacement; checks each answer and
+                                                  prints wall and step times; exit 1 if one is wrong
   battery.py keep RUN --as workstation|server     copy a finished open/inhouse run's report.md, results.json
                                                   and manifest.json into results/<role>/<run>/ (committed);
                                                   refuses a private run
@@ -85,7 +88,7 @@ import score   # noqa: E402
 import model_check  # noqa: E402  the deposited model (--model); REFMAC check (--model-check)
 import nullmodel  # noqa: E402  a wrong --model changes nothing and is recognised as wrong
 import shelx_check  # noqa: E402  SHELXL refinement of a small molecule's COD structure
-import timing  # noqa: E402  the timing benchmark: three sets placed by molecular replacement
+import timing  # noqa: E402  the timing benchmarks: sets placed by molecular replacement
 import depdata_check  # noqa: E402  our merge vs the depositor's data, shell by shell
 
 PDB_ID = re.compile(r"^[0-9][A-Za-z0-9]{3}$")
@@ -103,6 +106,9 @@ COMMAND_DOC = {
 }
 
 LOCAL_SITE = os.path.join(HERE, "site.json")
+# what a site config may leave out for the two committed arms: then it needs only their data roots
+PUBLIC_ARM_DEFAULTS = {"open": {"manifest": "open.json", "reference": "deposition"},
+                       "inhouse": {"manifest": "inhouse.json", "reference": "xds"}}
 
 
 # ------------------------------------------------------------------ configuration and manifests
@@ -120,7 +126,9 @@ def load_site(path):
     for k in ("runs_root", "lock_file", "pdb_cache", "cod_cache"):
         if site.get(k):
             site[k] = os.path.join(base, site[k])
-    for arm in site["arms"].values():
+    for name, arm in site["arms"].items():
+        for k, v in PUBLIC_ARM_DEFAULTS.get(name, {}).items():
+            arm.setdefault(k, v)
         arm["manifest"] = os.path.join(HERE, arm["manifest"])   # relative to tools/battery/
         if arm.get("data_root"):
             arm["data_root"] = os.path.join(base, arm["data_root"])
@@ -748,10 +756,10 @@ def cmd_nullmodel(a, site):
 
 
 def cmd_timing(a, site):
-    """The timing benchmark (timing.json): its three sets, each with --model <a model it has to place>,
-    in a run directory of its own; prints the checks and the step times, exits 1 when a set failed."""
+    """A timing benchmark (timing.json): its sets, each with --model <a model it has to place>, in a
+    run directory of its own; prints the checks and the step times, exits 1 when a set failed."""
     sets = []
-    for t in timing.load():
+    for t in timing.load(a.benchmark):
         e = select(site, [t["arm"]], t["set"], None)[0]
         e["timing"] = t
         sets.append(e)
@@ -762,9 +770,9 @@ def cmd_timing(a, site):
             "runs_root": a.runs_root or site["runs_root"], "nullmodel": False}
     if a.gpulock and not site.get("gpulock"):
         sys.exit("--gpulock: the site config names no gpulock")
-    baseline = None if a.baseline == "none" else a.baseline or site.get("baseline_timing")
-    run_dir = run_sets(site, sets, sorted({e["arm"] for e in sets}), a.rugnux, a.build_dir, a.label or "timing",
-                       opts, "timing benchmark", baseline)
+    baseline = None if a.baseline == "none" else a.baseline or (site.get("baseline_timing") or {}).get(a.benchmark)
+    run_dir = run_sets(site, sets, sorted({e["arm"] for e in sets}), a.rugnux, a.build_dir,
+                       a.label or f"timing-{a.benchmark}", opts, f"timing benchmark {a.benchmark}", baseline)
     res = json.load(open(os.path.join(run_dir, "results.json")))
     print("\n" + timing.console(res), flush=True)
     bad = [r for r in res if r.get("bench_verdict") != "pass" or r["verdict"] == "crash"]
@@ -1171,10 +1179,12 @@ def main():
     nm.add_argument("--timeout", type=int, default=10800, help="per rugnux run, in s")
     nm.add_argument("--runs-root")
 
-    tb = sub.add_parser("timing", help="the timing benchmark: three sets placed by molecular replacement, "
+    tb = sub.add_parser("timing", help="the timing benchmark: sets placed by molecular replacement, "
                                        "checked and timed per step (timing.json); fail on any wrong answer")
     tb.add_argument("--rugnux", required=True)
-    tb.add_argument("--label", help="short name, part of the run directory's name (default: timing)")
+    tb.add_argument("--benchmark", default="standard", choices=timing.names(),
+                    help="which set list of timing.json: standard (open data only, the default) or inhouse")
+    tb.add_argument("--label", help="short name, part of the run directory's name (default: timing-<benchmark>)")
     tb.add_argument("--build-dir", help="build tree whose CMakeCache.txt records the flags")
     tb.add_argument("--gpulock", action="store_true",
                     help="queue each rugnux behind the site's GPU lock (timing is then not a reference)")
@@ -1182,7 +1192,7 @@ def main():
     tb.add_argument("--timeout", type=int, default=10800, help="per rugnux run, in s")
     tb.add_argument("--extra", default="", help="extra rugnux arguments, applied to every set")
     tb.add_argument("--baseline", help="earlier timing run to report the delta against (default: the "
-                                       "site's baseline_timing; 'none' for no delta)")
+                                       "site's baseline_timing for this benchmark; 'none' for no delta)")
     tb.add_argument("--runs-root")
 
     kp = sub.add_parser("keep", help="copy a run's report.md, results.json and manifest.json into "

tools/battery/model_check.py

@@ -56,6 +56,7 @@ import numpy as np
 DEFAULT_CACHE = "/data/battery/pdb_cache"
 CCP4_SETUP = os.environ.get("CCP4_SETUP", "/opt/xtal/ccp4-9/bin/ccp4.setup-sh")
 RCSB = "https://files.rcsb.org/download/"
+COMMITTED = os.path.join(os.path.dirname(os.path.abspath(__file__)), "models", "pdb")   # the timing benchmark's entries
 PDB_ID = re.compile(r"^[0-9][a-z0-9]{3}$")
 MAX_ATOMS = 200000          # REFMAC time grows with atoms x reflections
 CELL_TOL = 0.05             # Niggli cells compared loosely on purpose; the R score decides
@@ -78,7 +79,12 @@ def fetch(url, dest):
 
 
 def deposition(pdb_id, cache_dir):
-    """Coordinates path and the published refinement statistics, cached as <id>.json."""
+    """Coordinates path and the published refinement statistics: from the copy committed in
+    models/pdb/ where there is one (read from the file itself, no cache and no network), else
+    fetched from the RCSB into cache_dir and cached as <id>.json."""
+    committed = os.path.join(COMMITTED, pdb_id + ".cif.gz")
+    if os.path.exists(committed):
+        return refinement_record(pdb_id, committed)
     os.makedirs(cache_dir, exist_ok=True)
     meta_path = os.path.join(cache_dir, pdb_id + ".json")
     xyz = fetch(RCSB + pdb_id + ".cif.gz", os.path.join(cache_dir, pdb_id + ".cif.gz"))

tools/battery/models/AF-P17090-F1-predicted_aligned_error_v6.json

@@ -0,0 +1 @@
+[{"predicted_aligned_error":[[0,1,2,4,6,8,11,9,9,10,10,11,11,10,12,10,10,13,14,12,12,11,9,7,11,6,10,11,13,12,13,12,12,11,12,9,9,8,6,6,6,6,12,9,9,6,5,2,3,6,5,8,9,13,13,11,11,11,12,11,11,11,10,7,6,4,4,4,6,5,7,6,12,12,12,9,14,14,12,22,16,17,16,19,18,17,17,17,18,18,19,17,18,19,18,16,19,23,17,18,22,21,21,20,17,15,15,15,17,16,18,16,15,18,16,18,20,21,23,22,18,19,23,20,18,20,21,18,19,17,19,20,20,21,18,19,15,14,16,13,15,14,12,17,14,17,17,16,16,17,19,20,19,21,21,23,24,25,25,25,26],[1,0,1,2,3,3,4,5,6,5,6,8,6,7,8,6,7,9,10,11,10,8,6,3,4,3,4,5,6,6,5,5,5,5,4,3,3,3,3,3,4,2,4,4,4,3,3,2,3,2,2,4,4,5,6,4,6,6,7,6,7,6,6,3,3,2,2,2,4,2,4,3,6,5,7,6,7,8,7,10,9,9,9,9,10,10,12,11,11,11,12,10,11,11,10,9,11,11,9,8,10,11,11,9,8,7,9,8,9,9,10,9,8,10,8,9,8,9,11,12,10,10,12,12,10,13,11,11,11,10,13,10,10,10,8,10,8,8,8,7,8,7,7,8,10,10,11,11,12,14,14,16,17,18,21,22,23,23,25,25,25],[1,1,0,1,1,1,2,2,3,4,4,4,4,4,4,5,5,7,8,7,7,6,4,2,2,2,2,3,3,3,3,3,2,3,2,2,2,2,2,2,2,2,3,2,2,2,2,1,1,1,2,2,2,3,3,2,2,2,3,3,3,2,2,2,1,1,2,2,2,2,2,2,2,2,3,3,3,4,4,5,5,5,5,5,6,6,7,7,7,7,7,6,6,6,6,6,6,7,6,5,7,6,7,5,5,4,4,4,4,5,5,5,5,5,5,5,5,5,6,6,6,6,7,7,6,7,7,7,7,6,7,6,6,6,5,5,4,5,4,4,4,4,5,6,7,8,9,10,11,14,16,17,19,20,22,23,24,24,26,26,26],[1,1,1,0,1,1,1,1,2,3,3,2,2,3,3,3,4,5,6,7,6,5,4,2,2,2,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,2,2,2,2,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,4,5,5,5,4,4,4,4,4,4,4,4,4,4,4,4,4,3,3,3,4,4,4,4,4,3,3,4,3,4,4,4,4,4,4,5,4,5,5,5,5,5,5,4,4,4,3,3,3,3,3,3,3,3,4,5,7,8,8,9,11,13,14,16,18,19,20,22,23,23,24,25,24],[2,2,1,1,0,1,2,2,3,4,4,3,4,4,4,4,5,7,8,7,8,6,3,2,2,2,2,2,3,4,3,2,2,2,2,2,2,2,2,2,2,2,2,2,3,2,2,2,1,1,1,1,2,2,2,2,2,2,3,3,3,2,2,1,1,1,2,2,2,2,2,2,2,2,3,3,4,4,4,5,5,5,6,6,6,6,7,7,7,7,6,7,6,7,7,6,6,6,6,5,6,6,6,5,6,5,5,5,5,5,6,6,6,6,6,6,6,5,5,6,6,6,6,7,7,6,7,7,6,7,6,6,5,5,5,4,5,5,4,4,4,4,4,6,8,9,8,10,12,14,16,17,19,18,20,21,23,22,24,25,25],[3,3,2,2,1,0,1,2,2,3,3,4,3,4,4,6,5,8,8,8,8,7,4,4,3,3,3,3,5,5,4,3,3,3,3,3,2,3,3,3,3,4,4,4,4,3,3,3,2,2,2,3,2,3,4,3,3,3,2,3,2,2,2,1,2,3,2,2,3,3,3,3,2,3,3,4,5,5,6,6,6,6,6,6,6,7,7,7,7,7,7,7,7,8,7,7,7,7,7,6,7,8,7,6,7,6,6,5,5,5,5,6,6,6,6,7,7,6,7,8,7,7,7,8,7,7,8,8,7,7,7,7,6,7,6,5,5,5,5,4,5,4,5,6,8,9,8,9,11,13,14,15,17,18,20,19,22,21,23,24,24],[3,2,2,1,1,1,0,1,1,2,2,2,2,2,3,3,3,5,7,6,6,5,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,2,2,2,2,1,1,2,2,2,2,2,1,2,2,1,1,1,1,1,2,2,2,2,2,1,1,1,1,2,2,3,3,4,4,4,4,4,4,4,5,6,6,5,6,6,6,5,5,6,5,5,5,5,4,5,5,5,4,4,4,4,4,4,5,5,5,4,4,4,4,4,5,5,5,5,6,5,6,6,6,6,6,6,6,5,5,5,5,4,3,4,4,3,2,3,3,3,5,6,8,8,10,11,14,15,16,19,20,21,22,22,22,23,24,23],[3,3,2,2,2,1,1,0,1,2,2,2,2,3,4,3,4,6,7,7,7,7,4,3,2,2,3,3,3,3,3,3,3,2,2,2,2,2,3,3,3,3,4,3,4,3,3,3,2,2,2,2,2,2,3,2,2,2,2,2,2,1,1,1,1,2,2,3,2,2,2,2,2,2,3,3,3,4,4,4,6,6,6,6,7,7,8,7,8,8,7,8,8,7,7,7,7,7,7,7,7,6,6,5,5,5,5,5,6,5,6,6,6,6,6,5,5,6,6,6,7,7,6,8,8,7,7,7,7,8,7,7,7,6,6,4,5,6,4,4,4,4,4,6,7,8,8,9,12,14,15,16,17,18,20,20,22,22,24,25,26],[6,5,4,4,3,2,2,1,0,1,1,1,2,2,3,4,4,5,7,8,7,6,4,4,4,4,5,5,6,6,6,6,5,5,6,4,5,6,6,7,7,6,6,6,6,7,5,6,6,4,3,4,3,5,5,4,4,4,4,4,3,3,2,3,3,3,3,4,3,4,3,3,3,4,4,6,7,7,7,9,10,10,9,9,10,9,10,10,11,12,10,10,11,10,11,10,10,10,10,8,10,9,10,9,9,9,8,8,8,9,9,9,8,9,9,9,10,10,9,10,10,10,11,11,11,12,12,11,12,10,11,10,10,10,10,9,8,7,7,6,6,6,6,8,9,9,10,11,13,16,16,17,18,20,21,22,23,24,25,26,26],[7,6,6,5,4,4,3,2,1,0,1,2,1,1,2,3,3,5,7,8,7,8,6,4,5,5,6,6,7,6,6,6,7,6,6,5,6,6,7,7,7,7,7,7,8,6,6,7,6,7,5,5,5,5,5,4,4,5,4,4,3,3,2,3,3,5,4,4,4,4,3,3,4,5,5,7,7,9,8,9,10,9,9,10,10,11,11,11,10,11,11,11,11,11,10,10,9,10,9,9,8,9,8,8,8,9,8,9,8,7,8,8,9,9,8,9,10,9,10,9,9,11,11,12,11,13,12,12,12,12,12,11,12,10,11,10,11,9,7,6,7,7,7,8,10,9,10,11,13,16,16,17,20,20,22,23,23,24,24,25,25],[7,7,6,7,5,4,3,3,2,1,0,1,1,1,2,3,3,4,6,8,8,7,5,4,5,6,6,6,6,6,6,7,7,6,7,7,8,7,8,8,7,8,7,8,7,8,7,8,8,6,6,6,6,7,6,6,5,4,4,4,4,3,4,3,4,5,5,5,4,4,4,4,4,4,6,6,8,8,7,8,10,9,10,9,11,9,11,10,11,12,11,10,11,11,9,10,10,9,9,9,9,10,9,8,9,8,9,8,8,7,9,8,8,10,8,9,10,9,9,10,10,11,11,11,10,12,11,11,11,10,10,10,10,9,10,8,8,9,6,6,6,6,8,7,10,9,12,12,14,18,19,19,18,22,22,23,24,24,25,25,26],[5,6,5,4,4,3,2,2,2,1,1,0,1,1,2,2,2,4,6,8,7,8,5,4,5,4,5,7,6,7,6,6,7,6,6,6,6,5,6,6,6,7,6,6,6,5,5,6,5,6,5,6,6,5,6,6,5,5,4,4,3,3,3,3,3,5,4,4,4,3,3,3,4,4,6,5,7,8,7,8,9,9,10,8,10,9,11,10,11,10,10,10,11,10,10,11,9,10,10,10,10,10,9,8,10,8,10,7,9,9,10,9,8,9,8,9,9,10,9,10,11,11,11,13,10,12,11,11,11,10,10,10,11,10,10,8,10,10,7,7,7,6,8,8,10,10,11,12,14,17,17,18,18,20,21,21,24,23,26,26,27],[7,6,6,4,4,4,3,2,1,1,1,1,0,1,1,1,3,4,6,9,8,7,6,5,5,5,8,8,7,7,7,7,8,6,6,5,6,7,7,9,7,7,6,7,7,6,5,7,8,6,6,6,5,6,5,6,5,6,5,5,4,3,3,4,3,5,5,5,4,4,3,4,4,5,6,7,7,8,8,8,10,9,9,9,9,9,9,10,11,9,11,11,10,10,11,10,9,10,10,10,10,11,9,9,10,9,9,8,10,9,8,8,9,8,9,9,10,11,10,11,11,12,11,13,11,12,11,11,12,11,10,10,10,10,9,8,9,9,7,8,7,7,8,9,9,10,11,12,14,16,16,18,20,22,23,23,24,24,26,26,26],[7,9,6,6,5,5,3,4,2,1,1,1,1,0,1,2,2,4,7,8,7,8,5,5,5,6,6,6,6,6,6,6,6,6,6,6,6,6,7,8,7,9,7,7,7,7,6,9,7,7,6,6,7,6,6,6,5,5,4,5,4,3,3,3,4,6,5,5,4,4,4,4,4,4,6,6,7,8,8,8,10,9,10,9,11,10,11,12,13,11,11,12,11,11,10,11,9,9,9,9,10,10,8,8,8,8,10,9,10,9,9,9,8,Line truncated

tools/battery/models/AF-P32322-F1-predicted_aligned_error_v6.json

@@ -0,0 +1 @@
+[{"predicted_aligned_error":[[0,1,1,2,4,4,6,4,6,5,4,5,4,4,5,4,4,5,4,5,6,5,5,4,4,3,3,3,2,2,2,5,5,5,5,5,6,5,5,5,5,4,5,5,5,5,6,4,4,4,3,4,4,4,4,4,4,3,6,5,4,2,2,2,2,3,4,4,5,5,6,6,6,6,5,6,5,6,5,5,6,6,6,6,5,4,5,4,3,3,3,7,4,5,6,5,6,8,6,6,7,7,7,6,6,7,7,6,6,6,7,6,7,6,5,6,6,7,6,6,7,5,6,6,6,7,5,7,7,8,7,6,7,6,5,7,7,6,7,7,7,6,6,3,4,5,4,3,4,5,4,4,5,6,5,6,6,7,6,7,7,7,8,7,8,7,6,8,8,7,7,8,8,7,7,6,8,7,7,8,9,8,8,9,9,10,10,9,10,11,10,10,12,10,12,11,10,11,10,11,11,9,8,10,10,8,9,10,10,8,10,11,8,9,10,9,9,9,9,9,12,9,8,10,10,11,11,10,13,11,12,10,12,11,11,11,11,11,11,12,12,11,11,12,10,12,11,10,10,9,12,9,9,10,10,9,8,9,9,8,8,9,8,8,6,8,7,7,7,8,7,7,9,10,9,11,11,13,16,15,13,14,16,16,15,16,19,19,19,19,22,23,23,24,25,26,27,27,27,27,28,28,28,29,29,29,29,28,28,29,29,29,29,29,29,29,29,29,29],[1,0,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,2,2,2,3,3,3,3,3,2,2,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,1,2,1,1,1,1,1,1,2,2,2,2,2,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,1,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,2,2,2,2,2,2,2,2,2,3,3,3,2,2,3,3,3,2,3,3,2,2,2,2,2,3,2,2,3,3,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,6,5,6,5,5,6,5,5,5,5,4,5,5,4,4,4,5,4,4,4,4,4,4,5,4,4,4,5,5,5,4,5,5,5,6,5,6,5,5,6,7,7,6,6,6,6,5,6,5,5,6,6,5,5,6,5,5,5,4,5,4,4,4,4,4,4,4,4,4,4,4,3,3,4,4,3,4,4,4,4,5,5,6,7,8,9,10,11,12,13,13,13,15,14,16,15,16,17,19,20,22,23,24,26,27,28,28,28,29,29,29,30,29,30,30,29,29,30,30,30,30,30,30,30,30,29,29],[1,1,0,1,1,1,1,1,1,2,2,1,1,2,1,1,1,1,1,1,2,2,2,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,1,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,1,1,2,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,1,2,2,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,2,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,4,4,4,4,4,4,4,3,3,4,3,3,3,4,3,3,3,3,3,3,4,3,3,3,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,4,4,5,4,4,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,5,5,6,9,9,10,11,10,12,13,13,14,14,17,15,17,18,20,20,21,23,24,25,27,27,27,27,28,28,29,29,29,29,29,28,28,29,29,29,29,29,29,29,29,29,28],[1,1,1,0,1,1,1,1,1,2,1,1,1,2,1,1,1,2,1,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,1,2,2,1,2,2,2,1,2,2,2,1,2,2,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,4,3,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,5,5,4,4,5,5,5,5,5,4,4,4,4,4,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,3,3,3,3,3,3,3,4,4,5,7,9,10,11,12,11,10,12,11,14,13,15,14,14,18,18,19,21,23,24,25,27,27,27,28,29,28,29,29,29,29,29,29,29,29,29,29,29,29,29,29,29,29,28],[1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,1,1,1,2,2,1,1,2,2,1,2,2,2,2,2,2,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,1,2,1,1,1,2,2,2,2,2,2,2,1,1,2,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,4,3,3,3,3,3,4,3,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,3,3,3,4,4,4,4,4,4,4,5,5,5,4,4,4,4,4,4,4,4,3,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,3,2,3,3,3,4,4,5,6,7,10,11,11,11,11,11,10,12,13,15,14,15,20,19,20,21,24,24,25,26,27,27,28,28,28,28,29,29,29,29,29,29,29,29,29,30,30,30,30,30,29,29],[2,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,2,2,2,2,1,1,2,2,1,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,1,1,1,2,1,1,1,1,1,1,1,1,2,2,1,2,1,2,2,1,2,2,2,2,2,2,2,2,2,2,2,1,1,1,1,2,2,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,3,3,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,4,4,4,4,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,3,3,2,3,3,3,3,4,5,7,8,8,9,10,9,9,10,10,12,12,14,13,13,18,18,19,21,23,25,25,27,28,28,28,28,29,29,29,29,29,29,29,29,29,30,30,30,30,30,30,30,29,29],[2,2,2,1,1,1,0,1,1,1,2,1,1,2,2,2,2,2,2,2,2,2,3,2,2,2,2,2,2,2,1,1,1,1,1,2,2,2,2,2,1,1,2,2,2,2,2,2,2,2,2,1,1,1,2,1,2,2,1,1,2,2,2,2,2,1,1,1,1,1,1,2,2,1,1,2,2,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,1,2,2,2,3,2,2,2,3,2,2,2,3,2,2,2,3,2,2,3,3,2,2,2,2,2,2,2,3,2,2,2,2,2,2,3,3,3,3,2,2,2,2,3,2,3,2,3,2,3,3,2,3,3,2,2,3,3,2,2,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,3,3,3,3,4,3,3,4,4,4,4,4,4,4,5,5,5,5,5,5,5,6,6,6,6,5,4,4,5,5,4,4,5,4,4,4,4,4,4,5,4,4,4,5,5,5,4,5,6,5,5,5,6,6,6,6,7,7,6,6,7,7,8,8,7,5,6,7,6,6,7,6,5,6,6,6,6,5,5,5,5,4,5,5,4,4,5,4,3,3,4,3,3,3,4,4,3,4,4,4,4,5,7,7,8,9,9,10,9,10,10,11,12,13,14,14,16,20,19,20,22,23,24,25,27,27,27,27,28,28,29,29,29,29,29,28,29,30,30,30,30,30,30,30,30,29,29],[2,2,2,2,1,1,1,0,1,1,1,1,1,1,2,2,2,2,2,3,2,2,3,3,3,3,3,3,2,2,2,2,1,2,2,2,2,2,2,2,1,1,2,1,2,2,2,2,2,2,2,1,2,2,2,1,2,2,2,2,2,3,3,2,2,2,1,1,1,1,2,2,2,2,2Line truncated

tools/battery/models/AF-Q65MG6-F1-predicted_aligned_error_v6.json

@@ -0,0 +1 @@
+[{"predicted_aligned_error":[[0,1,2,5,3,3,7,8,6,8,12,9,12,19,15,16,19,18,17,19,19,20,19,21,21,22,21,23,18,21,21,20,22,17,21,18,16,17,23,20,16,20,24,17,21,21,22,22,24,22,24,23,21,24,21,23,23,21,24,23,24,24,22,23,23,23,21,22,24,23,20,20,21,21,20,22,20,21,22,23,21,21,22,21,21,19,20,21,22,24,22,23,23,23,23,23,23,23,24,24,23,24,23,24,23,24,23,24,24,23,24,22,24,23,23,24,22,22,23,23,23,24,23,23,23,24,24,23,23,24,22,23,24,23,22,23,24,23,23,23,24,24,24,23,22,24,23,24,23,23,25,21,21,19,19,18,17,21,16,15,19,19,18,18,12,12,16,19,18,17,19,19,22,23,20,22,22,25,24,23,23,24,24,25,23,23,23,24,24,24,21,25,23,23,24,23,22,22,24,22,20,17,21,18,20,20,21,20,20,18,18,17,19,18,18,21,19,19,17,18,19,20,18,18,17,17,17],[1,0,1,2,2,2,3,4,4,5,6,7,6,10,10,10,9,12,8,10,9,9,10,10,11,9,12,10,12,13,11,11,11,10,12,10,9,11,11,9,10,13,12,9,13,11,12,13,14,12,14,11,11,13,12,12,12,11,13,13,13,13,13,12,13,13,12,12,13,13,11,11,11,10,11,11,10,11,11,13,12,13,12,10,13,10,13,12,12,13,13,13,14,14,14,13,13,13,14,14,15,14,13,14,14,13,13,14,14,13,14,13,15,14,14,15,14,13,13,14,13,14,14,13,13,16,15,14,13,15,13,14,14,14,13,14,14,14,13,14,15,14,13,13,13,13,13,13,12,12,13,11,12,10,11,10,9,12,10,9,8,10,9,9,7,9,9,11,11,10,10,10,12,12,13,13,13,13,13,13,14,15,14,14,13,14,13,14,14,13,13,14,14,14,14,14,13,13,12,12,11,10,10,11,11,12,12,10,10,12,11,10,13,12,11,11,13,13,8,11,12,10,11,10,11,9,10],[2,1,0,1,1,1,1,2,2,2,2,2,4,3,3,4,6,5,6,5,5,5,7,6,7,6,6,6,6,7,6,7,7,6,8,6,5,6,6,5,7,7,8,6,8,8,9,11,8,10,7,8,9,8,8,8,8,7,9,9,8,8,8,8,8,8,8,8,8,8,7,7,7,6,6,6,7,7,7,7,8,9,7,7,7,7,7,7,7,8,8,8,9,8,9,9,9,10,9,10,9,9,9,9,9,10,9,10,8,9,11,10,13,11,11,13,9,10,11,10,9,10,10,10,10,14,10,9,10,11,8,9,10,10,9,8,9,10,10,9,9,8,8,9,8,8,9,8,8,7,8,7,7,7,6,8,6,5,6,6,5,4,5,4,3,5,5,5,5,6,6,7,7,7,7,8,7,8,8,8,9,8,8,9,9,9,9,9,9,9,8,10,10,8,9,9,8,8,7,7,7,6,7,6,7,6,5,6,5,5,6,5,5,5,4,6,5,6,5,6,5,8,7,6,5,6,5],[4,1,1,0,1,1,1,1,1,2,2,2,2,3,2,3,3,3,4,4,4,5,5,5,6,5,6,5,5,6,6,6,4,4,5,5,4,4,4,4,4,5,5,4,7,5,6,6,6,6,6,6,6,6,6,6,6,6,6,7,7,7,7,7,7,8,7,6,8,8,6,7,5,5,6,5,5,6,6,5,6,6,5,5,6,6,6,6,5,6,7,7,9,7,7,8,8,8,9,9,8,8,8,8,7,7,8,7,6,6,7,7,8,8,8,8,8,8,7,9,8,9,9,8,8,9,8,8,7,8,7,8,8,7,7,7,8,9,8,9,9,7,7,7,7,6,6,6,6,5,5,5,5,5,5,5,5,5,4,4,4,4,4,3,3,4,3,4,4,4,5,5,5,5,6,7,7,7,7,8,7,7,8,8,8,8,8,8,9,7,7,8,9,9,8,7,7,6,5,5,5,5,5,5,5,5,5,4,4,4,5,4,4,4,4,4,5,5,4,5,5,5,5,5,5,5,5],[3,2,1,1,0,1,2,1,1,2,2,2,2,2,3,3,3,4,4,4,5,5,6,6,6,6,6,5,5,6,5,5,4,4,5,5,3,4,4,4,4,5,5,4,6,5,6,7,8,6,6,6,5,5,6,6,6,6,6,7,7,7,7,6,7,8,7,6,8,8,6,6,6,6,6,6,5,6,6,6,6,5,5,5,6,6,6,6,5,6,7,8,9,7,7,8,8,8,9,8,8,8,8,8,7,7,8,7,7,7,9,8,10,8,8,9,8,8,8,8,9,9,10,9,8,9,8,9,8,9,7,9,9,8,7,8,8,9,8,9,9,8,8,7,7,7,7,6,6,6,6,5,5,5,5,5,4,4,4,3,3,3,3,2,3,3,3,3,4,4,4,5,6,6,6,7,7,7,8,8,7,8,8,8,7,9,9,8,8,6,7,8,9,8,8,6,7,6,6,6,5,5,5,4,5,5,5,5,5,5,5,5,5,5,5,5,6,5,6,5,6,6,7,6,7,6,6],[3,1,1,1,1,0,1,1,1,1,1,1,1,1,2,2,2,3,3,3,3,3,4,4,4,4,4,4,4,4,4,3,4,4,4,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,4,4,5,5,4,4,5,5,4,4,4,3,4,4,3,4,4,4,4,4,4,4,3,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,4,5,5,5,4,5,5,5,6,6,6,6,5,5,5,5,5,6,6,6,5,5,5,5,5,5,4,5,5,5,4,4,4,5,5,5,5,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,2,2,3,2,2,2,3,2,2,2,3,3,3,3,3,3,4,3,4,4,4,4,5,5,4,4,5,5,4,5,5,5,5,5,5,5,6,5,5,4,4,4,4,3,4,3,4,3,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4],[3,1,1,1,1,1,0,1,1,1,1,1,1,1,1,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,3,3,3,3,3,4,3,3,4,4,4,4,4,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,4,5,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,5,4,5,5,5,5,5,5,5,5,5,4,4,5,5,4,5,5,5,5,4,4,5,4,5,5,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,4,4,4,4,4,4,4,5,4,4,4,5,4,4,4,4,4,5,4,4,4,4,4,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,2,2,2,3,2,2,3,3,3,3,3,3,3,3],[5,2,1,1,1,1,1,0,1,1,1,1,1,1,1,1,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,3,4,4,3,3,3,4,4,4,4,3,4,4,4,3,4,4,4,4,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,3,4,4,4,5,5,4,5,5,5,5,5,5,4,5,5,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,5,5,4,5,4,5,4,5,5,4,4,4,4,5,5,5,5,4,4,4,4,3,4,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,4,4,4,4,4,4,4,5,4,4,5,5,4,5,4,4,5,5,5,4,4,4,4,4,3,3,3,3,2,2,2,2,2,3,2,2,3,2,2,2,2,3,3,2,3,3,3,3,3,3,3,3],[4,2,1,1,1,1,1,1,0,1,1,1,1,1,1,1,1,2,2,2,2,3,3,3,3,3,3,3,2,3,3,3,2,2,3,2,2,2,2,2,2,2,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,3,4,4,3,3,4,4,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,4,4,4,4,4,4,4,5,4,4,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,5,4,5,4,5,4,5,5,5,5,4,4,5,4,5,4,5,5,4,4,4,4,5,4,4,4,4,4,4,3,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,2,2,2,3,3,3,4,4,3,4,4,4,4,4,4,4,4,5,4,4,4,4,4,5,4,4,3,4,3,3,3,3,3,3,2,2,2,2,3,3,2,2,3,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4],[3,2,1,1,1,1,1,1,1,0,1,1,1,1,1,1,1,2,2,2,2,2,2,2,3,3,3,2,2,2,2,3,2,2,3,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,3,3,2,3,3,3,2,2,2,2,2,3,3,3,3,3,3,3,4,3,3,3,4,4,4,4,3,3,4,4,3,4,4,4,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,3,4,4,4,3,3,3,4,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,2,2,2,2,1,2,2,1,1,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4,4,3,4,3,3,4,4,4,3,3,3,3,3,3,2,2,2,2,2,2,2,2,2,2,2,Line truncated

tools/battery/models/LICENSE.txt

@@ -1,12 +1,20 @@
-AlphaFold Protein Structure Database predictions (tools/battery/models/)
+AlphaFold Protein Structure Database predictions (tools/battery/models/AF-*)
 
   AF-P68082-F1-model_v6.cif, AF-P68082-F1-predicted_aligned_error_v6.json
       myoglobin, Equus caballus (UniProt P68082), model version 6
   AF-P62894-F1-model_v6.cif, AF-P62894-F1-predicted_aligned_error_v6.json
       cytochrome c, Bos taurus (UniProt P62894), model version 6
+  AF-P17090-F1-model_v6.cif, AF-P17090-F1-predicted_aligned_error_v6.json
+      urease accessory protein UreE, Proteus mirabilis HI4320 (UniProt P17090), model version 6
+  AF-P32322-F1-model_v6.cif, AF-P32322-F1-predicted_aligned_error_v6.json
+      pyrroline-5-carboxylate reductase 1, Homo sapiens (UniProt P32322), model version 6
+  AF-Q65MG6-F1-model_v6.cif, AF-Q65MG6-F1-predicted_aligned_error_v6.json
+      NADH-dependent nitro/flavin oxidoreductase, Bacillus licheniformis ATCC 14580 (UniProt Q65MG6),
+      model version 6
 
 Source: AlphaFold Protein Structure Database, https://alphafold.ebi.ac.uk/ (EMBL-EBI and Google
-DeepMind), downloaded 2026-10-15 from https://alphafold.ebi.ac.uk/files/. The files are unmodified.
+DeepMind), downloaded from https://alphafold.ebi.ac.uk/files/ on 2026-10-15 (P68082, P62894) and
+2026-10-16 (P17090, P32322, Q65MG6). The files are unmodified.
 
 Licence: Creative Commons Attribution 4.0 International (CC BY 4.0),
 https://creativecommons.org/licenses/by/4.0/ - the full legal code follows below.

tools/battery/results/timing_platforms/timing_table.py

@@ -4,7 +4,8 @@
     python3 -I timing_table.py platforms.json [-o out.md]
 
 platforms.json: {"title", "intro", "timing_json", "stands_out": [str], "platforms": [entry]}.
-"timing_json" is a copy of tools/battery/timing.json (path relative to platforms.json), for the data
+"timing_json" is a copy of tools/battery/timing.json (path relative to platforms.json) and "benchmark"
+(default inhouse) the benchmark in it the platforms ran, for the data
 section. An entry is {"platform" (short name + CPU), "gpu", "role", "conditions", "os", "build",
 "battery_cmd", "notes": [str], "peak_mem": {set: str}, "table2_note"} plus ONE source:
   "run":     a battery run directory (results.json + manifest.json; logs under logs/<set>/ or
@@ -205,7 +206,7 @@ def main():
           "is started from a model it has to place by molecular replacement:", "",
           "| set | arm | start model (--model) | expected: MR group (sgno), R_free max, placement CC min |",
           "|:--|:--|:--|:--|"]
-    for t in tj["sets"]:
+    for t in tj["benchmarks"][cfg.get("benchmark", "inhouse")]:
         st = t["start"]
         if "alphafold" in st:
             sm = f"`tools/battery/{st['alphafold']}`: {st['why']}"

tools/battery/site.timing.example.json

@@ -0,0 +1,7 @@
+{
+ "_comment": "The least a site config needs for battery.py timing (README: 'The timing benchmark'): where run directories go and the open arm's data root, holding 9zlo/, 9s02/, 9gjx/ and 4nwv/ for --benchmark standard. Add an inhouse data root (myob_x10sa/, cytc_x10sa/) for --benchmark inhouse, and gpulock for --gpulock. Relative paths are taken from this file's directory. Models and references are in the repository: no pdb_cache, no network.",
+ "runs_root": "/path/to/runs",
+ "arms": {
+  "open": {"data_root": "/path/to/open"}
+ }
+}

tools/battery/timing.json

@@ -1,17 +1,16 @@
 {
- "about": "Timing benchmark (README: 'The timing benchmark'): three sets, each started from a model rugnux has to place by molecular replacement, timed per step and checked for a right answer. 'start' is the model given to --model: an AlphaFold DB prediction committed in models/ (with its predicted aligned error beside it), or a deposited entry from the site's pdb_cache moved by a fixed rigid-body transform (strict NCS expanded first, then x' = R (x - c) + c + t, R a rotation by 'deg' about the unit 'axis', c the centroid of the expanded asymmetric unit, t 'shift' in A). The reference model the placement is checked against is the one the battery itself uses for the set (its manifest row's 'model', else the PDB entry of the set id). 'expect': 'mr_sgno' the space group the placed model is in (the model decides the hand), 'rfree_max' the highest R_FREE accepted, 'cc_min' the lowest correlation of the placed model's intensities with the reference model's (shell-normalised, 10-4 A, best over the lattice's reindexing operators).",
- "sets": [
-  {"set": "myob_x10sa", "arm": "inhouse",
-   "start": {"alphafold": "models/AF-P68082-F1-model_v6.cif",
-             "why": "AlphaFold DB AF-P68082-F1 v6: myoglobin, Equus caballus (UniProt P68082), the species of the battery's reference model 4DC8 (horse heart). No heme"},
-   "expect": {"mr_sgno": 4, "rfree_max": 0.50, "cc_min": 0.30}},
-  {"set": "cytc_x10sa", "arm": "inhouse",
-   "start": {"alphafold": "models/AF-P62894-F1-model_v6.cif",
-             "why": "AlphaFold DB AF-P62894-F1 v6: cytochrome c, Bos taurus (UniProt P62894), the protein of the battery's reference model (bovine heart, placed with 6FF5). No heme; three copies to place, related by a pseudo-translation"},
-   "expect": {"mr_sgno": 154, "rfree_max": 0.50, "cc_min": 0.30}},
-  {"set": "4nwv", "arm": "open",
-   "start": {"moved": {"pdb": "4nwv", "axis": [0.36, 0.48, 0.80], "deg": 30.0, "shift": [12.0, -18.0, 25.0]},
-             "why": "the deposited model of 4NWV (I222, a virus capsid: 3 chains and 14 strict NCS operators, 45 chains once expanded), turned 30 deg and shifted 33 A: far outside what the rigid body corrects, so it is rejected and molecular replacement places the whole asymmetric unit"},
-   "expect": {"mr_sgno": 23, "rfree_max": 0.35, "cc_min": 0.30}}
- ]
+  "about": "Timing benchmarks (README: 'The timing benchmark'), chosen with battery.py timing --benchmark NAME: each set started from a model rugnux has to place by molecular replacement, timed per step and checked for a right answer. 'standard' needs only open data (the open arm's 9zlo, 9s02, 9gjx and 4nwv); 'inhouse' is the earlier set of two in-house standard proteins and 4nwv. 'start' is the model given to --model: an AlphaFold DB prediction committed in models/ (with its predicted aligned error beside it), or a deposited entry (committed in models/pdb/) moved by a fixed rigid-body transform (strict NCS expanded first, then x' = R (x - c) + c + t, R a rotation by 'deg' about the unit 'axis', c the centroid of the expanded asymmetric unit, t 'shift' in A). The reference model the placement is checked against is the one the battery itself uses for the set (its manifest row's 'model', else the PDB entry of the set id; committed in models/pdb/ for every set here but cytc_x10sa, whose reference_model.cif sits beside its data). 'expect': 'mr_sgno' the space group the placed model is in (the model decides the hand), 'rfree_max' the highest R_FREE accepted, 'cc_min' the lowest correlation of the placed model's intensities with the reference model's (shell-normalised, 10-4 A, best over the lattice's reindexing operators).",
+  "benchmarks": {
+   "standard": [
+    {"set": "9zlo", "arm": "open", "start": {"alphafold": "models/AF-P17090-F1-model_v6.cif", "why": "AlphaFold DB AF-P17090-F1 v6: urease accessory protein UreE, Proteus mirabilis (UniProt P17090), the deposited entity of 9ZLO; two copies to place (P 21 21 21, 2.0 A, EIGER 16M)"}, "expect": {"mr_sgno": 19, "rfree_max": 0.5, "cc_min": 0.3}},
+    {"set": "9s02", "arm": "open", "start": {"alphafold": "models/AF-P32322-F1-model_v6.cif", "why": "AlphaFold DB AF-P32322-F1 v6: pyrroline-5-carboxylate reductase 1, Homo sapiens (UniProt P32322; 9S02 deposits isoform 3); five copies to place (P 21 21 2, 1.65 A, EIGER2 16M)"}, "expect": {"mr_sgno": 18, "rfree_max": 0.5, "cc_min": 0.3}},
+    {"set": "9gjx", "arm": "open", "start": {"alphafold": "models/AF-Q65MG6-F1-model_v6.cif", "why": "AlphaFold DB AF-Q65MG6-F1 v6: NADH-dependent nitro/flavin oxidoreductase, Bacillus licheniformis (UniProt Q65MG6), the deposited entity of 9GJX; six copies to place (P 1 21 1, 2.4 A, EIGER 16M)"}, "expect": {"mr_sgno": 4, "rfree_max": 0.5, "cc_min": 0.3}},
+    {"set": "4nwv", "arm": "open", "start": {"moved": {"pdb": "4nwv", "axis": [0.36, 0.48, 0.8], "deg": 30.0, "shift": [12.0, -18.0, 25.0]}, "why": "the deposited model of 4NWV (I222, a virus capsid: 3 chains and 14 strict NCS operators, 45 chains once expanded), turned 30 deg and shifted 33 A: far outside what the rigid body corrects, so it is rejected and molecular replacement places the whole asymmetric unit"}, "expect": {"mr_sgno": 23, "rfree_max": 0.35, "cc_min": 0.3}}
+   ],
+   "inhouse": [
+    {"set": "myob_x10sa", "arm": "inhouse", "start": {"alphafold": "models/AF-P68082-F1-model_v6.cif", "why": "AlphaFold DB AF-P68082-F1 v6: myoglobin, Equus caballus (UniProt P68082), the species of the battery's reference model 4DC8 (horse heart). No heme"}, "expect": {"mr_sgno": 4, "rfree_max": 0.5, "cc_min": 0.3}},
+    {"set": "cytc_x10sa", "arm": "inhouse", "start": {"alphafold": "models/AF-P62894-F1-model_v6.cif", "why": "AlphaFold DB AF-P62894-F1 v6: cytochrome c, Bos taurus (UniProt P62894), the protein of the battery's reference model (bovine heart, placed with 6FF5). No heme; three copies to place, related by a pseudo-translation"}, "expect": {"mr_sgno": 154, "rfree_max": 0.5, "cc_min": 0.3}},
+    {"set": "4nwv", "arm": "open", "start": {"moved": {"pdb": "4nwv", "axis": [0.36, 0.48, 0.8], "deg": 30.0, "shift": [12.0, -18.0, 25.0]}, "why": "the deposited model of 4NWV (I222, a virus capsid: 3 chains and 14 strict NCS operators, 45 chains once expanded), turned 30 deg and shifted 33 A: far outside what the rigid body corrects, so it is rejected and molecular replacement places the whole asymmetric unit"}, "expect": {"mr_sgno": 23, "rfree_max": 0.35, "cc_min": 0.3}}
+   ]
+  }
 }

tools/battery/timing.py

@@ -1,11 +1,12 @@
-"""The timing benchmark: three sets, each started from a model rugnux has to place by molecular
+"""The timing benchmarks: a few sets, each started from a model rugnux has to place by molecular
 replacement, timed per step and checked for a right answer.
 
-timing.json lists the sets (README, 'The timing benchmark'). Two are in-house standard proteins
-started from their AlphaFold DB prediction (models/, committed with its predicted aligned error);
-one is an open-arm entry started from its deposited model moved by a fixed rigid-body transform,
-made at run time from the site's pdb_cache (`start_model`). Each runs the battery's own command for
-its arm with --model <start model>. A set passes (`check`) when every one of these holds:
+timing.json lists the sets of each benchmark (README, 'The timing benchmark'): 'standard', three
+open-arm EIGER 16M sets and 4nwv, needs nothing but the open data; 'inhouse' is two in-house standard
+proteins and 4nwv. A set is started from its AlphaFold DB prediction (models/, committed with its
+predicted aligned error) or from its deposited model moved by a fixed rigid-body transform, made at
+run time from the entry committed in models/pdb/ (`start_model`). Each runs the battery's own command
+for its arm with --model <start model>. A set passes (`check`) when every one of these holds:
 
   space_group   the battery's own verdict (score.judge: lattice and space group against the
                 reference, the hand excepted) is a pass
@@ -41,8 +42,12 @@ CC_SHELLS = 8                      # equal-count shells for the normalisation
 STEP_LINE = re.compile(r"TIMING step (.+?)(?:, \d+ images)? ([\d.]+)$")
 
 
-def load():
-    return json.load(open(MANIFEST))["sets"]
+def names():
+    return list(json.load(open(MANIFEST))["benchmarks"])
+
+
+def load(benchmark):
+    return json.load(open(MANIFEST))["benchmarks"][benchmark]
 
 
 def read_model(path):
@@ -66,8 +71,8 @@ def rotation(axis, deg):
 
 
 def start_model(start, wd, pdb_cache):
-    """The file given to --model: the committed prediction, or the deposited model moved by the
-    manifest's transform, written into the set's work directory."""
+    """The file given to --model: the committed prediction, or the deposited model (models/pdb/,
+    else the pdb_cache) moved by the manifest's transform, written into the set's work directory."""
     if start.get("alphafold"):
         return os.path.join(HERE, start["alphafold"])
     mv = start["moved"]