diff --git a/tools/battery/README.md b/tools/battery/README.md index 75458fb0c..3ff7df72d 100644 --- a/tools/battery/README.md +++ b/tools/battery/README.md @@ -529,19 +529,22 @@ with the deposition, that needs CCP4: `tools/battery/model_check.py` scores each (`p.mtz`) against the deposited model, unmodified (ligands, waters and hydrogens kept; only atoms of unknown element, UNX, are left out because REFMAC stops on them): REFMAC's rigid-body mode, R-factors taken before any shift, the data reindexed into the model's setting and reduced to the -deposited space group. The depositor's own structure factors go through the same setting choice -(they can sit in the other branch of a merohedral ambiguity) and are scored by the same protocol. +deposited space group. The model is never refined. The depositor's own structure factors go through +the same setting choice (they can sit in the other branch of a merohedral ambiguity) and are scored +by the same protocol. -All REFMAC numbers use the same high-resolution limit, `d_min_used` = max(our d_min, the deposited -d_min), i.e. the lower of the two resolutions, so neither data set is scored past what the other -reaches. The four R-free fields, all of the same deposited model: +Both data sets are scored on the same reflections: the hkl present in both, at `d_min_used` = +max(our d_min, the deposited d_min), i.e. the lower of the two resolutions, so neither data set is +scored past what the other reaches and a cut either one made (low-resolution limit, anisotropic +cut) applies to both. Bulk solvent and the overall anisotropic scale are REFMAC's defaults, fitted +to each data set. The four R-free fields, all of the same deposited model: | field | data | free set | reads as | |---|---|---|---| | `refmac_rfree` | ours | ours | close to a work R: most of our free reflections were work reflections to the depositor's refinement | | `refmac_rfree_depflags` | ours | the depositor's | **the fair number for our data** | | `refmac_rfree_depdata` | the depositor's | the depositor's | the baseline: same model, same protocol, the data the model was refined against | -| `refmac_rfree_ratio` | | | `refmac_rfree_depflags / refmac_rfree_depdata`; below 1 = our data fit the model better; the report plots it | +| `refmac_rfree_ratio` | | | `refmac_rfree_depflags / refmac_rfree_depdata`: **the headline**; below 1 = our data fit the model better; the report plots it | `refmac_rwork` is R on our own work reflections. The depositor-data numbers carry a home advantage (the model was refined against those data), and `refmac_rfree_depdata` is not the @@ -560,39 +563,6 @@ set (`refmac_reason` says why). The runner records `refmac_rfree`, `refmac_rwork A `--model-check` run records the REFMAC version in its manifest (`refmac`), as does every recheck, and the report lists it. -**Refined, shared free set (second protocol).** The first-cycle numbers above score the model as -deposited at the common resolution limit. The second protocol refines the same deposited model -against each data set at that data set's OWN resolution and compares R-free on a free set both -share: - -- same starting model, same REFMAC protocol for both data sets: restrained refinement, 10 cycles, - automatic weight, isotropic B for every atom (a deposited ANISOU is often TLS-derived, and - refining it atom by atom diverged), twin refinement where the entry declares twinning; a ligand - whose code the monomer library describes with other atoms is renamed to a code the library - lacks, so REFMAC makes its restraints from the model's own coordinates (`ligands_renamed` in - the check's output); -- each data set over its whole resolution range; its work set is every reflection it has except - the shared free set; -- shared free set = the depositor's free reflections present in BOTH data sets (after the change - of basis), which takes the lower of the two limits in every direction, anisotropic cuts and - rejected reflections included; depositor free reflections outside it are removed from both; -- R-free is REFMAC's own: each data set's free set IS the shared set. It is not recomputed from - the output MTZ, whose F carries REFMAC's anisotropic scale - on strongly anisotropic data that - scale multiplies the noise of the weak direction and the recomputed R comes out far too high. - -| field | meaning | -|---|---| -| `refmac_refined_rfree_shared` | our data, refined, R-free on the shared set - **the refined number for our data** | -| `refmac_refined_rfree_shared_depdata` | the depositor's data, same protocol, same shared set | -| `refmac_refined_ratio` | the ratio of the two; below 1 = our data give the better refined model | -| `refmac_refined_rwork`, `refmac_refined_rwork_depdata` | REFMAC's R-work after refinement, each data set (their own work sets and resolutions: not comparable with each other) | -| `refmac_refined_rfree_refmac`, `refmac_refined_rfree_refmac_depdata` | REFMAC's own R-free, for reference | -| `refmac_shared_free_n` | size of the shared free set | -| `refmac_refined_reason` | why the refined protocol gave nothing, when it did not (the first-cycle numbers stand) | - -The refined numbers need the depositor's structure factors with a free set, like the ratio; they -take minutes to an hour per set. `model_check.py --no-refine` skips them. - ### Rechecking an existing run `battery.py recheck RUN [--only a,b] [--jobs N]` reruns the model check and the depositor-data @@ -758,7 +728,7 @@ was run `--unforced`), so its forced XDS-arm rows are left out the same way. | `rfree`, `rwork`, `cc_model`, `model_fit`, `rfree_deposited`, `rfree_ratio` | any row with a model: R_FREE, R_WORK, CC_MODEL_OVERALL and MODEL_FIT as rugnux reports them (placement-only, own free set: trend fields), the published R-free, and rfree / rfree_deposited | | `anom_sigma` | any row with a model: ANOMALOUS_SCATTERER_MEAN_SIGMA, the anomalous difference map (F+ - F- on the model's phases) read at every model atom from phosphorus up (S of Met/Cys, metals, Cl, I, ...) and averaged, in map sigma - how much anomalous signal the merge carries; null where the merge kept no Bijvoet split or the model has no such atom | | `rmodel`, `rmodel_shell_scaled`, `radial_misfit` | any row with a model: R_MODEL, R_MODEL_SHELL_SCALED and MODEL_RADIAL_MISFIT - R over all the reflections, with and without one free scale per resolution shell, and how big that rescale was. `rmodel_shell_scaled` is the R that compares between runs (below) | -| `refmac_rfree`, `refmac_rwork`, `refmac_rfree_depflags`, `refmac_rfree_depdata`, `refmac_rfree_ratio`, `refmac_twin`, `refmac_untwinned`, `refmac_status`, `refmac_reason`, `refmac_refined_*`, `refmac_shared_free_n` | the REFMAC check (`--model-check`, open arm; see its section for which R-free is which) | +| `refmac_rfree`, `refmac_rwork`, `refmac_rfree_depflags`, `refmac_rfree_depdata`, `refmac_rfree_ratio`, `refmac_twin`, `refmac_untwinned`, `refmac_status`, `refmac_reason` | the REFMAC check (`--model-check`, open arm; see its section for which R-free is which) | | `dep_status`, `dep_reason`, `dep_kind`, `dep_d_min`, `dep_n_common`, `dep_cc_delta_all`, `dep_cc_delta_outer`, `dep_beyond_cc`, `dep_beyond_d` | open arm with a model: our merge against the depositor's data per shell (`depdata_check.py`, above) | | `shx_*` | small molecule with a `cod` structure: the SHELXL check (`shelx_check.py`, above) | | `wall_s`, `rugnux_wall_s`, `elapsed_s`, `exit_code`, `gpu_others`, `rugnux_verdict` | timing (`wall_s` is rugnux's own WALL_TIME, not the time spent waiting for the GPU queue), the exit code, other GPU processes seen before the set, and rugnux's own verdict line | @@ -911,11 +881,9 @@ on the server (`20261011-1236_549939_1.0.0-rc.175`, and `20261014-0141_6e9b89_1. ## Figures for publication -`paper_plots.py RUN [RUN-REDO ...] --out DIR [--time-runs RUN ...] [--uncontended RUN] [--rfree refined|first-cycle]` -draws three open-arm figures from a run's `results.json`: R-free of the deposited model after the REFMAC -check (`--model-check`) on the depositor's data against Rugnux's data (by default the refined protocol: -each data set at its own resolution, R-free on the shared free set; `--rfree first-cycle` for the -first-cycle numbers at the common resolution), the deposited high-resolution limit against Rugnux's cut, and a +`paper_plots.py RUN [RUN-REDO ...] --out DIR [--time-runs RUN ...] [--uncontended RUN]` +draws three open-arm figures from a run's `results.json`: R-free of the deposited model by the REFMAC +check (`--model-check`) on the depositor's data against Rugnux's data, the deposited high-resolution limit against Rugnux's cut, and a histogram of the processing time per dataset with the median marked. Later runs fill in sets an earlier one did not run (a `-redo`). `--time-runs` takes the time histogram from other runs - one whose sets had the GPU to themselves - and `summary.txt` lists the provenance of both. Beside the PDF/PNG figures it writes the @@ -949,11 +917,10 @@ once and its row says what the merge is worth and what it cost: | facility, beamline, detector, dataset, PDB | from the manifest | | images | frames in the input files (rugnux's `IMAGES_PROCESSED`); images (deposition) the deposition's own count, where it is known | | indexed %, d_min, R_split | rugnux's report (`INDEXING_RATE`, `INCLUDE_RESOLUTION_RANGE`, `R_SPLIT`: whole frames go to one half, as in CrystFEL) | -| R_free rugnux (dimple) | `dimple` of the deposited model against our merge, reindexed into the deposition's space group, with the deposition's own free set, at the deposition's resolution | -| R_free deposition (dimple) | the same protocol on the deposited data (the entry's `-sf.cif`: `intensity_meas`, else the mean of the anomalous pair, else `F_meas_au` squared) | -| R_free published | the entry's own R-free (dark and ground-state entries only), from its own refinement program | -| target | which of the two is lower (`dimple` or `published`): the R-free to beat | -| difference, quality | ours minus the target; win where ours is lower by more than 0.01, draw within 0.01, loss otherwise | +| R_free rugnux | the deposited model against our merge, not refined: the rotation battery's REFMAC check (`model_check.py`, see above), i.e. `refmac_rfree_depflags` - REFMAC's first cycle, the deposition's own free set, the common reflections at the lower of the two resolutions | +| R_free deposition | the same protocol on the deposited data (`refmac_rfree_depdata`): the R-free to beat | +| R_free published | the entry's own R-free (dark and ground-state entries only), from its own refinement program; for information | +| difference, quality | ours minus the deposition's; win where ours is lower by more than 0.01, draw within 0.01, loss otherwise | | time, ms/image | wall clock of the rugnux command, which ends with the model validation | | I/O floor | the input's bytes over `--io-gbps` (default 5 GB/s): the time reading alone would take from an SSD | | page cache | share of the input already in memory when the run started (`fincore`): cold, warm or partial | @@ -969,17 +936,15 @@ still_battery.py run --rugnux build/rugnux/rugnux [--only pal90,sacla236 | --tie still_battery.py compare RUN_A RUN_B still_battery.py report RUN [--out DIR] still_battery.py list -still_battery.py reference [--only pal90,sacla236] still_battery.py keep RUN --as workstation|server ``` -The deposited-data reference is computed once per entry and resolution and kept in the PDB cache -(`/-dimple-/reference.json`, with the dimple run beside it); `run` and -`report` compute a missing one on the way (so `report` of an older run gains the column), and -`reference` computes or lists them all up front. +Both R-free values come from one `model_check.py` call per set (seconds), in `work//model_check/`. +`report RUN --out DIR` of a run scored by an older protocol rescores its merges into `DIR` (rugnux is +not run again). A run writes `/__still-