From 7b1bc3a3ea8be07c47da6f147f8b71e1b8a5e866 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Mon, 26 Oct 2026 14:45:12 +0100 Subject: [PATCH] docs: report headline keys, model section for every way a model is given, still battery RUGNUX_REPORT: a table of the headline keys (checked against ResultReport.cpp); section 5 appears for --model, --mr or a model among the inputs and holds the MR and REFINE_ keys; a --mode refine report has one section; stills ambiguity resolved by the crystals' own cells first; the developer-only CELL_ESD_OBSERVATIONS / CELL_DSPACING_FIT; the "nothing here is refined" paragraph points at REFINE_*. BENCHMARKING: the still battery, the refined R-free beside the placement one, the Refinement and Doubled-cell steps in the timing table. EXTERNAL_TEST_DATA_STILLS: the gain is measured, not passed; how the battery runs these sets. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit e9f5e3b63fb6208d6bd5984c1ecd0eb0da7c29ab) --- docs/EXTERNAL_TEST_DATA_STILLS.md | 14 +++++++-- docs/RUGNUX_BENCHMARKING.md | 17 ++++++++-- docs/RUGNUX_REPORT.md | 52 ++++++++++++++++++++++++------- 3 files changed, 67 insertions(+), 16 deletions(-) diff --git a/docs/EXTERNAL_TEST_DATA_STILLS.md b/docs/EXTERNAL_TEST_DATA_STILLS.md index 490d4d97c..b68d40972 100644 --- a/docs/EXTERNAL_TEST_DATA_STILLS.md +++ b/docs/EXTERNAL_TEST_DATA_STILLS.md @@ -37,6 +37,15 @@ Every set of the manifest is in the table and every set in the table is in the m same CXIDB entry, PDB entry, space group and dmin; `tools/battery/doc_check.py` checks this. The beamline of `pal90` is the CXIDB entry's (NCI); the PDB entry 6IRK records CSI. +## How the battery runs them + +Each set is one rugnux command, the way a serial user runs: the deposited cell and space group given +(`-C`, `-S`), the wavelength where the files do not state it, `--crystfel-geometry` for the AGIPD and +JUNGFRAU sets (the geometry file of the entry; the SACLA, PAL and LCLS files are placed without one), +and the deposited model with `--model`, so that the report carries its R-free against the merge. The +exact arguments are the `args` of each set in `tools/battery/still_manifest.json`; the +[stills section of the tutorial](RUGNUX_TUTORIAL.md#still--serial-data) explains them. + ## The files we use Paths are relative to the still data root of the site config (`tools/battery/README.md`). CXIDB @@ -69,8 +78,9 @@ that line differs. **The `_int16` sets.** The SwissFEL files store converted JUNGFRAU data as uncompressed float32 keV, 68 MB (JF16M) or 17 MB (JF4M) a frame, which caps reading at about 74 or 295 frames/s at 5 GB/s. The `_int16` sets are copies with the same layout in which `/data/data` holds whole photons (keV divided -by the photon energy, rounded) as int16 with bitshuffle/LZ4, about 25 times smaller; rugnux reads them -with `--gain 1` and merges them to the same numbers. Make them with +by the photon energy, rounded) as int16 with bitshuffle/LZ4, about 25 times smaller; the battery runs +them with the same arguments as the float32 originals (rugnux measures the gain from the frames), and +they merge to the same numbers. Make them with `tools/battery/still/float32_to_int16.py ` (h5py and hdf5plugin), into `swissfel_103/conv/` (4.57 keV) and `swissfel_213/conv/` (12.037 keV, keeping the tarball's directory tree), named `_int16.h5`. One MbCO dark run (`run_000238`) holds no frames and has no copy. diff --git a/docs/RUGNUX_BENCHMARKING.md b/docs/RUGNUX_BENCHMARKING.md index e6785bd7e..0175538d7 100644 --- a/docs/RUGNUX_BENCHMARKING.md +++ b/docs/RUGNUX_BENCHMARKING.md @@ -26,6 +26,13 @@ the old ones were retired. and also checks that the answers are right. It answers *how fast, on this machine*. It takes minutes, and needs nothing on the machine but four open datasets. +Serial stills have a battery of their own, `tools/battery/still_battery.py`, on the public datasets +of [External test data: serial crystallography](EXTERNAL_TEST_DATA_STILLS.md). It shares the site +config, the verbs and the run-directory layout, but scores nothing about the lattice or the symmetry +(a serial user gives the cell and the space group): each row says what the merge is worth (R_split, +the deposited model's R-free against it beside the deposition's own) and what it cost. The README's +*The still battery* describes it. + The two are kept apart on purpose. A battery run reads hundreds of gigabytes once, so its wall clock mostly measures the disk and the page cache (see [the page-cache trap](#the-page-cache)). Use the battery to judge correctness and the timing benchmark, run under controlled conditions, to judge @@ -189,9 +196,11 @@ noise, and one that reproduces is a real change. ## The REFMAC model check -R-free from `rugnux --model` is a **trend number**. It is a rigid-body placement scored on rugnux's +R-free from `rugnux --model` is a **trend number**. `R_FREE=` is the placed model scored on rugnux's own free set, with a deliberately simple scale model, so it shows how a set moves between runs, but -it cannot be compared with the deposition's R-free. `--model-check` adds an independent number that +it cannot be compared with the deposition's R-free. rugnux then refines the placed model as well +(`REFINE_R_FREE=`, recorded as `rfree_refined` in `results.json`, reported and never scored), still +on its own free set, so that number is not comparable with the deposition's either. `--model-check` adds an independent number that can be. It runs `tools/battery/model_check.py` on every open-arm set with a deposited X-ray model, after rugnux has finished, so it cannot influence the processing. It needs CCP4: the runner sources `$CCP4_SETUP`, by default `/opt/xtal/ccp4-9/bin/ccp4.setup-sh`, and calls the `refmac5` found there @@ -288,7 +297,7 @@ $B timing --rugnux $R [--benchmark standard|inhouse] [--threads N] [--label L] [ The benchmark runs a few sets. Each starts from a model that the run has to place by molecular replacement, so a single run times every stage: reading, pre-scan, indexing, integration, scaling, -the space-group search, MR and model validation. It also checks that every answer is right. +the space-group search, MR, model validation and the refinement of the placed model. It also checks that every answer is right. `--benchmark standard`, the default, is the one to run on a new machine. It uses open data only: three EIGER 16M rotation sets of ordinary size and quality, and a virus capsid as the large case. @@ -574,6 +583,8 @@ section. For a rotation run the steps are, in order: | Analyses | twinning, anisotropy and the other report-only analyses | | Model validation | `--model` only: structure factors, scale fits, rigid-body placement, maps | | Molecular replacement | only where the rigid body did not place the model (always, in the timing benchmark) | +| Refinement | the placed model refined against the merge (wherever a model is placed, unless `--no-refine`) | +| Doubled-cell hypothesis | only where an index-2 superstructure qualifies: the doubled cell integrated and merged as well ([Results report](RUGNUX_REPORT.md#index-2-superstructure)) | | Writing files | MTZ, mmCIF, HKL, report | A step that is slower on one machine than on another points at the code, and a step that is slower diff --git a/docs/RUGNUX_REPORT.md b/docs/RUGNUX_REPORT.md index 38d3fb6ab..d88d3e004 100644 --- a/docs/RUGNUX_REPORT.md +++ b/docs/RUGNUX_REPORT.md @@ -119,10 +119,12 @@ The header block above section 1 records **how the result was produced**: `RUGNU Rates, per-image costs and progress remain on stdout only. -Sections, in order: the `SUMMARY`, then `1. DATA SET AND GEOMETRY`, `2. CRYSTAL`, -`3. MERGED DATA`, `4. DIAGNOSTICS` and `5. MODEL VALIDATION` only with `--model` or `--mr` - which -also holds molecular replacement's keys where it ran ([below](#molecular-replacement)) -(see [The summary](#the-summary-and-what-the-run-decided) below). The numbering is contiguous, and +Sections, in order: the `SUMMARY` ([below](#the-summary-and-what-the-run-decided)), then +`1. DATA SET AND GEOMETRY`, `2. CRYSTAL`, `3. MERGED DATA`, `4. DIAGNOSTICS` and, only where a model +was given (`--model`, `--mr`, or a model file among the inputs), `5. MODEL VALIDATION` - which also +holds the keys of [molecular replacement](#molecular-replacement) and of the +[refinement](#refinement-refine_) where they ran. A `--mode refine` report has one section, +`1. REFINEMENT`. The numbering is contiguous, and the fifth section appearing renumbers nothing; a stage that did not run states that inside its section — `MERGE= NOT_PERFORMED` — rather than the section disappearing. @@ -207,6 +209,29 @@ is ambiguous about which geometry produced it. **Not in the report:** timing, frame rates, thread counts, per-image progress and library banners. Those are process, not result, and stay on stdout. +## The headline keys + +The keys most scripts read, by section; the rest of this page explains the ones that need it. + +| key | meaning | +|---|---| +| `EXPERIMENT_TYPE` | `ROTATION` or `STILLS` (section 1) | +| `IMAGES_PROCESSED`, `WAVELENGTH` | Images the run read, and the wavelength (Å) | +| `OSCILLATION_RANGE`, `STARTING_ANGLE`, `ROTATION_AXIS` | Rotation only: the per-image rotation the run integrated at (deg: the file's, scaled where the run adopted a goniometer rotation scale), the first image's angle and the axis | +| `DETECTOR_DISTANCE`, `BEAM_CENTRE`, `DETECTOR_TILT` | The geometry the run integrated at (mm; the point of normal incidence in pixels; the three PONI rotations in degrees) - on a rotation run the post-refined one, not the header's | +| `DIRECT_BEAM` | Where the direct beam meets the detector under that tilt (pixels) | +| `LATTICE_FOUND`, `UNIT_CELL_CONSTANTS`, `CELL_ESD` | Section 2: whether a lattice was found, the cell (a b c α β γ, Å and degrees) and its e.s.d.s ([below](#cell-esds)) | +| `SPACE_GROUP_NAME`, `SPACE_GROUP_NUMBER` | The space group the files are written in ([below](#the-space-group-and-the-sohncke-answer-beside-it)) | +| `MOSAICITY_DEG` | Rotation only ([below](#mosaicity)) | +| `MERGE` | Section 3: `PERFORMED` or `NOT_PERFORMED` (`--no-merge`) | +| `INCLUDE_RESOLUTION_RANGE` | `dmax dmin` of the written reflections (Å), as XDS's keyword | +| `FITTED_RESOLUTION` | Where the CC1/2 fall-off crosses the `--resolution-cc-target`; absent where the curve is not a fall-off (a `RESOLUTION_FIT` warning says why) | +| `UNIQUE_REFLECTIONS`, `TOTAL_OBSERVATIONS`, `COMPLETENESS`, `MULTIPLICITY` | Over the written range; completeness in percent | +| `I_OVER_SIGMA`, `R_MEAS`, `CC_HALF`, `WILSON_B` | The overall merging statistics; the shell table beside them is in the comment lines | +| `SIGANO`, `CC_ANOM`, `FRIEDELS_LAW` | The anomalous signal, and whether Friedel pairs were merged (`FALSE` with `-A`) | +| `ERROR_MODEL_A`, `ERROR_MODEL_B`, `ISA` | The error model in XDS's convention and ISa = 1/sqrt(a·b); `ISA= undetermined` where b is zero within its error, so 1/b would be noise | +| `R_WORK`, `R_FREE` | Section 5: the model as placed against these data ([below](#model-validation)); the refined model's are `REFINE_R_WORK`/`REFINE_R_FREE` | + ## The summary, and what the run decided The file opens with a **`SUMMARY`** section, above everything it summarises. It exists because the @@ -239,7 +264,8 @@ and a run that produced a textbook data set read identically for their first thr admits alternative indexing: one sweep is indexed consistently, so it matters only against other data; the operators are **`INDEXING_AMBIGUITY_OPERATORS=`** in section 2). On serial stills the same ambiguity mixes the hands in the merge. Where the space group was given (`-S`), the crystals are made - consistent from the data alone and section 2 says so: **`INDEXING_AMBIGUITY_RESOLVED= data`**, + consistent from the data alone - by their own cells where those decide the indexing, and by the + intensities otherwise - and section 2 says so: **`INDEXING_AMBIGUITY_RESOLVED= data`**, **`AMBIGUITY_CRYSTALS=`** (the crystals that took part), **`AMBIGUITY_REINDEXED=`** (of them, the ones moved), **`AMBIGUITY_SEPARATION_SIGMA=`** (per crystal, how far its chosen indexing beats the best other one in correlation with all the other crystals, in sigma; the median) and @@ -257,7 +283,7 @@ and a run that produced a textbook data set read identically for their first thr always were; they have moved from the bottom of the file to the top. Then the numbered sections: **1. DATA SET AND GEOMETRY**, **2. CRYSTAL**, -**3. MERGED DATA**, **4. DIAGNOSTICS**, and **5. MODEL VALIDATION** only with `--model`. +**3. MERGED DATA**, **4. DIAGNOSTICS**, and **5. MODEL VALIDATION** only where a model was given. ## The reference-range table @@ -327,8 +353,8 @@ per-zone screw table. reindexing operator from the axes the space group was determined on (CCP4 style, `h,k,l` where nothing moved) and what chose the setting - `STANDARD`, `CELL` (`-C`), `SPACE_GROUP` (a non-standard `-S`), `MODEL` (a `--model` that fits) or `REFERENCE` (`-z`). `SPACE_GROUP_NAME`, the cell and every -file the run writes are in that setting (see RUGNUX_ADVANCED.md, "The setting the files are written -in"). +file the run writes are in that setting (see +[The setting the files are written in](RUGNUX_ADVANCED.md#the-setting-the-files-are-written-in)). `SOHNCKE_SPACE_GROUP=` in section 2 is written on every run whose space group was determined by the search; a run given its group with `-S` has no Sohncke candidate to name and omits the key. Where the @@ -744,7 +770,9 @@ scatter of their cells, over √blocks, is **`CELL_ESD_BLOCKS=`** (where at leas fitted; **`CELL_ESD_BLOCK_COUNT=`** says how many). `CELL_ESD=` is, per parameter, the larger of the two. **`CELL_ESD_REFLECTIONS=`** is the number of distinct reflections used and **`CELL_ESD_THETA_RANGE_DEG=`** their θ range, written as `_cell_measurement.reflns_used`, -`theta_min` and `theta_max` in the mmCIF. +`theta_min` and `theta_max` in the mmCIF. With `--developer`, **`CELL_ESD_OBSERVATIONS=`** is the +number of observations behind them and **`CELL_DSPACING_FIT=`** the cell the d-spacing fit itself +returned. Neither estimate includes what is common to every observation - the wavelength and the detector distance scale the whole cell - so the e.s.d. is a precision, not an accuracy. Stills runs have no @@ -908,7 +936,8 @@ says so, since refinement and map interpretation should allow for it. ## Model validation -Section 5 appears only with `--model`. It reports the supplied model against the merged data — +Section 5 appears only where a model was given (`--model`, `--mr`, or a model file among the +inputs). It reports the supplied model against the merged data — R-factors, maps, anomalous sites — and, separately, whether the data accepted the model at all. The two are not the same question, and the report keeps them apart. **The R-factors, the maps and the @@ -983,7 +1012,8 @@ between those two runs cannot be read; `R_MODEL_SHELL_SCALED` can. Both R values are over **all** the reflections, not the free 5%. Nothing here is refined — the coordinates, the B-factors and the occupancies are the model's own, and only the scale and the -placement are fitted — so work and free estimate the same quantity (they differ by a median 0.001 +placement are fitted (the refinement that follows is reported apart, under +[`REFINE_*`](#refinement-refine_)) — so work and free estimate the same quantity (they differ by a median 0.001 over our corpus) and the split buys no cross-validation while costing a factor of √20 in precision. None of this touches the maps: the map coefficients and `R_WORK` / `R_FREE` are