diff --git a/docs/CPU_DATA_ANALYSIS_DECISIONS.md b/docs/CPU_DATA_ANALYSIS_DECISIONS.md
index ecc059048..f248d736d 100644
--- a/docs/CPU_DATA_ANALYSIS_DECISIONS.md
+++ b/docs/CPU_DATA_ANALYSIS_DECISIONS.md
@@ -126,6 +126,8 @@ $
This is the standard, few-parameter scaling model used by refinement programs. No free-form per-resolution-shell rescale is applied: such a rescale is dataset-specific and reshapes each map's radial amplitude profile differently, which would make maps from a multi-dataset campaign no longer directly comparable.
+That decision has a cost, and it is paid by the R-factors rather than by the maps. $k\exp(-\mathbf{s}^\mathsf{T} B \mathbf{s})$ can only bend one way with resolution, so whatever a dataset's radial amplitude profile does that this shape cannot follow is reported as R — which makes R a reading of *this* dataset and not a quantity comparable with another reduction of the same crystal. Both uses are wanted, so both are served, separately: the maps and `R_WORK` / `R_FREE` keep the scale above, and a second reading, `R_MODEL_SHELL_SCALED`, applies one free scale per resolution shell to the *same* fit and is reported beside them, with `MODEL_RADIAL_MISFIT` saying how much that rescale had to do. Nothing is written from the second reading; it never reaches a map, a map coefficient or a decision (§14.3, and RUGNUX_REPORT).
+
The fit sees the **working reflections only**; the parameters it returns are then applied to every reflection, free ones included, so that R-free (§14.3) is computed against them. The parameters are few, but they are fitted by minimising the very sum R is made of, and a free reflection that helped choose them is no longer held out.
### 14.3 R-work and R-free
diff --git a/docs/RUGNUX_ADVANCED.md b/docs/RUGNUX_ADVANCED.md
index 9df974f79..077cdf1fd 100644
--- a/docs/RUGNUX_ADVANCED.md
+++ b/docs/RUGNUX_ADVANCED.md
@@ -143,7 +143,9 @@ cell or no usable space group, does not fail the run: it is logged, and the resu
maps cannot be mistaken for one that was never given `--model`.
Either way the results report carries a **`5. MODEL VALIDATION`** section: `R_WORK=` / `R_FREE=`
-with their reflection counts, the bulk-solvent and overall scale parameters, the mean 2mFo−DFc density
+with their reflection counts, `R_MODEL_SHELL_SCALED=` and `MODEL_RADIAL_MISFIT=` beside them (the R
+that compares between two runs, and why the other two do not — see
+[RUGNUX_REPORT](RUGNUX_REPORT.md)), the bulk-solvent and overall scale parameters, the mean 2mFo−DFc density
at the atom centres, the reindexing operators the written reflections were brought into the model's
frame with, and `MAPS_PREFIX=`; or `MODEL_VALIDATION= NOT_PERFORMED` with
`MODEL_VALIDATION_REASON=` when the model could not be used. A run given no `--model` has no such
diff --git a/docs/RUGNUX_REPORT.md b/docs/RUGNUX_REPORT.md
index 9b50f8697..cc0ffa34b 100644
--- a/docs/RUGNUX_REPORT.md
+++ b/docs/RUGNUX_REPORT.md
@@ -551,6 +551,38 @@ The null's own numbers — `MODEL_FIT_STATISTIC`, `MODEL_FIT_VALUE`, the `MODEL_
the `MODEL_INDEXING_MARGIN*` keys — are written with `--developer`; the default report carries the
verdict (`MODEL_FIT`, `MODEL_FIT_SIGMA`, `MODEL_DECISIONS_TAKEN` and the two decisions).
+### Comparing two runs: `R_MODEL_SHELL_SCALED`
+
+`R_WORK` and `R_FREE` describe *this* dataset against *this* model, and that is all they describe.
+They are **not comparable with another run's**. The model is scaled to the data by an overall factor
+and a symmetry-constrained anisotropic *B* — a shape that can only bend one way with resolution, kept
+that way on purpose so that a batch of maps stays on one scale (see
+[CPU_DATA_ANALYSIS_DECISIONS](CPU_DATA_ANALYSIS_DECISIONS.md)). Whatever the amplitudes' own radial
+profile does that this shape cannot follow is then reported as R. Two reductions of one crystal whose
+merged amplitudes have different radial profiles therefore differ in `R_FREE` for a reason that has
+nothing to do with either fitting the model better, and by more than a real change in the data moves it.
+
+`R_MODEL_SHELL_SCALED` is the same sum with that taken out: one free scale per resolution shell — the
+merge table's own shells — fitted on the shell's reflections, so only the agreement *inside* each
+shell is left. `R_MODEL` is the same sum without the per-shell scale, so the gap between the two is
+what the radial profile cost, and `MODEL_RADIAL_MISFIT` is the size of the rescale that closed it
+(the RMS of ln *k*shell about its mean). When the misfit moves between two runs, `R_FREE`
+between those two runs cannot be read; `R_MODEL_SHELL_SCALED` can.
+
+Both R values are over **all** the reflections, not the free 5%. Nothing here is refined — the
+coordinates, the B-factors and the occupancies are the model's own, and only the scale and the
+placement are fitted — so work and free estimate the same quantity (they differ by a median 0.001
+over our corpus) and the split buys no cross-validation while costing a factor of √20 in precision.
+
+None of this touches the maps: the map coefficients, the `.ccp4` files and `R_WORK` / `R_FREE` are
+computed from the scale described above and are unchanged by these keys.
+
+| key | meaning |
+|---|---|
+| `R_MODEL`, `R_MODEL_REFLECTIONS` | R over every reflection, on the scale the maps use |
+| `R_MODEL_SHELL_SCALED` | The same, with one free scale per resolution shell — **the number to compare between runs** |
+| `MODEL_RADIAL_MISFIT` | How large that per-shell rescale had to be: RMS of its logarithm. Near zero means the maps' own scale already described the radial profile, and then the two R values agree |
+
### CC(model, data)
Beside the R-factors, section 5 carries the **correlation of the merged intensities with the placed,
diff --git a/rugnux/ModelValidation.cpp b/rugnux/ModelValidation.cpp
index 783b19849..df9ac2c7c 100644
--- a/rugnux/ModelValidation.cpp
+++ b/rugnux/ModelValidation.cpp
@@ -916,6 +916,72 @@ ModelValidationResult ValidateAgainstModel(const std::vector &
result.cc_model_overall, result.cc_model_n,
result.cc_model_shells.back().d_min, result.cc_model_shells.back().cc,
result.cc_model_shells.back().n, result.cc_model_shells.back().sigma);
+
+ // --- the same R with the radial profile taken out, and how much of it there was ---
+ // R above is what the maps are made of and is read WITHIN a run; this is the one to read
+ // BETWEEN two. The scale the maps use can only bend as k*exp(-B s^2) (see the note in
+ // fit_model, and the reason it stays that way), so where two reductions of one crystal differ
+ // in the radial profile of their amplitudes, the part of the difference that shape cannot
+ // follow lands in R - measured on this corpus, enough to move R by more than a real change in
+ // the data does. Giving the scale one free factor per shell removes the radial profile and
+ // only the radial profile: what is left is the agreement inside each shell, which is the
+ // question "do these data fit this model better or worse than those did".
+ //
+ // On ALL the reflections, not the free set. Nothing here is refined - the coordinates, the B
+ // factors and the occupancies are the model's own - so work and free estimate the same
+ // quantity (measured: the two differ by a median 0.001 over the corpus), and the 5% split
+ // therefore buys no cross-validation while costing a factor of sqrt(20) in precision. Read on
+ // all of them, the number is worth about +-0.002 per set instead of +-0.010, which is the
+ // difference between seeing a real change and seeing the sampling of a free set.
+ //
+ // Nothing is written from any of this. fmodel, the maps, the MTZ and the R-factors above are
+ // untouched; it is a second reading of the fit that has already happened.
+ double num = 0, den = 0, plain_num = 0, plain_den = 0;
+ int n_scored = 0;
+ std::vector sum_oc(report_shell_d_min.size(), 0), sum_cc(report_shell_d_min.size(), 0);
+ for (const auto &hv : fmodel.v) {
+ const auto it = obs_by_hkl.find(hkl_key(hv.hkl));
+ if (it == obs_by_hkl.end()) continue;
+ const size_t bin = shell_of(it->second.d);
+ if (bin >= sum_oc.size()) continue;
+ sum_oc[bin] += it->second.F * std::abs(hv.value);
+ sum_cc[bin] += static_cast(std::abs(hv.value)) * std::abs(hv.value);
+ }
+ // The shell's scale by least squares on its own reflections. One parameter per shell against
+ // the thousands of reflections in it, so scoring the same reflections it was fitted on lowers
+ // R by far less than the difference it is there to expose.
+ std::vector ln_k;
+ ln_k.reserve(report_shell_d_min.size());
+ for (size_t i = 0; i < sum_oc.size(); i++)
+ if (sum_cc[i] > 0 && sum_oc[i] > 0)
+ ln_k.push_back(std::log(sum_oc[i] / sum_cc[i]));
+ for (const auto &hv : fmodel.v) {
+ const auto it = obs_by_hkl.find(hkl_key(hv.hkl));
+ if (it == obs_by_hkl.end()) continue;
+ const size_t bin = shell_of(it->second.d);
+ if (bin >= sum_oc.size()) continue;
+ const double Fo = it->second.F, Fc = std::abs(hv.value);
+ const double k = (sum_cc[bin] > 0 && sum_oc[bin] > 0) ? sum_oc[bin] / sum_cc[bin] : 1.0;
+ num += std::fabs(Fo - k * Fc);
+ den += Fo;
+ plain_num += std::fabs(Fo - Fc);
+ plain_den += Fo;
+ ++n_scored;
+ }
+ if (den > 0 && ln_k.size() >= 2) {
+ result.r_model = plain_num / plain_den;
+ result.r_model_shell_scaled = num / den;
+ result.n_model = n_scored;
+ result.shell_scale_bins = static_cast(ln_k.size());
+ const double mean_ln_k = std::accumulate(ln_k.begin(), ln_k.end(), 0.0) / ln_k.size();
+ double var = 0;
+ for (double v : ln_k) var += (v - mean_ln_k) * (v - mean_ln_k);
+ result.radial_misfit = std::sqrt(var / ln_k.size());
+ logger.Info("Model validation: R(all reflections) {:.4f}, with a per-shell scale {:.4f} "
+ "over {} shells; radial misfit {:.3f}",
+ result.r_model, result.r_model_shell_scaled, result.shell_scale_bins,
+ result.radial_misfit);
+ }
}
// --- sigma_A weighting: the maps are 2mFo-DFc and mFo-DFc, not 2Fo-Fc and Fo-Fc ---
diff --git a/rugnux/ModelValidation.h b/rugnux/ModelValidation.h
index 7fc6831a8..4aceab134 100644
--- a/rugnux/ModelValidation.h
+++ b/rugnux/ModelValidation.h
@@ -3,6 +3,7 @@
#pragma once
+#include
#include
#include
#include
@@ -31,6 +32,28 @@ struct ModelValidationResult {
double k_overall = 0.0;
int n_work = 0;
int n_free = 0;
+
+ // R over ALL the reflections - r_model as the maps are scaled, r_model_shell_scaled with one free
+ // scale per resolution shell instead - and how far those per-shell scales had to depart from a
+ // flat one. The pair exists to be read BETWEEN runs, where r_work / r_free cannot be.
+ //
+ // The scale the maps carry can only bend as k*exp(-B s^2), deliberately (see ModelValidation.cpp),
+ // so where two reductions of one crystal differ in the radial profile of their amplitudes, the
+ // part of the difference that shape cannot follow is reported as R - by more, measured, than a
+ // real change in the data moves it. A per-shell scale removes the radial profile and only the
+ // radial profile, leaving the agreement inside each shell.
+ //
+ // All the reflections and not the free set, because nothing here is refined: the coordinates, the
+ // B factors and the occupancies are the model's own, so work and free estimate the same quantity
+ // and the 5% split buys no cross-validation while costing a factor of sqrt(20) in precision.
+ double r_model = NAN;
+ double r_model_shell_scaled = NAN;
+ int n_model = 0;
+ // The RMS of ln(k_shell) about its own mean. Near zero says k*exp(-B s^2) already described this
+ // dataset's radial profile, and then the two R-factors agree; a large value says it did not, and
+ // is worth seeing on its own - it is the gap between the two R-factors, named.
+ double radial_misfit = NAN;
+ int shell_scale_bins = 0;
double mean_atom_density_sigma = 0.0; // mean 2mFo-DFc value at atom centres, in sigma
std::string maps_prefix; // where the .ccp4 / _maps.mtz were written
diff --git a/rugnux/ResultReport.cpp b/rugnux/ResultReport.cpp
index 9e9960675..08b3d8ff3 100644
--- a/rugnux/ResultReport.cpp
+++ b/rugnux/ResultReport.cpp
@@ -34,6 +34,9 @@ namespace {
// whether choosing the partner reflections differently could repair it.
// 13: CC_MODEL_* in section 5 - the correlation of the merged intensities with the placed model,
// by resolution shell, which only a run given a model can report.
+ // 14: R_MODEL / R_MODEL_SHELL_SCALED / MODEL_RADIAL_MISFIT in section 5 - R over all the
+ // reflections, with and without one free scale per shell, and the size of that rescale.
+ // R_WORK and R_FREE are unchanged and still describe the model the maps were made from.
// 8: every line that is not `KEY= value` and not blank starts with `#`, warnings included
// (`# WARNING:`); the header gains RUGNUX_DOWNLOAD and BUILD_CXX_FLAGS, and RUGNUX_GIT is
// the full build-time hash.
@@ -1631,6 +1634,15 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
Add(s, KeyReal("R_FREE", mv.r_free, "{:.4f}"));
Add(s, KeyInt("R_WORK_REFLECTIONS", mv.n_work));
Add(s, KeyInt("R_FREE_REFLECTIONS", mv.n_free));
+ // The pair to read BETWEEN two runs, where R_WORK / R_FREE cannot be - see the prose at
+ // the end of this section and the note in ModelValidation.h.
+ if (std::isfinite(mv.r_model_shell_scaled)) {
+ Add(s, KeyReal("R_MODEL", mv.r_model, "{:.4f}"));
+ Add(s, KeyReal("R_MODEL_SHELL_SCALED", mv.r_model_shell_scaled, "{:.4f}"));
+ Add(s, KeyInt("R_MODEL_REFLECTIONS", mv.n_model));
+ Add(s, KeyReal("MODEL_RADIAL_MISFIT", mv.radial_misfit, "{:.3f}"));
+ Add(s, KeyInt("MODEL_RADIAL_SHELLS", mv.shell_scale_bins, true));
+ }
Add(s, KeyReal("BULK_SOLVENT_K_SOL", mv.k_sol, "{:.3f}", true));
Add(s, KeyReal("BULK_SOLVENT_B_SOL", mv.b_sol, "{:.1f}", true));
Add(s, KeyReal("SCALE_OVERALL", mv.k_overall, "{:.4f}", true));
@@ -1746,6 +1758,26 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
" structure's and are a data-quality reading, not a refinement result. The scale and the\n"
" placement are both fitted on the working reflections alone, so the free set is held out\n"
" of the fit and not only out of the sum."));
+ if (std::isfinite(mv.r_model_shell_scaled))
+ Add(s, Prose(fmt::format(
+ " R_WORK and R_FREE above describe THIS dataset against this model, and that is what they\n"
+ " are for. They are not comparable with another dataset's, or with the same dataset\n"
+ " reduced differently: the model is scaled to the data by an overall factor and an\n"
+ " anisotropic B, a shape that can only bend one way with resolution, and whatever the\n"
+ " amplitudes' own radial profile does that this shape cannot follow is reported as R.\n"
+ " Two reductions with different radial profiles therefore differ in R for a reason that\n"
+ " has nothing to do with how well either fits the model.\n"
+ " R_MODEL_SHELL_SCALED ({:.4f}) is the same R with that removed - one free scale per\n"
+ " resolution shell of the table above, so only the agreement INSIDE each shell is left -\n"
+ " and it is the number to compare between runs. R_MODEL ({:.4f}) is the same sum without\n"
+ " the per-shell scale, so the gap between the two is what the radial profile cost.\n"
+ " Both are over every reflection, not the free set: nothing here is refined, so work and\n"
+ " free measure the same thing and using all of them is simply more precise.\n"
+ " MODEL_RADIAL_MISFIT ({:.3f}) is the size of that per-shell rescale, as the RMS of its\n"
+ " logarithm over {} shells: near zero means the scale already described the profile and\n"
+ " the two R values agree. Nothing is written from any of this - the maps, the map\n"
+ " coefficients and the R-factors above are untouched by it.",
+ mv.r_model_shell_scaled, mv.r_model, mv.radial_misfit, mv.shell_scale_bins)));
if (mv.fit_tested)
Add(s, Prose(fmt::format(
" MODEL_FIT says whether these data accept the model: the same model was refitted from\n"
diff --git a/tests/ResultReportTest.cpp b/tests/ResultReportTest.cpp
index 12d61a22b..037a92118 100644
--- a/tests/ResultReportTest.cpp
+++ b/tests/ResultReportTest.cpp
@@ -421,8 +421,11 @@ TEST_CASE("ResultReport_ModelValidationSection", "[Diagnostics]") {
CHECK(dev_text.find("\nMODEL_FIT_NULL_REPLICATES= 5\n") != std::string::npos);
}
CHECK(text.find("\nMODEL_FIT_SIGMA= +4.12\n") != std::string::npos);
- // A run that measured no CC(model, data) writes no key for it.
+ // A run that measured no CC(model, data) writes no key for it, and the shell-scaled R shares
+ // its shells, so it is absent too.
CHECK(text.find("CC_MODEL_OVERALL=") == std::string::npos);
+ CHECK(text.find("R_MODEL_SHELL_SCALED=") == std::string::npos);
+ CHECK(text.find("MODEL_RADIAL_MISFIT=") == std::string::npos);
CHECK(text.find("CC_MODEL_CONFIRMED_TO_D_MIN=") == std::string::npos);
CHECK(text.find("\nMODEL_DECISIONS_TAKEN= ENANTIOMORPH\n") != std::string::npos);
// The hand is ASSUMED from the model, never determined: merged intensities cannot see it.
@@ -473,6 +476,32 @@ TEST_CASE("ResultReport_ModelValidationSection", "[Diagnostics]") {
CHECK(cc_text.find(" 1.80 0.0412 2110 +1.9") != std::string::npos);
CHECK(cc_text.find("The table reads in ONE direction only") != std::string::npos);
+ // The comparable R rides on the same shells. R_WORK and R_FREE are what they always were - the
+ // maps' own scale - and the new keys sit beside them rather than in place of them.
+ ModelValidationResult with_r = with_cc;
+ with_r.r_model = 0.2011;
+ with_r.r_model_shell_scaled = 0.1734;
+ with_r.n_model = 17610;
+ with_r.radial_misfit = 0.312;
+ with_r.shell_scale_bins = 3;
+ result.model_validation = with_r;
+ const auto r_text = RenderResultReport("p", "in.h5", x, result);
+ CHECK(r_text.find("\nR_WORK= 0.1823\n") != std::string::npos);
+ CHECK(r_text.find("\nR_FREE= 0.2145\n") != std::string::npos);
+ CHECK(r_text.find("\nR_MODEL= 0.2011\n") != std::string::npos);
+ CHECK(r_text.find("\nR_MODEL_SHELL_SCALED= 0.1734\n") != std::string::npos);
+ CHECK(r_text.find("\nR_MODEL_REFLECTIONS= 17610\n") != std::string::npos);
+ CHECK(r_text.find("\nMODEL_RADIAL_MISFIT= 0.312\n") != std::string::npos);
+ CHECK(r_text.find("the number to compare between runs") != std::string::npos);
+ // The shell count is a working detail of the rescale, not part of the reading.
+ CHECK(r_text.find("MODEL_RADIAL_SHELLS=") == std::string::npos);
+ {
+ RunProvenance dev;
+ dev.developer = true;
+ CHECK(RenderResultReport("p", "in.h5", x, result, dev)
+ .find("\nMODEL_RADIAL_SHELLS= 3\n") != std::string::npos);
+ }
+
// Nothing significant anywhere: the key still has to be written, saying so.
ModelValidationResult no_signal = with_cc;
for (auto &sh : no_signal.cc_model_shells)
diff --git a/tools/battery/README.md b/tools/battery/README.md
index b4d932462..a9b8eeffc 100644
--- a/tools/battery/README.md
+++ b/tools/battery/README.md
@@ -57,6 +57,20 @@ arms `cc_half_noise_ratio` = (1/CC1/2_rugnux - 1) / (1/(CC1/2_XDS - 0.0005) - 1)
range is shown as a guide: CC1/2 = S/(S+E), so 1/CC1/2 - 1 = E/S, and 2 means a merge as noisy as
XDS's would be with half of its observations.
+**R_free does not compare between runs; `rmodel_shell_scaled` does.** rugnux scales the model to
+the data with an overall factor and a symmetry-constrained anisotropic B, which can only bend one
+way with resolution - deliberately, so that the maps of a batch stay comparable. Whatever the
+amplitudes' own radial profile does that this shape cannot follow is reported as R. So two runs
+whose merged amplitudes have different radial profiles have R_free values that differ for a reason
+that has nothing to do with the data fitting the model: measured on one such pair, R_free moved a
+median +0.042 over eleven sets where `phenix.model_vs_data`, which scales per shell, moved +0.002.
+`rmodel_shell_scaled` is the same R with one free scale per resolution shell, over every reflection
+rather than the free 5% (nothing is refined here, so the free set buys no cross-validation and only
+costs precision): on that same pair its median was +0.001, and it still flagged the one set that
+was a real regression. `radial_misfit` is the size of that per-shell rescale, the RMS of its
+logarithm; when it moves, R_free beside it cannot be read. All three are reported for a human,
+never scored.
+
**Low-resolution R_meas** is reported, not scored: the lowest-resolution shell of rugnux's own
table (`lowres_r_meas`), of the reference-range table (`refres_lowres_r_meas`) and of XDS's
`CORRECT.LP` (`lowres_r_meas_ref`, stored in the manifest as `r_meas_low`), with the shells'
@@ -292,6 +306,7 @@ was run `--unforced`), so its forced XDS-arm rows are left out the same way.
| `lowres_d`, `lowres_r_meas`, `refres_lowres_d`, `refres_lowres_r_meas`, `lowres_d_ref`, `lowres_r_meas_ref`, `lowres_r_meas_ratio` | the lowest-resolution shell's high-resolution limit and R_meas: rugnux's own table, the reference-range table, XDS's; and refres_lowres_r_meas / lowres_r_meas_ref |
| `model`, `model_note` | open arm: the deposited coordinates given to `--model`, or why there were none |
| `rfree`, `rwork`, `cc_model`, `model_fit`, `rfree_deposited`, `rfree_ratio` | open arm with a model: R_FREE, R_WORK, CC_MODEL_OVERALL and MODEL_FIT as rugnux reports them (placement-only, own free set: trend fields), the published R-free, and rfree / rfree_deposited |
+| `rmodel`, `rmodel_shell_scaled`, `radial_misfit` | open arm with a model: R_MODEL, R_MODEL_SHELL_SCALED and MODEL_RADIAL_MISFIT - R over all the reflections, with and without one free scale per resolution shell, and how big that rescale was. `rmodel_shell_scaled` is the R that compares between runs (below) |
| `refmac_rfree`, `refmac_rwork`, `refmac_rfree_depflags`, `refmac_rfree_depdata`, `refmac_rfree_ratio`, `refmac_status`, `refmac_reason` | the REFMAC check (`--model-check`, open arm) |
| `wall_s`, `rugnux_wall_s`, `elapsed_s`, `exit_code`, `gpu_others`, `rugnux_verdict` | timing (`wall_s` is rugnux's own WALL_TIME, not the time spent waiting for the GPU queue), the exit code, other GPU processes seen before the set, and rugnux's own verdict line |
diff --git a/tools/battery/report.py b/tools/battery/report.py
index aef3ddd89..a15089300 100644
--- a/tools/battery/report.py
+++ b/tools/battery/report.py
@@ -18,6 +18,8 @@ NOISE = {
"completeness": ("abs", 1.0), # percentage points
"cell_dev_pct": ("abs", 0.2), # percentage points
"rfree": ("abs", 0.005),
+ "rmodel_shell_scaled": ("abs", 0.005),
+ "radial_misfit": ("abs", 0.05),
"refres_r_meas": ("rel", 0.05),
"refres_isa": ("rel", 0.05),
"refres_lowres_r_meas": ("rel", 0.05),
@@ -188,6 +190,11 @@ def changes(ra, rb):
for name in ("d_min", "isa", "r_meas", "cc_half", "completeness", "cell_dev_pct", "rfree"):
if beyond_noise(name, ra.get(name), rb.get(name)):
out.append(name)
+ # The R that is comparable between runs, and the radial profile that decides whether `rfree`
+ # beside it can be read at all (both only where both runs carry them).
+ for name in ("rmodel_shell_scaled", "radial_misfit"):
+ if ra.get(name) is not None and rb.get(name) is not None and beyond_noise(name, ra[name], rb[name]):
+ out.append(name)
# the like-for-like table only where both runs have one (a pre-schema-3 run has none)
for name in ("refres_r_meas", "refres_isa", "refres_lowres_r_meas"):
if ra.get(name) is not None and rb.get(name) is not None and beyond_noise(name, ra[name], rb[name]):
@@ -215,7 +222,8 @@ def arrow(x, y, fmt=lambda v: str(v)):
def compare_table(doc, rows, only_changed=True):
head = ["set", "arm", "verdict", "space group", "d_min", "ISa", "R_meas", "CC1/2",
- "cell dev %", "R_free", "ref-range R_meas", "ref-range low-res R_meas", "time s", "beyond noise"]
+ "cell dev %", "R_free", "R_model (shell-scaled)", "radial misfit",
+ "ref-range R_meas", "ref-range low-res R_meas", "time s", "beyond noise"]
out = []
for (arm, s), ra, rb, ch in rows:
if only_changed and not ch:
@@ -229,6 +237,8 @@ def compare_table(doc, rows, only_changed=True):
arrow(ra.get("cc_half"), rb.get("cc_half"), lambda v: f"{v:.3f}"),
arrow(ra.get("cell_dev_pct"), rb.get("cell_dev_pct"), lambda v: f"{v:.2f}"),
arrow(ra.get("rfree"), rb.get("rfree"), lambda v: f"{v:.3f}"),
+ arrow(ra.get("rmodel_shell_scaled"), rb.get("rmodel_shell_scaled"), lambda v: f"{v:.3f}"),
+ arrow(ra.get("radial_misfit"), rb.get("radial_misfit"), lambda v: f"{v:.2f}"),
arrow(ra.get("refres_r_meas"), rb.get("refres_r_meas"), lambda v: f"{100 * v:.1f}%"),
arrow(ra.get("refres_lowres_r_meas"), rb.get("refres_lowres_r_meas"),
lambda v: f"{100 * v:.1f}%"),
@@ -342,6 +352,8 @@ def build(run_dir, baseline=None, allow_incomplete=False):
("lowres_r_meas_ratio", "ref-range low-res R_meas / XDS", "{:.3f}"),
("cc_half_noise_ratio", "ref-range CC1/2 noise / XDS", "{:.2f}"),
("rfree", "R_free (placement)", "{:.3f}"),
+ ("rmodel_shell_scaled", "R_model, shell-scaled", "{:.3f}"),
+ ("radial_misfit", "radial misfit", "{:.2f}"),
("rfree_ratio", "R_free ratio", "{:.3f}"), ("wall_s", "time s", "{:.0f}")):
q = quartiles(r.get(name) for r in rs)
if q:
@@ -456,7 +468,18 @@ def build(run_dir, baseline=None, allow_incomplete=False):
doc.h(3, "Per set")
doc.p("Space group: on the open arm the data's own determination. Where --model put the model's "
"enantiomorph on the label, the label is in the note and the group scored is the search's.")
- model = [("R_free", "rfree"), ("R_work", "rwork"), ("CC model", "cc_model"), ("dep R_free", "rfree_deposited"),
+ if any(r.get("rmodel_shell_scaled") for r in res):
+ doc.p("R_free and R_work describe one run against its model and do not carry across runs: the "
+ "model is scaled to the data by an overall factor and an anisotropic B, so whatever the "
+ "amplitudes' radial profile does that this shape cannot follow is reported as R, and a "
+ "change in the profile alone moves R_free further than a real change in the data does. "
+ "R_model (shell-scaled) is the same R with one free scale per resolution shell removing "
+ "exactly that, over every reflection rather than the free 5%, and it is the column to "
+ "read in a delta table. Radial misfit is how big that per-shell rescale had to be (the "
+ "RMS of its logarithm): when it moves between two runs, R_free between them means "
+ "little. Both are reported for a human to judge; nothing is scored on them.")
+ model = [("R_free", "rfree"), ("R_work", "rwork"), ("R_model (shell-scaled)", "rmodel_shell_scaled"),
+ ("radial misfit", "radial_misfit"), ("CC model", "cc_model"), ("dep R_free", "rfree_deposited"),
("R_free ratio", "rfree_ratio")] if any(r.get("rfree") for r in res) else []
model += [("REFMAC R_free", "refmac_rfree_depflags"), ("REFMAC dep data", "refmac_rfree_depdata"),
("REFMAC ratio", "refmac_rfree_ratio")] if any(r.get("refmac_status") for r in res) else []
diff --git a/tools/battery/score.py b/tools/battery/score.py
index 8742d1bab..c44b3bb40 100644
--- a/tools/battery/score.py
+++ b/tools/battery/score.py
@@ -197,6 +197,13 @@ def judge(entry, rep, run_note):
# on rugnux's own free set, so a trend number, not the depositor's R-free
"rfree": fnum(rep.get("R_FREE")), "rwork": fnum(rep.get("R_WORK")),
"model_fit": rep.get("MODEL_FIT"), "cc_model": fnum(rep.get("CC_MODEL_OVERALL")),
+ # R over ALL the reflections, with and without one free scale per resolution shell, and the
+ # size of that rescale. rfree above describes one run and is not comparable with another's:
+ # the model's scale can only bend as k*exp(-B s^2), so a change in the amplitudes' radial
+ # profile alone moves it. rmodel_shell_scaled has that taken out and IS comparable; it is
+ # the R to read in a delta table. radial_misfit says how much there was to take out.
+ "rmodel": fnum(rep.get("R_MODEL")), "rmodel_shell_scaled": fnum(rep.get("R_MODEL_SHELL_SCALED")),
+ "radial_misfit": fnum(rep.get("MODEL_RADIAL_MISFIT")),
"sg_label": None, "accepted_alt": None, "accepted_alt_why": None,
}
# the same merge binned over the reference's range (--report-resolution, XDS arms)