Stills: the error model is reported as its terms a, b, c, not as an ISa

A stills merge no longer reports ISA. On stills 1/b is the I/sigma ceiling of a single still
observation under the stills model - the same frames processed as a rotation sweep read several
times higher - so it describes the processing model more than the instrument and the crystal, and
read on CORRECT.LP's scale it calls good serial data useless. The report gives the error model as
ERROR_MODEL_A, ERROR_MODEL_B (XDS convention, as before) and a new ERROR_MODEL_C, the partiality
term, in the same convention: sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2).
ERROR_MODEL_B= undetermined where b is zero within its error (ISA= said it before).

The same on every other place that showed the stills ISa: the summary's Signal row, the mmCIF
(_reflns.jfjoch_error_model_c instead of _reflns.jfjoch_diffrn_ISa), the log line, the live status
the broker serves, and the viewer's merge-statistics window (no ISa card and no ISa diagnostic for
a stills merge). MergeStatistics::stills marks a merge made by the stills path. The 1/b the
space-group search reads (merge_isa) is unchanged. Rotation output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-11-01 10:34:55 +01:00
1 parent cf874196b0
commit 3caa11caa4
17 files changed
+135 -39

No files matched your search

+2 -2
View File
@@ -388,9 +388,9 @@ Per-shell and overall merging statistics are computed on corrected intensities,
itself count as missing instead of leaving the denominator along with the data,
- the anomalous signal-to-noise $\mathrm{SigAno}$ and the half-set anomalous correlation $\mathrm{CC}_\mathrm{anom}$ (below).
The error model is refined as $\sigma_\mathrm{corr}^2 = a\,\sigma^2 + (b\,\langle I\rangle)^2$, with $a$ set by the scatter of weak (counting-limited) reflections and $b$ the intensity-proportional systematic scatter of the strong ones. On the **rotation** path, **ISa** is the asymptotic ($I\to\infty$) signal-to-noise — by definition the reproducibility limit of the strongest reflections (Diederichs, *Acta Cryst.* **D66** (2010) 733) — and is read directly from the strong symmetry equivalents as the counting-subtracted fractional scatter of well-measured reflection groups (a robust median over strong groups; the $I/\sigma$ threshold is relaxed on weak or radiation-damaged data that has few strong reflections), rather than as $1/b$ of the whole-range fit, whose $b$ is raised slightly by an intermediate-intensity excess and so understates the limit. The asymptotic value is **report-only** — nothing downstream reads it, and the merged $\sigma$ is not floored at $b|I|$ (that floor was removed). The per-observation $\sigma_\mathrm{corr}$ (the merge weights) uses the whole-range $a,b$. The **stills** path has no asymptotic estimate and reports $\mathrm{ISa}=1/b$ directly.
The error model is refined as $\sigma_\mathrm{corr}^2 = a\,\sigma^2 + (b\,\langle I\rangle)^2$, with $a$ set by the scatter of weak (counting-limited) reflections and $b$ the intensity-proportional systematic scatter of the strong ones. On the **rotation** path, **ISa** is the asymptotic ($I\to\infty$) signal-to-noise — by definition the reproducibility limit of the strongest reflections (Diederichs, *Acta Cryst.* **D66** (2010) 733) — and is read directly from the strong symmetry equivalents as the counting-subtracted fractional scatter of well-measured reflection groups (a robust median over strong groups; the $I/\sigma$ threshold is relaxed on weak or radiation-damaged data that has few strong reflections), rather than as $1/b$ of the whole-range fit, whose $b$ is raised slightly by an intermediate-intensity excess and so understates the limit. The asymptotic value is **report-only** — nothing downstream reads it, and the merged $\sigma$ is not floored at $b|I|$ (that floor was removed). The per-observation $\sigma_\mathrm{corr}$ (the merge weights) uses the whole-range $a,b$. The **stills** path adds a partiality term, $\sigma_\mathrm{corr}^2 = a\,\sigma^2 + (b\,\langle I\rangle)^2 + c^2\left(\frac{1-p}{p}\right)^2\langle I\rangle^2$, and reports **no ISa**: there $1/b$ is the $I/\sigma$ ceiling of one fully recorded still under the stills model - the same frames processed as a rotation sweep read several times higher - which describes the processing model more than the instrument and the crystal. It reports $a$, $b$ and $c$ instead (`ERROR_MODEL_A/B/C`, `_reflns.jfjoch_error_model_a/b/c`); $1/b$ is still used inside, by the space-group search.
$a$ and $b$ are **reported in XDS's convention**, which is $\sigma^2 = a(\sigma_0^2 + b I^2)$ with $\mathrm{ISa}=1/\sqrt{ab}$, so the printed pair can be read straight against a `CORRECT.LP`. The internal fit keeps the form above; only the report converts, as $b_\mathrm{XDS} = b^2/a$. Note that $a$ is the same in both conventions and that the two ISa expressions are the same number, $1/\sqrt{a\cdot b^2/a} = 1/b$ — so the rotation log prints **two** ISa, the whole-range $1/b$ (XDS's meaning) and the strong-reflection asymptote beside it, which can only ever be the more optimistic of the two. The mmCIF follows the same split: `_reflns.jfjoch_diffrn_ISa` is the whole-range value, directly comparable with a `CORRECT.LP`, and the asymptote is written separately as `_reflns.jfjoch_diffrn_ISa_asymptotic`, with `_reflns.jfjoch_error_model_a` and `_b` alongside so the number can be re-derived. Note that a file written before this change carries the *asymptote* under the plain `ISa` name. A third, unrelated $b$ appears in the space-group search (§13.1); it is fitted with the $\sigma^2$ coefficient held at 1 and its gate constants are calibrated in that convention.
$a$ and $b$ are **reported in XDS's convention**, which is $\sigma^2 = a(\sigma_0^2 + b I^2)$ with $\mathrm{ISa}=1/\sqrt{ab}$, so the printed pair can be read straight against a `CORRECT.LP`. The internal fit keeps the form above; only the report converts, as $b_\mathrm{XDS} = b^2/a$ (and on stills $c_\mathrm{XDS} = c^2/a$). Note that $a$ is the same in both conventions and that the two ISa expressions are the same number, $1/\sqrt{a\cdot b^2/a} = 1/b$ — so the rotation log prints **two** ISa, the whole-range $1/b$ (XDS's meaning) and the strong-reflection asymptote beside it, which can only ever be the more optimistic of the two. The mmCIF follows the same split: `_reflns.jfjoch_diffrn_ISa` is the whole-range value, directly comparable with a `CORRECT.LP`, and the asymptote is written separately as `_reflns.jfjoch_diffrn_ISa_asymptotic`, with `_reflns.jfjoch_error_model_a` and `_b` alongside so the number can be re-derived. Note that a file written before this change carries the *asymptote* under the plain `ISa` name. A third, unrelated $b$ appears in the space-group search (§13.1); it is fitted with the $\sigma^2$ coefficient held at 1 and its gate constants are calibrated in that convention.
**Anomalous signal-to-noise (SigAno).** The strength of the anomalous signal is reported per shell and overall as $\mathrm{SigAno}=\langle|\Delta I|\rangle / \langle\sigma(\Delta I)\rangle$, where $\Delta I = I(+)-I(-)$ over acentric reflections measured in both Bijvoet hands and $\sigma(\Delta I)=\sqrt{\sigma_+^2+\sigma_-^2}$. It is computed from the **full-multiplicity** inverse-variance $I(+)/I(-)$ split (the same one written to the output), i.e. from all observations rather than a half-set. For pure noise $\mathrm{SigAno}$ approaches the half-normal value $\sqrt{2/\pi}\approx0.8$, and it rises above $1$ once a real anomalous difference is present. A half-set anomalous correlation ($\mathrm{CC}_\mathrm{anom}$) is reported beside it: $\Delta I$ is formed once per half-set and the two are correlated over the acentric pairs where **both** hands split into two non-empty halves, per shell and overall as one correlation rather than a mean of shells. Unlike SigAno it is not a ratio against the error model, so an optimistic $\sigma$ cannot inflate it. It has no floor either: subtracting the two Bijvoet hands cancels the large common intensity that keeps $\mathrm{CC}_{1/2}$ non-negative, so on data with little anomalous signal and about two observations per mate it goes strongly negative. That is a property of the statistic and is reported as measured. It agrees with AIMLESS's `CCanom` and `phenix.merging_statistics`' `cc_anom`; XDS's `Anomal Corr` is a **different quantity and is not comparable with it** — on the same observations it reads two to three times higher in the low shells. $\mathrm{CC}_\mathrm{anom}$ is a rotation-path statistic (the stills merge forms no half-set anomalous difference), and it is reported in the `CCanom` column of the printed merge-statistics table and as `CC_ANOM=` in the results report — not in the mmCIF. SigAno is emitted only when an anomalous split was made, using the standard PDBx items `_reflns.pdbx_absDiff_over_sigma_anomalous` (overall) and `_reflns_shell.pdbx_absDiff_over_sigma_anomalous` (per shell), and appears as the `SigAno` column of the same table. Where either could not be measured at all — a Friedel-merged run that split no Bijvoet pair, a shell too thin to split one in both hands — the table prints `-` and the results report writes no key, which is not the same statement as a value measured to be zero.
+2 -1
View File
@@ -24,9 +24,10 @@ ignores them, and one that does can find them without guessing.
| item | meaning |
|---|---|
| `_reflns.jfjoch_diffrn_ISa` | Asymptotic I/σ in **XDS's sense**: the whole-range `1/√(a·b)` of the error model, so it can be read directly against a `CORRECT.LP` |
| `_reflns.jfjoch_diffrn_ISa` | Asymptotic I/σ in **XDS's sense**: the whole-range `1/√(a·b)` of the error model, so it can be read directly against a `CORRECT.LP`. Rotation only: a stills merge writes the error-model terms instead |
| `_reflns.jfjoch_diffrn_ISa_asymptotic` | The **strong-reflection** tier — the counting-subtracted scatter of well-measured groups. XDS has no equivalent, and it can only ever be the more optimistic of the two. Rotation path only |
| `_reflns.jfjoch_error_model_a`, `_b` | The error model in XDS's convention, `σ² = a(σ₀² + b·I²)`, so the ISa above is re-derivable from the file rather than taken on trust |
| `_reflns.jfjoch_error_model_c` | Stills only: the partiality term, `σ² = a(σ₀² + b·I² + c·((1−p)/p)²·I²)` with p the observation's partiality |
| `_reflns.jfjoch_second_moment_I` | Twinning second moment ⟨I²⟩/⟨I⟩² — 2.00 untwinned, 1.50 for a perfect twin |
| `_reflns.jfjoch_L_test_mean_abs_L`, `_L_test_mean_L_squared` | Padilla–Yeates L-test. ⟨\|L\|⟩ is 0.500 untwinned / 0.375 for a perfect twin; ⟨L²⟩ is 0.333 / 0.200. Written only when the test found pairs |
| `_reflns.jfjoch_radiation_damage_relative_B` | Relative *B* from the first to the last rotation batch (Ų); positive is the usual direction, high-resolution intensity fading with dose |
+13 -1
View File
@@ -231,10 +231,22 @@ The keys most scripts read, by section; the rest of this page explains the ones
| `UNIQUE_REFLECTIONS`, `TOTAL_OBSERVATIONS`, `COMPLETENESS`, `MULTIPLICITY` | Over the written range; completeness in percent |
| `I_OVER_SIGMA`, `R_MEAS`, `CC_HALF`, `WILSON_B` | The overall merging statistics; the shell table beside them is in the comment lines |
| `SIGANO`, `CC_ANOM`, `FRIEDELS_LAW` | The anomalous signal, and whether Friedel pairs were merged (`FALSE` with `-A`) |
| `ERROR_MODEL_A`, `ERROR_MODEL_B`, `ISA` | The error model in XDS's convention and ISa = 1/sqrt(a·b); `ISA= undetermined` where b is zero within its error, so 1/b would be noise |
| `ERROR_MODEL_A`, `ERROR_MODEL_B`, `ISA` | The error model in XDS's convention and ISa = 1/sqrt(a·b); `ISA= undetermined` where b is zero within its error, so 1/b would be noise. Rotation only for `ISA` |
| `ERROR_MODEL_C` | Stills only: the partiality term of the stills error model, σ² = a(σ₀² + b·I² + c·((1−p)/p)²·I²) in the same convention. A stills report writes no `ISA` ([below](#the-stills-error-model)), and `ERROR_MODEL_B= undetermined` where b is zero within its error |
| `R_WORK`, `R_FREE` | Section 5: the model as placed against these data ([below](#model-validation)); the refined model's are `REFINE_R_WORK`/`REFINE_R_FREE` |
| `R_FREE_FLAGS` | Section 5: the free set both are read on - `DRAWN` by this run, inherited from the `-z` `REFERENCE`, or from the `--free-flags` `FILE` |
### The stills error model
A stills merge reports its error model as its terms, `ERROR_MODEL_A`, `ERROR_MODEL_B` and
`ERROR_MODEL_C`, with σ² = a(σ₀² + b·I² + c·((1−p)/p)²·I²), p being the partiality of the observation:
b is the systematic error of a fully recorded still and c the error a partial one adds. It writes no
`ISA`. On stills 1/sqrt(a·b) is the I/σ ceiling of a single still observation under the stills model -
the same frames processed as a rotation sweep read several times higher - so it describes the
processing model more than the instrument and the crystal, and read on `CORRECT.LP`'s scale it would
call good serial data useless. The serial figures of merit are `R_SPLIT=`, `CC_HALF=` and `CC_STAR=`.
The mmCIF follows: `_reflns.jfjoch_error_model_c` instead of `_reflns.jfjoch_diffrn_ISa`.
## The summary, and what the run decided
The file opens with a **`SUMMARY`** section, above everything it summarises. It exists because the
+6 -2
View File
@@ -428,8 +428,9 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
// ISa in XDS's sense: the whole-range 1/sqrt(a*b) of the error model below, so this item can
// be read straight against a CORRECT.LP. The strong-reflection asymptote - a tier XDS does not
// have, and always the more optimistic of the two - is written separately rather than here.
out << "_reflns.jfjoch_diffrn_ISa " << CifStr(error_model.isa)
<< " # 1/sqrt(a*b), the XDS convention\n";
if (!statistics.stills)
out << "_reflns.jfjoch_diffrn_ISa " << CifStr(error_model.isa)
<< " # 1/sqrt(a*b), the XDS convention\n";
if (!error_model.isa_asymptotic.empty())
out << "_reflns.jfjoch_diffrn_ISa_asymptotic " << CifStr(error_model.isa_asymptotic)
<< " # strong-reflection asymptote (Diederichs); rotation path only\n";
@@ -438,6 +439,9 @@ void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
<< " # sigma^2 = a*(sigma0^2 + b*I^2), XDS convention\n";
if (!error_model.b.empty())
out << "_reflns.jfjoch_error_model_b " << CifStr(error_model.b) << "\n";
if (!error_model.c.empty())
out << "_reflns.jfjoch_error_model_c " << CifStr(error_model.c)
<< " # stills partiality term: + c*((1-p)/p)^2*I^2\n";
// Dataset-wide isotropic Wilson B-factor estimate (standard PDBx item), analogous to XDS's
// "WILSON LINE ... B=". Emitted only when the log-linear fit succeeded.
if (std::isfinite(statistics.wilson_b) && statistics.wilson_b > 0.0)
+4 -1
View File
@@ -21,12 +21,15 @@ struct CellEsd; // geom_refinement/CellEsd.h
// The error model as it is reported, already formatted. `isa` is the whole-range 1/sqrt(a*b), the
// same quantity XDS's ISa denotes, so a file written here is directly comparable with a CORRECT.LP;
// `isa_asymptotic` is the strong-reflection tier, which only the rotation path has. `a` and `b` are
// in XDS's convention, sigma^2 = a*(sigma0^2 + b*I^2). Empty strings are written as unknown.
// in XDS's convention, sigma^2 = a*(sigma0^2 + b*I^2), and `c` is the stills partiality term in the same
// convention, sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2). Empty strings are written as unknown.
// A stills merge writes no ISa (MergeStatistics::stills).
struct ErrorModelReport {
std::string isa;
std::string isa_asymptotic;
std::string a;
std::string b;
std::string c;
};
// What the reflection files record about where they came from, beyond the experiment itself and the
+1
View File
@@ -949,6 +949,7 @@ MergeStatistics MergeOnTheFly::MergeStats(const std::vector<MergedReflection> &m
};
MergeStatistics out;
out.stills = true;
out.shells.resize(n_shells);
out.n_observations_rejected = reject_count;
+10
View File
@@ -182,6 +182,10 @@ struct MergeStatistics {
std::vector<MergeStatisticsShell> shells;
MergeStatisticsShell overall;
AnomalousMergeStatistics per_hand;
// Made by the stills merge (MergeOnTheFly). Its error model is reported as its terms, not as an
// ISa: on stills 1/b is the ceiling of one still observation under the stills model, which is not
// the instrument and crystal error CORRECT.LP's ISa measures.
bool stills = false;
// Dataset-wide isotropic Wilson B-factor estimate (A^2) from the log-linear fit of the shell-mean
// merged intensity against 1/d^2 (CalcGlobalWilsonB) - the analogue of XDS's "WILSON LINE ... B=".
@@ -280,6 +284,12 @@ inline XdsErrorModel ToXdsErrorModel(double a, double b) {
return {a, b * b / a, 1.0 / b};
}
// The stills partiality term c (MergeOnTheFly::ErrorModelC) in the same convention:
// sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2).
inline double ToXdsPartialityTerm(double a, double c) {
return a > 0.0 ? c * c / a : 0.0;
}
// Why the stills merge took or left one observation: a bit for every test it fails, 0 = merged
// (MergeOnTheFly::Verdicts; a diagnostic, --export-observations).
enum ObservationFlag : uint32_t {
+1 -1
View File
@@ -426,7 +426,7 @@ void JFJochReceiverRugnux::RugnuxThread() {
std::optional<double> d_min, isa;
if (result.has_merge_statistics)
SetIfFinite(d_min, result.merge_statistics.overall.d_min);
if (result.error_model_isa_resolved && result.error_model_isa > 0.0)
if (!result.merge_statistics.stills && result.error_model_isa_resolved && result.error_model_isa > 0.0)
isa = result.error_model_isa;
live_status.Headline(result.space_group ? result.space_group->xhm() : "", d_min, isa);
// The placed model and a copy of its map coefficients (the end message takes the originals), and the
+37 -14
View File
@@ -47,6 +47,8 @@ namespace {
// the XDS GEOMETRY KEYWORDS section; OVERLOADED_* / OVERLAPPED_* and the OVERLOADS / OVERLAPS flags.
// 17: INDEXING_AMBIGUITY_RESOLVED and the AMBIGUITY_* keys in section 2 - serial stills, the
// indexing ambiguity resolved from the data alone (or measured against the reference's answer).
// 18: stills report their error model as ERROR_MODEL_A / _B / _C (C the partiality term) and no
// ISA (nor ISa in the summary's Signal row).
// 8: every line that is not `KEY= value` and not blank starts with `#`, warnings included
// (`# WARNING:`); the header gains RUGNUX_DOWNLOAD and BUILD_CXX_FLAGS, and RUGNUX_GIT is
// the full build-time hash.
@@ -1543,7 +1545,14 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
// The error model in XDS's convention, so the numbers are directly comparable with a
// CORRECT.LP. Kept in the default report because beamline monitoring scripts read them.
Add(s, KeyReal("ERROR_MODEL_A", result.error_model_a, "{:.4f}"));
Add(s, KeyReal("ERROR_MODEL_B", result.error_model_b, "{:.4e}"));
// On stills the terms are the whole report of the error model, so a b that is zero within
// its error says so here; on rotation ISA= says it.
if (ms.stills && !result.error_model_isa_resolved)
Add(s, KeyText("ERROR_MODEL_B", "undetermined"));
else
Add(s, KeyReal("ERROR_MODEL_B", result.error_model_b, "{:.4e}"));
if (ms.stills)
Add(s, KeyReal("ERROR_MODEL_C", result.error_model_c, "{:.4e}"));
// A b that is zero within its error gives a 1/b that is the noise in b, not an I/sigma.
Add(s, KeyText("SCALING_ITERATIONS",
fmt::format("{} partials, {} fulls{}", result.scaling_iterations_partials,
@@ -1558,12 +1567,17 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
? "the space-group search merge and the final merge"
: !result.scaling_converged ? "the final merge"
: "the space-group search merge"));
if (result.error_model_isa_resolved)
Add(s, KeyReal("ISA", result.error_model_isa, "{:.2f}"));
else
Add(s, KeyText("ISA", "undetermined"));
if (result.error_model_isa_resolved && result.error_model_isa_asymptotic > 0.0)
Add(s, KeyReal("ISA_ASYMPTOTIC", result.error_model_isa_asymptotic, "{:.2f}", true));
// No ISA on stills: there 1/b is the ceiling of one still observation under the stills
// model, a property of the processing more than of the instrument, and read on CORRECT.LP's
// scale it calls good serial data useless.
if (!ms.stills) {
if (result.error_model_isa_resolved)
Add(s, KeyReal("ISA", result.error_model_isa, "{:.2f}"));
else
Add(s, KeyText("ISA", "undetermined"));
if (result.error_model_isa_resolved && result.error_model_isa_asymptotic > 0.0)
Add(s, KeyReal("ISA_ASYMPTOTIC", result.error_model_isa_asymptotic, "{:.2f}", true));
}
// Only when TRUE: FALSE is the ordinary case and says nothing.
if (result.has_reference) {
Add(s, KeyBool("REFERENCE_DATA_USED", true));
@@ -1626,9 +1640,16 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
" both hands the quantity does not exist: the key is then absent here and the column is\n"
" a dash in the table below, which is not the same claim as a signal measured to be\n"
" zero.\n\n"
" ERROR_MODEL_A / ERROR_MODEL_B are in XDS's convention, sigma^2 = a*(sigma0^2 + b*I^2),\n"
" so ISA = 1/sqrt(a*b) means what CORRECT.LP's ISa means. ISA_ASYMPTOTIC, where present,\n"
" is the strong-reflection tier only.", true));
+ std::string(ms.stills
? " ERROR_MODEL_A / ERROR_MODEL_B / ERROR_MODEL_C are the stills error model in XDS's\n"
" convention, sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2), with p the partiality\n"
" of the observation: b is the systematic error of a fully recorded still, c the added\n"
" error of a partial one. No ISa is given: 1/sqrt(a*b) here is the I/sigma ceiling of\n"
" a single still observation under the stills model, not what CORRECT.LP's ISa\n"
" measures; R_SPLIT, CC_HALF and CC_STAR are the numbers to quote."
: " ERROR_MODEL_A / ERROR_MODEL_B are in XDS's convention, sigma^2 = a*(sigma0^2 + b*I^2),\n"
" so ISA = 1/sqrt(a*b) means what CORRECT.LP's ISa means. ISA_ASYMPTOTIC, where present,\n"
" is the strong-reflection tier only."), true));
{
std::ostringstream shells;
@@ -3340,10 +3361,12 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
sig += fmt::format(" R_meas {:.1f} %", 100.0 * o.r_meas);
if (std::isfinite(o.cc_half))
sig += fmt::format(" CC1/2 {:.3f}", o.cc_half);
if (!result.error_model_isa_resolved)
sig += " ISa undetermined";
else if (result.error_model_isa > 0.0)
sig += fmt::format(" ISa {:.1f}", result.error_model_isa);
if (!result.merge_statistics.stills) { // no ISa on stills (section 3)
if (!result.error_model_isa_resolved)
sig += " ISa undetermined";
else if (result.error_model_isa > 0.0)
sig += fmt::format(" ISa {:.1f}", result.error_model_isa);
}
row("Signal", sig);
if (std::isfinite(o.cc_anom))
row("Anomalous", o.cc_anom > CC_ANOM_SIGNAL
+5 -1
View File
@@ -564,11 +564,15 @@ struct ProcessResult {
// the error-model asymptotic I/sigma (the strong-reflection tier; the whole-range XDS-equivalent is 1/b); has_reference is true when a reference MTZ drove CCref.
bool has_merge_statistics = false;
MergeStatistics merge_statistics;
double error_model_isa = 0.0; // whole-range 1/sqrt(a*b), XDS's meaning
double error_model_isa = 0.0; // whole-range 1/sqrt(a*b), XDS's meaning; on stills 1/b of
// the stills model, used inside (space-group search), not reported
double error_model_isa_asymptotic = 0.0; // strong-reflection tier; rotation path only, 0 on stills
bool error_model_isa_resolved = true; // false: b is zero within its error, ISa printed "undetermined"
double error_model_a = 0.0; // XDS convention: sigma^2 = a*(sigma0^2 + b*I^2)
double error_model_b = 0.0;
// Stills only: the partiality term, in the same convention,
// sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2).
double error_model_c = 0.0;
bool has_reference = false;
// How the reference MTZ was matched: the operator the merged indices were put through to reach
// its frame (h,k,l where none), the intensity CC and matched fraction in that frame, and whether
+8 -5
View File
@@ -1788,11 +1788,12 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
merge_engine.RefineErrorModel(merge_input);
if (merge_engine.ErrorModelActive())
{
// XDS convention, so the pair reads directly against a CORRECT.LP; the stills ISa is
// already the whole-range 1/b, there being no asymptote tier on this path.
// XDS convention, sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2). No ISa: on stills 1/b is
// the ceiling of one still observation, not what CORRECT.LP's ISa measures.
const auto em = ToXdsErrorModel(merge_engine.ErrorModelA(), merge_engine.ErrorModelB());
logger.Info("Error model (XDS convention): a={:.3f} b={:.3e} ISa={:.1f} partiality c={:.3f} chi2={:.2f}",
em.a, em.b, em.isa, merge_engine.ErrorModelC(), merge_engine.ErrorModelChi2());
logger.Info("Error model (XDS convention): a={:.3f} b={:.3e} c={:.3e} chi2={:.2f}",
em.a, em.b, ToXdsPartialityTerm(merge_engine.ErrorModelA(), merge_engine.ErrorModelC()),
merge_engine.ErrorModelChi2());
}
merge_engine.AddImages(merge_input);
if (config_.export_observations && !for_search && write_output)
@@ -1828,6 +1829,7 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
result.error_model_isa_resolved = merge_engine.ErrorModelBResolved();
result.error_model_a = em.a;
result.error_model_b = em.b;
result.error_model_c = ToXdsPartialityTerm(merge_engine.ErrorModelA(), merge_engine.ErrorModelC());
}
out.cc_half_before_corrections = out.statistics.overall.cc_half; // no surfaces on this path
logger.Info("Merge complete ({} unique reflections, {})", out.merged.size(), label);
@@ -5602,7 +5604,8 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
result.error_model_isa_asymptotic > 0 && result.error_model_isa_resolved
? fmt::format("{:.2f}", result.error_model_isa_asymptotic) : std::string(),
result.error_model_a > 0 ? fmt::format("{:.3f}", result.error_model_a) : std::string(),
result.error_model_b > 0 ? fmt::format("{:.4e}", result.error_model_b) : std::string()};
result.error_model_b > 0 ? fmt::format("{:.4e}", result.error_model_b) : std::string(),
result.error_model_c > 0 ? fmt::format("{:.4e}", result.error_model_c) : std::string()};
MtzTable merged_mtz = MergedMtzTable(sm.merged, *result.consensus_cell, experiment_, config_.provenance);
// The unmerged MTZ reads the fulls until it is made.
if (unmerged_mtz.valid())
+10 -5
View File
@@ -2503,6 +2503,7 @@ int RugnuxInputRun::PrepareRugnuxInput() {
std::optional<double> resolution_fit_A; // CC1/2 crossing, without the one-shell extension
double error_model_a = 0.0;
double error_model_b = 0.0;
double error_model_c = 0.0; // stills only
std::optional<MergeStatistics> reference_statistics; // --report-resolution
double reference_isa = 0.0;
@@ -2586,11 +2587,12 @@ int RugnuxInputRun::PrepareRugnuxInput() {
merge_engine.RefineErrorModel(reflections);
if (merge_engine.ErrorModelActive())
{
// XDS convention, so the pair reads directly against a CORRECT.LP; the stills ISa is
// already the whole-range 1/b, there being no asymptote tier on this path.
// XDS convention, sigma^2 = a*(sigma0^2 + b*I^2 + c*((1-p)/p)^2*I^2). No ISa: on stills 1/b is
// the ceiling of one still observation, not what CORRECT.LP's ISa measures.
const auto em = ToXdsErrorModel(merge_engine.ErrorModelA(), merge_engine.ErrorModelB());
logger.Info("Error model (XDS convention): a={:.3f} b={:.3e} ISa={:.1f} partiality c={:.3f} chi2={:.2f}",
em.a, em.b, em.isa, merge_engine.ErrorModelC(), merge_engine.ErrorModelChi2());
logger.Info("Error model (XDS convention): a={:.3f} b={:.3e} c={:.3e} chi2={:.2f}",
em.a, em.b, ToXdsPartialityTerm(merge_engine.ErrorModelA(), merge_engine.ErrorModelC()),
merge_engine.ErrorModelChi2());
}
merge_engine.AddImages(reflections);
merged_reflections = merge_engine.ExportReflections();
@@ -2615,6 +2617,7 @@ int RugnuxInputRun::PrepareRugnuxInput() {
error_model_isa_resolved = merge_engine.ErrorModelBResolved();
error_model_a = em.a;
error_model_b = em.b;
error_model_c = ToXdsPartialityTerm(merge_engine.ErrorModelA(), merge_engine.ErrorModelC());
}
}
@@ -2804,7 +2807,8 @@ int RugnuxInputRun::PrepareRugnuxInput() {
error_model_isa_asymptotic > 0 && error_model_isa_resolved
? fmt::format("{:.2f}", error_model_isa_asymptotic) : std::string(),
error_model_a > 0 ? fmt::format("{:.3f}", error_model_a) : std::string(),
error_model_b > 0 ? fmt::format("{:.4e}", error_model_b) : std::string()};
error_model_b > 0 ? fmt::format("{:.4e}", error_model_b) : std::string(),
error_model_c > 0 ? fmt::format("{:.4e}", error_model_c) : std::string()};
WriteReflections(merged_reflections, *experiment.GetUnitCell(), experiment, merged_statistics,
em_report, twinning, output_provenance, output_prefix, static_cast<size_t>(nthreads),
&scaled_fulls);
@@ -2842,6 +2846,7 @@ int RugnuxInputRun::PrepareRugnuxInput() {
scale_result.error_model_isa_resolved = error_model_isa_resolved;
scale_result.error_model_a = error_model_a;
scale_result.error_model_b = error_model_b;
scale_result.error_model_c = error_model_c;
scale_result.resolution_fit_A = resolution_fit_A;
scale_result.reference_merge_statistics = reference_statistics;
scale_result.reference_isa = reference_isa;
+26
View File
@@ -832,6 +832,32 @@ TEST_CASE("ResultReport_IsaUndetermined", "[Diagnostics]") {
CHECK(text.find("ISA_ASYMPTOTIC") == std::string::npos);
}
// A stills merge reports its error model as its three terms and no ISa anywhere.
TEST_CASE("ResultReport_StillsErrorModelTerms", "[Diagnostics]") {
DiffractionExperiment x(DetJF(1));
ProcessResult result;
result.consensus_cell = UnitCell{.a = 79.0f, .b = 79.0f, .c = 38.0f,
.alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f};
result.has_merge_statistics = true;
result.merge_statistics.stills = true;
result.merge_statistics.overall.d_max = 50.0f;
result.merge_statistics.overall.d_min = 1.50f;
result.merge_statistics.overall.cc_half = 0.99;
result.error_model_a = 1.1;
result.error_model_b = 0.1;
result.error_model_c = 0.02;
result.error_model_isa = 3.0;
const auto text = RenderResultReport("p", "in.h5", x, result);
CHECK(text.find("\nERROR_MODEL_B= 1.0000e-01\n") != std::string::npos);
CHECK(text.find("\nERROR_MODEL_C= 2.0000e-02\n") != std::string::npos);
CHECK(text.find("ISA=") == std::string::npos);
CHECK(text.find("ISa 3.0") == std::string::npos);
result.error_model_isa_resolved = false;
CHECK(RenderResultReport("p", "in.h5", x, result).find("\nERROR_MODEL_B= undetermined\n") != std::string::npos);
}
// The completeness of the range where the signal is, beside the completeness of the range that was
// written. Users compare the single overall figure against another program's and conclude the run
// lost data, when it is the deliberately generous cut diluting the denominator - so the report says
+2 -2
View File
@@ -201,7 +201,7 @@ def parse_cif(path):
"isig": _f(val("_reflns.pdbx_netI_over_sigmaI")),
"rmeas": _f(val("_reflns.pdbx_Rrim_I_all"), 100.0),
"cc": _f(val("_reflns.pdbx_CC_half"), 100.0),
"isa": _f(val("_reflns.jfjoch_diffrn_ISa")),
"em_b": _f(val("_reflns.jfjoch_error_model_b")), # the stills error model writes no ISa
"dmin": _f(val("_reflns.d_resolution_high")),
"dmax": _f(val("_reflns.d_resolution_low")),
}
@@ -535,7 +535,7 @@ SUMMARY_COLS = [
("<I/sig>", "isig", 8, 2),
("Rmeas%", "rmeas", 7, 2),
("CC1/2%", "cc", 7, 2),
("ISa", "isa", 6, 2),
("EM b", "em_b", 7, 4),
]
+3 -2
View File
@@ -149,7 +149,8 @@ JFJochMergeStatsWindow::JFJochMergeStatsWindow(const QString &title, const Merge
// by the CC1/2 logistic cutoff, so it varies per run - show it first.
if (o.d_min < o.d_max)
hero->addWidget(MakeCard(QString::asprintf("%.2f", o.d_min), "Resolution [Å]", this));
hero->addWidget(MakeCard(num(isa, 1), "ISa", this));
if (!std::isnan(isa)) // NaN: a stills merge, which reports no ISa
hero->addWidget(MakeCard(num(isa, 1), "ISa", this));
hero->addWidget(MakeCard(pct(o.cc_half * 100.0), "CC½", this));
hero->addWidget(MakeCard(pct(o.r_meas * 100.0), "R-meas", this));
hero->addWidget(MakeCard(pct(completeness), "Completeness", this));
@@ -324,7 +325,7 @@ JFJochMergeStatsWindow::JFJochMergeStatsWindow(const QString &title, const Merge
table_->verticalHeader()->setVisible(false);
stack_->addWidget(chart_); // page 0
stack_->addWidget(table_); // page 1
isa_page_ = buildIsaDiagnostic(merged_i_sigma, isa);
isa_page_ = std::isnan(isa) ? nullptr : buildIsaDiagnostic(merged_i_sigma, isa);
if (isa_page_)
stack_->addWidget(isa_page_); // page 2
else
+1 -1
View File
@@ -31,7 +31,7 @@ class JFJochMergeStatsWindow : public QWidget {
public:
// `report` is the rugnux result report text (<prefix>_report.txt), shown as its own view.
JFJochMergeStatsWindow(const QString &title, const MergeStatistics &stats,
double isa, bool has_reference,
double isa, bool has_reference, // isa NaN: none (stills)
const TwinningAnalysisResult &twinning,
const std::optional<gemmi::SpaceGroup> &space_group,
const std::optional<SearchSpaceGroupResult> &space_group_search,
@@ -9,6 +9,8 @@
#include "../../writer/RugnuxModelFile.h"
#include "../../common/ThreadAffinity.h"
#include <cmath>
#include <QApplication>
#include <QCheckBox>
#include <QClipboard>
@@ -797,7 +799,8 @@ void JFJochProcessingJobsWindow::onFinished(ProcessResult result, QString report
jobs_[row].report = report;
jobs_[row].checks = std::move(checks);
jobs_[row].merge_stats = result.merge_statistics;
jobs_[row].isa = result.error_model_isa;
// A stills merge reports its error model as terms, not as an ISa (the report says why): NaN, no card.
jobs_[row].isa = result.merge_statistics.stills ? NAN : result.error_model_isa;
jobs_[row].merge_has_reference = result.has_reference;
jobs_[row].twinning = result.twinning;
jobs_[row].space_group = result.space_group;