From 8a8c992ccadd1196eb3c7d828ba91ecaf643f510 Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 26 Sep 2026 10:12:51 +0200 Subject: [PATCH] Rugnux report: warn only where the evidence supports the claim Report-layer fixes from the warnings audit of the rc173 full battery. Processing is unchanged (merged MTZ/HKL bit-identical on the sets checked). - SUPERCELL_POSSIBLE stays a key and a summary line; a warning only where the rocking part reaches 20% (sub-cells that refine normally rocked at 2.1-16.2%, the accepted doubled cells at 4.2% and 29.1%). The summary says where further lattice domains may put spots on the half-integer nodes. - LATTICE_TRANSLATION only where the Patterson at the vector reaches 90% of the origin (UNDECLARED_LATTICE_TRANSLATION_PCT, new); the two admitted vectors on the battery read 79% and 85% on crystals refining in the deposited cell, and are now reported as a very strong pseudo-translation. PSEUDO_TRANSLATION warns from a 20% peak (xtriage convention; detected peaks were 8.1% on one set, 24.8-63.2% on the rest). - <|L|> outside 0.375-0.55 (on the adopted or the pre-search merge) is TWINNING_VERDICT= NOT_READABLE with no warning, instead of a twin or SYMMETRY_SUSPECT reading. - NO_LATTICE no longer fires on a rotation run that indexed the sweep with no frame indexed on its own; INDEXING_RATE is --developer on rotation. - A rotation run that finds no lattice writes a report (VERDICT= FAILED, NO_LATTICE) before exiting 1, instead of an input-parameter error and no report (NoLatticeFound). - Powder and Ice summary lines, ICE_* keys, and a POWDER_RINGS flag from a 0.5 spot fraction. - SWEEP_GAPS warns only where the degraded ranges cover 10% of the sweep (77 -> 48 sets). - A single rotation sweep no longer carries an INDEXING_AMBIGUITY warning (one orientation matrix, consistent hand); INDEXING_AMBIGUITY_OPERATORS and an Indexing choice line instead. - A DOMAIN more than 10 deg away is a second crystal; the MULTIPLE_LATTICES sum is documented. - Mosaicity line and docs name XDS's suggested REFLECTING_RANGE_E.S.D. as the comparable number (0.90x median over 34 in-house sets; per-image SIGMAR runs ~1.35x higher). - HARMONIC_CONTAMINATION, SCALING_NOT_CONVERGED, POWDER_RINGS in the documented vocabulary. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C --- docs/CHANGELOG.md | 9 + docs/RUGNUX_REPORT.md | 96 ++++-- .../scale_merge/TranslationalNCS.cpp | 16 +- image_analysis/scale_merge/TranslationalNCS.h | 4 + rugnux/ReportDocument.h | 1 + rugnux/ResultReport.cpp | 292 +++++++++++++++--- rugnux/Rugnux.cpp | 31 +- rugnux/Rugnux.h | 25 ++ rugnux/rugnux_cli.cpp | 20 +- tests/ResultReportTest.cpp | 195 ++++++++++++ 10 files changed, 606 insertions(+), 83 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 81186c6dc..e91ec3229 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -33,6 +33,15 @@ * Rugnux reports further lattices found in the spots the main lattice leaves over on rotation data - split-crystal and twin domains, a crystal that changes along the sweep, a related or an unrelated cell (`EXTRA_LATTICE_*` keys, a `Multiple lattices` summary line) - and warns (`MULTIPLE_LATTICES`) where domains of the crystal hold at least 10 % of the spot intensity; processing is unchanged. * Rugnux reports the mosaicity the merge used (`MOSAICITY_DEG=` with `MOSAICITY_DEG_P10=`/`_P90=` over the frames, and a summary line), as XDS's sigma_M (`REFLECTING_RANGE_E.S.D.`). * `rugnux --polarization` is documented as the polarization degree (XDS `FRACTION_OF_POLARIZATION` = (1 + p)/2); the default 0.99 suits undulators. +* Rugnux warns about a possible supercell (`SUPERCELL_POSSIBLE`) only where the half-integer class rocks at 20 % or more; weaker cases are stated on the summary's `Supercell` line. +* Rugnux warns about an undeclared lattice translation only where the Patterson reaches 90 % of the origin there (`UNDECLARED_LATTICE_TRANSLATION_PCT=`), and about a pseudo-translation only from a 20 % peak. +* Rugnux reports `TWINNING_VERDICT= NOT_READABLE`, without a warning, where <|L|> lies outside its physical range (below 0.375 or above 0.55). +* Rugnux no longer reports `NO_LATTICE` on a rotation run that indexed the sweep although no single frame indexed on its own; `INDEXING_RATE=` is a `--developer` key on rotation runs. +* Rugnux writes a report with `VERDICT= FAILED` and `NO_LATTICE` when a rotation run finds no crystal lattice, instead of stopping with an input-parameter error and no report. +* Rugnux adds `Powder` and `Ice` summary lines and `ICE_*` keys, and warns (`POWDER_RINGS`) where powder rings hold at least half of the spots. +* Rugnux warns about degraded sweep ranges (`SWEEP_GAPS`) only where together they cover at least 10 % of the sweep. +* Rugnux no longer warns about an indexing ambiguity on a rotation sweep, which is indexed consistently; it reports the operators (`INDEXING_AMBIGUITY_OPERATORS=`, an `Indexing choice` summary line). +* Rugnux calls a further lattice more than 10° from the main one a second crystal rather than a split crystal. ### 1.0.0-rc.172 diff --git a/docs/RUGNUX_REPORT.md b/docs/RUGNUX_REPORT.md index c12490e4f..72cb0f2cd 100644 --- a/docs/RUGNUX_REPORT.md +++ b/docs/RUGNUX_REPORT.md @@ -138,7 +138,8 @@ report used to have no evaluative line anywhere until its last section: a run th and a run that produced a textbook data set read identically for their first three hundred lines. - **`VERDICT=`** is a closed vocabulary — `OK`, `WARNINGS`, `UNUSABLE`, `FAILED`. `FAILED` means no - lattice was determined or the run was cancelled; `UNUSABLE` means the data merged but carry no + lattice was determined or the run was cancelled (a sample with no crystal in the beam ends here, with + `NO_LATTICE`, and the run exits with status 1); `UNUSABLE` means the data merged but carry no usable signal; `WARNINGS` means something else needs attention; `OK` means nothing did. It is decided from the warnings the rest of the report produced, so it introduces no new analysis and cannot disagree with the sections below it. @@ -148,9 +149,20 @@ and a run that produced a textbook data set read identically for their first thr `SYMMETRY_AMBIGUITY`, `CENTERING_UNTESTED`, `UNUSABLE_MERGE`, `LOW_COMPLETENESS`, `SWEEP_GAPS`, `GONIO_SCALE`, `SPINDLE_CAP`, `ANISOTROPY`, `TWINNING`, `PSEUDO_TRANSLATION`, `LATTICE_TRANSLATION`, `MODEL_HAND`, `MODEL_NOT_VALIDATED`, `REFERENCE_MISMATCH`, `CANCELLED`, `RESOLUTION_FIT`, - `FLIGHT_PATH`, `GEOMETRY_NOT_CONVERGED`, `SUPERCELL_POSSIBLE`, `MULTIPLE_LATTICES`. `NONE` when nothing fired. A code appears if and only if its + `FLIGHT_PATH`, `GEOMETRY_NOT_CONVERGED`, `SCALING_NOT_CONVERGED`, `HARMONIC_CONTAMINATION`, + `SUPERCELL_POSSIBLE`, `MULTIPLE_LATTICES`, `POWDER_RINGS`. `NONE` when nothing fired. A code appears if and only if its warning fired, so the flags and the `WARNING:` lines are two renderings of one list — the closed type for machinery, the open sentence for a person. +- Below them, one line each, the facts a reader needs before reading further: space group, cell, + mosaicity, the powder and ice rings, multiple lattices, resolution, completeness, signal, anomalous + signal, anisotropy, pseudo-symmetry, twinning, supercell, radiation damage and the sweep. A line + states a finding that is not a condition too - weak powder rings, a possible supercell no stronger than + correct cells show, an arbitrary indexing choice (`Indexing choice`, a rotation sweep whose cell + admits alternative indexing: one sweep is indexed consistently, so it matters only against other data; + the operators are **`INDEXING_AMBIGUITY_OPERATORS=`** in section 2). On serial stills the same ambiguity + mixes the hands in the merge, and is an `INDEXING_AMBIGUITY` warning. `INDEXING_RATE=` - the share of + frames the per-image indexer took - is a stills measure: a rotation run indexes the sweep as a whole, so + there it is written only with `--developer`, and `NO_LATTICE` rests on whether the sweep was indexed. - **`WARNING_COUNT=`** and the **`WARNING:`** lines follow, in the same section. They are what they always were; they have moved from the bottom of the file to the top. @@ -283,6 +295,10 @@ experiment was useless". It is reported beside `ROTATION_REJECTED_DEG` on purpos *frames* moves when the same experiment is re-sliced, and a percentage of the *rotation* does not. The prose headline above the table states both. +A `SWEEP_GAPS` warning is given, one per range, only where the degraded ranges together cover at least +10 % of the sweep's rotation; below that they are in the table and the summary's `Sweep` line, and the +warning lines in `--developer`. One or two frames rejected on a long sweep do not change what the data are. + `SWEEP_QUALITY_STATUS` distinguishes **`COMPUTED`** (the diagnostic ran; a count of 0 means the sweep was clean throughout) from **`NOT_COMPUTED`** (it did not run — no scaling and merging, or stills data). A consumer must not read a missing table or a zero count as "clean" without checking it. @@ -419,7 +435,27 @@ can be asked to trust on such a pattern; it is absent where they stay separable which is the ordinary case. `POWDER_EXCLUDED_FROM_INDEXING` says whether the run needed them left out to index at all (with `POWDER_INDEXING_D_MIN` the resolution the retried first pass used). Rings are detected far more often than they are excluded: exclusion happens only where a first pass -found no usable lattice. The measurement is described in +found no usable lattice. The summary's `Powder` line states the rings on every run that found a lattice; +a warning under the `POWDER_RINGS` flag is given only where they hold at least half the spots +(`POWDER_SPOT_FRACTION=` 0.5 or more) - where the contaminant diffracts more than the crystal. + +Hexagonal ice is also measured by the merge itself, which leaves reflections on the ice rings out of +scaling (and keeps them in the merge) where it finds ice: + +``` +ICE_RINGS_DETECTED= TRUE +ICE_RING_SCORE= 2.69 +ICE_SPOT_RATIO= 1.08 +ICE_REFLECTIONS_ON_RINGS_PCT= 25.7 +``` + +**`ICE_RING_SCORE=`** is the strongest ice ring over the smooth radial background (fine-grained ice +that makes a smooth ring), **`ICE_SPOT_RATIO=`** the pile-up of found spots on the ring positions +against the ice-free flanks beside them (ice in large crystallites); 1 is no ice on either. +**`ICE_RINGS_DETECTED=`** is `TRUE` where either passes its gate (1.5 and 2.0), and +**`ICE_REFLECTIONS_ON_RINGS_PCT=`** is then the share of the integrated reflections set aside from +scaling. The summary has an `Ice` line. Ice handling is automatic and costs the merge nothing, so it is +not a warning. The measurement is described in [CPU/GPU data analysis ▸ Resolution and ice-ring handling](CPU_DATA_ANALYSIS_IMAGE.md#33-resolution-and-ice-ring-handling). ## Index-2 superstructure @@ -441,14 +477,17 @@ and rocks is a superstructure whose reflections this run did not integrate; **`S is the Niggli-reduced cell the lattice would double to, to give with `-C` to process on it. One that is occupied but hardly rocks is diffuse or disordered intensity rather than Bragg reflections. -**`SUPERCELL_POSSIBLE=`** is `TRUE`, with a warning under the `SUPERCELL_POSSIBLE` flag, where the -class is measured (`SUPERCELL_I_OVER_SIGMA=` at least 0.5) and part of it rocks like Bragg reflections -(`SUPERCELL_ROCK_PCT=` at least 2 %, three standard errors clear of zero). It is advice to check, not a -finding: the same numbers come from a real doubled cell and from a correct cell with weak ordered -intensity between its reflections, and which of the two a structure is decided by refinement. Process -both settings - the run's cell, and `SUPERCELL_DOUBLED_CELL=` given with `-C` - and compare them there. The -summary at the top of the report carries the same advice on its `Supercell` line, beside -pseudo-symmetry and twinning. +**`SUPERCELL_POSSIBLE=`** is `TRUE` where the class is measured (`SUPERCELL_I_OVER_SIGMA=` at least +0.5) and part of it rocks like Bragg reflections (`SUPERCELL_ROCK_PCT=` at least 2 %, three standard +errors clear of zero). It is advice to check, not a finding: the same numbers come from a real doubled +cell and from a correct cell with weak ordered intensity between its reflections - or with further +lattice domains whose spots land on the half-integer positions - and which of the two a structure is +decided by refinement. Process both settings - the run's cell, and `SUPERCELL_DOUBLED_CELL=` given with +`-C` - and compare them there. The summary at the top of the report carries that advice on its +`Supercell` line, and says so where further lattice domains were found. A warning under the +`SUPERCELL_POSSIBLE` flag is given only where the rocking part reaches 20 %: on a battery of rotation +data sets, correct sub-cells rocked at 2-16 %, and a real doubled cell can read anywhere in that range +too, so below 20 % nothing measured here tells the two apart. ## Further lattices @@ -467,7 +506,7 @@ non-ice spots of those frames and of their summed intensity; **`EXTRA_LATTICE_CO lattices listed. For each, numbered from 1 in the order found: - **`EXTRA_LATTICE__KIND=`** — `DOMAIN`: the same cell, misoriented, over the whole sweep (a split - crystal); `TWIN_DOMAIN`: the same, turned by 180° (within 3°) — a non-merohedral twin domain; + crystal; more than 10° from the main lattice, the report calls it a second crystal); `TWIN_DOMAIN`: the same, turned by 180° (within 3°) — a non-merohedral twin domain; `SEGMENTED`: the same cell, with at least 60 % of its spots in 2 of 8 equal blocks of the sweep — the crystal in the beam changes along the rotation; `RELATED_CELL`: a cell whose volume is 1 to 8 times, or a 1/2 to 1/8 of, the main one's — a question about the main cell (see the supercell keys above and @@ -487,8 +526,8 @@ lattices listed. For each, numbered from 1 in the order found: **`EXTRA_LATTICE_INTENSITY_PCT=`** sums the intensity share of the domains (`DOMAIN`, `TWIN_DOMAIN`, `SEGMENTED`); a `RELATED_CELL` or `FOREIGN` lattice is listed but not counted, since it is as often a wrong main lattice or tNCS as a second crystal, -and **`MULTIPLE_LATTICES=`** is `TRUE`, with a warning under the `MULTIPLE_LATTICES` flag, where that is -at least 10 %. It is information, not a failure: a split crystal whose second domain is half as strong +and **`MULTIPLE_LATTICES=`** is `TRUE`, with a warning under the `MULTIPLE_LATTICES` flag, where that +sum is at least 10 % - the domains together, not the strongest alone. It is information, not a failure: a split crystal whose second domain is half as strong as the main one can still give data as good as XDS's, since the domains' reflections are only a problem where they overlap the main lattice's. Look at the crystal, and at the result. The summary at the top of the report has a `Multiple lattices` line, `none found` where nothing was listed. @@ -498,7 +537,10 @@ the report has a `Multiple lattices` line, `none found` where nothing was listed **`MOSAICITY_DEG=`** (section 2, and a `Mosaicity` line in the summary) is the mosaic spread the merge computed partiality from, on rotation data: Kabsch's σ_M, the standard deviation (not the FWHM) of a reflection's rocking curve in degrees, in the same partiality model as XDS - so it is the number to set -beside XDS's `REFLECTING_RANGE_E.S.D.`. It is fitted on every frame by maximum likelihood from the +beside XDS's `REFLECTING_RANGE_E.S.D.`: the value under *SUGGESTED VALUES FOR INPUT PARAMETERS* at the +end of `INTEGRATE.LP`, which `CORRECT.LP` repeats. On in-house rotation data it reads 0.90× that value +(median over 34 data sets). XDS's per-image `SIGMAR` column and the per-block *CRYSTAL MOSAICITY* in +`INTEGRATE.LP` are a different number - they run about 1.35× higher - and are not the comparison. It is fitted on every frame by maximum likelihood from the rocking offsets of that frame's 250 strongest indexed spots, with the energy bandwidth taken out where the beam has one (XDS keeps it in), and smoothed in frame order; the value is the median over the frames, with **`MOSAICITY_DEG_P10=`** and **`MOSAICITY_DEG_P90=`** for its spread. The per-frame values are the @@ -518,6 +560,11 @@ all, `INCONCLUSIVE` means the Patterson was measured but too few acentric reflec whether the vector it names modulates the intensities. Neither is a statement that the crystal has no pseudo-symmetry. +A warning under the `PSEUDO_TRANSLATION` flag is given where `TNCS_DETECTED= TRUE` and the Patterson +peak is at least 20 % of the origin - the height at which phenix.xtriage, too, calls a +pseudo-translation one molecular replacement must be told about. A detection below that is significant +for these data but its modulation is small; the summary calls it weak. + `TRUE` requires **both** of two tests, because either alone over-calls by about a factor of two: - **`TNCS_PATTERSON_PEAK_PCT=`** — the largest off-origin peak of the native Patterson, as a @@ -541,9 +588,13 @@ smaller cell would contradict it. Only `TNCS_SUBLATTICE= NEAR_EXTINCT_CLASS` — almost gone — says the reported cell may be a supercell. **`UNDECLARED_LATTICE_TRANSLATION=`** is a different finding, and is reported instead of a -pseudo-symmetry rather than as one. The merged data are *exactly* invariant under the translation, and -a translation the data are exactly invariant under is a lattice vector by definition — so the centring -or the cell is wrong, not the packing. It is what a centred lattice merged in P1 looks like, which +pseudo-symmetry rather than as one: a translation at which the Patterson reaches at least 75 % of the +origin, **`UNDECLARED_LATTICE_TRANSLATION_PCT=`**. Near 100 % the merged data are invariant under it, and +a translation the data are invariant under is a lattice vector by definition — so the centring or the +cell is wrong, not the packing; an undeclared centring reads 83-102 %. A warning under the +`LATTICE_TRANSLATION` flag is given from 90 %. Between 75 and 90 % it is a pseudo-translation so strong it +is nearly a lattice translation: the warning is then under `PSEUDO_TRANSLATION`, since molecular +replacement needs it declared unless refinement shows the cell to be a supercell. It is what a centred lattice merged in P1 looks like, which `--mode scale` on a file with no space group produces by design. The pseudo-symmetry search continues underneath it, so a real pseudo-translation sitting under an undeclared centring is still found. @@ -555,6 +606,15 @@ that do preserve the class (`REPAIRED`), and where no step does, `UNREADABLE` sa dropped from the twin verdict **in both directions**: it can no longer indicate a twin, and it can no longer be read as proof that there is none. The second moment then decides alone. +**`TWINNING_VERDICT=`** is `NO_INDICATION`, `INDICATED` (⟨|L|⟩ below 0.44, the phenix.xtriage convention, +or the second moment low, in a Laue class that admits a twin law), `SYMMETRY_SUSPECT` (the same low +reading in a holohedral class, where no twin law exists and a false adopted operator gives the same +distribution) or `NOT_READABLE`. ⟨|L|⟩ has a physical range: 0.5 untwinned, 0.375 a perfect twin, +whatever the law. Where it reads below 0.375, or above 0.55 (on this merge or on the one before the +space-group search), no twin fraction explains it - the statistic is distorted, by tNCS, anisotropy, +overlapping reflections of a very long cell, or too few reflections - and the verdict is +`NOT_READABLE`, with no warning: it says nothing about twinning, nor about the space group. + ## Diffraction anisotropy Section 4 also reports how much the fall-off with resolution depends on **direction**, and whether that is diff --git a/image_analysis/scale_merge/TranslationalNCS.cpp b/image_analysis/scale_merge/TranslationalNCS.cpp index fc7f6cd32..5c8ba4779 100644 --- a/image_analysis/scale_merge/TranslationalNCS.cpp +++ b/image_analysis/scale_merge/TranslationalNCS.cpp @@ -370,6 +370,7 @@ TranslationalNCSResult AnalyzeTranslationalNCS(const std::vector 0.0) diff --git a/image_analysis/scale_merge/TranslationalNCS.h b/image_analysis/scale_merge/TranslationalNCS.h index 02ff20e69..dea828cdd 100644 --- a/image_analysis/scale_merge/TranslationalNCS.h +++ b/image_analysis/scale_merge/TranslationalNCS.h @@ -72,6 +72,10 @@ struct TranslationalNCSResult { // exclusion and the search continues past them, so a real pseudo-translation sitting underneath // one is still found. std::vector> undeclared_lattice_translations; + // The Patterson at each of them, % of the origin (the same order). At least the 75% that admits a + // vector here; an undeclared centring reads 83-102%, and a very strong pseudo-translation can + // reach the lower part of that band too. + std::vector undeclared_lattice_translation_pct; // ---- (b) modulation depth ----------------------------------------------------------------- // False when the Patterson was measured but the modulation test could not run (too few acentric diff --git a/rugnux/ReportDocument.h b/rugnux/ReportDocument.h index 1c9d058e7..b3d5daf61 100644 --- a/rugnux/ReportDocument.h +++ b/rugnux/ReportDocument.h @@ -48,6 +48,7 @@ namespace PathologyCode { constexpr const char *MULTIPLE_LATTICES = "MULTIPLE_LATTICES"; // a second lattice carries much of the spot intensity constexpr const char *GEOMETRY_NOT_CONVERGED = "GEOMETRY_NOT_CONVERGED"; // the geometry walk ran out of rounds constexpr const char *SCALING_NOT_CONVERGED = "SCALING_NOT_CONVERGED"; // the per-frame scales hit their cap still moving + constexpr const char *POWDER_RINGS = "POWDER_RINGS"; // most of the spots are on powder rings } // A closed type plus an open description. The type is what machinery switches on; the text is what a diff --git a/rugnux/ResultReport.cpp b/rugnux/ResultReport.cpp index ce28d6439..0f8a4d002 100644 --- a/rugnux/ResultReport.cpp +++ b/rugnux/ResultReport.cpp @@ -37,6 +37,9 @@ namespace { // 14: R_MODEL / R_MODEL_SHELL_SCALED / MODEL_RADIAL_MISFIT in section 5 - R over all the // reflections, with and without one free scale per shell, and the size of that rescale. // R_WORK and R_FREE are unchanged and still describe the model the maps were made from. + // 15: TWINNING_VERDICT gains NOT_READABLE (<|L|> outside its physical range); ICE_* keys, + // UNDECLARED_LATTICE_TRANSLATION_PCT and INDEXING_AMBIGUITY_OPERATORS; INDEXING_RATE is a + // --developer key on rotation runs; the POWDER_RINGS flag. // 8: every line that is not `KEY= value` and not blank starts with `#`, warnings included // (`# WARNING:`); the header gains RUGNUX_DOWNLOAD and BUILD_CXX_FLAGS, and RUGNUX_GIT is // the full build-time hash. @@ -59,6 +62,65 @@ namespace { return sc.i_over_sigma >= 0.5 && sc.rock_pct >= 2.0 && sc.rock_pct > 3.0 * sc.rock_se_pct; } + // ...and a warning only where the rocking part is larger than any correct sub-cell showed. On the + // rc173 battery SUPERCELL_POSSIBLE named 15 rotation sets: 13 refine normally in the sub-cell and + // rock at 2.1-16.2%, where twin or split domains often put spots on the half-integer nodes; the + // two accepted doubled cells rock at 4.2% and 29.1%. Nothing measured here separates the first of + // those from the sub-cells, so it stays a summary line; only a rocking part no sub-cell reached + // is a condition. + constexpr double SUPERCELL_WARNING_ROCK_PCT = 20.0; + + // A translation the Patterson puts at this share of the origin is a lattice translation. The + // analysis admits a vector from 75%, and an undeclared centring reads 83-102% (TranslationalNCS.cpp); + // on the rc173 battery the only two admitted vectors read 79% and 85%, on crystals that refine + // normally in the cell as it stands. Below this it is reported as a pseudo-translation so strong it + // is nearly a lattice translation, and refinement is what tells the two apart. + constexpr double LATTICE_TRANSLATION_WARNING_PCT = 90.0; + + // A pseudo-translation below this Patterson peak is real where TNCS_DETECTED says so, but not what + // a molecular-replacement program has to be told about: its intensity modulation is small. The + // convention phenix.xtriage uses (a peak above 20% of the origin). On the rc173 battery the + // detected peaks are 8.1% on one set and 24.8-63.2% on the other 21. + constexpr double PSEUDO_TRANSLATION_WARNING_PCT = 20.0; + + // The physical range of <|L|>: 0.5 untwinned, 0.375 a perfect twin, whatever the twin law. A + // reading outside it is not a twin fraction - tNCS, anisotropy, overlapping reflections of a long + // cell or too few reflections distort the statistic. Above 0.5 the battery reads up to 0.54 on + // pseudo-centred crystals, so the upper bound leaves that room. + bool LTestOutsidePhysicalRange(double mean_abs_l) { + return mean_abs_l < 0.375 || mean_abs_l > 0.55; + } + + // Powder rings holding at least this share of the pre-scan's spots are a condition; fewer are + // stated in the summary only. POWDER_RINGS_DETECTED is TRUE on 63 of the rc173 battery's 235 sets, + // 35 of them at 0.2 or more and 13 at 0.5 or more - where the rings hold more spots than the crystal. + constexpr double POWDER_WARNING_SPOT_FRACTION = 0.5; + + // Degraded ranges that together cover less than this share of the sweep are listed in the table and + // the summary but not warned about: one or two frames rejected, or a short weak stretch, does not + // change what the data are. On the rc173 battery the warning fired on 77 of 235 sets; this keeps 48. + constexpr double SWEEP_GAPS_WARNING_FRACTION = 0.10; + + // Whether <|L|> - on the merge reported, or on the one the space-group search was given - reads + // outside its physical range. Then no twin verdict is read from it, in either direction. + bool TwinStatisticsAbnormal(const ProcessResult &r) { + const auto abnormal = [](const TwinningAnalysisResult &t) { + return t.l_test_pairs > 0 && !t.l_test_contaminated_by_tncs && LTestOutsidePhysicalRange(t.mean_abs_l); + }; + return abnormal(r.twinning) || (r.pre_promotion_twinning && abnormal(*r.pre_promotion_twinning)); + } + + std::string TwinVerdict(const ProcessResult &r) { + if (!TwinStatisticsAbnormal(r)) + return TwinningVerdictLine(r.twinning); + std::string l = fmt::format("<|L|> {:.3f}", r.twinning.mean_abs_l); + if (r.pre_promotion_twinning) + l += fmt::format(", {:.3f} before the space-group search", r.pre_promotion_twinning->mean_abs_l); + return fmt::format("not readable ({}: outside the 0.375-0.5 any twin fraction gives, so the " + "intensity statistics are distorted - by tNCS, anisotropy, overlapping reflections " + "of a long cell or too few reflections - and say nothing about twinning)", l); + } + // Domains of the crystal itself - misoriented, turned by 180 deg, or holding a segment of the sweep - // that together hold at least this share of the non-ice spot intensity are worth the user's look. A // related cell is a question about the crystal's own cell (left to the supercell keys), and an @@ -82,7 +144,10 @@ namespace { std::string LeftoverLatticeLine(const LeftoverLattice &l) { switch (l.kind) { case LeftoverLatticeKind::Domain: - return fmt::format("split crystal, {:.1f} deg from the main lattice", l.misorientation_deg); + // A few degrees is one crystal split in two; tens of degrees is a second crystal. + return fmt::format("{}, {:.1f} deg from the main lattice", + l.misorientation_deg > 10.0 ? "second crystal" : "split crystal", + l.misorientation_deg); case LeftoverLatticeKind::TwinDomain: return fmt::format("non-merohedral twin domain, {:.1f} deg from the main lattice", l.misorientation_deg); @@ -535,9 +600,14 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, { ReportSection s; s.title = "2. CRYSTAL"; + // The share of frames the per-image indexer took. A stills measure: a rotation run indexes the + // sweep as a whole and integrates every frame on that lattice, so on sparse frames it can read + // 0 while the whole sweep merges. if (result.indexing_rate.has_value()) - Add(s, KeyReal("INDEXING_RATE", *result.indexing_rate, "{:.4f}")); + Add(s, KeyReal("INDEXING_RATE", *result.indexing_rate, "{:.4f}", rotation)); Add(s, KeyBool("LATTICE_FOUND", result.consensus_cell.has_value())); + if (!result.indexing_ambiguity_operators.empty()) + Add(s, KeyText("INDEXING_AMBIGUITY_OPERATORS", result.indexing_ambiguity_operators)); if (result.consensus_cell.has_value()) Add(s, KeyText("UNIT_CELL_CONSTANTS", CellString(*result.consensus_cell))); // Kabsch's sigma_M, the Gaussian e.s.d. of the rocking width in degrees - XDS's @@ -572,7 +642,8 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, result.space_group_search->sohncke_space_group->xhm())); // This fired on every --mode scale run, falsely: that path never fills indexing_rate, because // the lattice comes from the input file, and a missing optional read as a measured zero. - if (result.indexing_rate.has_value() && *result.indexing_rate <= 0.0f) + // ...and a rotation run can index the sweep with no single frame indexing on its own. + if (result.indexing_rate.has_value() && *result.indexing_rate <= 0.0f && !result.rotation_lattice_found) Warn(doc, PathologyCode::NO_LATTICE, "No image indexed - no crystal lattice was determined from this dataset"); @@ -1157,7 +1228,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, Add(s, KeyText("SUPERCELL_DOUBLED_CELL", CellString(sc.doubled_cell))); const bool possible = SupercellPossible(sc); Add(s, KeyBool("SUPERCELL_POSSIBLE", possible)); - if (possible) + if (possible && sc.rock_pct >= SUPERCELL_WARNING_ROCK_PCT) Warn(doc, PathologyCode::SUPERCELL_POSSIBLE, fmt::format("Half-integer reflections of parity {} {} {} carry Bragg-like intensity ({:.0f}% " "of the lattice's, rocking part {:.1f}%): the true cell may be twice as large. " @@ -1175,7 +1246,10 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, " empty. One that is occupied and rocks is an index-2 superstructure the lattice does not\n" " index: its reflections are not integrated. To process on the doubled lattice, give\n" " SUPERCELL_DOUBLED_CELL with -C. An occupied class that does not rock is diffuse or\n" - " disordered intensity rather than Bragg reflections.")); + " disordered intensity rather than Bragg reflections. SUPERCELL_POSSIBLE is TRUE where\n" + " the class is measured and rocks; it is a warning only where the rocking part reaches\n" + " 20%, since correct cells with weak ordered intensity, or with further lattice domains\n" + " whose spots land on the half-integer positions, read up to about 16%.")); Add(s, Blank()); } @@ -1212,8 +1286,8 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, if (CountsAsDomain(l) && (!strongest || l.intensity_pct > strongest->intensity_pct)) strongest = &l; Warn(doc, PathologyCode::MULTIPLE_LATTICES, - fmt::format("Further domains of the crystal carry {:.0f}% of the spot intensity against {:.0f}% on the " - "main one (the strongest: {}); only the main lattice is integrated. This does " + fmt::format("Further lattices of the same cell carry {:.0f}% of the spot intensity together, against " + "{:.0f}% on the main one (the strongest: {}); only the main lattice is integrated. This does " "not by itself make the data worse - check the crystal, and the result", second, c.primary_intensity_pct, LeftoverLatticeLine(*strongest))); } @@ -1223,12 +1297,15 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, " times; a lattice is listed when the leftover spots sit on it far more often than at a\n" " displaced spindle angle (Z >= 5). Percentages are of the non-ice spots of the spread and\n" " validation frames and of their summed intensity. KIND: DOMAIN - the same cell misoriented,\n" - " over the whole sweep (a split crystal); TWIN_DOMAIN - the same, turned by 180 deg (a\n" + " over the whole sweep (a split crystal; more than 10 deg away, a second crystal); TWIN_DOMAIN -\n" + " the same, turned by 180 deg (a\n" " non-merohedral twin); SEGMENTED - the same cell, but SWEEP_CONCENTRATION >= 0.6 of its\n" " spots fall in 2 of 8 blocks of the sweep (the crystal in the beam changes); RELATED_CELL -\n" " a cell an integer multiple or fraction of the main one's volume, a question about the\n" " main cell (see SUPERCELL_* and the harmonic above); FOREIGN - an unrelated cell (a second\n" - " crystal, or a wrong main lattice). Only the main lattice is integrated.")); + " crystal, or a wrong main lattice). MULTIPLE_LATTICES is TRUE where DOMAIN, TWIN_DOMAIN\n" + " and SEGMENTED lattices together (EXTRA_LATTICE_INTENSITY_PCT) hold 10% or more of the\n" + " intensity. Only the main lattice is integrated.")); Add(s, Blank()); } @@ -1256,6 +1333,12 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, {"TRUE", "FALSE"})); if (result.powder_indexing_d_min_A) Add(s, KeyReal("POWDER_INDEXING_D_MIN", *result.powder_indexing_d_min_A, "{:.2f}")); + if (pw.spot_fraction >= POWDER_WARNING_SPOT_FRACTION) + Warn(doc, PathologyCode::POWDER_RINGS, fmt::format( + "Powder rings hold {:.0f}% of the spots found ({} rings): a crystalline phase other " + "than the crystal diffracts more than the crystal does. Reflections of the crystal " + "that fall on a ring carry its intensity too", + 100.0f * pw.spot_fraction, pw.rings_q_recipA.size())); Add(s, Blank()); Add(s, Prose(fmt::format( " Powder contamination: {} rings, holding {:.0f}% of the spots the pre-scan found\n" @@ -1271,6 +1354,27 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, Add(s, Blank()); } + // ---- hexagonal ice, as the merge's ice-ring handling measured it + if (result.ice_ring_score || result.ice_spot_ratio) { + Add(s, KeyBool("ICE_RINGS_DETECTED", result.ice_rings_detected)); + if (result.ice_ring_score) + Add(s, KeyReal("ICE_RING_SCORE", *result.ice_ring_score, "{:.2f}")); + if (result.ice_spot_ratio) + Add(s, KeyReal("ICE_SPOT_RATIO", *result.ice_spot_ratio, "{:.2f}")); + if (result.ice_rings_detected) + Add(s, KeyReal("ICE_REFLECTIONS_ON_RINGS_PCT", 100.0 * result.ice_flagged_fraction, "{:.1f}")); + Add(s, Blank()); + Add(s, Prose(result.ice_rings_detected + ? fmt::format(" Ice rings: DETECTED. The {:.1f}% of the integrated reflections that lie on the\n" + " hexagonal-ice rings were left out of scaling and kept in the merge.", + 100.0 * result.ice_flagged_fraction) + : std::string(" Ice rings: none detected, so no reflection was set aside for them."))); + Add(s, Prose(" ICE_RING_SCORE is the strongest ice ring over the smooth radial background (smooth\n" + " ice), ICE_SPOT_RATIO the pile-up of found spots on the ring positions against the\n" + " flanks beside them (ice in large crystallites); 1 is no ice on either.", true)); + Add(s, Blank()); + } + // ---- translational pseudo-symmetry // Printed immediately before the twinning block, and in the same section, because the two // interact: a pseudo-translation that is not a half-integer vector biases <|L|> upwards and @@ -1297,9 +1401,20 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, // A different finding from a pseudo-symmetry, and reported as one: the data are // EXACTLY invariant under these, so the centring or the cell is wrong rather than the // molecules being packed unusually. - for (const auto &v : t.undeclared_lattice_translations) + for (size_t i = 0; i < t.undeclared_lattice_translations.size(); ++i) { + const auto &v = t.undeclared_lattice_translations[i]; Add(s, KeyText("UNDECLARED_LATTICE_TRANSLATION", fmt::format("{:.3f} {:.3f} {:.3f}", v[0], v[1], v[2]))); + if (i < t.undeclared_lattice_translation_pct.size()) + Add(s, KeyReal("UNDECLARED_LATTICE_TRANSLATION_PCT", t.undeclared_lattice_translation_pct[i], + "{:.1f}")); + } + // The first one, at the height it was found at: a lattice translation where the Patterson + // there is close to the origin's, a very strong pseudo-translation below that. + const bool undeclared = !t.undeclared_lattice_translations.empty(); + const double undeclared_pct = t.undeclared_lattice_translation_pct.empty() + ? 100.0 : t.undeclared_lattice_translation_pct.front(); + const bool lattice_translation = undeclared && undeclared_pct >= LATTICE_TRANSLATION_WARNING_PCT; Add(s, KeyReal("TNCS_PATTERSON_PEAK_PCT", t.peak_percent, "{:.1f}")); Add(s, KeyReal("TNCS_PATTERSON_PEAK_Z", t.peak_z, "{:.1f}")); Add(s, KeyText("TNCS_PATTERSON_NULL_PCT", @@ -1353,15 +1468,31 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, " The reported cell is therefore a supercell candidate rather than a settled result,\n" " and the rival hypothesis is a cell with this translation as a lattice vector.")); - if (!t.undeclared_lattice_translations.empty()) { + if (lattice_translation) { const auto &v = t.undeclared_lattice_translations.front(); Add(s, ProseDefaultOnly(fmt::format( - "\n The merged data are EXACTLY invariant under ({:.3f}, {:.3f}, {:.3f}), which the space\n" - " group in use does not declare. A translation the data are exactly invariant under is a\n" - " lattice vector by definition, so it is the centring or the cell that is wrong, not the\n" - " packing. The pseudo-symmetry search continues underneath it, so the numbers above still\n" - " describe the crystal and not the missing centring.", v[0], v[1], v[2]))); + "\n The merged data are invariant under ({:.3f}, {:.3f}, {:.3f}) - the Patterson there is\n" + " {:.0f}% of the origin - which the space group in use does not declare. A translation the\n" + " data are invariant under is a lattice vector by definition, so it is the centring or the\n" + " cell that is wrong, not the packing. The pseudo-symmetry search continues underneath it,\n" + " so the numbers above still describe the crystal and not the missing centring.", + v[0], v[1], v[2], undeclared_pct))); + } else if (undeclared) { + const auto &v = t.undeclared_lattice_translations.front(); + Add(s, ProseDefaultOnly(fmt::format( + "\n The Patterson at ({:.3f}, {:.3f}, {:.3f}) is {:.0f}% of the origin: short of the ~100%\n" + " of a lattice translation, far above the ordinary pseudo-translation, so the class it\n" + " suppresses is nearly empty over 20-5 A. Either the cell is a supercell of one with this\n" + " vector as a lattice translation, or the crystal has a very strong pseudo-translation,\n" + " which molecular replacement needs declared; refinement in this cell tells them apart.\n" + " The pseudo-symmetry numbers above were measured underneath it.", + v[0], v[1], v[2], undeclared_pct))); } + if (t.detected && t.peak_percent < PSEUDO_TRANSLATION_WARNING_PCT) + Add(s, ProseDefaultOnly(fmt::format( + "\n The peak is significant but weak: below the {:.0f}% of the origin at which a\n" + " pseudo-translation is conventionally declared to molecular replacement.", + PSEUDO_TRANSLATION_WARNING_PCT))); Add(s, Prose("\n" + TranslationalNCSToText(t), true)); if (t.modulation_measured && !t.detected) @@ -1373,21 +1504,29 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, " intensities. Either test alone over-calls by about a factor of two on a corpus of merged\n" " datasets.", true)); - if (t.detected) + if (t.detected && t.peak_percent >= PSEUDO_TRANSLATION_WARNING_PCT) Warn(doc, PathologyCode::PSEUDO_TRANSLATION, fmt::format( "Translational pseudo-symmetry: Patterson off-origin peak {:.1f}% of the origin at " "({:.3f}, {:.3f}, {:.3f}), {:.1f} A - the molecular-replacement program needs the " "vector declared, or it will be searching against a modulation it does not model", t.peak_percent, t.vector_frac[0], t.vector_frac[1], t.vector_frac[2], t.vector_length_A)); - if (!t.undeclared_lattice_translations.empty()) + if (lattice_translation) Warn(doc, PathologyCode::LATTICE_TRANSLATION, fmt::format( - "The merged data are exactly invariant under the translation ({:.3f}, {:.3f}, {:.3f}), " - "which the space group in use does not declare - the lattice centring or the unit cell " - "is wrong", + "The merged data are invariant under the translation ({:.3f}, {:.3f}, {:.3f}) - the " + "Patterson there is {:.0f}% of the origin - which the space group in use does not " + "declare: the lattice centring or the unit cell is wrong", t.undeclared_lattice_translations[0][0], t.undeclared_lattice_translations[0][1], - t.undeclared_lattice_translations[0][2])); + t.undeclared_lattice_translations[0][2], undeclared_pct)); + else if (undeclared) + Warn(doc, PathologyCode::PSEUDO_TRANSLATION, fmt::format( + "Very strong translational pseudo-symmetry: the Patterson at ({:.3f}, {:.3f}, {:.3f}) is " + "{:.0f}% of the origin - the molecular-replacement program needs the vector declared, " + "unless refinement shows the cell to be a supercell with this vector as a lattice translation", + t.undeclared_lattice_translations[0][0], + t.undeclared_lattice_translations[0][1], + t.undeclared_lattice_translations[0][2], undeclared_pct)); if (t.detected && t.near_extinct_class) Warn(doc, PathologyCode::LATTICE_TRANSLATION, fmt::format( "The pseudo-translation at {:.1f} A suppresses its phase class almost completely, so the " @@ -1413,11 +1552,15 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, // One verdict, in the words every other place uses (TwinningVerdictLine): INDICATED is a twin // in a class that admits one; SYMMETRY_SUSPECT is the same low reading in a holohedral class, // where it says an adopted operator averages unequal intensities. + // NOT_READABLE where <|L|> reads outside the range any twin fraction gives: then neither + // twinning nor a false operator is read from it. + const bool abnormal = TwinStatisticsAbnormal(result); Add(s, KeyEnum("TWINNING_VERDICT", - tw.adopted_operators_suspect ? "SYMMETRY_SUSPECT" + abnormal ? "NOT_READABLE" + : tw.adopted_operators_suspect ? "SYMMETRY_SUSPECT" : tw.twinning_suspected ? "INDICATED" : "NO_INDICATION", - {"NO_INDICATION", "INDICATED", "SYMMETRY_SUSPECT"})); + {"NO_INDICATION", "INDICATED", "SYMMETRY_SUSPECT", "NOT_READABLE"})); // The fraction comes from the statistic that carries the verdict, and the key beside it says // which. None is quoted where it cannot be measured: merged under a false operator, or from a // second moment a pseudo-translation inflates. @@ -1441,8 +1584,8 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, {"UNAFFECTED", "REPAIRED", "UNREADABLE", "NOT_MEASURED"})); } Add(s, Blank()); - Add(s, Prose(" Twinning: " + TwinningVerdictLine(tw) + ".")); - if (tw.adopted_operators_suspect) + Add(s, Prose(" Twinning: " + TwinVerdict(result) + ".")); + if (!abnormal && tw.adopted_operators_suspect) Add(s, Prose(fmt::format( " The Laue class is holohedral, so no twin law exists within it - but merging under a\n" " false operator gives exactly this distribution, whatever the twin fraction. {} Either\n" @@ -1451,7 +1594,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, tw.laue_class_was_chosen_by_promotion ? "The class was chosen by the space-group search, so that promotion is suspect." : "The space group in use is higher than these intensities support."))); - else if (tw.twinning_suspected) + else if (!abnormal && tw.twinning_suspected) Add(s, Prose(" Refinement against these data needs a twin law.")); else if (tw.l_test_contaminated_by_tncs) // <|L|> is the number this sentence would otherwise rest on, and on this crystal it is @@ -1497,14 +1640,14 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, tz.control.mean_abs_e2_minus_1, tz.control.standard_error))); Add(s, Prose("\n" + TwinImmuneZonesToText(tz))); } - if (tw.adopted_operators_suspect) + if (!abnormal && tw.adopted_operators_suspect) Warn(doc, PathologyCode::TWINNING, fmt::format( "An adopted symmetry operator relates unequal intensities (<|L|> = {:.3f} in a holohedral " "Laue class, where a genuine operator reads about 0.5 and a false one 0.375): the " "space group may be too high, or the crystal is a twin whose law the point group " "absorbed - refine in a subgroup", tw.mean_abs_l)); - else if (tw.twinning_suspected) + else if (!abnormal && tw.twinning_suspected) Warn(doc, PathologyCode::TWINNING, fmt::format( "Twinning is indicated ({}) - refinement against the merged data needs a twin law", TwinningVerdictLine(tw))); @@ -1533,6 +1676,12 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, // ---- sweep quality { const auto &sq = result.merge_statistics.sweep_quality; + // Whether the degraded ranges together are a condition of the data set, or a few frames the + // table and the summary line account for. + double degraded_deg = 0.0; + for (const auto &r : sq.ranges) + degraded_deg += r.rotation_deg; + const bool material = degraded_deg >= SWEEP_GAPS_WARNING_FRACTION * sq.sweep_deg; const int n_sweep_frames = sq.frames_merged + sq.frames_downgraded + sq.frames_rejected; auto pct = [&](int n) { return n_sweep_frames > 0 ? 100.0 * n / n_sweep_frames : 0.0; }; // A delta-CC1/2 or its standard error that could not be measured prints as a dash - a @@ -1648,7 +1797,7 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, "Frames {}-{} {} ({:.1f} deg, scale {:.2f} and CC {:.2f} of the run, {:.0f}% scaled){}", r.first_image, r.last_image, SweepQualityReasonText(r.reason), r.rotation_deg, r.mean_relative_scale, r.mean_relative_cc, - 100.0 * r.indexed_fraction, cost)); + 100.0 * r.indexed_fraction, cost), !material); } Add(s, std::move(e)); } @@ -2052,7 +2201,9 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, const auto &o = result.merge_statistics.overall; const std::string group = result.space_group ? result.space_group->xhm() : std::string("P 1"); - if (no_lattice) + if (no_lattice && result.images_processed == 0) + doc.verdict_text = "No crystal lattice was found in these frames; nothing was integrated or merged."; + else if (no_lattice) doc.verdict_text = fmt::format( "No crystal lattice was determined from {} images; nothing was integrated or merged.", result.images_processed); @@ -2108,9 +2259,26 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, if (result.consensus_cell) row("Unit cell", CellString(*result.consensus_cell)); if (std::isfinite(result.mosaicity_deg)) - row("Mosaicity", fmt::format("{:.3f} deg (sigma_M as XDS; {:.3f}-{:.3f} over 80% of the frames)", + row("Mosaicity", fmt::format("{:.3f} deg (sigma_M, comparable with XDS's REFLECTING_RANGE_E.S.D.; " + "{:.3f}-{:.3f} over 80% of the frames)", result.mosaicity_deg, result.mosaicity_deg_p10, result.mosaicity_deg_p90)); + if (!result.indexing_ambiguity_operators.empty()) + row("Indexing choice", fmt::format("alternative indexing {} - one sweep is indexed consistently, " + "so it matters only against other data: index against those " + "with -z or --model", result.indexing_ambiguity_operators)); + if (!result.powder.rings_q_recipA.empty() || result.consensus_cell) { + const auto &pw = result.powder; + row("Powder", pw.rings_q_recipA.empty() ? std::string("no rings detected") + : fmt::format("{} rings holding {:.0f}% of the spots found{}", pw.rings_q_recipA.size(), + 100.0f * pw.spot_fraction, + result.powder_excluded_from_indexing ? "; left out of indexing" : "")); + } + if (result.ice_ring_score || result.ice_spot_ratio) + row("Ice", result.ice_rings_detected + ? fmt::format("rings DETECTED: the {:.1f}% of the reflections on them were left out of " + "scaling, kept in the merge", 100.0 * result.ice_flagged_fraction) + : std::string("no rings detected")); if (result.harmonic.detected) row("Beam", fmt::format("higher harmonic INDICATED ({:.0f} % of the unindexed spots " "are lattice points at lambda/3)", result.harmonic.fraction)); @@ -2175,23 +2343,51 @@ ReportDocument BuildReportDocument(const std::string &output_prefix, : fmt::format("{}: deltaB {:.1f} A^2", sev, an.delta_b)); } } - if (result.tncs.measurable) - row("Pseudo-symmetry", result.tncs.detected - ? fmt::format("INDICATED (Patterson peak {:.1f}% of origin, {:.1f} A)", - result.tncs.peak_percent, result.tncs.vector_length_A) - : std::string("no indication")); + if (result.tncs.measurable) { + const auto &t = result.tncs; + std::string ps; + if (!t.undeclared_lattice_translations.empty()) { + const auto &v = t.undeclared_lattice_translations.front(); + const double pct = t.undeclared_lattice_translation_pct.empty() + ? 100.0 : t.undeclared_lattice_translation_pct.front(); + ps = fmt::format("{} ({:.3f} {:.3f} {:.3f}, Patterson {:.0f}% of origin)", + pct >= LATTICE_TRANSLATION_WARNING_PCT + ? "undeclared LATTICE translation" + : "VERY STRONG, nearly a lattice translation", + v[0], v[1], v[2], pct); + } + if (t.detected) + ps += (ps.empty() ? "" : "; ") + fmt::format( + "{} (Patterson peak {:.1f}% of origin, {:.1f} A)", + t.peak_percent >= PSEUDO_TRANSLATION_WARNING_PCT + ? "INDICATED" : "weak - below the 20% that matters for molecular replacement", + t.peak_percent, t.vector_length_A); + row("Pseudo-symmetry", ps.empty() ? std::string("no indication") : ps); + } if (result.twinning.l_test_pairs > 0) - row("Twinning", TwinningVerdictLine(result.twinning)); - if (result.supercell) - row("Supercell", SupercellPossible(*result.supercell) - ? fmt::format("POSSIBLE - class {} {} {} rocks like Bragg reflections ({:.1f}%); " - "process also with -C \"{:.2f},{:.2f},{:.2f},{:.2f},{:.2f},{:.2f}\" " - "and compare", result.supercell->h, result.supercell->k, - result.supercell->l, result.supercell->rock_pct, - result.supercell->doubled_cell.a, result.supercell->doubled_cell.b, - result.supercell->doubled_cell.c, result.supercell->doubled_cell.alpha, - result.supercell->doubled_cell.beta, result.supercell->doubled_cell.gamma) - : std::string("no indication")); + row("Twinning", TwinVerdict(result)); + if (result.supercell) { + const auto &sc = *result.supercell; + const std::string doubled = fmt::format( + "process also with -C \"{:.2f},{:.2f},{:.2f},{:.2f},{:.2f},{:.2f}\" and compare", + sc.doubled_cell.a, sc.doubled_cell.b, sc.doubled_cell.c, + sc.doubled_cell.alpha, sc.doubled_cell.beta, sc.doubled_cell.gamma); + // Spots of further domains of the crystal can land on the half-integer positions. + const bool domains = result.leftover_lattices && DomainIntensityPct(*result.leftover_lattices) >= 5.0; + if (!SupercellPossible(sc)) + row("Supercell", "no indication"); + else if (sc.rock_pct >= SUPERCELL_WARNING_ROCK_PCT) + row("Supercell", fmt::format("POSSIBLE - class {} {} {} rocks like Bragg reflections " + "({:.1f}%); {}", sc.h, sc.k, sc.l, sc.rock_pct, doubled)); + else + row("Supercell", fmt::format("weak Bragg-like intensity in class {} {} {} (rocking part " + "{:.1f}%) - as correct cells with weak ordered intensity " + "show{}; to test a doubled cell, {}", sc.h, sc.k, sc.l, + sc.rock_pct, + domains ? ", and the further lattice domains found may put " + "spots there" : "", + doubled)); + } const double db = result.merge_statistics.radiation_damage_delta_b; if (std::isfinite(db)) row("Radiation damage", fmt::format("relative B {:+.2f} A^2 over the sweep", db)); diff --git a/rugnux/Rugnux.cpp b/rugnux/Rugnux.cpp index c2c23fbea..08b3f577d 100644 --- a/rugnux/Rugnux.cpp +++ b/rugnux/Rugnux.cpp @@ -4717,8 +4717,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b "data - free the resource it needs (a GPU shared with " "another process is the usual cause) and run again.", *best.indexer_error)); - throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, - std::string("Two-pass rotation indexing found no lattice.") + next_steps); + throw NoLatticeFound(std::string("Two-pass rotation indexing found no lattice.") + next_steps); } // Short-axis hypothesis - the small-molecule case. // @@ -4980,8 +4979,7 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // reflection by the small angle the constraint snapped away - and without the cell in // the message there is nothing to see that from. const auto &c = best.result->search_result.conventional.GetUnitCell(); - throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, - fmt::format("Two-pass rotation indexing found only a lattice that " + throw NoLatticeFound(fmt::format("Two-pass rotation indexing found only a lattice that " "takes {:.1f}% of the validation spots against the " "{:.1f}% a wrong spindle angle takes - it is not this " "crystal's lattice. It was {}-centred {}, " @@ -5790,6 +5788,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b const bool ice_present = (!ice_score.has_value() || *ice_score >= ice_min_score) || (ice_min_spot_ratio > 0.0f && ice_spot_ratio.has_value() && *ice_spot_ratio >= ice_min_spot_ratio); + result.ice_ring_score = ice_score; + result.ice_spot_ratio = ice_spot_ratio; if (experiment_.IsDetectIceRings() && !ice_present) { logger.Info("Ice-ring handling: measured ice score {:.2f} < {:.2f} and spot ratio {:.2f} < " "{:.2f}, no ice detected - ice-ring handling skipped entirely", @@ -5805,6 +5805,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b ++flagged; } } + result.ice_rings_detected = true; + result.ice_flagged_fraction = total > 0 ? static_cast(flagged) / total : 0.0; logger.Info("Ice-ring handling: ice score {:.2f} (gate {:.2f}), spot ratio {:.2f} (gate " "{:.2f}); flagged {} of {} reflections on ice rings (half-width {:.3f} A^-1); " "excluded from scaling, kept for merging", @@ -7940,14 +7942,21 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b // been fitted (validation step below), so the warning stands here exactly as in a run // without a model, and is withdrawn there if the model's indexing probe decided it. // On serial stills with -C and -S the model is already in `ambiguity_reference`. - if (!ambiguity_reference) { + if (!ambiguity_reference && experiment_.IsRotationIndexing()) { + // One sweep has one orientation matrix, so every frame is in the same hand and the + // merge mixes nothing. The hand is arbitrary, which matters only against other + // data: the report states it, and it is not a condition of this data set. + logger.Info("Indexing ambiguity: this cell / space group admits alternative indexing " + "(reindex operator(s): {}); one sweep is indexed in one hand, arbitrary " + "against other data sets", laws); + result.indexing_ambiguity_operators = laws; + } else if (!ambiguity_reference) { const std::string msg = fmt::format( "Indexing ambiguity: this cell / space group admits alternative indexing " - "(reindex operator(s): {}). {} are indexed in one hand at random, and rugnux can only " - "break this against an external reference. WITHOUT one the merge mixes the hands and " - "CC1/2 is degraded; a reference MTZ (-z) or a model ({}) resolves it.", - laws, experiment_.IsRotationIndexing() ? "Lattices" : "Serial-stills crystals", - experiment_.IsRotationIndexing() ? "--model" : "--model, which needs -C and -S here"); + "(reindex operator(s): {}). Serial-stills crystals are indexed in one hand at random, " + "and rugnux can only break this against an external reference. WITHOUT one the merge " + "mixes the hands and CC1/2 is degraded; a reference MTZ (-z) or a model (--model, " + "which needs -C and -S here) resolves it.", laws); logger.Warning("{}", msg); ambiguity_warning_text = " !! " + msg + "\n\n"; stats_text << ambiguity_warning_text; @@ -8212,6 +8221,8 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b "Model validation did not run: " + validation.failure_reason}); // The ambiguity warning above was issued before the model was read; take it back only // where the model actually settled the indexing. + if (validation.indexing_decided) + result.indexing_ambiguity_operators.clear(); if (validation.indexing_decided && !ambiguity_warning_text.empty()) { std::erase_if(result.warnings, [](const ReportWarning &w) { return w.code == PathologyCode::INDEXING_AMBIGUITY; diff --git a/rugnux/Rugnux.h b/rugnux/Rugnux.h index 7a1a99d81..6f533702d 100644 --- a/rugnux/Rugnux.h +++ b/rugnux/Rugnux.h @@ -14,6 +14,7 @@ #include "../common/DiffractionExperiment.h" #include "../common/PixelMask.h" #include "../common/CrystalLattice.h" +#include "../common/JFJochException.h" #include "../common/Reflection.h" // MergedReflection #include "../common/JFJochMessages.h" // DataMessage #include "../common/JFJochReceiverPlots.h" // MeanProcessingTime @@ -230,6 +231,16 @@ RotationScaleWalk WalkRotationScale(double first_fit, const std::function(float)> &refit_at, int max_rounds); +// The first pass found no lattice of the crystal in the frames - what a sample with no crystal in the +// beam ends in. A statement about the data, not an error in the input: the CLI writes a report with +// a FAILED verdict for it. `reason` is the explanation without the exception category. +class NoLatticeFound : public JFJochException { +public: + std::string reason; + explicit NoLatticeFound(const std::string &r) + : JFJochException(JFJochExceptionCategory::InputParameterInvalid, r), reason(r) {} +}; + struct ProcessResult { bool cancelled = false; uint64_t images_processed = 0; @@ -412,6 +423,18 @@ struct ProcessResult { // the indexing spot list at where it did. bool powder_excluded_from_indexing = false; std::optional powder_indexing_d_min_A; + // Hexagonal ice, as the merge's ice-ring handling measured it: the ring score on the radial + // profile and the spot pile-up on the ring positions (1 = no ice on either), whether either + // passed its gate, and the share of the integrated reflections then flagged as on a ring (left + // out of scaling, kept in the merge). Report only; unset where the merge did not run. + std::optional ice_ring_score; + std::optional ice_spot_ratio; + bool ice_rings_detected = false; + double ice_flagged_fraction = 0.0; + // The reindex operators a rotation run's cell and space group admit (" ; "-joined), where no + // reference or model fixed the choice. Within one sweep the choice is arbitrary but consistent; + // it matters only against other data. Empty where there is none, or where it was resolved. + std::string indexing_ambiguity_operators; // The same analysis measured BEFORE the space group was decided, on the subgroup merge the search // was given. It is the only twinning number a promotion cannot contaminate: once a higher Laue @@ -732,6 +755,8 @@ public: // Runs the configured workflow to completion or until Cancel(). Throws on setup failure. ProcessResult Run(RugnuxObserver *observer = nullptr); + // The pre-scan's powder measurement, for a report on a run that ended without a result. + const PowderRings &GetPowderRings() const { return powder_; } // Request cancellation; safe to call from any thread (the worker loop checks between images). void Cancel() { cancelled_ = true; } diff --git a/rugnux/rugnux_cli.cpp b/rugnux/rugnux_cli.cpp index c31c0aab8..ffee0a521 100644 --- a/rugnux/rugnux_cli.cpp +++ b/rugnux/rugnux_cli.cpp @@ -2794,6 +2794,23 @@ static int RunRugnux(int argc, char **argv) { ProcessResult result; try { result = process.Run(); + } catch (const NoLatticeFound &e) { + // No crystal in these frames is a result, and the report says so - VERDICT= FAILED with + // NO_LATTICE - before the run exits as failed. + logger.Error("No crystal lattice found: {}", e.reason); + ProcessResult failed; + failed.warnings.push_back({PathologyCode::NO_LATTICE, "No crystal lattice was found: " + e.reason}); + failed.powder = process.GetPowderRings(); + failed.used_beam_x_pxl = experiment.GetBeamX_pxl(); + failed.used_beam_y_pxl = experiment.GetBeamY_pxl(); + failed.used_distance_mm = experiment.GetDetectorDistance_mm(); + const auto direct = experiment.GetDiffractionGeometry().GetDirectBeam_pxl(); + failed.used_direct_beam_x_pxl = direct.first; + failed.used_direct_beam_y_pxl = direct.second; + provenance.wall_time_s = std::chrono::duration( + std::chrono::steady_clock::now() - invocation_start).count(); + WriteResultReport(output_prefix, input_file, experiment, failed, logger, provenance); + exit(EXIT_FAILURE); } catch (const std::exception &e) { logger.Error("Processing failed: {}", e.what()); exit(EXIT_FAILURE); @@ -2834,7 +2851,8 @@ static int RunRugnux(int argc, char **argv) { // Only when something actually indexed: with a zero indexing rate the cell is whatever the lattice // search happened to return and no reflection was measured on it, so printing it as the run's answer // states a result the data do not support. - const bool anything_indexed = result.indexing_rate.value_or(0.0f) > 0.0f; + // A rotation run can index the sweep as a whole with no frame indexing on its own. + const bool anything_indexed = result.indexing_rate.value_or(0.0f) > 0.0f || result.rotation_lattice_found; if (result.space_group.has_value() && anything_indexed) { std::string line = fmt::format("Space group: {} (No. {})", result.space_group->short_name(), result.space_group->number); diff --git a/tests/ResultReportTest.cpp b/tests/ResultReportTest.cpp index ad0ccc899..9fd4b3475 100644 --- a/tests/ResultReportTest.cpp +++ b/tests/ResultReportTest.cpp @@ -816,11 +816,31 @@ TEST_CASE("ResultReport_TranslationalNCS", "[Diagnostics]") { result.tncs.detected = false; result.tncs.near_extinct_class = false; result.tncs.undeclared_lattice_translations.push_back({0.5, 0.5, 0.0}); + result.tncs.undeclared_lattice_translation_pct.push_back(98.0); const auto lat = RenderResultReport("prefix", "in.h5", x, result); CHECK(lat.find("\nTNCS_DETECTED= FALSE\n") != std::string::npos); CHECK(lat.find("\nUNDECLARED_LATTICE_TRANSLATION= 0.500 0.500 0.000\n") != std::string::npos); + CHECK(lat.find("\nUNDECLARED_LATTICE_TRANSLATION_PCT= 98.0\n") != std::string::npos); CHECK(lat.find("\nPATHOLOGY_FLAGS= LATTICE_TRANSLATION\n") != std::string::npos); CHECK(lat.find("\nTNCS_SUBLATTICE=") == std::string::npos); + + // Admitted at 75% but short of a lattice translation's height: the claim is a very strong + // pseudo-translation, which is what molecular replacement has to be told. + result.tncs.undeclared_lattice_translation_pct = {80.0}; + const auto strong = RenderResultReport("prefix", "in.h5", x, result); + CHECK(strong.find("\nPATHOLOGY_FLAGS= PSEUDO_TRANSLATION\n") != std::string::npos); + CHECK(strong.find("VERY STRONG, nearly a lattice translation") != std::string::npos); + CHECK(strong.find("invariant under the translation") == std::string::npos); + + // Significant, but a peak this weak is not one molecular replacement needs declared. + result.tncs.undeclared_lattice_translations.clear(); + result.tncs.undeclared_lattice_translation_pct.clear(); + result.tncs.detected = true; + result.tncs.peak_percent = 8.1; + const auto weak = RenderResultReport("prefix", "in.h5", x, result); + CHECK(weak.find("\nTNCS_DETECTED= TRUE\n") != std::string::npos); + CHECK(weak.find("PSEUDO_TRANSLATION") == std::string::npos); + CHECK(weak.find("weak - below the 20% that matters for molecular replacement") != std::string::npos); } TEST_CASE("ResultReport_GeometryNotConverged", "[Diagnostics]") { @@ -961,3 +981,178 @@ TEST_CASE("ResultReport_Mosaicity", "[Diagnostics]") { CHECK(text.find("MOSAICITY_DEG_P90= 0.105") != std::string::npos); CHECK(text.find("Mosaicity 0.091 deg") != std::string::npos); } + +TEST_CASE("ResultReport_SupercellDemoted", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + ProcessResult result; + result.consensus_cell = UnitCell{.a = 50.0f, .b = 60.0f, .c = 70.0f, + .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + result.has_merge_statistics = true; + ProcessResult::SupercellClass sc; + sc.h = 0; sc.k = 0; sc.l = 1; + sc.occupancy_pct = 18.0; + sc.rock_pct = 5.0; + sc.rock_se_pct = 0.4; + sc.i_over_sigma = 1.2; + sc.doubled_cell = UnitCell{.a = 50.0f, .b = 60.0f, .c = 140.0f, .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + result.supercell = sc; + + // Possible, as correct sub-cells also read: stated, not warned about. + auto text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nSUPERCELL_POSSIBLE= TRUE\n") != std::string::npos); + CHECK(text.find("\nPATHOLOGY_FLAGS= NONE\n") != std::string::npos); + CHECK(text.find("weak Bragg-like intensity in class 0 0 1") != std::string::npos); + CHECK(text.find("may put spots there") == std::string::npos); + + // Further domains of the crystal can put spots on the half-integer nodes, and the line says so. + LeftoverLatticeCensus census; + census.primary_intensity_pct = 70.0; + census.lattices.push_back(LeftoverLattice{.kind = LeftoverLatticeKind::TwinDomain, .cell = *result.consensus_cell, + .volume_ratio = 1.0, .misorientation_deg = 179.5, .z = 50.0, + .spots_pct = 8.0, .intensity_pct = 7.0, .sweep_concentration = 0.3}); + result.leftover_lattices = census; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("the further lattice domains found may put spots there") != std::string::npos); + + // A rocking part no correct sub-cell reached is a condition. + result.supercell->rock_pct = 29.0; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nPATHOLOGY_FLAGS= SUPERCELL_POSSIBLE\n") != std::string::npos); + CHECK(text.find("POSSIBLE - class 0 0 1 rocks like Bragg reflections (29.0%)") != std::string::npos); +} + +TEST_CASE("ResultReport_TwinningOutsidePhysicalRange", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + ProcessResult result; + result.consensus_cell = UnitCell{.a = 70.0f, .b = 70.0f, .c = 70.0f, + .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + result.has_merge_statistics = true; + result.twinning.l_test_pairs = 5000; + result.twinning.mean_abs_l = 0.355; + result.twinning.second_moment = 1.60; + result.twinning.twinning_suspected = true; + result.twinning.adopted_operators_suspect = true; + + // Below the 0.375 of a perfect twin: no twin fraction can give it, so neither a twin nor a false + // operator is read from it. + auto text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nTWINNING_VERDICT= NOT_READABLE\n") != std::string::npos); + CHECK(text.find("TWINNING ") == std::string::npos); + CHECK(text.find("\nPATHOLOGY_FLAGS= NONE\n") != std::string::npos); + CHECK(text.find("Twinning not readable (<|L|> 0.355") != std::string::npos); + CHECK(text.find("SYMMETRY SUSPECT") == std::string::npos); + + // Inside the range on the adopted merge but far above 0.5 on the merge before the search: the + // statistic is distorted on these data, and the low adopted reading is not evidence either. + result.twinning.mean_abs_l = 0.395; + TwinningAnalysisResult pre; + pre.l_test_pairs = 5000; + pre.mean_abs_l = 0.590; + result.pre_promotion_twinning = pre; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nTWINNING_VERDICT= NOT_READABLE\n") != std::string::npos); + CHECK(text.find("0.590 before the space-group search") != std::string::npos); + + // An ordinary reading keeps its verdict. + result.pre_promotion_twinning.reset(); + result.twinning.adopted_operators_suspect = false; + result.twinning.mean_abs_l = 0.419; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nTWINNING_VERDICT= INDICATED\n") != std::string::npos); + CHECK(text.find("\nPATHOLOGY_FLAGS= TWINNING\n") != std::string::npos); +} + +TEST_CASE("ResultReport_RotationLatticeWithoutIndexedFrames", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + // A rotation run that indexed the sweep as a whole while no frame indexed on its own. + ProcessResult result; + result.images_processed = 1800; + result.indexing_rate = 0.0f; + result.rotation_lattice_found = true; + result.consensus_cell = UnitCell{.a = 50.0f, .b = 60.0f, .c = 70.0f, + .alpha = 90.0f, .beta = 100.0f, .gamma = 90.0f}; + const auto text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("NO_LATTICE") == std::string::npos); + CHECK(text.find("No image indexed") == std::string::npos); + CHECK(text.find("\nLATTICE_FOUND= TRUE\n") != std::string::npos); + + // The no-crystal exit: the CLI hands over only the reason. The verdict has no image count to quote. + ProcessResult none; + none.warnings.push_back({PathologyCode::NO_LATTICE, "No crystal lattice was found: none"}); + const auto failed = RenderResultReport("p", "in.h5", x, none); + CHECK(failed.find("\nVERDICT= FAILED\n") != std::string::npos); + CHECK(failed.find("\nPATHOLOGY_FLAGS= NO_LATTICE\n") != std::string::npos); + CHECK(failed.find("No crystal lattice was found in these frames") != std::string::npos); + CHECK(failed.find("UNIT_CELL_CONSTANTS=") == std::string::npos); + CHECK(failed.find("SPACE_GROUP_NUMBER=") == std::string::npos); +} + +TEST_CASE("ResultReport_PowderAndIce", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + ProcessResult result; + result.consensus_cell = UnitCell{.a = 79.0f, .b = 79.0f, .c = 38.0f, + .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + result.has_merge_statistics = true; + + auto text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("Powder no rings detected") != std::string::npos); + CHECK(text.find("ICE_RINGS_DETECTED") == std::string::npos); + + result.powder.rings_q_recipA = {1.61f, 1.71f, 1.82f}; + result.powder.spot_fraction = 0.20f; + result.ice_ring_score = 2.69f; + result.ice_spot_ratio = 1.08f; + result.ice_rings_detected = true; + result.ice_flagged_fraction = 0.257; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("Powder 3 rings holding 20% of the spots found") != std::string::npos); + CHECK(text.find("\nICE_RINGS_DETECTED= TRUE\n") != std::string::npos); + CHECK(text.find("\nICE_REFLECTIONS_ON_RINGS_PCT= 25.7\n") != std::string::npos); + CHECK(text.find("Ice rings DETECTED: the 25.7% of the reflections") != std::string::npos); + CHECK(text.find("POWDER_RINGS ") == std::string::npos); + CHECK(text.find("\nPATHOLOGY_FLAGS= NONE\n") != std::string::npos); + + // Most of the spots on rings is a condition. + result.powder.spot_fraction = 0.74f; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nPATHOLOGY_FLAGS= POWDER_RINGS\n") != std::string::npos); +} + +TEST_CASE("ResultReport_SweepGapsMaterial", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + ProcessResult result; + result.consensus_cell = UnitCell{.a = 79.0f, .b = 79.0f, .c = 38.0f, + .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + result.has_merge_statistics = true; + result.merge_statistics.sweep_quality = TestSweepQuality(); + // Two frames of a long sweep: in the table and the summary, not a condition of the data set. + result.merge_statistics.sweep_quality.sweep_deg = 360.0f; + result.merge_statistics.sweep_quality.ranges.resize(1); + result.merge_statistics.sweep_quality.ranges[0].rotation_deg = 1.0f; + auto text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("SWEEP_GAPS") == std::string::npos); + CHECK(text.find("Frames 100-149") == std::string::npos); + RunProvenance dev; + dev.developer = true; + CHECK(RenderResultReport("p", "in.h5", x, result, dev).find("Frames 100-149") != std::string::npos); + + result.merge_statistics.sweep_quality.ranges[0].rotation_deg = 40.0f; + text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("\nPATHOLOGY_FLAGS= SWEEP_GAPS\n") != std::string::npos); +} + +TEST_CASE("ResultReport_SecondCrystal", "[Diagnostics]") { + DiffractionExperiment x(DetJF(1)); + ProcessResult result; + result.consensus_cell = UnitCell{.a = 79.0f, .b = 79.0f, .c = 38.0f, + .alpha = 90.0f, .beta = 90.0f, .gamma = 90.0f}; + LeftoverLatticeCensus census; + census.primary_intensity_pct = 80.0; + census.lattices.push_back(LeftoverLattice{.kind = LeftoverLatticeKind::Domain, .cell = *result.consensus_cell, + .volume_ratio = 1.0, .misorientation_deg = 38.9, .z = 50.0, + .spots_pct = 12.0, .intensity_pct = 14.0, .sweep_concentration = 0.3}); + result.leftover_lattices = census; + const auto text = RenderResultReport("p", "in.h5", x, result); + CHECK(text.find("second crystal, 38.9 deg from the main lattice") != std::string::npos); + CHECK(text.find("split crystal, ") == std::string::npos); +}