Refinement: the solvent audit of the refined model, as REFINE_SOLVENT_ keys and <prefix>_solvent.json

RefineModel runs AnalyseSolventSites on the refined model (and with --refine-score-only on the model
as given) after the blob search, in the pipeline after a merge and in --mode refine alike. The maps
step now also makes the 2mFo-DFc grid, which the audit reads at each site and which is released
straight after. Nothing reads the audit: the refined model, maps, blobs and every R are unchanged
(1HR2 --mode refine: refined model file identical to the build without it).

The report gets REFINE_SOLVENT_SITES, _LABELS, _BEST, _DISAGREE, _AMBIGUOUS, _IONS and
_IONS_DISAGREE and a paragraph listing the ions (label -> best-supported species, bands,
neighbours, the first 40); rugnux writes every site to <prefix>_solvent.json beside _blobs.json.
The broker's live path does not carry it to the writer yet.

Docs: CPU_DATA_ANALYSIS_REFINEMENT.md 15.10 describes what is measured and every term of the score;
RUGNUX_REPORT.md the keys and the JSON; ACKNOWLEDGEMENT.md credits the Cat_Wiz bands and the
Auffinger-group surveys, Echols 2014, Morshed 2015, Carugo 2014, Brown & Altermatt 1985, Brese &
O'Keeffe 1991 (R0 as tabulated by Zheng et al. 2017), Mueller et al. 2003 and CheckMyMetal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-27 13:20:01 +01:00
1 parent ca660416a5
commit 85e0d2da95
10 files changed
+226 -8

No files matched your search

+39
View File
@@ -601,6 +601,45 @@ a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI'
(`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development
of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493).
**Solvent audit** — the waters and ions of the refined model are scored by their surroundings with the
stereochemical criteria of the surveys of ions at nucleic acids by P. Auffinger's group: the coordination bands
1.9-2.3 / 2.3-2.6 / 2.6-3.2 Å, the Mg²⁺ exclusion zone and the clash shells of N. Naleem, A. Henning-Knechtel,
S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and
annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120
[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); the Mg-O and Mg-N distances and the
2.3-3.4 Å exclusion zone of F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar and P. Auffinger,
"Principles of ion binding to RNA inferred from the analysis of a 1.55 Å resolution bacterial ribosome
structure – Part I: Mg2+" (2025), Nucleic Acids Res. 53, gkae1148
[doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); the Na⁺, K⁺ and water distances and
coordination numbers of F. Leonarski, L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor
Mg2+ inner-sphere binders but excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192
[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118), and the distance bands of their
2017 survey (above). The ligand sphere of the bond-valence sums and the reading of halides by the donors
around them follow N. Echols, N. Morshed, P. V. Afonine, A. J. McCoy, M. D. Miller, R. J. Read,
J. S. Richardson, T. C. Terwilliger and P. D. Adams, "Automated identification of elemental ions in
macromolecular crystal structures" (2014), Acta Cryst. D70, 1104-1114
[doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308), and the halide's positive and
negative neighbours N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental
ion identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158
[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241); the chloride's shell of donors
at up to 3.4 Å is O. Carugo's, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC
Struct. Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8). The
bond-valence sums are I. D. Brown and D. Altermatt's, "Bond-valence parameters obtained from a systematic
analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247
[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of N. E. Brese and
M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197
[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by H. Zheng,
K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, "Data mining of
iron(II) and iron(III) bond-valence parameters, and their relevance for macromolecular crystallography"
(2017), Acta Cryst. D73, 316-325 [doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584),
weighted by occupancy as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method
able to identify metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37
[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); their normalised vector sum and
its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, I. G. Shabalin, K. B. Handing
and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation tool" (2017), Acta Cryst. D73, 223-233
[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). The score that combines them is
our own.
## Software and computing methods
Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading
+54 -3
View File
@@ -281,7 +281,53 @@ in the report. A blob read as `LIGAND` is density of ligand size where the model
molecule or an unmodelled side chain is that as well as the compound that was soaked. The readings point a
person at the density; they do not identify a compound.
### 15.10 GPU and CPU
### 15.10 The solvent audit
Every water and every single-atom ion of the refined model (and, with `--refine-score-only`, of the model as
given) is measured in its surroundings and scored as each of water, Mg²⁺, Na⁺, K⁺, Ca²⁺ and Cl⁻ (NH₄⁺ is
scored too, and only reported). It is an audit: the model is not changed, and nothing else reads it.
**What is measured.** The model's N and O atoms around the site, symmetry mates included, counted in the
coordination bands of Cat_Wiz - 1.9-2.3 Å (Mg-like), 2.3-2.6 Å (Na, Ca), 2.6-3.2 Å (K, water) - and in the
clash shells 3.2-3.4 and 3.4-3.6 Å beyond them. Each neighbour is typed from the dictionary's energy type
(donor, acceptor or both) and from the chemistry of its residue: Lys NZ, Arg NE/NH1/NH2 and a polypeptide's
N-terminal N are positive; Asp/Glu carboxylate O, a C-terminal carboxylate and a nucleotide's phosphate O are
negative; His ring N and the Asn/Gln amide ends are both donor and acceptor, as the model's orientation of
them is a guess. Also measured: the shortest distance between two neighbours closer than 2.6 Å (the
octahedron's cis O…O), the nearest positive and negative group, the other ions within 4.2 and 6 Å, the
bond-valence sum $V = \sum_j q_j \exp((R_0 - d_j)/0.37)$ over the N and O within 3.5 Å for each cation
($R_0$ of Brese & O'Keeffe, $q_j$ the neighbour's occupancy) with its normalised vector sum (CheckMyMetal's
nVECSUM), and the 2mFo-DFc, mFo-DFc and anomalous maps at the site, in their rms.
**The score.** For each species, a plain sum of terms in log units:
| Term | What it is |
|---|---|
| first shell | the neighbours in the species' band against the coordination number it holds (Mg 6, Na 5-6, K 6-10, Ca 6-8, Cl 3-5, NH₄⁺ 4), quadratic; for water only more than four, or none at all, costs; for Mg, Na and Ca a missing ligand costs little (0.5), as the model may simply not have that water |
| distances | each neighbour in the band against the species' own distance (Mg-O 2.07 ± 0.04 Å, Mg-N 2.19 ± 0.08 Å, Na 2.41, Ca 2.37, K and water 2.8, NH₄⁺ 2.9, Cl 3.2 Å) |
| contacts it does not make | 2 per N or O closer than the species' shell (closer than 2.9 Å for Cl), and for Mg per N or O in its exclusion zone 2.3-3.4 Å |
| chemistry | a cation: 2 per donor-only neighbour in its shell and per positive group within 3.5 Å; Cl: 2 per acceptor-only neighbour, per negative group within 3.2 Å, and 2 where there is neither a positive group nor a donor N within 3.5 Å; water: 2 per hydrogen bond beyond two donated or two accepted (neighbours at 2.6-3.4 Å) |
| bond valence | for a cation, $V$ against its charge $z$, Gaussian with a spread of $0.25z$ widened by the neighbours' distance errors, and the vector sum against CheckMyMetal's outlier bound 0.23 |
| electrons | where the difference map at the site is above +3 σ, +1 for the species with more electrons than the label (18 against 10) and -1 for the others; below -3 σ the reverse; in between -0.5 for a species of another electron count, as B and occupancy can hide eight electrons |
| anomalous | an anomalous peak of 4 σ or more: +2 for K, Ca and Cl, -2 for the others |
A site bound to another cation (closer than 2.6 Å) shares that cation's other ligands as neighbours - the
cis waters of a Mg(H₂O)₆ are 2.9 Å apart - and those are left out of its score. **Coordinate error**: each
atom's is taken as its rms displacement $\sqrt{B/8\pi^2}$ times $d_\mathrm{min}/10$ Š(about 0.1 Šat B = 20 Ų
and 2 Å); a neighbour's band is widened by the error of the distance and its distance term is widened by it
too, so that where the coordinates are poor the species' shells overlap and the score stops telling them
apart instead of telling them apart wrongly. The best of water, Mg, Na, K, Ca and Cl is reported, and
`AMBIGUOUS` where the runner-up is within 2. The audit counts a site as disagreeing with its label where
the best species is another one, not ambiguously.
**Caveats.** The refined model's metal-site distances are restrained (§15.5) where its LINK records name
the bonds, which pulls an ion's shell towards the label it carries - the first pitfall of every survey of
ions in the PDB. `--mode refine --refine-score-only` audits the model as given instead. The electron count
is read only from the residual difference density, which a B factor or an occupancy can absorb, and the map
values are not yet compared with the neighbouring atoms'. Per site, everything above is in
`<prefix>_solvent.json`; the counts are the `REFINE_SOLVENT_` keys of the report.
### 15.11 GPU and CPU
On a CUDA build the target, the gradient and the restraints are evaluated on the card - the density gather,
cuFFT, the composition of the symmetry and the target per reflection, the adjoint through a CSR list, the inverse transform and one warp per atom for its gradient; the restraints in one kernel whose
@@ -292,7 +338,7 @@ the card's budget, or with `--no-gpu`, or on a build without CUDA (macOS include
Metal path), the CPU engine computes the same target, to rounding, several times more slowly (seconds on a
GPU, typically 2-15 minutes on the CPU).
### 15.11 Not yet
### 15.12 Not yet
What the refinement does not do yet - each a known gap, not a design choice:
@@ -304,7 +350,8 @@ What the refinement does not do yet - each a known gap, not a design choice:
refined.
- **NCS restraints**: copies are refined independently.
- **Twin refinement**: the target is not a twinned one, also where the twin hypothesis' merge was written.
- **Water building**: waters are neither picked nor removed; the blob finder only points at candidates.
- **Water building and ion identification**: waters are neither picked nor removed and no ion is relabelled;
the blob finder points at candidates and the solvent audit (§15.10) scores the waters and ions there are.
- **Alternative conformations and occupancies**: existing conformers are refined in place; no occupancy is
refined and no conformer is added.
- **Rotamer and peptide-flip search**, and **Ramachandran restraints**.
@@ -332,6 +379,10 @@ What the refinement does not do yet - each a known gap, not a design choice:
- P. V. Afonine et al., "Towards automated crystallographic structure refinement with phenix.refine", *Acta Cryst.* **D68** (2012), 352-367 (weights chosen among refinements by R-free within geometry bounds, and the overall B moved into the atoms, §15.3, §15.7).
- A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures", *Nature* **355** (1992), 472-475 (R-free).
- P. Emsley, B. Lohkamp, W. G. Scott & K. Cowtan, "Features and development of Coot", *Acta Cryst.* **D66** (2010), 486-501 (unmodelled blobs, §15.9).
- N. Naleem, A. Henning-Knechtel, S. Kirmizialtin & P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and annotating Mg2+ ions in RNA structures", *Nucleic Acids Res.* **54** (2026), gkag120; F. Leonarski, L. D'Ascenzo & P. Auffinger, *Nucleic Acids Res.* **45** (2017), 987-1004 and *RNA* **25** (2019), 173-192; F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar & P. Auffinger, *Nucleic Acids Res.* **53** (2025), gkae1148 (the coordination bands, distances and exclusion zone of the solvent audit, §15.10).
- N. Echols et al., "Automated identification of elemental ions in macromolecular crystal structures", *Acta Cryst.* **D70** (2014), 1104-1114, and N. Morshed, N. Echols & P. D. Adams, "Using support vector machines to improve elemental ion identification in macromolecular crystal structures", *Acta Cryst.* **D71** (2015), 1147-1158 (the ligand sphere and the halide rules, §15.10).
- H. Zheng et al., "CheckMyMetal: a macromolecular metal-binding validation tool", *Acta Cryst.* **D73** (2017), 223-233 (the vector sum, §15.10); O. Carugo, "Buried chloride stereochemistry in the Protein Data Bank", *BMC Struct. Biol.* **14** (2014), 19 (the chloride's shell, §15.10).
- I. D. Brown & D. Altermatt, *Acta Cryst.* **B41** (1985), 244-247, and N. E. Brese & M. O'Keeffe, "Bond-valence parameters for solids", *Acta Cryst.* **B47** (1991), 192-197 (the bond-valence sums, §15.10).
- M. Abramowitz & I. A. Stegun, *Handbook of Mathematical Functions*, National Bureau of Standards (1964), 9.8.1-9.8.4 (the Bessel functions of the target, §15.2).
The full citations, with DOIs, are in [ACKNOWLEDGEMENT.md](ACKNOWLEDGEMENT.md).
+1
View File
@@ -137,6 +137,7 @@ myrun_model.cif / .pdb the model as placed, in the cell and space group of my
myrun_refined_model.cif the refined model (and .pdb)
myrun_refined_maps.mtz map coefficients on the refined model
myrun_blobs.json the unexplained difference-density blobs
myrun_solvent.json the refined model's waters and ions, each scored by its surroundings
```
A stills run writes the same merged files (its `.hkl` holds the merged reflections) and, on a
+14 -1
View File
@@ -1202,13 +1202,26 @@ Where a model was placed and refined ([Refinement](RUGNUX_MODEL_AND_REFERENCE.md
| `REFINE_BULK_SOLVENT_K_SOL`, `REFINE_BULK_SOLVENT_B_SOL` | *dev*. The flat bulk solvent of the refined model |
| `REFINE_BLOBS` | The blobs of the refined mFo-DFc map (above 3 σ, peaking above 4 σ) |
| `REFINE_BLOBS_LIGAND`, `REFINE_BLOBS_WATER`, `REFINE_BLOBS_ION`, `REFINE_BLOBS_MODEL_ERROR` | How many of them are read as each ([§15.9](CPU_DATA_ANALYSIS_REFINEMENT.md#159-unexplained-density-blobs)); the rest are unassigned |
| `REFINE_SOLVENT_SITES` | The waters and single-atom ions of the refined model the solvent audit scored ([§15.10](CPU_DATA_ANALYSIS_REFINEMENT.md#1510-the-solvent-audit)) |
| `REFINE_SOLVENT_LABELS`, `REFINE_SOLVENT_BEST` | How many of them carry each label (`HOH 265 MG 42`), and how many each species is the best-supported one for |
| `REFINE_SOLVENT_DISAGREE`, `REFINE_SOLVENT_AMBIGUOUS` | The sites whose best-supported species is not their label (water, Mg, Na, K, Ca or Cl), not ambiguously; the sites where the runner-up is within 2 of the best |
| `REFINE_SOLVENT_IONS`, `REFINE_SOLVENT_IONS_DISAGREE` | The same for the sites not labelled water |
A prose paragraph after them lists the five strongest blobs with their reading, peak, volume, position and
A prose paragraph after the blob keys lists the five strongest blobs with their reading, peak, volume, position and
nearest atom; all of them are in `<prefix>_blobs.json`, strongest first, one object per blob: `kind`,
`centroid` (Å, Cartesian, the copy nearest the model), `peak_sigma`, `volume_A3`, `score_sigma_A3` (the
density above the cutoff integrated), `nearest_atom`, `nearest_atom_distance_A`, `polar_contacts`,
`metal_contacts` and `anomalous_sigma`.
A second paragraph, after the solvent keys, lists the ions, label → best-supported species, their N and O in the bands
1.9-2.3/2.3-2.6/2.6-3.2 Å and their neighbours closer than 3.4 Å. Every site is in `<prefix>_solvent.json`,
in model order, one object per site: `label`, `chain`, `seqid`, `altloc`, `xyz`, `occupancy`, `b_iso`,
`sigma_xyz_A` (the coordinate error assumed), `bands` (N and O by distance), `donors`, `acceptors`,
`donor_acceptors` (closer than 3.4 Å), `min_ligand_distance_A`, `nearest_positive_A`, `nearest_negative_A`,
`positive_within_3.5`, `negative_within_3.2`, `nearest_ion_A`, `ions_within_4.2`, `ions_within_6`, `bvs` and
`nvecsum` (per cation: `MG`, `NA`, `K`, `CA`), `map_sigma`, `difference_sigma`, `anomalous_sigma` (null where
there is no such map), `score` (per species), `best`, `ambiguous`, `agrees` and `neighbours`.
**The score of the model as given** follows, every term under `REFINE_SCORE_`: `REFINE_SCORE_R_WORK`,
`REFINE_SCORE_R_FREE`, `REFINE_SCORE_N_WORK` and `REFINE_SCORE_N_FREE` by default, and with `--developer` the
overall scale (`REFINE_SCORE_K_OVERALL`), its isotropic and anisotropic B (`REFINE_SCORE_B_ISO`,
@@ -869,9 +869,9 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
{"PHDELWT", 'P'}, {"FOM", 'W'}, {"FREE", 'I'}};
for (const auto &[label, type] : columns)
mtz.columns.push_back({label, type, type == 'H' ? 0 : 1, {}});
gemmi::AsuData<std::complex<float>> diff;
diff.unit_cell_ = ctx.cell;
diff.spacegroup_ = ctx.sg;
gemmi::AsuData<std::complex<float>> diff, fwt_data;
diff.unit_cell_ = fwt_data.unit_cell_ = ctx.cell;
diff.spacegroup_ = fwt_data.spacegroup_ = ctx.sg;
for (size_t m = 0; m < ctx.refl.Size(); m++) {
const double fo = ctx.refl.fo[m], fc = std::abs(fmodel[m]);
const double mw = fom[m], d = ctx.rows.d[m];
@@ -889,13 +889,16 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
for (size_t c = 0; c < mtz.columns.size(); c++)
mtz.columns[c].values.push_back(row[c]);
diff.v.push_back({ctx.refl.hkl[m], std::polar(static_cast<float>(delfwt), static_cast<float>(phi))});
fwt_data.v.push_back({ctx.refl.hkl[m], std::polar(static_cast<float>(fwt), static_cast<float>(phi))});
}
result.maps = maps;
// The difference map beside the anomalous one below: neither reads the other.
std::future<void> difference_map = std::async(std::launch::async, [&diff, &result] {
std::future<void> difference_map = std::async(std::launch::async, [&diff, &fwt_data, &result] {
const std::array<int, 3> size = gemmi::get_size_for_hkl(diff, {{0, 0, 0}}, 3.0);
result.difference_map = std::make_shared<gemmi::Grid<float>>(MapFromFPhi(gemmi::get_f_phi_on_grid<float>(diff, size, true)));
result.difference_map_rms = gemmi::calculate_data_statistics(result.difference_map->data).rms;
result.map = std::make_shared<gemmi::Grid<float>>(MapFromFPhi(gemmi::get_f_phi_on_grid<float>(fwt_data, size, true)));
result.map_rms = gemmi::calculate_data_statistics(result.map->data).rms;
});
// The anomalous difference map: F(+) - F(-) on the model's phase turned back by 90 degrees, read on the +
@@ -1301,6 +1304,13 @@ RefinementResult RefineModel(const gemmi::Structure &placed, const std::vector<M
result.blobs = FindDensityBlobs(*result.difference_map, result.difference_map_rms, ctx.st, result.d_min,
result.anomalous_map.get(), result.anomalous_map_rms);
}
{
const RefinementStage stage("solvent");
const SolventMaps maps{result.map.get(), result.map_rms, result.difference_map.get(),
result.difference_map_rms, result.anomalous_map.get(), result.anomalous_map_rms};
result.solvent = AnalyseSolventSites(ctx.st, result.d_min, maps);
result.map.reset();
}
result.k_sol = ctx.k_sol;
result.b_sol = ctx.b_sol;
result.model = std::make_shared<gemmi::Structure>(ctx.st);
@@ -20,6 +20,7 @@
#include "DensityBlobs.h"
#include "ModelValidation.h" // ModelMaps
#include "RefinementRestraints.h" // GeometryStatistics
#include "SolventSites.h"
class Logger;
@@ -143,6 +144,10 @@ struct RefinementResult {
std::shared_ptr<gemmi::Structure> model; // the refined model, in the data's cell and frame
std::shared_ptr<ModelMaps> maps; // its 2mFo-DFc and mFo-DFc coefficients (as _maps.mtz holds them)
// The 2mFo-DFc map of the refined model and its rms, for reading the solvent sites in (SolventSites.h); released
// once they are read.
std::shared_ptr<gemmi::Grid<float>> map;
double map_rms = 0;
// The mFo-DFc map of the refined model and its rms, for finding what it leaves unexplained (DensityBlobs.h).
std::shared_ptr<gemmi::Grid<float>> difference_map;
double difference_map_rms = 0;
@@ -151,6 +156,9 @@ struct RefinementResult {
double anomalous_map_rms = 0;
// What the refined model leaves unexplained in the difference map (FindDensityBlobs).
std::vector<DensityBlob> blobs;
// The refined model's waters and ions and the species their surroundings support (AnalyseSolventSites): an
// audit, nothing in the model is changed by it.
std::vector<SolventSite> solvent;
};
// `placed`: the model in the data's cell and space group. `merged`: the merge, with French-Wilson amplitudes
+48
View File
@@ -4,6 +4,7 @@
#include <cmath>
#include <algorithm>
#include <cctype>
#include <map>
#include <sstream>
#include <spdlog/fmt/fmt.h>
@@ -455,6 +456,52 @@ namespace {
return h;
}
// The solvent audit of the refined model (SolventSites.h): REFINE_SOLVENT_ keys. Counts by label as modelled
// and by the species the surroundings support best; a site disagrees where the best is another species, not
// ambiguously.
void AddSolventKeys(ReportSection &s, const std::vector<SolventSite> &sites, const std::string &output_prefix) {
std::map<std::string, int> labels, best;
constexpr int LISTED_IONS = 40; // the rest are in the JSON
int disagree = 0, ambiguous = 0, ions = 0, ions_disagree = 0;
std::string ion_rows;
for (const SolventSite &site : sites) {
labels[site.label]++;
best[SolventSpeciesName(site.best)]++;
ambiguous += site.ambiguous;
const bool differs = SolventLabelScored(site) && !site.ambiguous && !SolventLabelAgrees(site);
disagree += differs;
if (!SolventLabelIsWater(site)) {
ions++;
ions_disagree += differs;
if (ions <= LISTED_IONS)
ion_rows += fmt::format("\n {:<4} {}{:<6} -> {:<3}{} bands {}/{}/{}, {}", site.label, site.chain,
site.seqid, SolventSpeciesName(site.best),
site.ambiguous ? " (ambiguous)" : "", site.n_19_23, site.n_23_26,
site.n_26_32, site.neighbours.empty() ? "no N/O within 3.4 A" : site.neighbours);
}
}
if (ions > LISTED_IONS)
ion_rows += fmt::format("\n ... and {} more", ions - LISTED_IONS);
auto counts = [](const std::map<std::string, int> &m) {
std::string t;
for (const auto &[name, n] : m)
t += fmt::format("{}{} {}", t.empty() ? "" : " ", name, n);
return t;
};
Add(s, KeyInt("REFINE_SOLVENT_SITES", static_cast<int64_t>(sites.size())));
Add(s, KeyText("REFINE_SOLVENT_LABELS", counts(labels)));
Add(s, KeyText("REFINE_SOLVENT_BEST", counts(best)));
Add(s, KeyInt("REFINE_SOLVENT_DISAGREE", disagree));
Add(s, KeyInt("REFINE_SOLVENT_AMBIGUOUS", ambiguous));
Add(s, KeyInt("REFINE_SOLVENT_IONS", ions));
Add(s, KeyInt("REFINE_SOLVENT_IONS_DISAGREE", ions_disagree));
Add(s, Prose(fmt::format(
" The waters and ions of the refined model, each scored by its surroundings (an audit: the\n"
" model is not changed). Per site in {}_solvent.json; the ions, label -> best-supported\n"
" species, N/O in the bands 1.9-2.3/2.3-2.6/2.6-3.2 A, and their neighbours:{}",
output_prefix, ion_rows.empty() ? std::string(" none") : ion_rows)));
}
// The refinement's keys (Refinement.h): REFINE_ of the refinement of a placed model and of --mode refine, with every term of the
// score of the model as given under REFINE_SCORE_.
void AddRefinementKeys(ReportSection &s, const RefinementResult &rf, const std::string &output_prefix) {
@@ -518,6 +565,7 @@ namespace {
" R_FREE above remain the model as given. The refinement decided nothing about the\n"
" processing. The strongest unexplained blobs of its difference map ({}_blobs.json):{}",
output_prefix, output_prefix, top.empty() ? std::string(" none") : top)));
AddSolventKeys(s, rf.solvent, output_prefix);
}
if (rf.ok) {
const RefinementScore &sc = rf.score_start;
+1
View File
@@ -418,6 +418,7 @@ struct RugnuxOutputs {
std::optional<PlacedModelOutput> refined_model; // <prefix>_refined_model.cif/.pdb (ProcessConfig::refine)
std::shared_ptr<ModelMaps> refined_maps; // <prefix>_refined_maps.mtz
std::optional<std::vector<DensityBlob>> blobs; // <prefix>_blobs.json
std::optional<std::vector<SolventSite>> solvent; // <prefix>_solvent.json
std::optional<MtzTable> unmerged; // <prefix>_unmerged.mtz
std::optional<MtzTable> unmerged_partials; // <prefix>_unmerged_partials.mtz
std::string detector_jpeg; // <prefix>_detector.jpg; empty: none drawn
+46
View File
@@ -3,9 +3,12 @@
#include "RugnuxOutputFiles.h"
#include <cmath>
#include <filesystem>
#include <fstream>
#include <nlohmann/json.hpp>
#include "../writer/FileWriter.h"
#include "../writer/WriteModel.h"
#include "../writer/WriteRugnuxFiles.h"
@@ -23,6 +26,46 @@ void WriteDensityBlobs(const std::vector<DensityBlob> &blobs, const std::string
logger.Info("Difference-map blobs written to {}", path);
}
// The solvent audit of the refined model (SolventSites.h), one entry per site in model order.
void WriteSolventSites(const std::vector<SolventSite> &sites, const std::string &path, Logger &logger) {
std::ofstream f(path, std::ofstream::out | std::ofstream::trunc);
if (!f) {
logger.Warning("Cannot open {} for writing", path);
return;
}
// NaN and infinity (no such map, no such neighbour) as null
auto num = [](double v) { return std::isfinite(v) ? nlohmann::json(v) : nlohmann::json(); };
const char *cations[BVS_CATIONS] = {"MG", "NA", "K", "CA"};
nlohmann::json list = nlohmann::json::array();
for (const SolventSite &s : sites) {
nlohmann::json bvs, nvecsum, score;
for (int c = 0; c < BVS_CATIONS; c++) {
bvs[cations[c]] = s.bvs[c];
nvecsum[cations[c]] = s.nvecsum[c];
}
for (int k = 0; k < SOLVENT_SPECIES; k++)
score[SolventSpeciesName(static_cast<SolventSpecies>(k))] = s.score[k];
list.push_back({{"label", s.label}, {"chain", s.chain}, {"seqid", s.seqid},
{"altloc", s.altloc ? std::string(1, s.altloc) : std::string()},
{"xyz", {s.pos.x, s.pos.y, s.pos.z}}, {"occupancy", s.occupancy}, {"b_iso", s.b_iso},
{"sigma_xyz_A", s.sigma_xyz},
{"bands", {{"below_1.9", s.n_below_19}, {"1.9-2.3", s.n_19_23}, {"2.3-2.6", s.n_23_26},
{"2.6-3.2", s.n_26_32}, {"3.2-3.4", s.n_32_34}, {"3.4-3.6", s.n_34_36}}},
{"donors", s.donors}, {"acceptors", s.acceptors}, {"donor_acceptors", s.donor_acceptors},
{"min_ligand_distance_A", num(s.min_ligand_distance)},
{"nearest_positive_A", num(s.nearest_positive)}, {"nearest_negative_A", num(s.nearest_negative)},
{"positive_within_3.5", s.positive_contacts}, {"negative_within_3.2", s.negative_contacts},
{"nearest_ion_A", num(s.nearest_ion)}, {"ions_within_4.2", s.ions_within_42},
{"ions_within_6", s.ions_within_6}, {"bvs", bvs}, {"nvecsum", nvecsum},
{"map_sigma", num(s.map_sigma)}, {"difference_sigma", num(s.difference_sigma)},
{"anomalous_sigma", num(s.anomalous_sigma)}, {"score", score},
{"best", SolventSpeciesName(s.best)}, {"ambiguous", s.ambiguous},
{"agrees", SolventLabelAgrees(s)}, {"neighbours", s.neighbours}});
}
f << nlohmann::json{{"sites", list}}.dump(1) << "\n";
logger.Info("Solvent sites written to {}", path);
}
// The files of one output pass at `prefix`, all but its _process.h5.
void WriteOutputs(const RugnuxOutputs &o, const std::string &prefix, const OutputProvenance &provenance,
size_t nthreads, Logger &logger) {
@@ -48,6 +91,8 @@ void WriteOutputs(const RugnuxOutputs &o, const std::string &prefix, const Outpu
WriteModelMaps(*o.refined_maps, prefix + "_refined", logger);
if (o.blobs)
WriteDensityBlobs(*o.blobs, prefix + "_blobs.json", logger);
if (o.solvent)
WriteSolventSites(*o.solvent, prefix + "_solvent.json", logger);
if (o.unmerged) {
WriteMtz(*o.unmerged, prefix + "_unmerged.mtz");
logger.Info("Unmerged observations written to {}_unmerged.mtz", prefix);
@@ -194,6 +239,7 @@ ReportDocument WriteRefineOutputs(const RefineOnlyResult &result, const std::str
WritePlacedModel(*rf.model, cell, *sg, prefix + "_refined", logger);
WriteModelMaps(*rf.maps, prefix + "_refined", logger);
WriteDensityBlobs(rf.blobs, prefix + "_blobs.json", logger);
WriteSolventSites(rf.solvent, prefix + "_solvent.json", logger);
}
try {
std::ofstream file(prefix + "_report.txt");
+1
View File
@@ -5114,6 +5114,7 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
result.outputs->refined_model = PlacedModelOutput{refined.model, p.cell, p.space_group};
result.outputs->refined_maps = refined.maps;
result.outputs->blobs = refined.blobs;
result.outputs->solvent = refined.solvent;
Note(fmt::format("refined{}: R-work {:.3f}, R-free {:.3f}, rmsd bonds {:.3f} A, {} blobs",
refined.cancelled ? " (cut short)" : "", refined.r_work, refined.r_free,
refined.geometry.rmsd_bonds, refined.blobs.size()));