Refinement: the solvent audit of the refined model, as REFINE_SOLVENT_ keys and <prefix>_solvent.json
RefineModel runs AnalyseSolventSites on the refined model (and with --refine-score-only on the model as given) after the blob search, in the pipeline after a merge and in --mode refine alike. The maps step now also makes the 2mFo-DFc grid, which the audit reads at each site and which is released straight after. Nothing reads the audit: the refined model, maps, blobs and every R are unchanged (1HR2 --mode refine: refined model file identical to the build without it). The report gets REFINE_SOLVENT_SITES, _LABELS, _BEST, _DISAGREE, _AMBIGUOUS, _IONS and _IONS_DISAGREE and a paragraph listing the ions (label -> best-supported species, bands, neighbours, the first 40); rugnux writes every site to <prefix>_solvent.json beside _blobs.json. The broker's live path does not carry it to the writer yet. Docs: CPU_DATA_ANALYSIS_REFINEMENT.md 15.10 describes what is measured and every term of the score; RUGNUX_REPORT.md the keys and the JSON; ACKNOWLEDGEMENT.md credits the Cat_Wiz bands and the Auffinger-group surveys, Echols 2014, Morshed 2015, Carugo 2014, Brown & Altermatt 1985, Brese & O'Keeffe 1991 (R0 as tabulated by Zheng et al. 2017), Mueller et al. 2003 and CheckMyMetal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37
This commit is contained in:
1 parent
ca660416a5
commit
85e0d2da95
10 files changed
+226
-8
No files matched your search
@@ -601,6 +601,45 @@ a cutoff, found by flood fill as Coot's "unmodelled blobs" finds them, in GEMMI'
|
||||
(`find_blobs_by_flood_fill`): P. Emsley, B. Lohkamp, W. G. Scott and K. Cowtan, "Features and development
|
||||
of Coot" (2010), Acta Cryst. D66, 486-501 [doi:10.1107/S0907444910007493](https://doi.org/10.1107/S0907444910007493).
|
||||
|
||||
**Solvent audit** — the waters and ions of the refined model are scored by their surroundings with the
|
||||
stereochemical criteria of the surveys of ions at nucleic acids by P. Auffinger's group: the coordination bands
|
||||
1.9-2.3 / 2.3-2.6 / 2.6-3.2 Å, the Mg²⁺ exclusion zone and the clash shells of N. Naleem, A. Henning-Knechtel,
|
||||
S. Kirmizialtin and P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and
|
||||
annotating Mg2+ ions in RNA structures" (2026), Nucleic Acids Res. 54, gkag120
|
||||
[doi:10.1093/nar/gkag120](https://doi.org/10.1093/nar/gkag120); the Mg-O and Mg-N distances and the
|
||||
2.3-3.4 Å exclusion zone of F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar and P. Auffinger,
|
||||
"Principles of ion binding to RNA inferred from the analysis of a 1.55 Å resolution bacterial ribosome
|
||||
structure – Part I: Mg2+" (2025), Nucleic Acids Res. 53, gkae1148
|
||||
[doi:10.1093/nar/gkae1148](https://doi.org/10.1093/nar/gkae1148); the Na⁺, K⁺ and water distances and
|
||||
coordination numbers of F. Leonarski, L. D'Ascenzo and P. Auffinger, "Nucleobase carbonyl groups are poor
|
||||
Mg2+ inner-sphere binders but excellent monovalent ion binders—a critical PDB survey" (2019), RNA 25, 173-192
|
||||
[doi:10.1261/rna.068437.118](https://doi.org/10.1261/rna.068437.118), and the distance bands of their
|
||||
2017 survey (above). The ligand sphere of the bond-valence sums and the reading of halides by the donors
|
||||
around them follow N. Echols, N. Morshed, P. V. Afonine, A. J. McCoy, M. D. Miller, R. J. Read,
|
||||
J. S. Richardson, T. C. Terwilliger and P. D. Adams, "Automated identification of elemental ions in
|
||||
macromolecular crystal structures" (2014), Acta Cryst. D70, 1104-1114
|
||||
[doi:10.1107/S1399004714001308](https://doi.org/10.1107/S1399004714001308), and the halide's positive and
|
||||
negative neighbours N. Morshed, N. Echols and P. D. Adams, "Using support vector machines to improve elemental
|
||||
ion identification in macromolecular crystal structures" (2015), Acta Cryst. D71, 1147-1158
|
||||
[doi:10.1107/S1399004715004241](https://doi.org/10.1107/S1399004715004241); the chloride's shell of donors
|
||||
at up to 3.4 Å is O. Carugo's, "Buried chloride stereochemistry in the Protein Data Bank" (2014), BMC
|
||||
Struct. Biol. 14, 19 [doi:10.1186/s12900-014-0019-8](https://doi.org/10.1186/s12900-014-0019-8). The
|
||||
bond-valence sums are I. D. Brown and D. Altermatt's, "Bond-valence parameters obtained from a systematic
|
||||
analysis of the Inorganic Crystal Structure Database" (1985), Acta Cryst. B41, 244-247
|
||||
[doi:10.1107/S0108768185002063](https://doi.org/10.1107/S0108768185002063), with the R₀ of N. E. Brese and
|
||||
M. O'Keeffe, "Bond-valence parameters for solids" (1991), Acta Cryst. B47, 192-197
|
||||
[doi:10.1107/S0108768190011041](https://doi.org/10.1107/S0108768190011041), as tabulated by H. Zheng,
|
||||
K. M. Langner, G. P. Shields, J. Hou, M. Kowiel, F. H. Allen, G. Murshudov and W. Minor, "Data mining of
|
||||
iron(II) and iron(III) bond-valence parameters, and their relevance for macromolecular crystallography"
|
||||
(2017), Acta Cryst. D73, 316-325 [doi:10.1107/S2059798317000584](https://doi.org/10.1107/S2059798317000584),
|
||||
weighted by occupancy as P. Müller, S. Köpke and G. M. Sheldrick weight them, "Is the bond-valence method
|
||||
able to identify metal atoms in protein structures?" (2003), Acta Cryst. D59, 32-37
|
||||
[doi:10.1107/S0907444902018000](https://doi.org/10.1107/S0907444902018000); their normalised vector sum and
|
||||
its outlier bound are CheckMyMetal's: H. Zheng, D. R. Cooper, P. J. Porebski, I. G. Shabalin, K. B. Handing
|
||||
and W. Minor, "CheckMyMetal: a macromolecular metal-binding validation tool" (2017), Acta Cryst. D73, 223-233
|
||||
[doi:10.1107/S2059798317001061](https://doi.org/10.1107/S2059798317001061). The score that combines them is
|
||||
our own.
|
||||
|
||||
## Software and computing methods
|
||||
|
||||
Decoding bitshuffle+LZ4 images on the GPU, rather than decompressing them on the host and uploading
|
||||
|
||||
@@ -281,7 +281,53 @@ in the report. A blob read as `LIGAND` is density of ligand size where the model
|
||||
molecule or an unmodelled side chain is that as well as the compound that was soaked. The readings point a
|
||||
person at the density; they do not identify a compound.
|
||||
|
||||
### 15.10 GPU and CPU
|
||||
### 15.10 The solvent audit
|
||||
|
||||
Every water and every single-atom ion of the refined model (and, with `--refine-score-only`, of the model as
|
||||
given) is measured in its surroundings and scored as each of water, Mg²⁺, Na⁺, K⁺, Ca²⁺ and Cl⁻ (NH₄⁺ is
|
||||
scored too, and only reported). It is an audit: the model is not changed, and nothing else reads it.
|
||||
|
||||
**What is measured.** The model's N and O atoms around the site, symmetry mates included, counted in the
|
||||
coordination bands of Cat_Wiz - 1.9-2.3 Å (Mg-like), 2.3-2.6 Å (Na, Ca), 2.6-3.2 Å (K, water) - and in the
|
||||
clash shells 3.2-3.4 and 3.4-3.6 Å beyond them. Each neighbour is typed from the dictionary's energy type
|
||||
(donor, acceptor or both) and from the chemistry of its residue: Lys NZ, Arg NE/NH1/NH2 and a polypeptide's
|
||||
N-terminal N are positive; Asp/Glu carboxylate O, a C-terminal carboxylate and a nucleotide's phosphate O are
|
||||
negative; His ring N and the Asn/Gln amide ends are both donor and acceptor, as the model's orientation of
|
||||
them is a guess. Also measured: the shortest distance between two neighbours closer than 2.6 Å (the
|
||||
octahedron's cis O…O), the nearest positive and negative group, the other ions within 4.2 and 6 Å, the
|
||||
bond-valence sum $V = \sum_j q_j \exp((R_0 - d_j)/0.37)$ over the N and O within 3.5 Å for each cation
|
||||
($R_0$ of Brese & O'Keeffe, $q_j$ the neighbour's occupancy) with its normalised vector sum (CheckMyMetal's
|
||||
nVECSUM), and the 2mFo-DFc, mFo-DFc and anomalous maps at the site, in their rms.
|
||||
|
||||
**The score.** For each species, a plain sum of terms in log units:
|
||||
|
||||
| Term | What it is |
|
||||
|---|---|
|
||||
| first shell | the neighbours in the species' band against the coordination number it holds (Mg 6, Na 5-6, K 6-10, Ca 6-8, Cl 3-5, NH₄⁺ 4), quadratic; for water only more than four, or none at all, costs; for Mg, Na and Ca a missing ligand costs little (0.5), as the model may simply not have that water |
|
||||
| distances | each neighbour in the band against the species' own distance (Mg-O 2.07 ± 0.04 Å, Mg-N 2.19 ± 0.08 Å, Na 2.41, Ca 2.37, K and water 2.8, NH₄⁺ 2.9, Cl 3.2 Å) |
|
||||
| contacts it does not make | 2 per N or O closer than the species' shell (closer than 2.9 Å for Cl), and for Mg per N or O in its exclusion zone 2.3-3.4 Å |
|
||||
| chemistry | a cation: 2 per donor-only neighbour in its shell and per positive group within 3.5 Å; Cl: 2 per acceptor-only neighbour, per negative group within 3.2 Å, and 2 where there is neither a positive group nor a donor N within 3.5 Å; water: 2 per hydrogen bond beyond two donated or two accepted (neighbours at 2.6-3.4 Å) |
|
||||
| bond valence | for a cation, $V$ against its charge $z$, Gaussian with a spread of $0.25z$ widened by the neighbours' distance errors, and the vector sum against CheckMyMetal's outlier bound 0.23 |
|
||||
| electrons | where the difference map at the site is above +3 σ, +1 for the species with more electrons than the label (18 against 10) and -1 for the others; below -3 σ the reverse; in between -0.5 for a species of another electron count, as B and occupancy can hide eight electrons |
|
||||
| anomalous | an anomalous peak of 4 σ or more: +2 for K, Ca and Cl, -2 for the others |
|
||||
|
||||
A site bound to another cation (closer than 2.6 Å) shares that cation's other ligands as neighbours - the
|
||||
cis waters of a Mg(H₂O)₆ are 2.9 Å apart - and those are left out of its score. **Coordinate error**: each
|
||||
atom's is taken as its rms displacement $\sqrt{B/8\pi^2}$ times $d_\mathrm{min}/10$ Š(about 0.1 Šat B = 20 Ų
|
||||
and 2 Å); a neighbour's band is widened by the error of the distance and its distance term is widened by it
|
||||
too, so that where the coordinates are poor the species' shells overlap and the score stops telling them
|
||||
apart instead of telling them apart wrongly. The best of water, Mg, Na, K, Ca and Cl is reported, and
|
||||
`AMBIGUOUS` where the runner-up is within 2. The audit counts a site as disagreeing with its label where
|
||||
the best species is another one, not ambiguously.
|
||||
|
||||
**Caveats.** The refined model's metal-site distances are restrained (§15.5) where its LINK records name
|
||||
the bonds, which pulls an ion's shell towards the label it carries - the first pitfall of every survey of
|
||||
ions in the PDB. `--mode refine --refine-score-only` audits the model as given instead. The electron count
|
||||
is read only from the residual difference density, which a B factor or an occupancy can absorb, and the map
|
||||
values are not yet compared with the neighbouring atoms'. Per site, everything above is in
|
||||
`<prefix>_solvent.json`; the counts are the `REFINE_SOLVENT_` keys of the report.
|
||||
|
||||
### 15.11 GPU and CPU
|
||||
|
||||
On a CUDA build the target, the gradient and the restraints are evaluated on the card - the density gather,
|
||||
cuFFT, the composition of the symmetry and the target per reflection, the adjoint through a CSR list, the inverse transform and one warp per atom for its gradient; the restraints in one kernel whose
|
||||
@@ -292,7 +338,7 @@ the card's budget, or with `--no-gpu`, or on a build without CUDA (macOS include
|
||||
Metal path), the CPU engine computes the same target, to rounding, several times more slowly (seconds on a
|
||||
GPU, typically 2-15 minutes on the CPU).
|
||||
|
||||
### 15.11 Not yet
|
||||
### 15.12 Not yet
|
||||
|
||||
What the refinement does not do yet - each a known gap, not a design choice:
|
||||
|
||||
@@ -304,7 +350,8 @@ What the refinement does not do yet - each a known gap, not a design choice:
|
||||
refined.
|
||||
- **NCS restraints**: copies are refined independently.
|
||||
- **Twin refinement**: the target is not a twinned one, also where the twin hypothesis' merge was written.
|
||||
- **Water building**: waters are neither picked nor removed; the blob finder only points at candidates.
|
||||
- **Water building and ion identification**: waters are neither picked nor removed and no ion is relabelled;
|
||||
the blob finder points at candidates and the solvent audit (§15.10) scores the waters and ions there are.
|
||||
- **Alternative conformations and occupancies**: existing conformers are refined in place; no occupancy is
|
||||
refined and no conformer is added.
|
||||
- **Rotamer and peptide-flip search**, and **Ramachandran restraints**.
|
||||
@@ -332,6 +379,10 @@ What the refinement does not do yet - each a known gap, not a design choice:
|
||||
- P. V. Afonine et al., "Towards automated crystallographic structure refinement with phenix.refine", *Acta Cryst.* **D68** (2012), 352-367 (weights chosen among refinements by R-free within geometry bounds, and the overall B moved into the atoms, §15.3, §15.7).
|
||||
- A. T. Brünger, "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures", *Nature* **355** (1992), 472-475 (R-free).
|
||||
- P. Emsley, B. Lohkamp, W. G. Scott & K. Cowtan, "Features and development of Coot", *Acta Cryst.* **D66** (2010), 486-501 (unmodelled blobs, §15.9).
|
||||
- N. Naleem, A. Henning-Knechtel, S. Kirmizialtin & P. Auffinger, "Cat_Wiz: a stereochemistry-guided toolkit for locating, diagnosing, and annotating Mg2+ ions in RNA structures", *Nucleic Acids Res.* **54** (2026), gkag120; F. Leonarski, L. D'Ascenzo & P. Auffinger, *Nucleic Acids Res.* **45** (2017), 987-1004 and *RNA* **25** (2019), 173-192; F. Leonarski, A. Henning-Knechtel, S. Kirmizialtin, E. Ennifar & P. Auffinger, *Nucleic Acids Res.* **53** (2025), gkae1148 (the coordination bands, distances and exclusion zone of the solvent audit, §15.10).
|
||||
- N. Echols et al., "Automated identification of elemental ions in macromolecular crystal structures", *Acta Cryst.* **D70** (2014), 1104-1114, and N. Morshed, N. Echols & P. D. Adams, "Using support vector machines to improve elemental ion identification in macromolecular crystal structures", *Acta Cryst.* **D71** (2015), 1147-1158 (the ligand sphere and the halide rules, §15.10).
|
||||
- H. Zheng et al., "CheckMyMetal: a macromolecular metal-binding validation tool", *Acta Cryst.* **D73** (2017), 223-233 (the vector sum, §15.10); O. Carugo, "Buried chloride stereochemistry in the Protein Data Bank", *BMC Struct. Biol.* **14** (2014), 19 (the chloride's shell, §15.10).
|
||||
- I. D. Brown & D. Altermatt, *Acta Cryst.* **B41** (1985), 244-247, and N. E. Brese & M. O'Keeffe, "Bond-valence parameters for solids", *Acta Cryst.* **B47** (1991), 192-197 (the bond-valence sums, §15.10).
|
||||
- M. Abramowitz & I. A. Stegun, *Handbook of Mathematical Functions*, National Bureau of Standards (1964), 9.8.1-9.8.4 (the Bessel functions of the target, §15.2).
|
||||
|
||||
The full citations, with DOIs, are in [ACKNOWLEDGEMENT.md](ACKNOWLEDGEMENT.md).
|
||||
@@ -137,6 +137,7 @@ myrun_model.cif / .pdb the model as placed, in the cell and space group of my
|
||||
myrun_refined_model.cif the refined model (and .pdb)
|
||||
myrun_refined_maps.mtz map coefficients on the refined model
|
||||
myrun_blobs.json the unexplained difference-density blobs
|
||||
myrun_solvent.json the refined model's waters and ions, each scored by its surroundings
|
||||
```
|
||||
|
||||
A stills run writes the same merged files (its `.hkl` holds the merged reflections) and, on a
|
||||
|
||||
+14
-1
@@ -1202,13 +1202,26 @@ Where a model was placed and refined ([Refinement](RUGNUX_MODEL_AND_REFERENCE.md
|
||||
| `REFINE_BULK_SOLVENT_K_SOL`, `REFINE_BULK_SOLVENT_B_SOL` | *dev*. The flat bulk solvent of the refined model |
|
||||
| `REFINE_BLOBS` | The blobs of the refined mFo-DFc map (above 3 σ, peaking above 4 σ) |
|
||||
| `REFINE_BLOBS_LIGAND`, `REFINE_BLOBS_WATER`, `REFINE_BLOBS_ION`, `REFINE_BLOBS_MODEL_ERROR` | How many of them are read as each ([§15.9](CPU_DATA_ANALYSIS_REFINEMENT.md#159-unexplained-density-blobs)); the rest are unassigned |
|
||||
| `REFINE_SOLVENT_SITES` | The waters and single-atom ions of the refined model the solvent audit scored ([§15.10](CPU_DATA_ANALYSIS_REFINEMENT.md#1510-the-solvent-audit)) |
|
||||
| `REFINE_SOLVENT_LABELS`, `REFINE_SOLVENT_BEST` | How many of them carry each label (`HOH 265 MG 42`), and how many each species is the best-supported one for |
|
||||
| `REFINE_SOLVENT_DISAGREE`, `REFINE_SOLVENT_AMBIGUOUS` | The sites whose best-supported species is not their label (water, Mg, Na, K, Ca or Cl), not ambiguously; the sites where the runner-up is within 2 of the best |
|
||||
| `REFINE_SOLVENT_IONS`, `REFINE_SOLVENT_IONS_DISAGREE` | The same for the sites not labelled water |
|
||||
|
||||
A prose paragraph after them lists the five strongest blobs with their reading, peak, volume, position and
|
||||
A prose paragraph after the blob keys lists the five strongest blobs with their reading, peak, volume, position and
|
||||
nearest atom; all of them are in `<prefix>_blobs.json`, strongest first, one object per blob: `kind`,
|
||||
`centroid` (Å, Cartesian, the copy nearest the model), `peak_sigma`, `volume_A3`, `score_sigma_A3` (the
|
||||
density above the cutoff integrated), `nearest_atom`, `nearest_atom_distance_A`, `polar_contacts`,
|
||||
`metal_contacts` and `anomalous_sigma`.
|
||||
|
||||
A second paragraph, after the solvent keys, lists the ions, label → best-supported species, their N and O in the bands
|
||||
1.9-2.3/2.3-2.6/2.6-3.2 Å and their neighbours closer than 3.4 Å. Every site is in `<prefix>_solvent.json`,
|
||||
in model order, one object per site: `label`, `chain`, `seqid`, `altloc`, `xyz`, `occupancy`, `b_iso`,
|
||||
`sigma_xyz_A` (the coordinate error assumed), `bands` (N and O by distance), `donors`, `acceptors`,
|
||||
`donor_acceptors` (closer than 3.4 Å), `min_ligand_distance_A`, `nearest_positive_A`, `nearest_negative_A`,
|
||||
`positive_within_3.5`, `negative_within_3.2`, `nearest_ion_A`, `ions_within_4.2`, `ions_within_6`, `bvs` and
|
||||
`nvecsum` (per cation: `MG`, `NA`, `K`, `CA`), `map_sigma`, `difference_sigma`, `anomalous_sigma` (null where
|
||||
there is no such map), `score` (per species), `best`, `ambiguous`, `agrees` and `neighbours`.
|
||||
|
||||
**The score of the model as given** follows, every term under `REFINE_SCORE_`: `REFINE_SCORE_R_WORK`,
|
||||
`REFINE_SCORE_R_FREE`, `REFINE_SCORE_N_WORK` and `REFINE_SCORE_N_FREE` by default, and with `--developer` the
|
||||
overall scale (`REFINE_SCORE_K_OVERALL`), its isotropic and anisotropic B (`REFINE_SCORE_B_ISO`,
|
||||
|
||||
@@ -869,9 +869,9 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
|
||||
{"PHDELWT", 'P'}, {"FOM", 'W'}, {"FREE", 'I'}};
|
||||
for (const auto &[label, type] : columns)
|
||||
mtz.columns.push_back({label, type, type == 'H' ? 0 : 1, {}});
|
||||
gemmi::AsuData<std::complex<float>> diff;
|
||||
diff.unit_cell_ = ctx.cell;
|
||||
diff.spacegroup_ = ctx.sg;
|
||||
gemmi::AsuData<std::complex<float>> diff, fwt_data;
|
||||
diff.unit_cell_ = fwt_data.unit_cell_ = ctx.cell;
|
||||
diff.spacegroup_ = fwt_data.spacegroup_ = ctx.sg;
|
||||
for (size_t m = 0; m < ctx.refl.Size(); m++) {
|
||||
const double fo = ctx.refl.fo[m], fc = std::abs(fmodel[m]);
|
||||
const double mw = fom[m], d = ctx.rows.d[m];
|
||||
@@ -889,13 +889,16 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
|
||||
for (size_t c = 0; c < mtz.columns.size(); c++)
|
||||
mtz.columns[c].values.push_back(row[c]);
|
||||
diff.v.push_back({ctx.refl.hkl[m], std::polar(static_cast<float>(delfwt), static_cast<float>(phi))});
|
||||
fwt_data.v.push_back({ctx.refl.hkl[m], std::polar(static_cast<float>(fwt), static_cast<float>(phi))});
|
||||
}
|
||||
result.maps = maps;
|
||||
// The difference map beside the anomalous one below: neither reads the other.
|
||||
std::future<void> difference_map = std::async(std::launch::async, [&diff, &result] {
|
||||
std::future<void> difference_map = std::async(std::launch::async, [&diff, &fwt_data, &result] {
|
||||
const std::array<int, 3> size = gemmi::get_size_for_hkl(diff, {{0, 0, 0}}, 3.0);
|
||||
result.difference_map = std::make_shared<gemmi::Grid<float>>(MapFromFPhi(gemmi::get_f_phi_on_grid<float>(diff, size, true)));
|
||||
result.difference_map_rms = gemmi::calculate_data_statistics(result.difference_map->data).rms;
|
||||
result.map = std::make_shared<gemmi::Grid<float>>(MapFromFPhi(gemmi::get_f_phi_on_grid<float>(fwt_data, size, true)));
|
||||
result.map_rms = gemmi::calculate_data_statistics(result.map->data).rms;
|
||||
});
|
||||
|
||||
// The anomalous difference map: F(+) - F(-) on the model's phase turned back by 90 degrees, read on the +
|
||||
@@ -1301,6 +1304,13 @@ RefinementResult RefineModel(const gemmi::Structure &placed, const std::vector<M
|
||||
result.blobs = FindDensityBlobs(*result.difference_map, result.difference_map_rms, ctx.st, result.d_min,
|
||||
result.anomalous_map.get(), result.anomalous_map_rms);
|
||||
}
|
||||
{
|
||||
const RefinementStage stage("solvent");
|
||||
const SolventMaps maps{result.map.get(), result.map_rms, result.difference_map.get(),
|
||||
result.difference_map_rms, result.anomalous_map.get(), result.anomalous_map_rms};
|
||||
result.solvent = AnalyseSolventSites(ctx.st, result.d_min, maps);
|
||||
result.map.reset();
|
||||
}
|
||||
result.k_sol = ctx.k_sol;
|
||||
result.b_sol = ctx.b_sol;
|
||||
result.model = std::make_shared<gemmi::Structure>(ctx.st);
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
#include "DensityBlobs.h"
|
||||
#include "ModelValidation.h" // ModelMaps
|
||||
#include "RefinementRestraints.h" // GeometryStatistics
|
||||
#include "SolventSites.h"
|
||||
|
||||
class Logger;
|
||||
|
||||
@@ -143,6 +144,10 @@ struct RefinementResult {
|
||||
|
||||
std::shared_ptr<gemmi::Structure> model; // the refined model, in the data's cell and frame
|
||||
std::shared_ptr<ModelMaps> maps; // its 2mFo-DFc and mFo-DFc coefficients (as _maps.mtz holds them)
|
||||
// The 2mFo-DFc map of the refined model and its rms, for reading the solvent sites in (SolventSites.h); released
|
||||
// once they are read.
|
||||
std::shared_ptr<gemmi::Grid<float>> map;
|
||||
double map_rms = 0;
|
||||
// The mFo-DFc map of the refined model and its rms, for finding what it leaves unexplained (DensityBlobs.h).
|
||||
std::shared_ptr<gemmi::Grid<float>> difference_map;
|
||||
double difference_map_rms = 0;
|
||||
@@ -151,6 +156,9 @@ struct RefinementResult {
|
||||
double anomalous_map_rms = 0;
|
||||
// What the refined model leaves unexplained in the difference map (FindDensityBlobs).
|
||||
std::vector<DensityBlob> blobs;
|
||||
// The refined model's waters and ions and the species their surroundings support (AnalyseSolventSites): an
|
||||
// audit, nothing in the model is changed by it.
|
||||
std::vector<SolventSite> solvent;
|
||||
};
|
||||
|
||||
// `placed`: the model in the data's cell and space group. `merged`: the merge, with French-Wilson amplitudes
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
#include <cmath>
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
#include <map>
|
||||
#include <sstream>
|
||||
|
||||
#include <spdlog/fmt/fmt.h>
|
||||
@@ -455,6 +456,52 @@ namespace {
|
||||
return h;
|
||||
}
|
||||
|
||||
// The solvent audit of the refined model (SolventSites.h): REFINE_SOLVENT_ keys. Counts by label as modelled
|
||||
// and by the species the surroundings support best; a site disagrees where the best is another species, not
|
||||
// ambiguously.
|
||||
void AddSolventKeys(ReportSection &s, const std::vector<SolventSite> &sites, const std::string &output_prefix) {
|
||||
std::map<std::string, int> labels, best;
|
||||
constexpr int LISTED_IONS = 40; // the rest are in the JSON
|
||||
int disagree = 0, ambiguous = 0, ions = 0, ions_disagree = 0;
|
||||
std::string ion_rows;
|
||||
for (const SolventSite &site : sites) {
|
||||
labels[site.label]++;
|
||||
best[SolventSpeciesName(site.best)]++;
|
||||
ambiguous += site.ambiguous;
|
||||
const bool differs = SolventLabelScored(site) && !site.ambiguous && !SolventLabelAgrees(site);
|
||||
disagree += differs;
|
||||
if (!SolventLabelIsWater(site)) {
|
||||
ions++;
|
||||
ions_disagree += differs;
|
||||
if (ions <= LISTED_IONS)
|
||||
ion_rows += fmt::format("\n {:<4} {}{:<6} -> {:<3}{} bands {}/{}/{}, {}", site.label, site.chain,
|
||||
site.seqid, SolventSpeciesName(site.best),
|
||||
site.ambiguous ? " (ambiguous)" : "", site.n_19_23, site.n_23_26,
|
||||
site.n_26_32, site.neighbours.empty() ? "no N/O within 3.4 A" : site.neighbours);
|
||||
}
|
||||
}
|
||||
if (ions > LISTED_IONS)
|
||||
ion_rows += fmt::format("\n ... and {} more", ions - LISTED_IONS);
|
||||
auto counts = [](const std::map<std::string, int> &m) {
|
||||
std::string t;
|
||||
for (const auto &[name, n] : m)
|
||||
t += fmt::format("{}{} {}", t.empty() ? "" : " ", name, n);
|
||||
return t;
|
||||
};
|
||||
Add(s, KeyInt("REFINE_SOLVENT_SITES", static_cast<int64_t>(sites.size())));
|
||||
Add(s, KeyText("REFINE_SOLVENT_LABELS", counts(labels)));
|
||||
Add(s, KeyText("REFINE_SOLVENT_BEST", counts(best)));
|
||||
Add(s, KeyInt("REFINE_SOLVENT_DISAGREE", disagree));
|
||||
Add(s, KeyInt("REFINE_SOLVENT_AMBIGUOUS", ambiguous));
|
||||
Add(s, KeyInt("REFINE_SOLVENT_IONS", ions));
|
||||
Add(s, KeyInt("REFINE_SOLVENT_IONS_DISAGREE", ions_disagree));
|
||||
Add(s, Prose(fmt::format(
|
||||
" The waters and ions of the refined model, each scored by its surroundings (an audit: the\n"
|
||||
" model is not changed). Per site in {}_solvent.json; the ions, label -> best-supported\n"
|
||||
" species, N/O in the bands 1.9-2.3/2.3-2.6/2.6-3.2 A, and their neighbours:{}",
|
||||
output_prefix, ion_rows.empty() ? std::string(" none") : ion_rows)));
|
||||
}
|
||||
|
||||
// The refinement's keys (Refinement.h): REFINE_ of the refinement of a placed model and of --mode refine, with every term of the
|
||||
// score of the model as given under REFINE_SCORE_.
|
||||
void AddRefinementKeys(ReportSection &s, const RefinementResult &rf, const std::string &output_prefix) {
|
||||
@@ -518,6 +565,7 @@ namespace {
|
||||
" R_FREE above remain the model as given. The refinement decided nothing about the\n"
|
||||
" processing. The strongest unexplained blobs of its difference map ({}_blobs.json):{}",
|
||||
output_prefix, output_prefix, top.empty() ? std::string(" none") : top)));
|
||||
AddSolventKeys(s, rf.solvent, output_prefix);
|
||||
}
|
||||
if (rf.ok) {
|
||||
const RefinementScore &sc = rf.score_start;
|
||||
|
||||
@@ -418,6 +418,7 @@ struct RugnuxOutputs {
|
||||
std::optional<PlacedModelOutput> refined_model; // <prefix>_refined_model.cif/.pdb (ProcessConfig::refine)
|
||||
std::shared_ptr<ModelMaps> refined_maps; // <prefix>_refined_maps.mtz
|
||||
std::optional<std::vector<DensityBlob>> blobs; // <prefix>_blobs.json
|
||||
std::optional<std::vector<SolventSite>> solvent; // <prefix>_solvent.json
|
||||
std::optional<MtzTable> unmerged; // <prefix>_unmerged.mtz
|
||||
std::optional<MtzTable> unmerged_partials; // <prefix>_unmerged_partials.mtz
|
||||
std::string detector_jpeg; // <prefix>_detector.jpg; empty: none drawn
|
||||
|
||||
@@ -3,9 +3,12 @@
|
||||
|
||||
#include "RugnuxOutputFiles.h"
|
||||
|
||||
#include <cmath>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
#include "../writer/FileWriter.h"
|
||||
#include "../writer/WriteModel.h"
|
||||
#include "../writer/WriteRugnuxFiles.h"
|
||||
@@ -23,6 +26,46 @@ void WriteDensityBlobs(const std::vector<DensityBlob> &blobs, const std::string
|
||||
logger.Info("Difference-map blobs written to {}", path);
|
||||
}
|
||||
|
||||
// The solvent audit of the refined model (SolventSites.h), one entry per site in model order.
|
||||
void WriteSolventSites(const std::vector<SolventSite> &sites, const std::string &path, Logger &logger) {
|
||||
std::ofstream f(path, std::ofstream::out | std::ofstream::trunc);
|
||||
if (!f) {
|
||||
logger.Warning("Cannot open {} for writing", path);
|
||||
return;
|
||||
}
|
||||
// NaN and infinity (no such map, no such neighbour) as null
|
||||
auto num = [](double v) { return std::isfinite(v) ? nlohmann::json(v) : nlohmann::json(); };
|
||||
const char *cations[BVS_CATIONS] = {"MG", "NA", "K", "CA"};
|
||||
nlohmann::json list = nlohmann::json::array();
|
||||
for (const SolventSite &s : sites) {
|
||||
nlohmann::json bvs, nvecsum, score;
|
||||
for (int c = 0; c < BVS_CATIONS; c++) {
|
||||
bvs[cations[c]] = s.bvs[c];
|
||||
nvecsum[cations[c]] = s.nvecsum[c];
|
||||
}
|
||||
for (int k = 0; k < SOLVENT_SPECIES; k++)
|
||||
score[SolventSpeciesName(static_cast<SolventSpecies>(k))] = s.score[k];
|
||||
list.push_back({{"label", s.label}, {"chain", s.chain}, {"seqid", s.seqid},
|
||||
{"altloc", s.altloc ? std::string(1, s.altloc) : std::string()},
|
||||
{"xyz", {s.pos.x, s.pos.y, s.pos.z}}, {"occupancy", s.occupancy}, {"b_iso", s.b_iso},
|
||||
{"sigma_xyz_A", s.sigma_xyz},
|
||||
{"bands", {{"below_1.9", s.n_below_19}, {"1.9-2.3", s.n_19_23}, {"2.3-2.6", s.n_23_26},
|
||||
{"2.6-3.2", s.n_26_32}, {"3.2-3.4", s.n_32_34}, {"3.4-3.6", s.n_34_36}}},
|
||||
{"donors", s.donors}, {"acceptors", s.acceptors}, {"donor_acceptors", s.donor_acceptors},
|
||||
{"min_ligand_distance_A", num(s.min_ligand_distance)},
|
||||
{"nearest_positive_A", num(s.nearest_positive)}, {"nearest_negative_A", num(s.nearest_negative)},
|
||||
{"positive_within_3.5", s.positive_contacts}, {"negative_within_3.2", s.negative_contacts},
|
||||
{"nearest_ion_A", num(s.nearest_ion)}, {"ions_within_4.2", s.ions_within_42},
|
||||
{"ions_within_6", s.ions_within_6}, {"bvs", bvs}, {"nvecsum", nvecsum},
|
||||
{"map_sigma", num(s.map_sigma)}, {"difference_sigma", num(s.difference_sigma)},
|
||||
{"anomalous_sigma", num(s.anomalous_sigma)}, {"score", score},
|
||||
{"best", SolventSpeciesName(s.best)}, {"ambiguous", s.ambiguous},
|
||||
{"agrees", SolventLabelAgrees(s)}, {"neighbours", s.neighbours}});
|
||||
}
|
||||
f << nlohmann::json{{"sites", list}}.dump(1) << "\n";
|
||||
logger.Info("Solvent sites written to {}", path);
|
||||
}
|
||||
|
||||
// The files of one output pass at `prefix`, all but its _process.h5.
|
||||
void WriteOutputs(const RugnuxOutputs &o, const std::string &prefix, const OutputProvenance &provenance,
|
||||
size_t nthreads, Logger &logger) {
|
||||
@@ -48,6 +91,8 @@ void WriteOutputs(const RugnuxOutputs &o, const std::string &prefix, const Outpu
|
||||
WriteModelMaps(*o.refined_maps, prefix + "_refined", logger);
|
||||
if (o.blobs)
|
||||
WriteDensityBlobs(*o.blobs, prefix + "_blobs.json", logger);
|
||||
if (o.solvent)
|
||||
WriteSolventSites(*o.solvent, prefix + "_solvent.json", logger);
|
||||
if (o.unmerged) {
|
||||
WriteMtz(*o.unmerged, prefix + "_unmerged.mtz");
|
||||
logger.Info("Unmerged observations written to {}_unmerged.mtz", prefix);
|
||||
@@ -194,6 +239,7 @@ ReportDocument WriteRefineOutputs(const RefineOnlyResult &result, const std::str
|
||||
WritePlacedModel(*rf.model, cell, *sg, prefix + "_refined", logger);
|
||||
WriteModelMaps(*rf.maps, prefix + "_refined", logger);
|
||||
WriteDensityBlobs(rf.blobs, prefix + "_blobs.json", logger);
|
||||
WriteSolventSites(rf.solvent, prefix + "_solvent.json", logger);
|
||||
}
|
||||
try {
|
||||
std::ofstream file(prefix + "_report.txt");
|
||||
|
||||
@@ -5114,6 +5114,7 @@ bool Rugnux::ScaleMergeAndSymmetry(PipelineLocals &p) {
|
||||
result.outputs->refined_model = PlacedModelOutput{refined.model, p.cell, p.space_group};
|
||||
result.outputs->refined_maps = refined.maps;
|
||||
result.outputs->blobs = refined.blobs;
|
||||
result.outputs->solvent = refined.solvent;
|
||||
Note(fmt::format("refined{}: R-work {:.3f}, R-free {:.3f}, rmsd bonds {:.3f} A, {} blobs",
|
||||
refined.cancelled ? " (cut short)" : "", refined.r_work, refined.r_free,
|
||||
refined.geometry.rmsd_bonds, refined.blobs.size()));
|
||||
|
||||
Reference in new issue
Block a user