SolventSites: score every water and ion of a model as water, Mg, Na, K, Ca or Cl by its surroundings

The first part of the solvent project (P1): an audit only, no model change. For every water and
single-atom ion it measures the N and O neighbours (symmetry mates included) in the Cat_Wiz bands
1.9-2.3 / 2.3-2.6 / 2.6-3.2 A and the clash shells to 3.6 A, types each neighbour as donor, acceptor
or both from the dictionary's hb_type and as positive or negative from its residue's chemistry (Lys,
Arg, N-terminus; Asp, Glu, C-terminus, nucleotide phosphate O; His and Asn/Gln amide ends marked
flip-ambiguous), takes the bond-valence sum and normalised vector sum for Mg, Na, K and Ca (Brese &
O'Keeffe R0, b = 0.37), the nearest like and unlike charges, the ion pairs within 6 A, and the
2mFo-DFc, mFo-DFc and anomalous maps at the site.

Each species then gets a plain sum of simple terms: the first shell against its coordination number,
the neighbours' distances against its own, contacts it does not make (too close, Mg's 2.3-3.4 A
exclusion zone, donors at a cation, acceptors at a halide, like charges), the bond valence against
the charge, and where the maps are there the residual difference density and the anomalous peak.
Every neighbour's coordinate error (from B and d_min) widens its bands and its distance term, so
poor coordinates make the species' shells overlap instead of separating them wrongly. A site bound to
another cation leaves out that cation's other ligands, which are no hydrogen bonds of its own (the
cis waters of a Mg(H2O)6). The best of water, Mg, Na, K, Ca and Cl is reported, AMBIGUOUS within 2;
NH4+ is scored but only reported.

Unit tests on synthetic geometry: an octahedral Mg(H2O)6, a K+ in eight carbonyl O at 2.8 A, a Cl-
with four N-H donors at 3.2 A, a tetrahedral water, the typing, the symmetry-mate and lattice
neighbours of AtomsNear, and the bands blurring with coordinate error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-27 13:19:49 +01:00
1 parent ef9c48a641
commit ca660416a5
5 files changed
+800

No files matched your search

@@ -23,6 +23,7 @@ ADD_LIBRARY(JFJochStructureRefinement STATIC
RefinementXray.cpp RefinementXray.h
Refinement.cpp Refinement.h
DensityBlobs.cpp DensityBlobs.h
SolventSites.cpp SolventSites.h
$<$<BOOL:${JFJOCH_CUDA_AVAILABLE}>:RefinementGPU.cpp>
$<$<BOOL:${JFJOCH_CUDA_AVAILABLE}>:RefinementGPU.cu>
RefinementGPU.h RefinementGPUEngine.h
@@ -0,0 +1,465 @@
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include "SolventSites.h"
#include <algorithm>
#include <cmath>
#include <spdlog/fmt/fmt.h>
#include "gemmi/neighbor.hpp"
#include "gemmi/resinfo.hpp"
#include "MonomerLibrary.h"
#include "RefinementRestraints.h" // AtomsNear
namespace {
constexpr double PI = 3.14159265358979;
constexpr double SEARCH_RADIUS = 6.0; // A: ion pairs are listed out to it (Cat_Wiz)
constexpr double SHELL_RADIUS = 3.4; // A: the N and O neighbours listed and typed
// A: the ligands bond valences are summed over, following Echols et al. (2014) Acta Cryst. D70, 1104-1114
constexpr double BVS_RADIUS = 3.5;
constexpr double LIKE_CHARGE_DISTANCE = 3.5; // A: like charges closer than this repel
// A: no negative group this close to a halide, following Morshed et al. (2015) Acta Cryst. D71, 1147-1158
constexpr double HALIDE_NEGATIVE_DISTANCE = 3.2;
constexpr double BV_B = 0.37; // A, the bond-valence softness
// A band's edges are soft by this much (A); a neighbour's coordinate error widens the band beyond it.
constexpr double BAND_EDGE = 0.05;
// A contact the species does not make (closer than its shell, in its exclusion zone, a donor at a cation or an
// acceptor at a halide, a like charge), in the score's log units, per contact.
constexpr double BAD_CONTACT = 2.0;
// A first-shell ligand fewer than a Mg, Na or Ca holds: a water the model may simply not have, so it costs little.
// Incomplete shells are common; they are no evidence for the ion, and not much against it.
constexpr double MISSING_LIGAND = 0.5;
// R0 (A) of the bond-valence parameters for Mg, Na, K, Ca to O and to N, b = 0.37 A, as tabulated by Zheng et al.
// (2017) Acta Cryst. D73, 316-325, Table 2.
// Following Brese & O'Keeffe (1991) Acta Cryst. B47, 192-197
constexpr double R0_O[BVS_CATIONS] = {1.69, 1.80, 2.13, 1.96};
constexpr double R0_N[BVS_CATIONS] = {1.85, 1.93, 2.26, 2.14};
// What a species' shell looks like: its first shell of N and O [lo, hi), how many it holds and at what distance
// (to O; to N where that differs), the N and O it cannot have nearer than `too_close`, an exclusion zone
// [exclude_lo, exclude_hi) it keeps empty (Mg only), its charge and electrons, and its row in SolventSite::bvs (-1
// none). Where the papers give no spread of the distance, its esd is a quarter of the band's width.
// Bands following Cat_Wiz, Naleem et al. (2026) Nucleic Acids Res. 54, gkag120; and Leonarski et al. (2017)
// Nucleic Acids Res. 45, 987-1004, (2019) RNA 25, 173-192, (2025) Nucleic Acids Res. 53, gkae1148
struct SpeciesRule {
double lo, hi;
double ideal, esd, ideal_n, esd_n;
double too_close;
double exclude_lo, exclude_hi;
double cn, cn_sigma;
int charge;
int electrons;
int bvs;
};
const SpeciesRule RULES[SOLVENT_SPECIES] = {
// water: 2.6-3.2 A, ~2.8 A, at most four (two donated, two accepted), at least one
{2.6, 3.2, 2.80, 0.15, 2.80, 0.15, 2.6, 0, 0, 4, 1.0, 0, 10, -1},
// Mg2+: six at 1.9-2.3 A, Mg-O 2.07 +- 0.04 A, Mg-N 2.19 +- 0.08 A, no N or O at 2.3-3.4 A
{1.9, 2.3, 2.07, 0.04, 2.19, 0.08, 1.9, 2.3, 3.4, 6, 0.7, 2, 10, 0},
// Na+: five or six at 2.3-2.6 A, ~2.41 A
{2.3, 2.6, 2.41, 0.08, 2.41, 0.08, 2.3, 0, 0, 5.5, 1.0, 1, 10, 1},
// K+: six to ten at 2.6-3.2 A, ~2.8 A
{2.6, 3.2, 2.80, 0.15, 2.80, 0.15, 2.6, 0, 0, 8, 1.5, 1, 18, 2},
// Ca2+: six to eight at 2.34-2.40 A
{2.3, 2.6, 2.37, 0.08, 2.37, 0.08, 2.3, 0, 0, 7, 1.0, 2, 18, 3},
// Cl-: three to five hydrogen-bond donors at 3.0-3.4 A, ~3.2 A, almost nothing nearer than 2.9 A
// Following Carugo (2014) BMC Struct. Biol. 14, 19
{3.0, 3.4, 3.20, 0.10, 3.20, 0.10, 2.9, 0, 0, 4, 1.0, -1, 18, -1},
// NH4+: four acceptors, at water's distances
{2.6, 3.2, 2.90, 0.15, 2.90, 0.15, 2.6, 0, 0, 4, 1.0, 1, 10, -1},
};
bool IsWater(const gemmi::Residue &r) {
return r.is_water();
}
bool IsHalogen(gemmi::El el) {
return el == gemmi::El::F || el == gemmi::El::Cl || el == gemmi::El::Br || el == gemmi::El::I;
}
bool Polar(gemmi::El el) {
return el == gemmi::El::N || el == gemmi::El::O;
}
// The species a residue name labels, -1 for one not scored here.
int LabelSpecies(const std::string &name) {
if (name == "HOH" || name == "WAT" || name == "DOD" || name == "H2O")
return static_cast<int>(SolventSpecies::Water);
for (int s = 1; s < SOLVENT_SPECIES; s++)
if (name == SolventSpeciesName(static_cast<SolventSpecies>(s)))
return s;
return -1;
}
// The one heavy atom of a single-atom ion, or of NH4; null for any other residue.
const gemmi::Atom *IonAtom(const gemmi::Residue &r) {
const gemmi::Atom *heavy = nullptr;
for (const gemmi::Atom &a : r.atoms)
if (!a.is_hydrogen()) {
if (heavy != nullptr && heavy->altloc == a.altloc)
return nullptr;
heavy = &a;
}
if (heavy == nullptr)
return nullptr;
if (r.name == "NH4" || heavy->element.is_metal() || IsHalogen(heavy->element.elem))
return heavy;
return nullptr;
}
// +1 a cation, -1 an anion, 0 a water or not a solvent site.
int IonCharge(const gemmi::Residue &r) {
if (IsWater(r))
return 0;
const gemmi::Atom *a = IonAtom(r);
if (a == nullptr)
return 0;
return IsHalogen(a->element.elem) ? -1 : 1;
}
// Probability that a distance measured as d with error sigma lies in [lo, hi).
double InBand(double d, double sigma, double lo, double hi) {
const double s = std::sqrt(2.0) * sigma;
return 0.5 * (std::erfc((lo - d) / s) - std::erfc((hi - d) / s));
}
// The coordinate error assumed for an atom: its rms displacement sqrt(B / 8 pi^2) times d_min / 10 A, a scale
// that gives ~0.1 A at B 20 A^2 and 2 A, typical of refined structures.
double CoordinateError(double b_iso, double d_min) {
return std::sqrt(std::max(b_iso, 0.0) / (8 * PI * PI)) * d_min / 10.0;
}
struct Ligand {
gemmi::Position pos;
double distance;
double sigma; // the distance's error from the two atoms' coordinate errors
double valence[BVS_CATIONS]; // its bond valence to each cation, occupancy weighted
AtomChemistry chem;
bool nitrogen;
};
// The score of species `s` at `site`, with the N and O neighbours `ligands`. A plain sum of terms in log units:
// the first shell's count against the species' coordination number and its distances against the species' own,
// contacts the species does not make, the shell's chemistry, the bond-valence sum against the charge with its
// vector sum, and - where the maps are there - the electrons the difference map says the label lacks and the
// anomalous signal. A neighbour's coordinate error widens the bands it is counted in and the distance it is
// compared with, so that where the coordinates are poor the species' shells overlap and the score cannot tell
// them apart, rather than telling them apart wrongly. `label`: the label's species, -1 for one not scored here.
double ScoreSpecies(int s, const SolventSite &site, const std::vector<Ligand> &ligands, int label) {
const SpeciesRule &rule = RULES[s];
const auto species = static_cast<SolventSpecies>(s);
double n = 0, distance = 0, too_close = 0, excluded = 0;
double donors_in = 0, acceptors_in = 0; // donor-only and acceptor-only in the first shell
double hbond_donors = 0, hbond_acceptors = 0; // the same at 2.6-3.4 A, a water's hydrogen bonds
bool donor_nitrogen = false;
for (const Ligand &l : ligands) {
const double e = l.sigma;
const double p = InBand(l.distance, BAND_EDGE, rule.lo - e, rule.hi + e);
n += p;
const double ideal = l.nitrogen ? rule.ideal_n : rule.ideal;
const double esd2 = std::pow(l.nitrogen ? rule.esd_n : rule.esd, 2) + e * e;
distance -= p * (l.distance - ideal) * (l.distance - ideal) / (2 * esd2);
too_close += InBand(l.distance, BAND_EDGE, 0.0, rule.too_close - e);
if (rule.exclude_hi > 0)
excluded += InBand(l.distance, BAND_EDGE, rule.exclude_lo + e, rule.exclude_hi);
const bool donor_only = l.chem.donor && !l.chem.acceptor;
const bool acceptor_only = l.chem.acceptor && !l.chem.donor;
donors_in += donor_only ? p : 0;
acceptors_in += acceptor_only ? p : 0;
const double hb = InBand(l.distance, BAND_EDGE, 2.6 - e, SHELL_RADIUS + e);
hbond_donors += donor_only ? hb : 0;
hbond_acceptors += acceptor_only ? hb : 0;
donor_nitrogen = donor_nitrogen || (l.nitrogen && l.chem.donor && l.distance < LIKE_CHARGE_DISTANCE);
}
double score = 0;
// the first shell
if (species == SolventSpecies::Water) {
if (n > rule.cn)
score -= (n - rule.cn) * (n - rule.cn) / (2 * rule.cn_sigma * rule.cn_sigma);
if (n < 1)
score -= BAD_CONTACT * (1 - n);
} else if (n < rule.cn && (species == SolventSpecies::Mg || species == SolventSpecies::Na ||
species == SolventSpecies::Ca))
score -= MISSING_LIGAND * (rule.cn - n);
else
score -= (n - rule.cn) * (n - rule.cn) / (2 * rule.cn_sigma * rule.cn_sigma);
score += distance;
score -= BAD_CONTACT * (too_close + excluded);
// the chemistry
if (rule.charge > 0) {
score -= BAD_CONTACT * donors_in; // an N-H does not point at a cation
score -= BAD_CONTACT * site.positive_contacts;
} else if (rule.charge < 0) {
score -= BAD_CONTACT * acceptors_in; // a halide is surrounded by donors
score -= BAD_CONTACT * site.negative_contacts;
// at least one positive group or one donor N, following Morshed et al. (2015) Acta Cryst. D71, 1147-1158
if (!(site.nearest_positive < LIKE_CHARGE_DISTANCE) && !donor_nitrogen)
score -= BAD_CONTACT;
} else {
// a water donates two hydrogen bonds and accepts two
score -= BAD_CONTACT * (std::max(0.0, hbond_donors - 2) + std::max(0.0, hbond_acceptors - 2));
}
// the bond valence, its expected spread widened by the ligands' distance errors
if (rule.bvs >= 0) {
double var = 0.25 * rule.charge * 0.25 * rule.charge;
for (const Ligand &l : ligands)
var += std::pow(l.valence[rule.bvs] * l.sigma / BV_B, 2);
const double v = site.bvs[rule.bvs] - rule.charge;
score -= v * v / (2 * var);
// nVECSUM an outlier above 0.23, following CheckMyMetal, Zheng et al. (2017) Acta Cryst. D73, 223-233
score -= 0.5 * std::pow(site.nvecsum[rule.bvs] / 0.23, 2);
}
// the electrons: what the difference map says the label lacks (positive) or has too many of (negative)
if (label >= 0 && std::isfinite(site.difference_sigma)) {
const int delta = rule.electrons - RULES[label].electrons;
if (site.difference_sigma >= 3)
score += delta > 0 ? 1 : -1;
else if (site.difference_sigma <= -3)
score += delta < 0 ? 1 : -1;
else if (delta != 0)
score -= 0.5; // nothing missing, nothing extra - though B and occupancy can hide 8 electrons
}
// the anomalous signal: K, Ca and Cl scatter anomalously, water, Mg and Na next to nothing
if (std::isfinite(site.anomalous_sigma) && site.anomalous_sigma >= 4)
score += rule.electrons == 18 ? 2 : -2;
return score;
}
std::string AtomName(const gemmi::Residue &r, const gemmi::Atom &a, const std::string &chain) {
return a.name + " " + r.name + " " + chain + r.seqid.str();
}
} // namespace
const char *SolventSpeciesName(SolventSpecies s) {
switch (s) {
case SolventSpecies::Water: return "HOH";
case SolventSpecies::Mg: return "MG";
case SolventSpecies::Na: return "NA";
case SolventSpecies::K: return "K";
case SolventSpecies::Ca: return "CA";
case SolventSpecies::Cl: return "CL";
case SolventSpecies::NH4: break;
}
return "NH4";
}
AtomChemistry ChemistryOf(const gemmi::MonLib &monlib, const gemmi::Residue &residue, const gemmi::Atom &atom,
bool n_terminus) {
AtomChemistry c;
if (!Polar(atom.element.elem))
return c;
if (IsWater(residue)) {
c.donor = c.acceptor = true;
return c;
}
const std::string &r = residue.name, &n = atom.name;
char hb = 'B'; // an N or O the dictionary does not describe: either
auto cc = monlib.monomers.find(r);
if (cc != monlib.monomers.end()) {
auto it = cc->second.find_atom(n);
if (it != cc->second.atoms.end()) {
auto e = monlib.ener_lib.atoms.find(it->chem_type);
if (e != monlib.ener_lib.atoms.end())
hb = e->second.hb_type;
}
}
c.donor = hb == 'D' || hb == 'B';
c.acceptor = hb == 'A' || hb == 'B';
c.positive = (r == "LYS" && n == "NZ") || (r == "ARG" && (n == "NE" || n == "NH1" || n == "NH2")) ||
(n_terminus && n == "N");
c.negative = (r == "ASP" && (n == "OD1" || n == "OD2")) || (r == "GLU" && (n == "OE1" || n == "OE2")) ||
n == "OXT" || (n == "O" && residue.find_atom("OXT", '*') != nullptr) ||
n == "OP1" || n == "OP2" || n == "OP3";
c.flip_ambiguous = (r == "HIS" && (n == "ND1" || n == "NE2")) || (r == "ASN" && (n == "OD1" || n == "ND2")) ||
(r == "GLN" && (n == "OE1" || n == "NE2"));
if (c.flip_ambiguous)
c.donor = c.acceptor = true;
return c;
}
bool SolventLabelIsWater(const SolventSite &s) {
return LabelSpecies(s.label) == static_cast<int>(SolventSpecies::Water);
}
bool SolventLabelScored(const SolventSite &s) {
const int label = LabelSpecies(s.label);
return label >= 0 && label != static_cast<int>(SolventSpecies::NH4);
}
bool SolventLabelAgrees(const SolventSite &s) {
return LabelSpecies(s.label) == static_cast<int>(s.best);
}
std::vector<SolventSite> AnalyseSolventSites(const gemmi::Structure &st, double d_min, const SolventMaps &maps) {
std::vector<SolventSite> out;
if (st.models.empty())
return out;
gemmi::Model model = st.models[0]; // NeighborSearch takes it non-const
gemmi::UnitCell cell = st.cell;
cell.set_cell_images_from_spacegroup(st.find_spacegroup());
gemmi::NeighborSearch ns(model, cell, SEARCH_RADIUS);
ns.populate(false);
const gemmi::MonLib &monlib = EmbeddedMonomerLibrary();
// the first residue of each polypeptide, whose N is charged
std::vector<const gemmi::Residue *> n_termini;
for (const gemmi::Chain &ch : model.chains) {
const gemmi::ConstResidueSpan polymer = ch.get_polymer();
if (!polymer.empty() && gemmi::find_tabulated_residue(polymer.front().name).is_amino_acid())
n_termini.push_back(&polymer.front());
}
auto is_n_terminus = [&](const gemmi::Residue &r) {
return std::find(n_termini.begin(), n_termini.end(), &r) != n_termini.end();
};
auto map_sigma = [](const gemmi::Grid<float> *g, double rms, const gemmi::Position &p) {
return g != nullptr && rms > 0 ? g->interpolate_value(p) / rms : NAN;
};
for (size_t ci = 0; ci < model.chains.size(); ci++)
for (size_t ri = 0; ri < model.chains[ci].residues.size(); ri++) {
const gemmi::Residue &res = model.chains[ci].residues[ri];
if (!IsWater(res) && IonAtom(res) == nullptr)
continue;
const int label = LabelSpecies(res.name);
for (size_t ai = 0; ai < res.atoms.size(); ai++) {
const gemmi::Atom &a = res.atoms[ai];
if (a.is_hydrogen())
continue;
SolventSite site;
site.label = res.name;
site.chain = model.chains[ci].name;
site.seqid = res.seqid.str();
site.altloc = a.altloc;
site.pos = a.pos;
site.occupancy = a.occ;
site.b_iso = a.b_iso;
site.sigma_xyz = CoordinateError(a.b_iso, d_min);
site.min_ligand_distance = NAN;
site.nearest_positive = site.nearest_negative = site.nearest_ion = INFINITY;
site.map_sigma = map_sigma(maps.map, maps.map_rms, a.pos);
site.difference_sigma = map_sigma(maps.difference, maps.difference_rms, a.pos);
site.anomalous_sigma = map_sigma(maps.anomalous, maps.anomalous_rms, a.pos);
std::vector<Ligand> ligands;
std::vector<gemmi::Position> bound_cations; // other cations closer than 2.6 A
std::vector<gemmi::Position> inner; // the N and O closer than 2.6 A
gemmi::Vec3 vec[BVS_CATIONS];
for (const ModelNeighbour &nb : AtomsNear(ns, model, a.pos, SEARCH_RADIUS)) {
// the site itself, its own hydrogens and its other conformers; its symmetry mates stay
if (nb.chain == static_cast<int>(ci) && nb.residue == static_cast<int>(ri) &&
(nb.atom != static_cast<int>(ai) || nb.distance < 0.5))
continue;
const gemmi::Chain &pch = model.chains[nb.chain];
const gemmi::Residue &pr = pch.residues[nb.residue];
const gemmi::Atom &pa = pr.atoms[nb.atom];
if (pa.is_hydrogen() || (a.altloc && pa.altloc && a.altloc != pa.altloc))
continue;
const double d = nb.distance;
const int charge = IonCharge(pr);
if (charge != 0) {
site.ions_within_6++;
site.ions_within_42 += d < 4.2;
site.nearest_ion = std::min(site.nearest_ion, d);
if (charge > 0) {
site.nearest_positive = std::min(site.nearest_positive, d);
site.positive_contacts += d < LIKE_CHARGE_DISTANCE;
if (d < 2.6)
bound_cations.push_back(nb.pos);
} else {
site.nearest_negative = std::min(site.nearest_negative, d);
site.negative_contacts += d < HALIDE_NEGATIVE_DISTANCE;
}
continue;
}
const AtomChemistry chem = ChemistryOf(monlib, pr, pa, is_n_terminus(pr));
if (chem.positive) {
site.nearest_positive = std::min(site.nearest_positive, d);
site.positive_contacts += d < LIKE_CHARGE_DISTANCE;
}
if (chem.negative) {
site.nearest_negative = std::min(site.nearest_negative, d);
site.negative_contacts += d < HALIDE_NEGATIVE_DISTANCE;
}
if (!Polar(pa.element.elem))
continue;
if (d < 1.9) site.n_below_19++;
else if (d < 2.3) site.n_19_23++;
else if (d < 2.6) site.n_23_26++;
else if (d < 3.2) site.n_26_32++;
else if (d < 3.4) site.n_32_34++;
else if (d < 3.6) site.n_34_36++;
if (d < SHELL_RADIUS) {
if (chem.donor && chem.acceptor)
site.donor_acceptors++;
else if (chem.donor)
site.donors++;
else if (chem.acceptor)
site.acceptors++;
site.neighbours += fmt::format("{}{} {:.2f}", site.neighbours.empty() ? "" : ", ",
AtomName(pr, pa, pch.name), d);
}
if (d < 2.6)
inner.push_back(nb.pos);
Ligand l{};
l.pos = nb.pos;
l.distance = d;
const double e = CoordinateError(pa.b_iso, d_min);
l.sigma = std::sqrt(site.sigma_xyz * site.sigma_xyz + e * e);
l.chem = chem;
l.nitrogen = pa.element == gemmi::El::N;
if (d < BVS_RADIUS)
for (int c = 0; c < BVS_CATIONS; c++) {
// Following Brown & Altermatt (1985) Acta Cryst. B41, 244-247, occupancy weighted as
// Muller et al. (2003) Acta Cryst. D59, 32-37
l.valence[c] = pa.occ * std::exp(((l.nitrogen ? R0_N[c] : R0_O[c]) - d) / BV_B);
site.bvs[c] += l.valence[c];
vec[c] += (nb.pos - a.pos) * (l.valence[c] / d);
}
ligands.push_back(l);
}
// A site bound to a cation shares that cation's other ligands as neighbours: the cis waters of a
// Mg(H2O)6 are 2.9 A from each other, and are no hydrogen bonds of theirs. The scores leave them out.
std::erase_if(ligands, [&](const Ligand &l) {
for (const gemmi::Position &c : bound_cations)
if (l.pos.dist(c) < 2.6)
return true;
return false;
});
for (int c = 0; c < BVS_CATIONS; c++)
site.nvecsum[c] = site.bvs[c] > 0 ? vec[c].length() / site.bvs[c] : 0;
for (size_t i = 0; i < inner.size(); i++)
for (size_t j = i + 1; j < inner.size(); j++)
site.min_ligand_distance = std::isnan(site.min_ligand_distance)
? inner[i].dist(inner[j])
: std::min(site.min_ligand_distance, inner[i].dist(inner[j]));
for (int s = 0; s < SOLVENT_SPECIES; s++)
site.score[s] = ScoreSpecies(s, site, ligands, label);
// NH4+ is reported, not decided: water, Mg, Na, K, Ca, Cl
int best = 0, second = -1;
for (int s = 1; s < static_cast<int>(SolventSpecies::NH4); s++)
if (site.score[s] > site.score[best]) {
second = best;
best = s;
} else if (second < 0 || site.score[s] > site.score[second])
second = s;
site.best = static_cast<SolventSpecies>(best);
site.ambiguous = site.score[best] - site.score[second] < SOLVENT_AMBIGUOUS_MARGIN;
out.push_back(site);
}
}
return out;
}
@@ -0,0 +1,101 @@
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <array>
#include <string>
#include <vector>
#include "gemmi/grid.hpp"
#include "gemmi/model.hpp"
#include "gemmi/monlib.hpp"
// What the solvent of a model - every water and every single-atom ion - is surrounded by, and which species
// that surrounding supports: water, Mg2+, Na+, K+, Ca2+ or Cl- (NH4+ scored too, but only reported). An audit
// of the model as it is: nothing here changes the model.
//
// Each site's neighbours (symmetry mates included) are measured in the coordination bands of Cat_Wiz and typed
// by their chemistry (AtomChemistry); bond-valence sums are taken for each cation; the maps are read at the
// site. Each species then gets a score, a plain sum of terms (SolventSites.cpp says which), and the best one
// is reported, AMBIGUOUS where the runner-up is close.
// What an atom can do in a hydrogen bond or a salt bridge.
struct AtomChemistry {
bool donor = false;
bool acceptor = false;
bool positive = false; // Lys NZ, Arg NE/NH1/NH2, a polypeptide's N-terminal N
bool negative = false; // Asp/Glu carboxylate O, a C-terminal carboxylate, a nucleotide's phosphate O
bool flip_ambiguous = false; // His ring N, Asn/Gln amide ends: which end is which is the model's guess
};
// The chemistry of `atom` of `residue`: donor or acceptor from the dictionary's energy type (hb_type D, A, B),
// charge from the residue's chemistry; a water is both; an N or O of a residue the dictionary does not
// describe is both. `n_terminus`: the residue begins a polypeptide.
AtomChemistry ChemistryOf(const gemmi::MonLib &monlib, const gemmi::Residue &residue, const gemmi::Atom &atom,
bool n_terminus);
enum class SolventSpecies { Water, Mg, Na, K, Ca, Cl, NH4 };
constexpr int SOLVENT_SPECIES = 7;
// The residue name of a species: HOH, MG, NA, K, CA, CL, NH4.
const char *SolventSpeciesName(SolventSpecies s);
// The cations bond valences are summed for, in this order.
constexpr int BVS_CATIONS = 4; // Mg, Na, K, Ca
struct SolventSite {
std::string label; // the residue name as modelled: HOH, MG, NA, ...
std::string chain, seqid;
char altloc = '\0';
gemmi::Position pos;
double occupancy = 1, b_iso = 0;
double sigma_xyz = 0; // the coordinate error assumed for the site, A
// N and O neighbours, by distance: the bands of Cat_Wiz, and the clash shells beyond
int n_below_19 = 0, n_19_23 = 0, n_23_26 = 0, n_26_32 = 0, n_32_34 = 0, n_34_36 = 0;
// the N and O neighbours closer than 3.4 A, by what they can do in a hydrogen bond
int donors = 0, acceptors = 0, donor_acceptors = 0;
// the shortest distance between two N or O neighbours closer than 2.6 A to the site; NaN with fewer than two
double min_ligand_distance = 0;
// the nearest positive and negative group (a charged atom of the model or an ion), within 6 A; INFINITY none
double nearest_positive = 0, nearest_negative = 0;
// the positive groups closer than 3.5 A, the negative ones closer than 3.2 A
int positive_contacts = 0, negative_contacts = 0;
// the other ions of the model (not waters) within 4.2 and within 6 A, and the nearest one (INFINITY none)
int ions_within_42 = 0, ions_within_6 = 0;
double nearest_ion = 0;
// bond-valence sum and normalised vector sum for each of the BVS cations (Mg, Na, K, Ca), from the N and
// O neighbours closer than 3.5 A
std::array<double, BVS_CATIONS> bvs{}, nvecsum{};
// the maps at the site, in their rms; NaN where there is no such map
double map_sigma = 0, difference_sigma = 0, anomalous_sigma = 0;
std::array<double, SOLVENT_SPECIES> score{}; // log-score per species, SolventSpecies order
SolventSpecies best = SolventSpecies::Water; // the best-scoring of water, Mg, Na, K, Ca and Cl
bool ambiguous = false; // the runner-up within SOLVENT_AMBIGUOUS_MARGIN of it
std::string neighbours; // the N and O neighbours closer than 3.4 A, nearest first
};
constexpr double SOLVENT_AMBIGUOUS_MARGIN = 2.0;
// The maps a site is read in: 2mFo-DFc, mFo-DFc and the anomalous difference map, each with its rms; any of
// them may be missing (null).
struct SolventMaps {
const gemmi::Grid<float> *map = nullptr;
double map_rms = 0;
const gemmi::Grid<float> *difference = nullptr;
double difference_rms = 0;
const gemmi::Grid<float> *anomalous = nullptr;
double anomalous_rms = 0;
};
// Every water and single-atom ion of the first model of `st` (in its cell and space group; hydrogens are
// ignored), in model order. `d_min`: the resolution the model was refined at, for the coordinate error
// (0 for a model with no data).
std::vector<SolventSite> AnalyseSolventSites(const gemmi::Structure &st, double d_min, const SolventMaps &maps);
// Whether the site is labelled a water; whether its label is one of the species decided here (water, Mg, Na, K,
// Ca, Cl); whether the best-supported species is its label.
bool SolventLabelIsWater(const SolventSite &s);
bool SolventLabelScored(const SolventSite &s);
bool SolventLabelAgrees(const SolventSite &s);
+1
View File
@@ -186,6 +186,7 @@ ADD_EXECUTABLE(jfjoch_test
ModelDensityGPUTest.cpp
RefinementTest.cpp
RefineOnlyTest.cpp
SolventSitesTest.cpp
SearchSpaceGroupTest.cpp
SearchSpaceGroupTwinTest.cpp
SyntheticMergedReflections.h
+232
View File
@@ -0,0 +1,232 @@
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include <array>
#include <cmath>
#include <string>
#include <vector>
#include "../image_analysis/structure_refinement/MonomerLibrary.h"
#include "../image_analysis/structure_refinement/RefinementRestraints.h"
#include "../image_analysis/structure_refinement/SolventSites.h"
namespace {
// A P1 box with nothing in it, large enough that no lattice translate comes near the middle.
gemmi::Structure empty_box(const char *group = "P 1", double edge = 40.0) {
gemmi::Structure st;
st.cell = gemmi::UnitCell(edge, edge, edge, 90, 90, 90);
st.spacegroup_hm = group;
st.setup_cell_images();
st.models.emplace_back(1);
st.models[0].chains.emplace_back("A");
return st;
}
// One atom as a residue of its own, numbered after the last.
void add_atom(gemmi::Structure &st, const std::string &resname, const std::string &atom, gemmi::El el,
const gemmi::Position &pos) {
gemmi::Chain &ch = st.models[0].chains[0];
gemmi::Residue r;
r.name = resname;
r.seqid = gemmi::SeqId(static_cast<int>(ch.residues.size()) + 1, ' ');
gemmi::Atom a;
a.name = atom;
a.element = gemmi::Element(el);
a.pos = pos;
a.occ = 1.0f;
a.b_iso = 20.0f;
r.atoms.push_back(a);
ch.residues.push_back(r);
}
const gemmi::Position CENTRE(20, 20, 20);
const SolventSite &site_labelled(const std::vector<SolventSite> &sites, const std::string &label) {
for (const SolventSite &s : sites)
if (s.label == label)
return s;
FAIL("no site labelled " << label);
return sites.front();
}
} // namespace
TEST_CASE("SolventSites_AtomsNear_symmetry_mate", "[Solvent]") {
// an atom near the 2-fold axis of P2: its mate (-x, y, -z) is 1.70 A away
gemmi::Structure st = empty_box("P 1 2 1", 30.0);
const gemmi::Position p = st.cell.orthogonalize(gemmi::Fractional(0.02, 0.3, 0.02));
add_atom(st, "HOH", "O", gemmi::El::O, p);
gemmi::UnitCell cell = st.cell;
cell.set_cell_images_from_spacegroup(st.find_spacegroup());
gemmi::NeighborSearch ns(st.models[0], cell, 5.0);
ns.populate(false);
const std::vector<ModelNeighbour> near = AtomsNear(ns, st.models[0], p, 3.0);
REQUIRE(near.size() == 2);
CHECK(near[0].distance == Catch::Approx(0.0).margin(1e-6));
CHECK(near[1].distance == Catch::Approx(std::sqrt(2.0) * 1.2).margin(1e-4));
CHECK(near[1].pos.dist(gemmi::Position(near[1].image.apply(p))) == Catch::Approx(0.0).margin(1e-6));
// and an atom across the face of the cell, a lattice translate away
gemmi::Structure box = empty_box("P 1", 20.0);
add_atom(box, "HOH", "O", gemmi::El::O, gemmi::Position(0.5, 10, 10));
add_atom(box, "HOH", "O", gemmi::El::O, gemmi::Position(19.5, 10, 10));
gemmi::UnitCell bcell = box.cell;
bcell.set_cell_images_from_spacegroup(box.find_spacegroup());
gemmi::NeighborSearch bns(box.models[0], bcell, 5.0);
bns.populate(false);
const std::vector<ModelNeighbour> across = AtomsNear(bns, box.models[0], gemmi::Position(0.5, 10, 10), 3.0);
REQUIRE(across.size() == 2);
CHECK(across[1].residue == 1);
CHECK(across[1].distance == Catch::Approx(1.0).margin(1e-6));
CHECK(across[1].pos.x == Catch::Approx(-0.5).margin(1e-6));
}
TEST_CASE("SolventSites_chemistry", "[Solvent]") {
const gemmi::MonLib &monlib = EmbeddedMonomerLibrary();
auto chem = [&](const std::string &res, const std::string &atom, gemmi::El el) {
gemmi::Residue r;
r.name = res;
gemmi::Atom a;
a.name = atom;
a.element = gemmi::Element(el);
r.atoms.push_back(a);
return ChemistryOf(monlib, r, r.atoms[0], false);
};
const AtomChemistry nz = chem("LYS", "NZ", gemmi::El::N);
CHECK((nz.donor && !nz.acceptor && nz.positive && !nz.negative));
const AtomChemistry nh1 = chem("ARG", "NH1", gemmi::El::N);
CHECK((nh1.donor && nh1.positive));
const AtomChemistry od1 = chem("ASP", "OD1", gemmi::El::O);
CHECK((od1.acceptor && !od1.donor && od1.negative));
const AtomChemistry op1 = chem("G", "OP1", gemmi::El::O);
CHECK((op1.acceptor && op1.negative));
const AtomChemistry n7 = chem("G", "N7", gemmi::El::N);
CHECK((n7.acceptor && !n7.donor && !n7.negative));
const AtomChemistry n3 = chem("U", "N3", gemmi::El::N);
CHECK((n3.donor && !n3.acceptor));
const AtomChemistry o2p = chem("A", "O2'", gemmi::El::O);
CHECK((o2p.donor && o2p.acceptor));
const AtomChemistry carbonyl = chem("GLY", "O", gemmi::El::O);
CHECK((carbonyl.acceptor && !carbonyl.donor && !carbonyl.negative));
const AtomChemistry nd2 = chem("ASN", "ND2", gemmi::El::N);
CHECK((nd2.flip_ambiguous && nd2.donor && nd2.acceptor));
const AtomChemistry ring = chem("HIS", "NE2", gemmi::El::N);
CHECK((ring.flip_ambiguous && ring.donor && ring.acceptor));
const AtomChemistry water = chem("HOH", "O", gemmi::El::O);
CHECK((water.donor && water.acceptor));
CHECK(!chem("GLY", "CA", gemmi::El::C).acceptor);
}
TEST_CASE("SolventSites_octahedral_Mg", "[Solvent]") {
// Mg(H2O)6, Mg-O 2.07 A
gemmi::Structure st = empty_box();
add_atom(st, "MG", "MG", gemmi::El::Mg, CENTRE);
for (int axis = 0; axis < 3; axis++)
for (double sign : {-1.0, 1.0}) {
gemmi::Position p = CENTRE;
(axis == 0 ? p.x : axis == 1 ? p.y : p.z) += sign * 2.07;
add_atom(st, "HOH", "O", gemmi::El::O, p);
}
const std::vector<SolventSite> sites = AnalyseSolventSites(st, 0.0, SolventMaps{});
REQUIRE(sites.size() == 7);
const SolventSite &mg = site_labelled(sites, "MG");
CHECK(mg.n_19_23 == 6);
CHECK(mg.n_23_26 + mg.n_26_32 + mg.n_32_34 == 0);
CHECK(mg.bvs[0] == Catch::Approx(6 * std::exp((1.69 - 2.07) / 0.37)).epsilon(1e-6));
CHECK(mg.nvecsum[0] == Catch::Approx(0.0).margin(1e-9));
CHECK(mg.min_ligand_distance == Catch::Approx(2.07 * std::sqrt(2.0)).epsilon(1e-6));
CHECK(mg.best == SolventSpecies::Mg);
CHECK(!mg.ambiguous);
CHECK(SolventLabelAgrees(mg));
// its waters are waters: each is bound to the Mg, and the other five are its cis and trans neighbours
for (const SolventSite &s : sites)
if (s.label == "HOH") {
CHECK(s.nearest_positive == Catch::Approx(2.07).epsilon(1e-6));
CHECK(s.best == SolventSpecies::Water);
}
}
TEST_CASE("SolventSites_K_eight_carbonyls", "[Solvent]") {
// K+ in a square antiprism of eight carbonyl O at 2.8 A
gemmi::Structure st = empty_box();
add_atom(st, "K", "K", gemmi::El::K, CENTRE);
const double h = 1.3, r = std::sqrt(2.8 * 2.8 - h * h);
for (int k = 0; k < 8; k++) {
const double phi = (k < 4 ? k * 90.0 : k * 90.0 + 45.0) * 3.14159265358979 / 180.0;
add_atom(st, "GLY", "O", gemmi::El::O,
gemmi::Position(CENTRE.x + r * std::cos(phi), CENTRE.y + r * std::sin(phi), CENTRE.z + (k < 4 ? h : -h)));
}
const std::vector<SolventSite> sites = AnalyseSolventSites(st, 0.0, SolventMaps{});
REQUIRE(sites.size() == 1);
const SolventSite &k = sites[0];
CHECK(k.n_26_32 == 8);
CHECK(k.acceptors == 8);
CHECK(k.donors == 0);
CHECK(k.bvs[2] == Catch::Approx(8 * std::exp((2.13 - 2.8) / 0.37)).epsilon(1e-6));
CHECK(k.best == SolventSpecies::K);
CHECK(!k.ambiguous);
CHECK(k.score[static_cast<int>(SolventSpecies::K)] > k.score[static_cast<int>(SolventSpecies::Water)] + 5);
}
TEST_CASE("SolventSites_Cl_four_donors", "[Solvent]") {
// Cl- with four backbone N-H donors at 3.2 A, tetrahedral
gemmi::Structure st = empty_box();
add_atom(st, "CL", "CL", gemmi::El::Cl, CENTRE);
const double t = 3.2 / std::sqrt(3.0);
for (const auto &[x, y, z] : std::vector<std::array<double, 3>>{{1, 1, 1}, {1, -1, -1}, {-1, 1, -1}, {-1, -1, 1}})
add_atom(st, "GLY", "N", gemmi::El::N, gemmi::Position(CENTRE.x + t * x, CENTRE.y + t * y, CENTRE.z + t * z));
const std::vector<SolventSite> sites = AnalyseSolventSites(st, 0.0, SolventMaps{});
REQUIRE(sites.size() == 1);
const SolventSite &cl = sites[0];
CHECK(cl.n_26_32 + cl.n_32_34 == 4);
CHECK(cl.donors == 4);
CHECK(cl.best == SolventSpecies::Cl);
CHECK(!cl.ambiguous);
// a water accepts two hydrogen bonds, not four
CHECK(cl.score[static_cast<int>(SolventSpecies::Cl)] > cl.score[static_cast<int>(SolventSpecies::Water)] + 2);
}
TEST_CASE("SolventSites_tetrahedral_water", "[Solvent]") {
// a water donating to two carbonyl O at 2.8 A and accepting from two N-H at 2.9 A, tetrahedral
gemmi::Structure st = empty_box();
add_atom(st, "HOH", "O", gemmi::El::O, CENTRE);
const std::vector<std::array<double, 3>> dirs = {{1, 1, 1}, {1, -1, -1}, {-1, 1, -1}, {-1, -1, 1}};
for (size_t i = 0; i < dirs.size(); i++) {
const double d = (i < 2 ? 2.8 : 2.9) / std::sqrt(3.0);
add_atom(st, "GLY", i < 2 ? "O" : "N", i < 2 ? gemmi::El::O : gemmi::El::N,
gemmi::Position(CENTRE.x + d * dirs[i][0], CENTRE.y + d * dirs[i][1], CENTRE.z + d * dirs[i][2]));
}
const std::vector<SolventSite> sites = AnalyseSolventSites(st, 0.0, SolventMaps{});
REQUIRE(sites.size() == 1);
const SolventSite &w = sites[0];
CHECK(w.n_26_32 == 4);
CHECK(w.donors == 2);
CHECK(w.acceptors == 2);
CHECK(w.best == SolventSpecies::Water);
CHECK(!w.ambiguous);
CHECK(SolventLabelAgrees(w));
}
TEST_CASE("SolventSites_coordinate_error_blurs_the_bands", "[Solvent]") {
// the octahedral Mg again, its B high and the resolution poor: the bands of Mg and Na overlap, so the score
// can no longer prefer Mg by much over Na - but still not water, whose shell is far too far out
gemmi::Structure st = empty_box();
add_atom(st, "MG", "MG", gemmi::El::Mg, CENTRE);
for (int axis = 0; axis < 3; axis++)
for (double sign : {-1.0, 1.0}) {
gemmi::Position p = CENTRE;
(axis == 0 ? p.x : axis == 1 ? p.y : p.z) += sign * 2.07;
add_atom(st, "HOH", "O", gemmi::El::O, p);
}
for (gemmi::Residue &r : st.models[0].chains[0].residues)
r.atoms[0].b_iso = 80.0f;
const SolventSite sharp = site_labelled(AnalyseSolventSites(st, 1.5, SolventMaps{}), "MG");
const SolventSite blurred = site_labelled(AnalyseSolventSites(st, 3.5, SolventMaps{}), "MG");
CHECK(blurred.sigma_xyz > sharp.sigma_xyz);
const int mg = static_cast<int>(SolventSpecies::Mg), na = static_cast<int>(SolventSpecies::Na);
CHECK(blurred.score[mg] - blurred.score[na] < sharp.score[mg] - sharp.score[na]);
CHECK(blurred.best != SolventSpecies::Water);
}