Files
leonarski_fandClaude Opus 5.5 2a0e91bd6a RESULT-CHANGING: Refinement: the solvent build within the weight search, two rounds each judged by refining the solvent
Each round of the build was a refinement of its own - the whole weight search again, and a rejected round cost a
full refinement - so the build took 2-4 times the refinement. Now the atom list carries placeholder waters from the
start (occupancy 0: no density, no mask, no contact, not refined; a fifteenth of the heavy atoms, at least 64), and
the build runs where the weight trajectory ends: a round reads the maps and the audit of that model, puts waters into
placeholders, takes waters out (occupancy 0) and relabels sites in place (RetypeAtom: element, flags, metal bonds,
exclusions; the GPU engine refreshes an atom's density on an occupancy or element change), in every trial slot; then
the solvent alone is refined at the trajectory's weights (B refined, 10 iterations a cycle) and the round is kept
where that lowers R-free, else undone. At most two rounds; the final ladder then refines what they kept with the
rest. The output model is the view with the placeholders moved into their chains (SolventView) and the relabelled
metals' inner-sphere records. One topology for the whole refinement, so the metal records are decided once by
construction (the outer_records plumbing of the previous commit is gone again). Contacts() leaves out atoms at
occupancy 0; FindSpecialPositions() too (placeholders at the origin sat on the 2-folds of I222/P43212/C2 and were
projected there - R-free 0.44 on 36gk until fixed); waters put on a special position get their site constraint.
The audit runs on the threads (its density tables made up front), and MakeMaps makes its three maps at once.

Timing (--mode refine on the depositions, GPU, each set's three runs under one lock, load 15-24 from other agents),
refinement seconds without the build / with the old build / with this one, R-free old / new:
1hr2 3.5/11.3/4.6 0.2462/0.2490, 2r8s 2.8/11.2/4.6 0.2238/0.2244, 193l 1.0/3.1/1.8 0.1713/0.1713,
1exr 1.6/5.2/1.9 0.2076/0.2084, 1k4c 2.5/4.9/3.2 0.2106/0.2106, 36gk 3.7/12.4/6.7 0.2559/0.2543,
7l6j 3.6/16.3/6.5 0.1530/0.1546, 8agq 2.1/6.7/2.7 0.1753/0.1761, 6hwj 1.9/8.1/3.6 0.2222/0.2232,
8sa8 8.5/36.2/14.1 0.1714/0.1710, 6f3p 4.8/22.3/8.3 0.1629/0.1630, 7os3 1.4/4.9/2.6 0.2122/0.2115.
Build overhead 0.2-0.9x the refinement (median 0.7), was 1.0-3.7x (median 2.3); R-free within 0.001 on 8/12.
Tried and dropped (P3.md): the trajectory again after the rounds (never gained), a weight ladder after the build
(never displaced), building after the ladder instead (same), accepting rounds within the R-free tie (worse on 4),
a full step as the round's judge (noisier), one macro-cycle for the solvent (6f3p's rounds refused).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVeDQqN9qRD57vnShzTC37
2026-10-27 18:39:47 +01:00

127 lines
7.2 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <array>
#include <string>
#include <vector>
#include "gemmi/grid.hpp"
#include "gemmi/model.hpp"
#include "gemmi/monlib.hpp"
// What the solvent of a model - every water and every single-atom ion - is surrounded by, and which species
// that surrounding supports: water, Mg2+, Na+, K+, Ca2+ or Cl- (NH4+ scored too, but only reported). An audit
// of the model as it is: nothing here changes the model.
//
// Each site's neighbours (symmetry mates included) are measured in the coordination bands of Cat_Wiz and typed
// by their chemistry (AtomChemistry); bond-valence sums are taken for each cation; the maps are read at the
// site. Each species then gets a score, a plain sum of terms (SolventSites.cpp says which), and the best one
// is reported, AMBIGUOUS where the runner-up is close.
// What an atom can do in a hydrogen bond or a salt bridge.
struct AtomChemistry {
bool donor = false;
bool acceptor = false;
bool positive = false; // Lys NZ, Arg NE/NH1/NH2, a polypeptide's N-terminal N
bool negative = false; // Asp/Glu carboxylate O, a C-terminal carboxylate, a nucleotide's phosphate O
bool flip_ambiguous = false; // His ring N, Asn/Gln amide ends: which end is which is the model's guess
};
// The chemistry of `atom` of `residue`: donor or acceptor from the dictionary's energy type (hb_type D, A, B),
// charge from the residue's chemistry; a water is both; an N or O of a residue the dictionary does not
// describe is both. `n_terminus`: the residue begins a polypeptide.
AtomChemistry ChemistryOf(const gemmi::MonLib &monlib, const gemmi::Residue &residue, const gemmi::Atom &atom,
bool n_terminus);
enum class SolventSpecies { Water, Mg, Na, K, Ca, Cl, NH4 };
constexpr int SOLVENT_SPECIES = 7;
// The residue name of a species: HOH, MG, NA, K, CA, CL, NH4.
const char *SolventSpeciesName(SolventSpecies s);
// The cations bond valences are summed for, in this order.
constexpr int BVS_CATIONS = 4; // Mg, Na, K, Ca
struct SolventSite {
std::string label; // the residue name as modelled: HOH, MG, NA, ...
std::string chain, seqid;
char altloc = '\0';
gemmi::Position pos;
double occupancy = 1, b_iso = 0;
double sigma_xyz = 0; // the coordinate error assumed for the site, A
// N and O neighbours, by distance: the bands of Cat_Wiz, and the clash shells beyond
int n_below_19 = 0, n_19_23 = 0, n_23_26 = 0, n_26_32 = 0, n_32_34 = 0, n_34_36 = 0;
// the N and O neighbours closer than 3.4 A, by what they can do in a hydrogen bond
int donors = 0, acceptors = 0, donor_acceptors = 0;
// the shortest distance between two N or O neighbours closer than 2.6 A to the site; NaN with fewer than two
double min_ligand_distance = 0;
// the nearest positive and negative group (a charged atom of the model or an ion), within 6 A; INFINITY none
double nearest_positive = 0, nearest_negative = 0;
// the positive groups closer than 3.5 A, the negative ones closer than 3.2 A
int positive_contacts = 0, negative_contacts = 0;
// the other ions of the model (not waters) within 4.2 and within 6 A, and the nearest one (INFINITY none)
int ions_within_42 = 0, ions_within_6 = 0;
double nearest_ion = 0;
// bond-valence sum and normalised vector sum for each of the BVS cations (Mg, Na, K, Ca), from the N and
// O neighbours closer than 3.5 A
std::array<double, BVS_CATIONS> bvs{}, nvecsum{};
// the maps at the site, in their rms; NaN where there is no such map
double map_sigma = 0, difference_sigma = 0, anomalous_sigma = 0;
// An ion is read where its own density puts it, not where the restraints of its label hold it: the highest point
// of 2mFo-DFc within SOLVENT_PEAK_REACH of the modelled position (its shell, bands and bond valences are measured
// from there), and how far that is from it (A). A water is read where it is modelled (shift 0).
gemmi::Position measured_at;
double peak_shift = 0;
// The mean B of the N and O closer than 3.5 A (the environment's B, as CheckMyMetal compares a metal's with its
// ligands'); NaN with none.
double b_env = 0;
// The electrons the 2mFo-DFc map shows at the site: its height there over the height of one electron of the
// neighbours' (each neighbour's height over what its element at its B gives, the median of them), at the
// environment's B and as a water's shape - 10 for a water as dense as its neighbours; NaN without the map or
// neighbours. `electron_spread` is the spread of the neighbours' calibration (rms of the log), floored.
double electrons = 0, electron_spread = 0;
// The anomalous map's height expected at the site for each species (f'' at the wavelength, on the scale the
// model's S and P show), in the map's rms; NaN where there is no anomalous map or nothing to put it on a scale.
std::array<double, SOLVENT_SPECIES> anomalous_expected{};
// Purine N7 among the first-shell ligands (closer than 2.6 A), and those of them a divalent ion could not hold:
// the site more than 1.0 A out of the base's plane, or not 3.5-3.9 A from its O6 or N6 (NAR 2017).
int n7_ligands = 0, n7_off_plane = 0;
std::array<double, SOLVENT_SPECIES> score{}; // log-score per species, SolventSpecies order
SolventSpecies best = SolventSpecies::Water; // the best-scoring of water, Mg, Na, K, Ca and Cl
bool ambiguous = false; // the runner-up within SOLVENT_AMBIGUOUS_MARGIN of it
std::string neighbours; // the N and O neighbours closer than 3.4 A, nearest first
};
constexpr double SOLVENT_AMBIGUOUS_MARGIN = 2.0;
// The maps a site is read in: 2mFo-DFc, mFo-DFc and the anomalous difference map, each with its rms; any of
// them may be missing (null). `wavelength_A`: the data's, for the anomalous scattering expected (0: unknown).
struct SolventMaps {
const gemmi::Grid<float> *map = nullptr;
double map_rms = 0;
const gemmi::Grid<float> *difference = nullptr;
double difference_rms = 0;
const gemmi::Grid<float> *anomalous = nullptr;
double anomalous_rms = 0;
double wavelength_A = 0;
};
// How far from an ion's modelled position its density's highest point is looked for (A).
constexpr double SOLVENT_PEAK_REACH = 0.6;
// Every water and single-atom ion of the first model of `st` (in its cell and space group; hydrogens are
// ignored), in model order. `d_min`: the resolution the model was refined at, for the coordinate error
// (0 for a model with no data).
// The sites are measured on `nthreads` threads; the result does not depend on their number.
std::vector<SolventSite> AnalyseSolventSites(const gemmi::Structure &st, double d_min, const SolventMaps &maps,
size_t nthreads = 1);
// Whether the site is labelled a water; whether its label is one of the species decided here (water, Mg, Na, K,
// Ca, Cl); whether the best-supported species is its label.
bool SolventLabelIsWater(const SolventSite &s);
bool SolventLabelScored(const SolventSite &s);
bool SolventLabelAgrees(const SolventSite &s);