mx/Jungfraujoch
3
2 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 87 Wiki Activity

Refinement score: the model's structure factors as REFMAC 5.8 scores them at 0 cycles

Browse Source 
Three terms of the score of a model - the same F_model at every stage, the as-given start and the
refined end alike - and a breakdown of it:

- Riding hydrogens from the start: the dictionary's hydrogens are added before the model is first
  scored (REFMAC's 0-cycle restores them; they move R by 0.010-0.013 at 1.1 A). The engine is made
  once, with them.
- The bulk-solvent mask on a grid of its own, min(d_min/3, 0.4 A), at most 2^26 points. On the
  density grid (1.1 A at 3.3 A) gemmi's 0.8 A shrink found no grid offset and was skipped, which put
  R 0.015-0.03 high at >= 2.5 A. 0.3 and 0.4 A agree to 0.001.
- The overall scale and anisotropic B by robust least squares (pseudo-Huber, iteratively
  reweighted, c = median |residual|) in the fine pass of the k_sol/b_sol grid (ModelScaleBox::robust,
  refinement only; MR and validation keep the plain fit). Plain least squares on |F| is the exact LS
  optimum but lets a few badly-fitted strong reflections set the scale (1.1 A set: R 0.1706 against
  REFMAC's 0.1600). The scale fit of refinement now runs on the CPU (no ModelScaleGPUPool).

RefinementSettings::score_only scores the model as given and stops; RefinementResult::score_start
carries every term (k, B, anisotropic B, k_sol, b_sol, mask spacing, R in 20 shells, -lnL work/free,
the likelihood FOM, hydrogens; per-reflection terms with score_only). GeometryStatistics gains rmsZ.
rugnux_refine_test: --score-only, --dump.

Measured against REFMAC 5.8.0431 ncyc 0 (refi bref iso) on the same model, data and flags, 14 open-arm
sets 1.1-3.6 A: R-work/R-free within 0.002 on 12 (was +0.003..+0.03); 7n2s and 9yl4 score 0.003/0.005
below REFMAC, which a 0.6-0.8 A mask grid reproduces. REFMAC with ANISOU stripped scores identically,
so isotropic B is not a parity term. Open: the likelihood's D/FOM (ours 0.01-0.11 higher at high
resolution); geometry differs only by REFMAC's metal LINK restraints and residues outside the
embedded dictionary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit 4f94b21611874ad9dcf89e6771fe311a0e10bb06)
(cherry picked from commit 00db6a6f539ac2da515c0a649474335f4c4b07b8)
This commit is contained in:
leonarski_f committed 2026-10-26 01:38:23 +01:00
1 parent 8933a72a89
commit c4d2d419ce
10 files changed
+310 -54
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

image_analysis/structure_refinement/ModelScaling.cpp
+49 -9
View File
@@ -25,6 +25,8 @@
namespace {
constexpr int ROBUST_ROUNDS = 4; // reweightings of the robust fit (ModelScaleBox::robust)
// gemmi::Scaling<float> as a grid point fits it: the same starting point, parameters, model values,
// derivatives and R. The solvent pair is fixed there, so |Fcalc + k_sol exp(-b_sol s^2) Fmask| of a
// reflection is the same at every evaluation of the solver; it is taken once per grid point here rather than
@@ -35,8 +37,9 @@ struct FixedSolventFit {
gemmi::Miller hkl;
float fobs;
float fcalc_abs; // std::abs(Scaling::get_fcalc(p))
double weight = 1.0; // 1, but for the robust fit's reweighting (RobustReweight)
double get_y() const { return fobs; }
double get_weight() const { return 1.0; }
double get_weight() const { return weight; }
};
const std::vector<gemmi::Vec6> &constraint_matrix;
double k_overall = 1.0;
@@ -61,7 +64,7 @@ struct FixedSolventFit {
// Scaling::get_fcalc() at (k_sol, b_sol)
const std::complex<float> fcalc =
s.use_solvent ? p.fcmol + (float) (k_sol * std::exp(-b_sol * p.stol2)) * p.fmask : p.fcmol;
points[i] = {p.hkl, p.fobs, std::abs(fcalc)};
points[i] = {p.hkl, p.fobs, std::abs(fcalc), 1.0};
if (p.fobs < 1 || p.fobs < p.sigma) // skip weak reflections
continue;
double fcalc_abs = points[i].fcalc_abs;
@@ -138,6 +141,27 @@ struct FixedSolventFit {
}
};
// The robust fit's weights at the current parameters: each reflection's residual r = |Fo| - |F_model| weighted by
// 1 / sqrt(1 + (r / c)^2), with c the median |r| of the unweighted fit. Iterated, the weighted least squares then
// minimises sum c^2 2 (sqrt(1 + (r / c)^2) - 1) - quadratic for the bulk of the reflections and linear for the few
// a model fits badly (a beam-stop shadow, an ice ring, a missing part of the model), which least squares would
// otherwise let set the scale of all the rest.
// Following Huber (1964) Ann. Math. Statist. 35, 73-101 for the M-estimator, as iteratively reweighted least squares
void RobustReweight(FixedSolventFit &fit, double &c) {
std::vector<double> r(fit.points.size());
for (size_t i = 0; i < fit.points.size(); i++)
r[i] = std::fabs(fit.points[i].fobs - fit.compute_value(fit.points[i]));
if (c <= 0) {
std::vector<double> sorted = r;
std::nth_element(sorted.begin(), sorted.begin() + sorted.size() / 2, sorted.end());
c = sorted[sorted.size() / 2];
}
if (c <= 0)
return;
for (size_t i = 0; i < fit.points.size(); i++)
fit.points[i].weight = 1.0 / std::sqrt(1.0 + gemmi::sq(r[i] / c));
}
// R-factor of the current parameters over the fitted reflections. This is what the grid is
// selected on, and it is the quantity the scale exists to make small.
double RFactor(const FixedSolventFit &fit) {
@@ -171,16 +195,17 @@ double ComputeLMMatrices(const FixedSolventFit &fit, const double *kaniso, doubl
const double k = kaniso ? kaniso[i] : std::exp(-0.25 * fit.b_star.r_u_r(gemmi::Vec3(p.hkl)));
double y = fit.compute_value_and_derivatives<NA>(p, k, dy_da);
double dy_sig = p.get_y() - y;
const double w = p.get_weight();
for (int j = 0; j != NA; ++j) {
const bool used = dy_da[j] != 0;
for (int k = 0; k < NA; ++k) {
const double a = dy_da[j] * dy_da[k];
const double a = dy_da[j] * dy_da[k] * w;
alpha[NA * j + k] += used ? a : 0.0;
}
const double b = dy_sig * dy_da[j];
const double b = dy_sig * dy_da[j] * w;
beta[j] += used ? b : 0.0;
}
wssr += gemmi::sq(dy_sig);
wssr += gemmi::sq(dy_sig) * w;
}
// The upper half of alpha is the lower half's mirror, as in gemmi.
for (int j = 1; j < NA; j++)
@@ -198,8 +223,8 @@ double ComputeWSSR(const FixedSolventFit &fit, std::vector<double> &kaniso) {
for (size_t i = 0; i < fit.points.size(); ++i) {
const auto &p = fit.points[i];
kaniso[i] = std::exp(-0.25 * fit.b_star.r_u_r(p.hkl));
// FixedSolventFit::compute_value(), weight 1
wssr += gemmi::sq(p.get_y() - p.fcalc_abs * (float) (fit.k_overall * kaniso[i]));
// FixedSolventFit::compute_value()
wssr += gemmi::sq(p.get_y() - p.fcalc_abs * (float) (fit.k_overall * kaniso[i])) * p.get_weight();
}
return (double) wssr;
}
@@ -343,10 +368,18 @@ ModelScaleReport FitModelScale(gemmi::Scaling<float> &scaling, ModelScaleBox box
// The scale the walk stands at, which a point starts from only where the data give it no start of its own.
double k_last = scaling.k_overall;
gemmi::SMat33<double> b_last = scaling.b_star;
auto fit_point = [&scaling](FixedSolventFit &fit, PointFit &pf, double k_start,
const gemmi::SMat33<double> &b_start) {
bool robust_pass = false; // the fine pass of a robust fit (ModelScaleBox::robust)
auto fit_point = [&scaling, &robust_pass](FixedSolventFit &fit, PointFit &pf, double k_start,
const gemmi::SMat33<double> &b_start) {
fit.Start(scaling, pf.k_sol, pf.b_sol, k_start, b_start);
LevMarFit(fit); // k_overall + anisotropic B only
if (robust_pass) {
double c = 0;
for (int round = 0; round < ROBUST_ROUNDS; round++) {
RobustReweight(fit, c);
LevMarFit(fit);
}
}
pf.r = RFactor(fit);
pf.k_overall = fit.k_overall;
pf.b_star = fit.b_star;
@@ -403,6 +436,13 @@ ModelScaleReport FitModelScale(gemmi::Scaling<float> &scaling, ModelScaleBox box
const double k_hi2 = std::min(box.k_hi, k0 + 0.05);
const double b_hi2 = std::min(box.b_hi, b0 + b_step);
std::vector<PointFit> fine;
// A robust fit takes the fine pass robustly: each of its points, and the coarse winner again, by robust least
// squares - the coarse pass only finds where to look.
if (box.robust) {
robust_pass = true;
fitted.clear();
best_r = -1;
}
for (double ks = std::max(box.k_lo, k0 - 0.05); ks <= k_hi2 + 1e-9; ks += 0.025)
for (double bs = std::max(box.b_lo, b0 - b_step); bs <= b_hi2 + 1e-9; bs += b_step / 2.0)
fine.push_back(PointFit{ks, bs});