Refinement: B ceiling from the data's Wilson B, weights balanced at the refined level

Two start-conditioned defects of the first round, found on a flat-B start of the
crystal model and on a placed prediction (pLDDT-derived B of ~6 A^2), both taken from
the B column as given:

- The ceiling of B was 2 x b_max of the model, i.e. 100 A^2 for a flat or pLDDT B
  column, and it bit through the TLS tensors (BoundedB scales them down to it) long
  before atoms reached it: on the flat start one round gave R-free 0.4040 at ceiling
  100, 0.3989 at 140, 0.3870 at 200, 0.3848 at 300, while the refined max B was 123.
  The ceiling is now set from at least 2.5 x the data's Wilson B as well (WilsonB,
  fitted against the model's own atoms from 4 A), which leaves a deposited model's
  ceiling where it was.
- The ADP weight was balanced before the overall B of the scale was moved into the
  atoms, which every trial does: the B similarity's gradient goes as 1/B, so a column
  at 6 A^2 balanced a weight ~100 x too stiff (2350 vs ~20 once refined). The move is
  now made once after the placement, before the weights are balanced.

Together these give in round 1 most of what the second and third rounds were adding:
flat start 0.4040 -> 0.3822 (base with rounds 0.3798 at 3 x the time), placed
prediction 0.4162 -> 0.4038 (0.3982 with rounds), crystal model with its own B
unchanged (0.3790). 13 AlphaFold starts from images with the rounds as they are: six
lower by 0.005-0.021, none higher beyond 0.0025, median -0.002, refinement time
unchanged.

Measured and not adopted: elastic-network and packing-density B starts (within noise
once the ceiling is right), and a coarse-to-fine B solve along chain / network-mode /
residue shifts before the per-atom one (same optimum to 0.0002 at +25 % evaluations).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdJDrM65gxsCn2PrbfK
This commit is contained in:
leonarski_fandClaude Fable 5.1 committed 2026-11-14 16:53:41 +01:00
1 parent 595a94d4dc
commit 058ba80fc7
2 files changed
+62 -3

No files matched your search

@@ -53,6 +53,8 @@ namespace {
constexpr double B_FLOOR = 2.0; // A^2: no atom's B goes below it
// nor above twice the model's largest B, within these bounds (the GPU's gather is reserved for it)
constexpr double B_CEILING_MIN = 100.0, B_CEILING_MAX = 300.0;
// The B the ceiling is set from is at least this times the data's Wilson B (WilsonB).
constexpr double B_CEILING_WILSON = 2.5;
// The ceiling of B for a model whose highest B is `b_max`: twice it, within the bounds - the bounds lifted for a
// model that comes with B above them (a structure refined at 4-5 A, B 400-600 A^2), which the ceiling would
// otherwise flatten (the gather is reserved for it).
@@ -2288,6 +2290,54 @@ double RefinementDMin(double max_inv_d2) {
return 0.999 / std::sqrt(max_inv_d2);
}
// The Wilson B of the data against the model's atoms: the slope of ln(<|F|^2> / sum_j f_j^2) in s^2 / 2 over shells
// from 4 A to the finest amplitude; 0 where the data do not reach 4 A. The B factors refine around it whatever the
// model's say, so the ceiling of B is set from it as well as from them (BCeiling): a model may come with B factors
// that say nothing of where they will refine - a flat B, a prediction's pLDDT.
// Following Wilson (1949) Acta Cryst. 2, 318-321
double WilsonB(const std::vector<MergedReflection> &merged, const std::vector<gemmi::Atom *> &atoms) {
constexpr int SHELLS = 10;
constexpr double S2_MIN = 1.0 / 16.0; // 4 A
const double s2_max = MaxInvD2(merged);
if (s2_max <= S2_MIN)
return 0;
std::map<std::pair<int, signed char>, int> composition;
for (const gemmi::Atom *a : atoms)
if (!a->is_hydrogen())
composition[{static_cast<int>(a->element.elem), a->charge}]++;
std::vector<double> f2_sum(SHELLS, 0.0), s2_sum(SHELLS, 0.0);
std::vector<int> n(SHELLS, 0);
for (const MergedReflection &r : merged) {
if (!(std::isfinite(r.F) && r.d > 0))
continue;
const double s2 = 1.0 / (double(r.d) * r.d);
if (s2 < S2_MIN)
continue;
const int k = std::min(SHELLS - 1, static_cast<int>((s2 - S2_MIN) / (s2_max - S2_MIN) * SHELLS));
f2_sum[k] += double(r.F) * r.F;
s2_sum[k] += s2;
n[k]++;
}
// the line y = ln k - B x through the shells (x = s^2 / 2), each weighted by its reflections
double sw = 0, sx = 0, sy = 0, sxx = 0, sxy = 0;
for (int k = 0; k < SHELLS; k++) {
if (n[k] < 20)
continue;
const double s2 = s2_sum[k] / n[k];
double f2 = 0;
for (const auto &[ion, count] : composition)
f2 += count * std::pow(IonFormFactor<double>(static_cast<gemmi::El>(ion.first), ion.second).calculate_sf(s2 / 4.0), 2);
const double x = s2 / 2.0, y = std::log(f2_sum[k] / n[k] / f2), w = n[k];
sw += w;
sx += w * x;
sy += w * y;
sxx += w * x * x;
sxy += w * x * y;
}
const double det = sw * sxx - sx * sx;
return det > 0 ? std::max(0.0, -(sw * sxy - sx * sy) / det) : 0.0;
}
// One refinement of a model, with the solvent build in it where settings.solvent asks for it.
// `twin_laws`: against the data as twinned by them (one domain each beside the model's own), empty for untwinned;
// `null_first`: the first of them the null (TwinNull) replaces the mates of.
@@ -2326,7 +2376,8 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
for (const TlsBody &body : tls_bodies)
for (int i : body.atoms)
b_max = std::max(b_max, ctx.atoms[i]->b_iso + gemmi::u_to_b() * body.U(ctx.atoms[i]->pos).trace() / 3.0);
ctx.b_ceiling = BCeiling(b_max);
result.wilson_b = WilsonB(merged, ctx.atoms);
ctx.b_ceiling = BCeiling(std::max(b_max, B_CEILING_WILSON * result.wilson_b));
if (ctx.atoms.empty()) {
result.failure_reason = "the model has no atoms";
return result;
@@ -2432,14 +2483,15 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
result.r_free_start = r.r_free;
result.score_start = ctx.Score();
logger.Info("Refinement{}: {} atoms ({} riding hydrogens), {} reflections to {:.2f} A ({} free), on the {}; as "
"given R-work {:.4f}, R-free {:.4f}; k_sol {:.3f}, b_sol {:.1f}, overall B {:.1f}, mask grid {:.2f} A",
"given R-work {:.4f}, R-free {:.4f}; k_sol {:.3f}, b_sol {:.1f}, overall B {:.1f}, Wilson B {:.0f}, B "
"ceiling {:.0f}, mask grid {:.2f} A",
ctx.refl.Twinned() ? fmt::format(" against the data twinned by {} (fraction {:.3f}, {} mates not "
"observed)", result.twin_law, FractionSum(ctx.rows),
ctx.refl.mate_hkl.size())
: std::string(),
ctx.atoms.size(), result.score_start.hydrogens, ctx.refl.Size(), result.d_min, r.n_free,
result.on_gpu ? "GPU" : "CPU", r.r_work, r.r_free, ctx.k_sol, ctx.b_sol, result.score_start.b_iso,
result.score_start.mask_spacing);
result.wilson_b, ctx.b_ceiling, result.score_start.mask_spacing);
// The model as given, scaled: a candidate of the final choice beside the trials (the restrained stage, 5.), with
// its atoms and residues as they are, which the solvent build edits in place.
@@ -2605,6 +2657,12 @@ RefinementResult RefineOnce(const gemmi::Structure &placed, const std::vector<Me
r = ctx.Refit();
result.r_work_placement = r.r_work;
result.r_free_placement = r.r_free;
// The overall B of the scale into the atoms (MoveOverallB, as every trial does), so that the weights are balanced
// and the trials started at the level the B factors refine at, not the one they came at: the B similarity's
// gradient goes as 1 / B, and from a prediction's B (6 A^2 under an overall B of 45) the balanced weight was a
// hundred times too stiff - what the second round of the weights (4b.) was mostly correcting.
if (settings.restrained)
ctx.MoveOverallB();
logger.Info("Refinement, placement ({} rigid bodies kept, B per residue {}): R-work {:.4f}, R-free {:.4f}",
result.rigid_bodies, result.group_b ? "kept" : "not kept", r.r_work, r.r_free);
@@ -217,6 +217,7 @@ struct RefinementResult {
// The refined model's waters and ions and the species their surroundings support (AnalyseSolventSites): an
// audit, nothing in the model is changed by it.
std::optional<std::vector<SolventSite>> solvent; // the solvent audit (settings.solvent)
double wilson_b = 0; // the data's Wilson B against the model's atoms (A^2), 0 where the data do not reach 4 A
double b_ceiling = 0; // the highest B the refinement allowed
SolventBuildResult solvent_build; // the solvent build's rounds (settings.solvent)