Files
Jungfraujoch/image_analysis/structure_refinement/ModelScaling.h
T
leonarski_f 46ad2c10ed Refinement GPU: the robust scale fit on the card, the pool back in every trial slot
The score's robust fine pass (pseudo-Huber IRLS of ModelScaleBox::robust) on the device, so that refinement on
a GPU no longer fits its scale on the host every cycle - with the trials side by side that was a robust grid
search on one thread per slot (13-133 s per refinement on the seven timing sets, against 1.7-8 s before the
score commit).

- ModelScaleGPU: the weights multiply the WSSR and J^T J / J^T r in scale_sums (a weight row per fit,
  reserved at the first robust fit); robust_residuals / robust_weights kernels; the median of each fit's
  first residuals taken on the host as FitModelScale takes it (nth_element at n/2, on the fits' threads);
  LevMarBatch split out of FitBatch so that the rounds restart from where a fit stands. FitSolvent(...,
  robust) refits the coarse winner in the fine pass, as FitModelScale's robust fit does; the plain fit is
  unchanged.
- ModelScaleGPUPool::Fit(..., robust); its streams at the highest priority: a fit is a chain of small
  launches the host waits on, and on a card shared with the trials' long kernels each wait queued behind
  them (refit.scale 58-87 ms -> 14-45 ms per cycle, the result the same to the bit).
- Refinement: the main context and every trial slot fit on their pool (robust); the CPU engine keeps
  FitModelScale's robust fit.

The card's fit is FitModelScale's to rounding (float residuals and weights, the steps' anisotropic factor in
float), deterministic run to run and whichever engine a fit takes: R-work/R-free of the 14 parity sets on
the GPU equal the CPU engine's to 4 digits (largest difference 1.4e-6 on the 3.6 A set).
Test: ModelScaleGPUPool_RobustMatchesFitModelScale (one reflection in fifty a 4x outlier, five cells).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit d15f048cbd7ed2d6b430f9f9cb5018ca284da8f0)
2026-10-26 01:38:24 +01:00

90 lines
4.7 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <condition_variable>
#include <cstddef>
#include <memory>
#include <mutex>
#include <vector>
#include "gemmi/model.hpp"
#include "gemmi/scaling.hpp"
// Fit the overall scale, the anisotropic B and the flat bulk solvent of a model to observed
// amplitudes, writing the answer into `scaling` (k_overall, b_star, k_sol, b_sol) so that
// gemmi's own scale_data() can then apply it.
//
// The caller decides which reflections are fitted: FitModelScale reads scaling.points, which
// prepare_points() filled. Pass the WORKING set to keep the free reflections out of the scale
// model - that is a property of the call site, not of this function.
//
// Replaces gemmi's fit_isotropic_b_approximately() + fit_parameters(), which is an unbounded
// Levenberg-Marquardt: nothing there stops k_sol and b_sol reaching values a flat solvent model
// is meaningless at (b_sol of 1707 A^2 was measured on this corpus). Here k_sol and b_sol come
// from a grid search over the physical ranges, so an unphysical answer cannot be represented.
// The box the solvent parameters are searched in. The defaults are the physically reasonable
// range reported by Fokine & Urzhumtsev and adopted as phenix's default grid: k_sol in (0.1, 0.8)
// and b_sol in (10, 80) A^2, of which phenix searches b_sol 10-80 in steps of 5. A model with high
// atomic B gets a higher b_sol ceiling (SolventBMax).
struct ModelScaleBox {
double k_lo = 0.10, k_hi = 0.60;
double b_lo = 10.0, b_hi = 80.0;
// The overall scale and anisotropic B of the points of the fine pass by robust least squares
// (ModelScaling.cpp, RobustReweight) rather than plain least squares on |F|: what refinement scores a model with.
bool robust = false;
};
// The b_sol ceiling for `model`: 80 A^2, or twice the model's mean atomic B where that is higher.
// b_sol is how far the solvent's edge is smeared, and the edge can be no sharper than the atoms it
// lies against: a model at B of 300 A^2 beside a solvent held at 80 gets a solvent term that reaches
// far too deep in resolution, which only a too-small k_sol can then offset. Fokine & Urzhumtsev's
// 10-80 A^2 is for models at the B of ordinary entries (their mean b_sol 46 A^2), where the ceiling
// stays at 80.
double SolventBMax(const gemmi::Model &model);
struct ModelScaleReport {
int n_points = 0; // reflections the fit actually used
int n_grid = 0; // grid points evaluated
double r_work_fit = 1.0; // R on the fitted reflections, at the chosen solution
};
// nthreads: the grid points are fitted in parallel on up to this many threads; the answer does not
// depend on it.
ModelScaleReport FitModelScale(gemmi::Scaling<float> &scaling, ModelScaleBox box = {}, size_t nthreads = 1);
#ifdef JFJOCH_USE_CUDA
// FitModelScale on a GPU (ModelScaleGPU::FitSolvent): the same grid over the same box, the same fits and
// the same winner, with the sums over the reflections taken in double on the device where the host takes
// them in long double - so the answer is FitModelScale's to rounding, not to the bit. A few engines,
// reserved once, each a stream and room for one fit at a time; a fit takes a free one and waits for one
// where all are taken. Every kernel is deterministic, so which engine a fit gets changes no number.
class ModelScaleGPUPool {
public:
// What `engines` engines for fits of up to `max_points` reflections take on the card.
static size_t DeviceBytes(size_t max_points, size_t engines, bool robust = false);
// On `device`. Throws JFJochException on a CUDA failure.
ModelScaleGPUPool(int device, size_t max_points, size_t engines);
~ModelScaleGPUPool();
// FitModelScale(scaling, {.b_hi = b_hi, .robust = robust}, nthreads). Where FitModelScale would walk the
// grid point to point (five or fewer reflections for the isotropic start), it is FitModelScale on the host.
// Safe to call from several threads. Throws JFJochException on a CUDA failure. The robust fit's reweighting
// runs on the device; it is FitModelScale's robust fit to rounding (float residuals and weights, the
// anisotropic factor of the steps in float), deterministic run to run.
ModelScaleReport Fit(gemmi::Scaling<float> &scaling, double b_hi, size_t nthreads, bool robust = false);
// What the pool holds on the card.
size_t Bytes() const { return DeviceBytes(max_points_, engines_.size()); }
private:
struct Engine;
int device_;
size_t max_points_;
std::vector<std::unique_ptr<Engine>> engines_;
std::vector<Engine *> idle_;
std::mutex m_;
std::condition_variable cv_;
};
#endif