Files
leonarski_f 4809c1622c
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 28m46s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 32m5s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 39m44s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m7s
Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m30s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 31m54s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m29s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 23m51s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 26m7s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m38s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 28m6s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m44s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 33m22s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 34m6s
Build Packages / Unit tests (push) Successful in 1h2m47s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 35m59s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 34m8s
Build Packages / Validation Battery (push) Successful in 38m31s
v1.0.0-rc.187 (#98)
* Rugnux: Refinement several times faster, one protocol on the CPU and the GPU, parts of it on the Apple GPU; its random draws are the same on every platform.
* Rugnux: Refinement follows common practice: the free set decides only the restraint weights and the displacement model (isotropic, TLS or anisotropic, by Hamilton's test), everything else is decided on the working set; lower R-free for placed and predicted models.
* Rugnux: Restraint dictionaries via `--restraints` (or among the inputs) and more built-in ligands; a model whose space group is a subgroup of the data's is refined in its own group; reflections plus a model select `--mode refine` on their own.
* Rugnux: `-C` fixes the crystal system on the axes as typed; `-S` alone fixes the space group only, the cell is still found and reduced; a space group or cell stored in the input file is ignored, with a line in the log.
* Rugnux: More reliable multi-sweep merging; `--mode scale` merges several stills runs as one; `_plot.txt` and the report name each input; a second crystal form is reported for stills given a cell.
* jfjoch_broker (experimental): Live Rugnux on FPGA detectors (JUNGFRAU) takes the same path as a DECTRIS stream, rotation and serial alike, model step included, with analysis threads per GPU; frames waiting for the analysis can be viewed; no host stage drops a frame - a source faster than the analysis is held at the card.
* jfjoch_broker: Summation of more than 256 frames per image is split between the card and the host (`summation_mode` is then `hybrid`), and live Rugnux runs on such collections; pedestal runs take a receiver of their own.
* jfjoch_broker (experimental): `stills_downsample` analyses 1 of every N serial images (chosen by a hash; the first 2000 all; the others written without analysis; reset to 1 by `/initialize`); `max_spot_count` left out means 250 for per-image indexing and up to 1000 for live stills; the live status gives the merge before the model work, the fitted resolution and the anomalous peak height of each unexplained blob.
* jfjoch_broker: Without a `pcie` entry the broker runs on a mock of the card instead of the HLS simulation; its spot, ROI and azimuthal results are zero then.
* Web frontend: Redesigned Rugnux live page; Vite 8, TypeScript 6, Mol* 5.13 and dependency security updates.
* Tools: `jfjoch_fpga_stills_test` measures live stills on the cards with a stored dataset; `jfjoch_fpga_test` loses the HLS-simulation option.
* Other: FFTW no longer fetched or linked (the CPU FFT indexer keeps the name `FFTW`); PCIe driver now GPL-2.0-or-later and third-party notices complete; several crash fixes.

Reviewed-on: #98
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-11-19 20:49:28 +01:00

101 lines
5.5 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <condition_variable>
#include <cstddef>
#include <memory>
#include <mutex>
#include <vector>
#include "gemmi/model.hpp"
#include "gemmi/scaling.hpp"
// Fit the overall scale, the anisotropic B and the flat bulk solvent of a model to observed
// amplitudes, writing the answer into `scaling` (k_overall, b_star, k_sol, b_sol) so that
// gemmi's own scale_data() can then apply it.
//
// The caller decides which reflections are fitted: FitModelScale reads scaling.points, which
// prepare_points() filled. Pass the WORKING set to keep the free reflections out of the scale
// model - that is a property of the call site, not of this function.
//
// Replaces gemmi's fit_isotropic_b_approximately() + fit_parameters(), which is an unbounded
// Levenberg-Marquardt: nothing there stops k_sol and b_sol reaching values a flat solvent model
// is meaningless at (b_sol of 1707 A^2 was measured on this corpus). Here k_sol and b_sol come
// from a grid search over the physical ranges, so an unphysical answer cannot be represented.
// The box the solvent parameters are searched in: k_sol in (0.10, 0.60) and b_sol in (10, 80) A^2.
// The b_sol range is the physically reasonable one reported by Fokine & Urzhumtsev and phenix's
// default grid (b_sol 10-80 in steps of 5); their k_sol range reaches 0.8, the box here stops at 0.6. A model with high
// atomic B gets a higher b_sol ceiling (SolventBMax).
struct ModelScaleBox {
double k_lo = 0.10, k_hi = 0.60;
double b_lo = 10.0, b_hi = 80.0;
// The overall scale and anisotropic B of the points of the fine pass by robust least squares
// (ModelScaling.cpp, RobustReweight) rather than plain least squares on |F|: what refinement scores a model with.
bool robust = false;
// Where a fit of the model as it was a moment ago found the solvent: the fine pass around (k_start, b_start)
// without the coarse pass. Negative: the coarse pass finds where to look.
double k_start = -1, b_start = -1;
};
// The b_sol ceiling for `model`: 80 A^2, or twice the model's mean atomic B where that is higher.
// b_sol is how far the solvent's edge is smeared, and the edge can be no sharper than the atoms it
// lies against: a model at B of 300 A^2 beside a solvent held at 80 gets a solvent term that reaches
// far too deep in resolution, which only a too-small k_sol can then offset. Fokine & Urzhumtsev's
// 10-80 A^2 is for models at the B of ordinary entries (their mean b_sol 46 A^2), where the ceiling
// stays at 80.
double SolventBMax(const gemmi::Model &model);
// The anisotropic part of an overall scale's b_star (gemmi::Scaling, fractional reciprocal basis): b_star less its
// isotropic B. exp(-r_u_r(h) / 4) of it is the factor by which the fit's overall anisotropy scales reflection h
// beyond its isotropic fall-off, which map coefficients divide out so that the map is not stretched along the
// data's weak direction (as phenix.refine's are).
gemmi::SMat33<double> AnisotropicPart(const gemmi::SMat33<double> &b_star, const gemmi::UnitCell &cell);
struct ModelScaleReport {
int n_points = 0; // reflections the fit actually used
int n_grid = 0; // grid points evaluated
double r_work_fit = 1.0; // R on the fitted reflections, at the chosen solution
};
// nthreads: the grid points are fitted in parallel on up to this many threads; the answer does not
// depend on it.
ModelScaleReport FitModelScale(gemmi::Scaling<float> &scaling, ModelScaleBox box = {}, size_t nthreads = 1);
#ifdef JFJOCH_USE_CUDA
// FitModelScale on a GPU (ModelScaleGPU::FitSolvent): the same grid over the same box, the same fits and
// the same winner, with the sums over the reflections taken in double on the device where the host takes
// them in long double - so the answer is FitModelScale's to rounding, not to the bit. A few engines,
// reserved once, each a stream and room for one fit at a time; a fit takes a free one and waits for one
// where all are taken. Every kernel is deterministic, so which engine a fit gets changes no number.
class ModelScaleGPUPool {
public:
// What `engines` engines for fits of up to `max_points` reflections take on the card.
static size_t DeviceBytes(size_t max_points, size_t engines, bool robust = false);
// On `device`. Throws JFJochException on a CUDA failure.
ModelScaleGPUPool(int device, size_t max_points, size_t engines);
~ModelScaleGPUPool();
// FitModelScale(scaling, {.b_hi = b_hi, .robust = robust}, nthreads). Where FitModelScale would walk the
// grid point to point (five or fewer reflections for the isotropic start), it is FitModelScale on the host.
// Safe to call from several threads. Throws JFJochException on a CUDA failure. The robust fit's reweighting
// runs on the device; it is FitModelScale's robust fit to rounding (float residuals and weights, the
// anisotropic factor of the steps in float), deterministic run to run.
// k_start, b_start: ModelScaleBox's.
ModelScaleReport Fit(gemmi::Scaling<float> &scaling, double b_hi, size_t nthreads, bool robust = false,
double k_start = -1, double b_start = -1);
// What the pool holds on the card.
size_t Bytes() const { return DeviceBytes(max_points_, engines_.size()); }
private:
struct Engine;
int device_;
size_t max_points_;
std::vector<std::unique_ptr<Engine>> engines_;
std::vector<Engine *> idle_;
std::mutex m_;
std::condition_variable cv_;
};
#endif