Refinement: the bulk-solvent mask's shrink over the threads; map stage timers

The map preparation (refine.maps, ~0.38 s per refinement) was the CPU bulk-
solvent mask of the missing reflections: 0.33 s of it, on the main thread
while the two FFT maps (0.07 s each) and Fcalc ran beside it. Inside the mask,
gemmi's serial shrink (set_margin_around) took 0.135 s, the FFTW transform
0.12 s and gemmi's island removal 0.06 s.

The shrink is now a gather over the w planes (ParallelChunks): every mark gemmi
makes is the same store on a masked point and a marked point still reads as
masked, so each masked point can read from the unchanged mask whether gemmi
would have marked it - the same mask bit for bit. The gridding test now also
runs a 0.5 A grid, where both rings of the shrink are active, against gemmi's
mask (5 groups, 1 and 4 threads). Sub-stage timers (maps.*, missing.*,
fmask.*) stay.

cytochrome C live refine: refine.maps 0.37-0.44 -> 0.28-0.35 s, fmask.put 0.22
-> 0.12 s. R-free identical on the 22-merge guard. The FFT is left serial: a
parallel decomposition of the r2c transform is not bit-identical to FFTW's 3D
plan (4e-10 of the largest coefficient, measured) and the island removal is a
serial flood fill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdJDrM65gxsCn2PrbfK
This commit is contained in:
leonarski_fandClaude Fable 5.1 committed 2026-11-14 13:12:15 +01:00
1 parent a9c009bbe0
commit 2d6f93f4ca
4 files changed
+86 -10

No files matched your search

@@ -467,12 +467,70 @@ void PutModelDensityOnGrid(gemmi::DensityCalculator<Table, float> &dc, const gem
SymmetrizeOrbits(dc.grid, orbit_leaders, nthreads, [](float a, float b) { return a + b; });
}
// SolventMasker(AtomicRadiiSet::Refmac).put_mask_on_grid() (gemmi solmask.hpp) with the atoms and the
// symmetry spread over the threads as above - the same mask, bit for bit. The island removal and the
// shrink that follow are gemmi's own, except that the shrink is skipped where it cannot change a point:
// it looks at the grid offsets within rshrink of each point (set_margin_around), and on a grid whose
// spacing is coarser than rshrink along all three axes there are none - which is most rigid-body zones -
// yet it still walks the whole grid.
namespace {
// gemmi's SolventMasker::shrink() - set_margin_around(r, 1, -1), then the margin set to 1 - with the planes of the
// grid spread over the threads, the same mask bit for bit. gemmi walks the unmasked points and marks the masked
// points within r of each: those among the nearest neighbours, and, where one of those was masked, the rest.
// Every mark is the same store on a point that was masked, and a marked point still reads as masked, so the order
// of the marks does not matter: here each masked point reads, from the mask as it was, whether gemmi would have
// marked it - an unmasked nearest neighbour, or an unmasked point further out within r that has a masked nearest
// neighbour of its own (the offsets are symmetric, so the points that reach q are the points q reaches).
void ShrinkMask(gemmi::Grid<float> &grid, double r, size_t nthreads) {
// the two rings of offsets, as set_margin_around() builds them: a positive offset stored wrapped (u - nu), so
// that a point's coordinate plus an offset stays within [-n, n), which is what index_near_zero() takes
const int du = static_cast<int>(std::floor(r / grid.spacing[0])), dv = static_cast<int>(std::floor(r / grid.spacing[1])),
dw = static_cast<int>(std::floor(r / grid.spacing[2]));
if (2 * du >= grid.nu || 2 * dv >= grid.nv || 2 * dw >= grid.nw)
gemmi::fail("grid operation failed: radius bigger than half the unit cell?");
const double max_spacing2 = gemmi::sq(std::max({grid.unit_cell.a / grid.nu, grid.unit_cell.b / grid.nv,
grid.unit_cell.c / grid.nw})) + 1e-6;
std::vector<std::array<int, 3>> near, far;
for (int w = -dw; w <= dw; ++w)
for (int v = -dv; v <= dv; ++v)
for (int u = -du; u <= du; ++u) {
const double r2 = grid.unit_cell.orthogonalize_difference(grid.get_fractional(u, v, w)).length_sq();
if (r2 <= r * r && r2 != 0.)
(r2 < max_spacing2 ? near : far).push_back({u <= 0 ? u : u - grid.nu, v <= 0 ? v : v - grid.nv,
w <= 0 ? w : w - grid.nw});
}
const std::vector<float> &in = grid.data;
const auto unmasked = [&](int u, int v, int w, const std::array<int, 3> &d) {
return in[grid.index_near_zero(u + d[0], v + d[1], w + d[2])] >= 1.0f;
};
std::vector<float> out(in.size());
ParallelChunks(grid.nw, nthreads, [&](int lo, int hi) {
for (int w = lo; w < hi; ++w)
for (int v = 0; v < grid.nv; ++v)
for (int u = 0; u < grid.nu; ++u) {
const size_t i = grid.index_q(u, v, w);
out[i] = in[i];
if (in[i] >= 1.0f)
continue;
bool margin = std::any_of(near.begin(), near.end(), [&](const auto &d) { return unmasked(u, v, w, d); });
for (size_t k = 0; k < far.size() && !margin; ++k) {
if (!unmasked(u, v, w, far[k]))
continue;
int pu = u + far[k][0], pv = v + far[k][1], pw = w + far[k][2]; // back into [0, n)
pu += pu < 0 ? grid.nu : 0;
pv += pv < 0 ? grid.nv : 0;
pw += pw < 0 ? grid.nw : 0;
margin = std::any_of(near.begin(), near.end(), [&](const auto &d) { return !unmasked(pu, pv, pw, d); });
}
if (margin)
out[i] = 1.0f;
}
});
grid.data = std::move(out);
}
} // namespace
// SolventMasker(AtomicRadiiSet::Refmac).put_mask_on_grid() (gemmi solmask.hpp) with the atoms, the
// symmetry and the shrink spread over the threads - the same mask, bit for bit. The island removal is
// gemmi's own. The shrink is skipped where it cannot change a point: it looks at the grid offsets within
// rshrink of each point (set_margin_around), and on a grid whose spacing is coarser than rshrink along all
// three axes there are none - which is most rigid-body zones - yet gemmi's still walks the whole grid.
void PutMaskOnGrid(gemmi::Grid<float> &grid, const gemmi::Model &model, const std::vector<size_t> &orbit_leaders,
size_t nthreads) {
const gemmi::SolventMasker masker(gemmi::AtomicRadiiSet::Refmac);
@@ -498,7 +556,7 @@ void PutMaskOnGrid(gemmi::Grid<float> &grid, const gemmi::Model &model, const st
for (int i = 0; i < 3; i++)
shrink_has_offsets = shrink_has_offsets || static_cast<int>(std::floor(masker.rshrink / grid.spacing[i])) > 0;
if (shrink_has_offsets)
masker.shrink(grid);
ShrinkMask(grid, masker.rshrink, nthreads);
}
gemmi::Grid<float> SolventMaskGrid(const gemmi::UnitCell &cell, const gemmi::SpaceGroup &sg, double d_min,
@@ -1202,8 +1202,14 @@ std::vector<MissingReflection> MissingReflections(Context &ctx, const gemmi::SMa
engine->SetAniso(&ctx.aniso);
// the atoms and the solvent side by side: neither reads the other
std::vector<std::complex<float>> fcalc, fmask;
std::future<void> atoms = std::async(std::launch::async, [&] { engine->Fcalc(ctx.Model(), fcalc); });
ctx.mask->Fmask(ctx.Model(), missing.hkl, fmask);
std::future<void> atoms = std::async(std::launch::async, [&] {
const RefinementStage s("missing.fcalc");
engine->Fcalc(ctx.Model(), fcalc);
});
{
const RefinementStage s("missing.fmask");
ctx.mask->Fmask(ctx.Model(), missing.hkl, fmask);
}
atoms.get();
std::vector<std::pair<double, double>> d_by_s; // (1/d^2, D) of the observed reflections, by resolution
@@ -1235,9 +1241,11 @@ std::vector<MissingReflection> MissingReflections(Context &ctx, const gemmi::SMa
// them, and the difference map is never filled.
void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double wavelength_A, RefinementResult &result) {
const RefinementStage stage("maps");
std::optional<RefinementStage> sub("maps.refit");
ctx.Refit();
const std::vector<std::complex<float>> fmodel = ctx.FModel();
const gemmi::GroupOps gops = ctx.sg->operations();
sub.emplace("maps.rows");
std::vector<double> fom(ctx.refl.Size());
double fom_sum = 0;
int fom_n = 0;
@@ -1299,17 +1307,21 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
fwt_data.v.push_back({ctx.refl.hkl[m], std::polar(static_cast<float>(fwt), static_cast<float>(phi))});
}
result.maps = maps;
sub.emplace("maps.anom");
// The reflections the data do not have, the difference map and the 2mFo-DFc map beside the anomalous one below:
// none reads another.
std::future<std::vector<MissingReflection>> missing = std::async(std::launch::async, [&ctx, &b_aniso] {
const RefinementStage s("maps.missing");
return MissingReflections(ctx, b_aniso);
});
const std::array<int, 3> size = gemmi::get_size_for_hkl(diff, {{0, 0, 0}}, 3.0);
std::future<void> difference_map = std::async(std::launch::async, [&diff, &size, &result] {
const RefinementStage s("maps.diff_map");
result.difference_map = std::make_shared<gemmi::Grid<float>>(MapFromFPhi(gemmi::get_f_phi_on_grid<float>(diff, size, true)));
result.difference_map_rms = gemmi::calculate_data_statistics(result.difference_map->data).rms;
});
std::future<void> map = std::async(std::launch::async, [&fwt_data, &size, &result] {
const RefinementStage s("maps.map");
result.map = std::make_shared<gemmi::Grid<float>>(MapFromFPhi(gemmi::get_f_phi_on_grid<float>(fwt_data, size, true)));
result.map_rms = gemmi::calculate_data_statistics(result.map->data).rms;
});
@@ -1358,6 +1370,7 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
}
}
// The filled reflections after the observed ones: FWT/PHWT and the model's FC/PHIC, every other column missing.
sub.emplace("maps.fill");
const float nan = std::numeric_limits<float>::quiet_NaN();
for (const MissingReflection &r : missing.get()) {
const double phi_deg = std::arg(r.f_model) * 180.0 / PI;
@@ -1368,6 +1381,7 @@ void MakeMaps(Context &ctx, const std::vector<MergedReflection> &merged, double
for (size_t c = 0; c < mtz.columns.size(); c++)
mtz.columns[c].values.push_back(c < std::size(row) ? row[c] : nan);
}
sub.emplace("maps.join");
difference_map.get();
map.get();
}
@@ -556,10 +556,13 @@ void BulkSolventMask::Fmask(const gemmi::Model &model, const std::vector<gemmi::
std::vector<std::complex<float>> &f) const {
if (orbit_leaders_.empty()) // at the first mask: a GPU engine's refinement never draws one here
orbit_leaders_ = OrbitLeaders(grid_, nthreads_);
std::optional<RefinementStage> sub("fmask.put");
gemmi::Grid<float> mask = grid_;
mask.data.resize(mask.point_count());
PutMaskOnGrid(mask, model, orbit_leaders_, nthreads_);
sub.emplace("fmask.fft");
const gemmi::FPhiGrid<float> fm = MapToFPhi(mask);
sub.reset();
f.clear();
for (const gemmi::Miller &h : hkl)
f.push_back(fm.get_value_by_hkl(h));
+2 -1
View File
@@ -1340,7 +1340,8 @@ TEST_CASE("ModelValidation_ParallelGriddingMatchesGemmi", "[ModelValidation]") {
return false;
return true;
};
for (double d_min : {6.0, 3.5}) {
// 1.5 A: a 0.5 A grid, where the mask's 0.8 A shrink reaches both of its rings of neighbours
for (double d_min : {6.0, 3.5, 1.5}) {
gemmi::DensityCalculator<gemmi::IT92<float>, float> gemmi_dc;
gemmi_dc.d_min = d_min;
gemmi_dc.rate = 1.5;