Files
Jungfraujoch/tests/AtomGradientMetalTest.cpp
Filip LeonarskiandClaude Opus 5.5 88330f395f Refinement gradient on the Apple GPU: the sums over the derivative map in Metal
RESULT-CHANGING (ulp level; owner approved for Metal ports where exactness costs speed)

AtomGradientsFromMap() takes a `gpu` flag; the CPU engine sets it, and on macOS the sums of every
atom's box against the FFT method's map then run on the Apple GPU (AtomGradientMetal.mm/.metal):
one SIMD group of 32 threads per atom, the lanes taking the box's points in turn, as
RefinementGPU.cu's atom_gradient_kernel does under CUDA. Each atom's box, radius and Gaussians are
the CPU's (built on the host as the CPU loops build them), and a point's place and the radius test
are spelled as GatherMapPoints() spells them with fast math and contraction off, so the GPU takes
the same points; the sums are in single precision and in another order, so the gradient is not the
CPU's bit for bit. Where the GPU does not take it (rugnux --no-gpu, no device, a box wider than the
cell) or fails (one warning, then the CPU from there on) the CPU loops run as before.

Parity (AtomGradientMetal_MatchesCPU, [portable][Metal], 2000 atoms, a quarter anisotropic, P1
triclinic and P212121): worst |GPU - CPU| over the largest |CPU| of that derivative 0.6-1.0e-6 for
x, y, z, B and 2.1-7.6e-6 for U; held to 1e-5 and 5e-5.

M6 Mac mini, rugnux --mode refine, an open-arm set (9zlo, its merged MTZ and refined model):
90.6 s -> 45.8 s wall; R-work 0.2697 both, R-free 0.2936 -> 0.2938. With --no-gpu the refined model
is byte-identical to the build before (115 s: --no-gpu also takes the density gather off the GPU).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UqXYdJDrM65gxsCn2PrbfK
2026-11-16 07:21:12 +01:00

103 lines
4.5 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#ifdef __APPLE__
#include <cmath>
#include <random>
#include <vector>
#include <gemmi/model.hpp>
#include "../image_analysis/structure_refinement/AtomGradientMetal.h"
#include "../image_analysis/structure_refinement/RefinementXray.h"
#include "../image_analysis/structure_refinement/RigidBodyRefine.h"
// The refinement gradient's sums on the Apple GPU (AtomGradientMetal.h) against the CPU's: the same points, summed
// in single precision on the GPU and in double on the CPU. A Mac without a Metal device fails rather than skips, as
// in ModelDensityMetalTest.cpp.
namespace {
// Atoms scattered over the whole cell - so that boxes wrap at every face - of a few elements, a quarter of them
// anisotropic, with occupancies below one.
gemmi::Model random_atoms(const gemmi::UnitCell &cell) {
std::mt19937 rng(17);
std::uniform_real_distribution<double> frac(0.0, 1.0), b(8.0, 60.0), occ(0.3, 1.0), u(0.05, 0.4);
const gemmi::El elements[] = {gemmi::El::C, gemmi::El::N, gemmi::El::O, gemmi::El::S};
gemmi::Model model(1);
gemmi::Chain chain("A");
gemmi::Residue res;
res.name = "UNK";
res.seqid = gemmi::SeqId(1, ' ');
for (int i = 0; i < 2000; i++) {
gemmi::Atom a;
a.name = "X";
a.element = elements[i % 4];
a.pos = cell.orthogonalize(gemmi::Fractional(frac(rng), frac(rng), frac(rng)));
a.b_iso = static_cast<float>(b(rng));
a.occ = static_cast<float>(occ(rng));
if (i % 4 == 3) {
const float d1 = static_cast<float>(u(rng)), d2 = static_cast<float>(u(rng)),
d3 = static_cast<float>(u(rng));
a.aniso = {d1, d2, d3, 0.3f * std::min(d1, d2), -0.2f * std::min(d1, d3), 0.1f * std::min(d2, d3)};
}
res.atoms.push_back(a);
}
chain.residues.push_back(res);
model.chains.push_back(chain);
return model;
}
// The largest difference of a derivative between the two, over the largest of that derivative on the CPU.
template <size_t N>
double worst(const std::vector<std::array<double, N>> &cpu, const std::vector<std::array<double, N>> &gpu) {
double out = 0;
for (size_t c = 0; c < N; c++) {
double max_g = 0, max_d = 0;
for (size_t i = 0; i < cpu.size(); i++) {
max_g = std::max(max_g, std::fabs(cpu[i][c]));
max_d = std::max(max_d, std::fabs(gpu[i][c] - cpu[i][c]));
}
out = std::max(out, max_d / max_g);
}
return out;
}
void check_gradient(const gemmi::UnitCell &cell, const char *group, double d_min) {
const gemmi::SpaceGroup &sg = *gemmi::find_spacegroup_by_name(group);
gemmi::Grid<float> map = RigidBodyZoneGrid(cell, sg, d_min);
map.data.resize(map.point_count());
// a smooth map with both signs, as the FFT method's derivative map is
for (int w = 0; w < map.nw; w++)
for (int v = 0; v < map.nv; v++)
for (int u = 0; u < map.nu; u++)
map.data[map.index_q(u, v, w)] = static_cast<float>(
std::sin(2 * M_PI * (3.0 * u / map.nu + 1.0 * v / map.nv)) +
std::cos(2 * M_PI * (2.0 * v / map.nv - 4.0 * w / map.nw)) +
0.5 * std::sin(2 * M_PI * (5.0 * u / map.nu + 3.0 * w / map.nw)));
const gemmi::Model model = random_atoms(cell);
const double blur = 10.0;
std::vector<std::array<double, 4>> gc, gg;
std::vector<std::array<double, 6>> uc, ug;
AtomGradientsFromMap(map, model, blur, 4, gc, &uc, false);
const size_t before = AtomGradientMetalCalls();
AtomGradientsFromMap(map, model, blur, 4, gg, &ug, true);
REQUIRE(AtomGradientMetalCalls() == before + 1); // the GPU summed it
REQUIRE(gg.size() == gc.size());
const double g = worst(gc, gg), u = worst(uc, ug);
CAPTURE(group, g, u);
CHECK(g < 1e-5);
CHECK(u < 5e-5);
}
}
TEST_CASE("AtomGradientMetal_MatchesCPU", "[portable][Metal]") {
// Triclinic, so that every term of the orthogonalization is there; then an orthorhombic group.
check_gradient(gemmi::UnitCell(41, 46, 52, 81, 96, 107), "P 1", 2.0);
check_gradient(gemmi::UnitCell(50, 56, 63, 90, 90, 90), "P 21 21 21", 1.6);
}
#endif