Files
leonarski_f 4809c1622c
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 28m46s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 32m5s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 39m44s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m7s
Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m30s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 31m54s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m29s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 23m51s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 26m7s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m38s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 28m6s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m44s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 33m22s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 34m6s
Build Packages / Unit tests (push) Successful in 1h2m47s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 35m59s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 34m8s
Build Packages / Validation Battery (push) Successful in 38m31s
v1.0.0-rc.187 (#98)
* Rugnux: Refinement several times faster, one protocol on the CPU and the GPU, parts of it on the Apple GPU; its random draws are the same on every platform.
* Rugnux: Refinement follows common practice: the free set decides only the restraint weights and the displacement model (isotropic, TLS or anisotropic, by Hamilton's test), everything else is decided on the working set; lower R-free for placed and predicted models.
* Rugnux: Restraint dictionaries via `--restraints` (or among the inputs) and more built-in ligands; a model whose space group is a subgroup of the data's is refined in its own group; reflections plus a model select `--mode refine` on their own.
* Rugnux: `-C` fixes the crystal system on the axes as typed; `-S` alone fixes the space group only, the cell is still found and reduced; a space group or cell stored in the input file is ignored, with a line in the log.
* Rugnux: More reliable multi-sweep merging; `--mode scale` merges several stills runs as one; `_plot.txt` and the report name each input; a second crystal form is reported for stills given a cell.
* jfjoch_broker (experimental): Live Rugnux on FPGA detectors (JUNGFRAU) takes the same path as a DECTRIS stream, rotation and serial alike, model step included, with analysis threads per GPU; frames waiting for the analysis can be viewed; no host stage drops a frame - a source faster than the analysis is held at the card.
* jfjoch_broker: Summation of more than 256 frames per image is split between the card and the host (`summation_mode` is then `hybrid`), and live Rugnux runs on such collections; pedestal runs take a receiver of their own.
* jfjoch_broker (experimental): `stills_downsample` analyses 1 of every N serial images (chosen by a hash; the first 2000 all; the others written without analysis; reset to 1 by `/initialize`); `max_spot_count` left out means 250 for per-image indexing and up to 1000 for live stills; the live status gives the merge before the model work, the fitted resolution and the anomalous peak height of each unexplained blob.
* jfjoch_broker: Without a `pcie` entry the broker runs on a mock of the card instead of the HLS simulation; its spot, ROI and azimuthal results are zero then.
* Web frontend: Redesigned Rugnux live page; Vite 8, TypeScript 6, Mol* 5.13 and dependency security updates.
* Tools: `jfjoch_fpga_stills_test` measures live stills on the cards with a stored dataset; `jfjoch_fpga_test` loses the HLS-simulation option.
* Other: FFTW no longer fetched or linked (the CPU FFT indexer keeps the name `FFTW`); PCIe driver now GPL-2.0-or-later and third-party notices complete; several crash fixes.

Reviewed-on: #98
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-11-19 20:49:28 +01:00

159 lines
7.8 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include "DensityBlobs.h"
#include <algorithm>
#include <cmath>
#include "gemmi/blob.hpp"
#include "gemmi/neighbor.hpp"
namespace {
constexpr double CUTOFF_SIGMA = 3.0;
constexpr double PEAK_SIGMA = 4.0;
// Closer than this to a model atom, the density is the model's own atom out of place.
constexpr double MODEL_ERROR_DISTANCE = 1.6;
// Waters and ions sit at least this far from the model's atoms.
constexpr double CONTACT_MIN = 2.2;
// An ion's peak, in sigma, where the anomalous map gives no help (with two metal-like contacts); an anomalous
// peak this high says heavier than water on its own.
constexpr double ION_PEAK_SIGMA = 8.0;
constexpr double ION_ANOMALOUS_SIGMA = 4.0;
// The smallest blob called ligand-like, A^3, and at low resolution one and a half of the resolution's spheres: at
// high resolution a strong single atom's blob already spans several of them.
constexpr double LIGAND_MIN_VOLUME = 8.0;
// How far from the difference map's peak the anomalous map's peak is looked for, A. The two maps are on different
// grids (each sampled at a third of its resolution, up to ~0.7 A apart at 2 A) and the anomalous scatterer need
// not be the difference density's highest point, so a reading at the peak alone can miss it; half the shortest
// metal-ligand bond (~2.0 A) keeps a neighbouring modelled sulfur or metal from being read as the blob's own.
constexpr double ANOMALOUS_PEAK_RADIUS = 1.0;
// The anomalous map's highest point within ANOMALOUS_PEAK_RADIUS of p, in its rms.
double AnomalousPeak(const gemmi::Grid<float> &anomalous, double rms, const gemmi::Position &p) {
double peak = anomalous.interpolate_value(p);
const gemmi::Fractional f = anomalous.unit_cell.fractionalize(p);
const int u0 = static_cast<int>(std::round(f.x * anomalous.nu));
const int v0 = static_cast<int>(std::round(f.y * anomalous.nv));
const int w0 = static_cast<int>(std::round(f.z * anomalous.nw));
const int du = static_cast<int>(std::ceil(ANOMALOUS_PEAK_RADIUS / anomalous.spacing[0]));
const int dv = static_cast<int>(std::ceil(ANOMALOUS_PEAK_RADIUS / anomalous.spacing[1]));
const int dw = static_cast<int>(std::ceil(ANOMALOUS_PEAK_RADIUS / anomalous.spacing[2]));
for (int u = u0 - du; u <= u0 + du; u++)
for (int v = v0 - dv; v <= v0 + dv; v++)
for (int w = w0 - dw; w <= w0 + dw; w++)
if (anomalous.get_position(u, v, w).dist(p) <= ANOMALOUS_PEAK_RADIUS)
peak = std::max(peak, static_cast<double>(anomalous.get_value(u, v, w)));
return peak / rms;
}
bool Polar(const gemmi::Element &el) {
return el == gemmi::El::N || el == gemmi::El::O;
}
} // namespace
const char *DensityBlobKindName(DensityBlob::Kind kind) {
switch (kind) {
case DensityBlob::Kind::Water: return "WATER";
case DensityBlob::Kind::Ion: return "ION";
case DensityBlob::Kind::Ligand: return "LIGAND";
case DensityBlob::Kind::ModelError: return "MODEL_ERROR";
case DensityBlob::Kind::Unassigned: break;
}
return "UNASSIGNED";
}
DensityBlobsMessage DensityBlobsToMessage(const std::vector<DensityBlob> &blobs) {
DensityBlobsMessage m;
for (const DensityBlob &b : blobs) {
m.kind.emplace_back(DensityBlobKindName(b.kind));
m.x.push_back(b.centroid.x);
m.y.push_back(b.centroid.y);
m.z.push_back(b.centroid.z);
m.peak_sigma.push_back(b.peak_sigma);
m.volume.push_back(b.volume);
m.score.push_back(b.score);
m.nearest_atom.push_back(b.nearest_atom);
m.nearest_atom_distance.push_back(b.nearest_atom_distance);
m.polar_contacts.push_back(b.polar_contacts);
m.metal_contacts.push_back(b.metal_contacts);
m.anomalous_sigma.push_back(b.anomalous_sigma);
}
return m;
}
// Following Coot's unmodelled blobs, Emsley et al. (2010) Acta Cryst. D66, 486-501
std::vector<DensityBlob> FindDensityBlobs(const gemmi::Grid<float> &difference, double rms, const gemmi::Structure &model,
double d_min, const gemmi::Grid<float> *anomalous, double anomalous_rms) {
std::vector<DensityBlob> out;
if (!(rms > 0) || model.models.empty())
return out;
gemmi::Grid<float> map = difference;
for (float &v : map.data)
v = static_cast<float>(v / rms);
gemmi::BlobCriteria criteria;
criteria.cutoff = CUTOFF_SIGMA;
criteria.min_volume = 0.0;
criteria.min_score = 0.0;
criteria.min_peak = PEAK_SIGMA;
const std::vector<gemmi::Blob> blobs = gemmi::find_blobs_by_flood_fill(map, criteria);
gemmi::Model copy = model.models[0];
gemmi::UnitCell cell = map.unit_cell;
cell.set_cell_images_from_spacegroup(map.spacegroup);
gemmi::NeighborSearch ns(copy, cell, 5.0);
ns.populate(false);
// A single atom's blob at this resolution: a sphere of half the resolution's radius.
const double atom_volume = 4.0 / 3.0 * 3.14159265358979 * std::pow(0.5 * d_min, 3);
for (const gemmi::Blob &b : blobs) {
DensityBlob d;
d.peak_sigma = b.peak_value;
d.volume = b.volume;
d.score = b.score;
d.centroid = b.centroid;
// the copy of the blob beside the atom nearest to it
if (gemmi::NeighborSearch::Mark *m = ns.find_nearest_atom(b.centroid, 12.0)) {
const gemmi::CRA cra = m->to_cra(copy);
const gemmi::FTransform t = ns.get_image_transformation(m->image_idx);
const gemmi::Fractional fc = cell.fractionalize(b.centroid);
const gemmi::Fractional fi = t.apply(cell.fractionalize(cra.atom->pos));
const gemmi::Fractional shifted(fc.x - std::round(fc.x - fi.x), fc.y - std::round(fc.y - fi.y),
fc.z - std::round(fc.z - fi.z));
d.centroid = cell.orthogonalize(gemmi::FTransform(t.inverse()).apply(shifted));
d.nearest_atom_distance = d.centroid.dist(cra.atom->pos);
d.nearest_atom = cra.atom->name + " " + cra.residue->name + " " + cra.chain->name +
cra.residue->seqid.str();
} else
d.nearest_atom_distance = INFINITY;
// The flood fill can find a blob that crosses the asymmetric unit's boundary again from a symmetry mate;
// beside the model the copies coincide, and the first (strongest) is kept.
if (std::any_of(out.begin(), out.end(), [&](const DensityBlob &o) { return o.centroid.dist(d.centroid) < 1.5; }))
continue;
ns.for_each(d.centroid, '\0', 3.4, [&](gemmi::NeighborSearch::Mark &m, double dist_sq) {
const double r = std::sqrt(dist_sq);
if (Polar(m.element) && r >= 2.4)
d.polar_contacts++;
if ((Polar(m.element) || m.element == gemmi::El::S) && r >= 1.8 && r <= 2.6)
d.metal_contacts++;
});
if (anomalous != nullptr && anomalous_rms > 0)
d.anomalous_sigma = AnomalousPeak(*anomalous, anomalous_rms, b.peak_pos);
if (d.nearest_atom_distance < MODEL_ERROR_DISTANCE)
d.kind = DensityBlob::Kind::ModelError;
else if (d.nearest_atom_distance >= 1.8 &&
(d.anomalous_sigma >= ION_ANOMALOUS_SIGMA || (d.metal_contacts >= 2 && d.peak_sigma >= ION_PEAK_SIGMA)))
d.kind = DensityBlob::Kind::Ion;
else if (d.volume > std::max(1.5 * atom_volume, LIGAND_MIN_VOLUME) && d.nearest_atom_distance >= CONTACT_MIN)
d.kind = DensityBlob::Kind::Ligand;
else if (d.nearest_atom_distance >= CONTACT_MIN && d.polar_contacts >= 1)
d.kind = DensityBlob::Kind::Water;
out.push_back(d);
}
std::sort(out.begin(), out.end(), [](const DensityBlob &a, const DensityBlob &b) { return a.score > b.score; });
return out;
}