Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 16m43s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 28m46s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 32m5s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 39m44s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 5m7s
Build Packages / Create release (push) Successful in 8s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 22m30s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 31m54s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 11m29s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 23m51s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 26m7s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 29m38s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 28m6s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 29m44s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 33m22s
Build Packages / Generate python client (push) Successful in 47s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 34m6s
Build Packages / Unit tests (push) Successful in 1h2m47s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 35m59s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 34m8s
Build Packages / Validation Battery (push) Successful in 38m31s
* Rugnux: Refinement several times faster, one protocol on the CPU and the GPU, parts of it on the Apple GPU; its random draws are the same on every platform. * Rugnux: Refinement follows common practice: the free set decides only the restraint weights and the displacement model (isotropic, TLS or anisotropic, by Hamilton's test), everything else is decided on the working set; lower R-free for placed and predicted models. * Rugnux: Restraint dictionaries via `--restraints` (or among the inputs) and more built-in ligands; a model whose space group is a subgroup of the data's is refined in its own group; reflections plus a model select `--mode refine` on their own. * Rugnux: `-C` fixes the crystal system on the axes as typed; `-S` alone fixes the space group only, the cell is still found and reduced; a space group or cell stored in the input file is ignored, with a line in the log. * Rugnux: More reliable multi-sweep merging; `--mode scale` merges several stills runs as one; `_plot.txt` and the report name each input; a second crystal form is reported for stills given a cell. * jfjoch_broker (experimental): Live Rugnux on FPGA detectors (JUNGFRAU) takes the same path as a DECTRIS stream, rotation and serial alike, model step included, with analysis threads per GPU; frames waiting for the analysis can be viewed; no host stage drops a frame - a source faster than the analysis is held at the card. * jfjoch_broker: Summation of more than 256 frames per image is split between the card and the host (`summation_mode` is then `hybrid`), and live Rugnux runs on such collections; pedestal runs take a receiver of their own. * jfjoch_broker (experimental): `stills_downsample` analyses 1 of every N serial images (chosen by a hash; the first 2000 all; the others written without analysis; reset to 1 by `/initialize`); `max_spot_count` left out means 250 for per-image indexing and up to 1000 for live stills; the live status gives the merge before the model work, the fitted resolution and the anomalous peak height of each unexplained blob. * jfjoch_broker: Without a `pcie` entry the broker runs on a mock of the card instead of the HLS simulation; its spot, ROI and azimuthal results are zero then. * Web frontend: Redesigned Rugnux live page; Vite 8, TypeScript 6, Mol* 5.13 and dependency security updates. * Tools: `jfjoch_fpga_stills_test` measures live stills on the cards with a stored dataset; `jfjoch_fpga_test` loses the HLS-simulation option. * Other: FFTW no longer fetched or linked (the CPU FFT indexer keeps the name `FFTW`); PCIe driver now GPL-2.0-or-later and third-party notices complete; several crash fixes. Reviewed-on: #98 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
159 lines
7.8 KiB
C++
159 lines
7.8 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include "DensityBlobs.h"
|
|
|
|
#include <algorithm>
|
|
#include <cmath>
|
|
|
|
#include "gemmi/blob.hpp"
|
|
#include "gemmi/neighbor.hpp"
|
|
|
|
namespace {
|
|
|
|
constexpr double CUTOFF_SIGMA = 3.0;
|
|
constexpr double PEAK_SIGMA = 4.0;
|
|
// Closer than this to a model atom, the density is the model's own atom out of place.
|
|
constexpr double MODEL_ERROR_DISTANCE = 1.6;
|
|
// Waters and ions sit at least this far from the model's atoms.
|
|
constexpr double CONTACT_MIN = 2.2;
|
|
// An ion's peak, in sigma, where the anomalous map gives no help (with two metal-like contacts); an anomalous
|
|
// peak this high says heavier than water on its own.
|
|
constexpr double ION_PEAK_SIGMA = 8.0;
|
|
constexpr double ION_ANOMALOUS_SIGMA = 4.0;
|
|
// The smallest blob called ligand-like, A^3, and at low resolution one and a half of the resolution's spheres: at
|
|
// high resolution a strong single atom's blob already spans several of them.
|
|
constexpr double LIGAND_MIN_VOLUME = 8.0;
|
|
// How far from the difference map's peak the anomalous map's peak is looked for, A. The two maps are on different
|
|
// grids (each sampled at a third of its resolution, up to ~0.7 A apart at 2 A) and the anomalous scatterer need
|
|
// not be the difference density's highest point, so a reading at the peak alone can miss it; half the shortest
|
|
// metal-ligand bond (~2.0 A) keeps a neighbouring modelled sulfur or metal from being read as the blob's own.
|
|
constexpr double ANOMALOUS_PEAK_RADIUS = 1.0;
|
|
|
|
// The anomalous map's highest point within ANOMALOUS_PEAK_RADIUS of p, in its rms.
|
|
double AnomalousPeak(const gemmi::Grid<float> &anomalous, double rms, const gemmi::Position &p) {
|
|
double peak = anomalous.interpolate_value(p);
|
|
const gemmi::Fractional f = anomalous.unit_cell.fractionalize(p);
|
|
const int u0 = static_cast<int>(std::round(f.x * anomalous.nu));
|
|
const int v0 = static_cast<int>(std::round(f.y * anomalous.nv));
|
|
const int w0 = static_cast<int>(std::round(f.z * anomalous.nw));
|
|
const int du = static_cast<int>(std::ceil(ANOMALOUS_PEAK_RADIUS / anomalous.spacing[0]));
|
|
const int dv = static_cast<int>(std::ceil(ANOMALOUS_PEAK_RADIUS / anomalous.spacing[1]));
|
|
const int dw = static_cast<int>(std::ceil(ANOMALOUS_PEAK_RADIUS / anomalous.spacing[2]));
|
|
for (int u = u0 - du; u <= u0 + du; u++)
|
|
for (int v = v0 - dv; v <= v0 + dv; v++)
|
|
for (int w = w0 - dw; w <= w0 + dw; w++)
|
|
if (anomalous.get_position(u, v, w).dist(p) <= ANOMALOUS_PEAK_RADIUS)
|
|
peak = std::max(peak, static_cast<double>(anomalous.get_value(u, v, w)));
|
|
return peak / rms;
|
|
}
|
|
|
|
bool Polar(const gemmi::Element &el) {
|
|
return el == gemmi::El::N || el == gemmi::El::O;
|
|
}
|
|
|
|
} // namespace
|
|
|
|
const char *DensityBlobKindName(DensityBlob::Kind kind) {
|
|
switch (kind) {
|
|
case DensityBlob::Kind::Water: return "WATER";
|
|
case DensityBlob::Kind::Ion: return "ION";
|
|
case DensityBlob::Kind::Ligand: return "LIGAND";
|
|
case DensityBlob::Kind::ModelError: return "MODEL_ERROR";
|
|
case DensityBlob::Kind::Unassigned: break;
|
|
}
|
|
return "UNASSIGNED";
|
|
}
|
|
|
|
DensityBlobsMessage DensityBlobsToMessage(const std::vector<DensityBlob> &blobs) {
|
|
DensityBlobsMessage m;
|
|
for (const DensityBlob &b : blobs) {
|
|
m.kind.emplace_back(DensityBlobKindName(b.kind));
|
|
m.x.push_back(b.centroid.x);
|
|
m.y.push_back(b.centroid.y);
|
|
m.z.push_back(b.centroid.z);
|
|
m.peak_sigma.push_back(b.peak_sigma);
|
|
m.volume.push_back(b.volume);
|
|
m.score.push_back(b.score);
|
|
m.nearest_atom.push_back(b.nearest_atom);
|
|
m.nearest_atom_distance.push_back(b.nearest_atom_distance);
|
|
m.polar_contacts.push_back(b.polar_contacts);
|
|
m.metal_contacts.push_back(b.metal_contacts);
|
|
m.anomalous_sigma.push_back(b.anomalous_sigma);
|
|
}
|
|
return m;
|
|
}
|
|
|
|
// Following Coot's unmodelled blobs, Emsley et al. (2010) Acta Cryst. D66, 486-501
|
|
std::vector<DensityBlob> FindDensityBlobs(const gemmi::Grid<float> &difference, double rms, const gemmi::Structure &model,
|
|
double d_min, const gemmi::Grid<float> *anomalous, double anomalous_rms) {
|
|
std::vector<DensityBlob> out;
|
|
if (!(rms > 0) || model.models.empty())
|
|
return out;
|
|
gemmi::Grid<float> map = difference;
|
|
for (float &v : map.data)
|
|
v = static_cast<float>(v / rms);
|
|
gemmi::BlobCriteria criteria;
|
|
criteria.cutoff = CUTOFF_SIGMA;
|
|
criteria.min_volume = 0.0;
|
|
criteria.min_score = 0.0;
|
|
criteria.min_peak = PEAK_SIGMA;
|
|
const std::vector<gemmi::Blob> blobs = gemmi::find_blobs_by_flood_fill(map, criteria);
|
|
|
|
gemmi::Model copy = model.models[0];
|
|
gemmi::UnitCell cell = map.unit_cell;
|
|
cell.set_cell_images_from_spacegroup(map.spacegroup);
|
|
gemmi::NeighborSearch ns(copy, cell, 5.0);
|
|
ns.populate(false);
|
|
// A single atom's blob at this resolution: a sphere of half the resolution's radius.
|
|
const double atom_volume = 4.0 / 3.0 * 3.14159265358979 * std::pow(0.5 * d_min, 3);
|
|
|
|
for (const gemmi::Blob &b : blobs) {
|
|
DensityBlob d;
|
|
d.peak_sigma = b.peak_value;
|
|
d.volume = b.volume;
|
|
d.score = b.score;
|
|
d.centroid = b.centroid;
|
|
// the copy of the blob beside the atom nearest to it
|
|
if (gemmi::NeighborSearch::Mark *m = ns.find_nearest_atom(b.centroid, 12.0)) {
|
|
const gemmi::CRA cra = m->to_cra(copy);
|
|
const gemmi::FTransform t = ns.get_image_transformation(m->image_idx);
|
|
const gemmi::Fractional fc = cell.fractionalize(b.centroid);
|
|
const gemmi::Fractional fi = t.apply(cell.fractionalize(cra.atom->pos));
|
|
const gemmi::Fractional shifted(fc.x - std::round(fc.x - fi.x), fc.y - std::round(fc.y - fi.y),
|
|
fc.z - std::round(fc.z - fi.z));
|
|
d.centroid = cell.orthogonalize(gemmi::FTransform(t.inverse()).apply(shifted));
|
|
d.nearest_atom_distance = d.centroid.dist(cra.atom->pos);
|
|
d.nearest_atom = cra.atom->name + " " + cra.residue->name + " " + cra.chain->name +
|
|
cra.residue->seqid.str();
|
|
} else
|
|
d.nearest_atom_distance = INFINITY;
|
|
// The flood fill can find a blob that crosses the asymmetric unit's boundary again from a symmetry mate;
|
|
// beside the model the copies coincide, and the first (strongest) is kept.
|
|
if (std::any_of(out.begin(), out.end(), [&](const DensityBlob &o) { return o.centroid.dist(d.centroid) < 1.5; }))
|
|
continue;
|
|
ns.for_each(d.centroid, '\0', 3.4, [&](gemmi::NeighborSearch::Mark &m, double dist_sq) {
|
|
const double r = std::sqrt(dist_sq);
|
|
if (Polar(m.element) && r >= 2.4)
|
|
d.polar_contacts++;
|
|
if ((Polar(m.element) || m.element == gemmi::El::S) && r >= 1.8 && r <= 2.6)
|
|
d.metal_contacts++;
|
|
});
|
|
if (anomalous != nullptr && anomalous_rms > 0)
|
|
d.anomalous_sigma = AnomalousPeak(*anomalous, anomalous_rms, b.peak_pos);
|
|
|
|
if (d.nearest_atom_distance < MODEL_ERROR_DISTANCE)
|
|
d.kind = DensityBlob::Kind::ModelError;
|
|
else if (d.nearest_atom_distance >= 1.8 &&
|
|
(d.anomalous_sigma >= ION_ANOMALOUS_SIGMA || (d.metal_contacts >= 2 && d.peak_sigma >= ION_PEAK_SIGMA)))
|
|
d.kind = DensityBlob::Kind::Ion;
|
|
else if (d.volume > std::max(1.5 * atom_volume, LIGAND_MIN_VOLUME) && d.nearest_atom_distance >= CONTACT_MIN)
|
|
d.kind = DensityBlob::Kind::Ligand;
|
|
else if (d.nearest_atom_distance >= CONTACT_MIN && d.polar_contacts >= 1)
|
|
d.kind = DensityBlob::Kind::Water;
|
|
out.push_back(d);
|
|
}
|
|
std::sort(out.begin(), out.end(), [](const DensityBlob &a, const DensityBlob &b) { return a.score > b.score; });
|
|
return out;
|
|
}
|