mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 81 Wiki Activity

Merge branch 'rc174-cand' into rc174

Browse Source 
Footprint integration, unmerged-fulls .hkl, SHELXL battery reporting,
data-quality fixes, the error-model refit reverted, 7UDI dropped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
This commit is contained in:
leonarski_f
2026-10-04 12:02:50 +02:00
co-authored by Claude Opus 5.5
parent 1b6429186c e72236b08d
commit b5c97829c0
30 changed files with 902 additions and 114 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CMakeLists.txt
+12
View File
@@ -41,6 +41,18 @@ IF (MSVC)
ADD_COMPILE_OPTIONS($<$<COMPILE_LANGUAGE:CUDA>:-Xcompiler=/Zc:preprocessor>)
ENDIF()
# No contraction of a*b+c into a fused multiply-add. GCC contracts whenever the target has FMA, so a
# -march=x86-64-v3 build (CI, production) and a baseline build rounded differently, and knife-edge
# decisions - which candidate cell a first pass indexes with, a resolution cut, a symmetry gate -
# came out differently from the same source and data. Off, every x86-64 level computes the same bits
# (measured: identical p.mtz on nine battery sets, no measurable time cost), and aarch64, where FMA is
# always there and GCC contracts by default, follows the same arithmetic. MSVC does not contract under
# its default /fp:precise. Host code of .cu sources gets it through nvcc; device code is not affected.
IF (NOT MSVC)
ADD_COMPILE_OPTIONS($<$<COMPILE_LANGUAGE:C,CXX>:-ffp-contract=off>)
ADD_COMPILE_OPTIONS($<$<COMPILE_LANGUAGE:CUDA>:-Xcompiler=-ffp-contract=off>)
ENDIF()
SET(JFJOCH_WRITER_ONLY OFF CACHE BOOL "Compile HDF5 writer only")
SET(JFJOCH_INSTALL_DRIVER_SOURCE OFF CACHE BOOL "Install kernel driver source (ignored if building writer only; necessary for RPM building)")
SET(JFJOCH_USE_CUDA ON CACHE BOOL "Compile Jungfraujoch with CUDA")
common/BraggIntegrationSettings.cpp
+13
View File
@@ -204,3 +204,16 @@ BraggIntegrationSettings &BraggIntegrationSettings::FlightPath(FlightPathMedium
FlightPathMedium BraggIntegrationSettings::GetFlightPath() const {
return flight_path;
}
BraggIntegrationSettings &BraggIntegrationSettings::Footprint(const SpotFootprint &input) {
if (input.sigma_rad.size() != input.sigma_tan.size()
|| input.sigma_rad.size() > static_cast<size_t>(SpotFootprint::MAX_BINS)
|| (!input.empty() && !(input.bin_px > 0.0f)))
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Invalid spot footprint table");
footprint = input;
return *this;
}
const SpotFootprint &BraggIntegrationSettings::GetFootprint() const {
return footprint;
}
common/BraggIntegrationSettings.h
+16
View File
@@ -4,6 +4,7 @@
#pragma once
#include <optional>
#include <vector>
// Spot-intensity extraction method used by the Bragg integration engine. ProfileGaussian (default)
// profile-fits with a measured-width Gaussian (Kabsch-style) - more accurate intensities than the
@@ -34,6 +35,18 @@ enum class FlightPathMedium { Air, Helium, Vacuum };
// the max_hkl default in broker/jfjoch_api.yaml, so an omitting client and an omitting config agree.
constexpr int BRAGG_ONLINE_DEFAULT_MAX_HKL = 100;
// The spot footprint measured on the data (image_analysis/bragg_integration/SpotFootprint.h): the
// radial and tangential standard deviations of a spot [px], tabulated against the distance from the
// beam centre in bins of bin_px. Empty until measured; the integrator then uses the fixed r1/r2/r3
// stencil alone.
struct SpotFootprint {
static constexpr int MAX_BINS = 16;
float bin_px = 0.0f;
std::vector<float> sigma_rad;
std::vector<float> sigma_tan;
[[nodiscard]] bool empty() const { return sigma_rad.empty(); }
};
class BraggIntegrationSettings {
IntegratorMode integrator_mode = IntegratorMode::ProfileGaussian;
float r_1 = 4;
@@ -148,6 +161,7 @@ class BraggIntegrationSettings {
// threshold fitted between two points rather than a physical criterion. The honest design is an
// explicit option, a stated default, and the assumption printed in the report.
FlightPathMedium flight_path = FlightPathMedium::Air;
SpotFootprint footprint;
public:
BraggIntegrationSettings& R1(float input);
@@ -165,6 +179,7 @@ public:
BraggIntegrationSettings& Overlap(OverlapMode input);
BraggIntegrationSettings& OverlapMinPeak(float input);
BraggIntegrationSettings& FlightPath(FlightPathMedium input);
BraggIntegrationSettings& Footprint(const SpotFootprint &input);
[[nodiscard]] IntegratorMode GetIntegrator() const;
@@ -186,4 +201,5 @@ public:
[[nodiscard]] float GetOverlapMinPeak() const;
// See flight_path: air unless the user said otherwise, which no file can say for them.
[[nodiscard]] FlightPathMedium GetFlightPath() const;
[[nodiscard]] const SpotFootprint &GetFootprint() const;
};
docs/CHANGELOG.md
+1
View File
@@ -5,6 +5,7 @@
* Rugnux: `.hkl` now holds unmerged, scaled full reflections (SHELX HKLF 4) on rotation data, so SHELXL computes Rint itself.
* Rugnux reads Rigaku d*TREK SMV images (Saturn CCD), including detector 2theta, image orientation and encoded pixel overflows.
* Rugnux integrates spots that grow wider than the integration disk away from the beam (typical of small molecules at high X-ray energy) over their measured footprint.
### 1.0.0-rc.173
docs/CPU_DATA_ANALYSIS_INTEGRATION.md
+2
View File
@@ -106,6 +106,8 @@ Only the ring moves. The signal disk $r_1$ stays circular, deliberately: it sets
What a circular $r_1$ loses is flux, and that loss is **not** a function of resolution alone: measured per reflection, it carries a directional component worth several Ų with a definite principal axis, on top of the isotropic part. Nor is there anything in the merge to absorb it. There is **no per-shell scale**, and there cannot usefully be one: every scale in §10 is fitted against a reference built from a reflection's own symmetry equivalents, and equivalents share $s^2$ exactly, so any function of $s^2$ lies in the exact null space of the whole scaling model — a per-shell parameter would have zero residual to fit against. (XDS and DIALS have the same null space, for the same reason.) The isotropic part of the loss is instead degenerate with the overall Wilson $B$ and is silently reported as part of it, so **the reported `WILSON_B` / `_reflns.B_iso_Wilson_estimate` carries an $r_1$-dependent contribution**: measured across a constant-ring-area radius sweep it falls monotonically as the disk grows, by 0.5 Ų on sharp strong data and by up to ~10 Ų on weak wide-spot data. What this costs the *data* is much less than what it costs the flux, because most of the loss is matched by a proportional $\sigma$: it moves no CC$_{1/2}$ and no $R_\text{meas}$, and — to within a few hundredths of an ångström — no resolution cut.
**Measured spot footprint (automatic).** The radii above are chosen from spots at 5 Å, which at high X-ray energy sit close to the beam. Away from it a spot can grow several times wider — radially from the sensor's parallax and the obliquity of the incidence, tangentially from the crystal's azimuthal spread, which rotates the diffracted beam about the incident one and smears the spot along its ring. On small-molecule data at 20–25 keV the standard deviation grows from ~1 px near the beam to ~5 px at the detector edge: the $r_1 = 4$ disk holds a quarter of the flux there, the $6\ldots13$ px ring a third of it, and the profile widths learned inside $r_1$ (§9.3) saturate near $r_1^2/4$. So the pre-scan measures every spot it finds with a window that follows the spot — three of its own standard deviations, iterated and re-centred — separately along and across the radius, and tabulates the median widths $\sigma_␍ho,\sigma_ au$ against the distance from the beam. Wherever $3\max(\sigma_␍ho,\sigma_ au)>r_1$ the integrator then (i) starts the background ring at $3\sigma$ along each axis, (ii) sums the reflection over the $r_1$ disk **and** the $3\sigma$ footprint ellipse, so the summation — the profile fit's seed and its fallback — holds the spot rather than its core, and (iii) builds the per-reflection Gaussian at the measured widths on a grid grown to hold them. Where every spot fits the disk nothing is installed and the integration is unchanged bit for bit, which is the case for compact protein spots; like the measured radius, the footprint applies to the canonical pass and not to the geometry pre-pass, and a canonical pass whose wider rings the neighbours starve falls back to the settings without it. Judged by refining the published structures with SHELXL, it removes the intensity loss that grew with resolution on the small-molecule sets (rugnux/model intensity in the outermost shell 0.81–0.91 → 0.98–1.02).
### 9.2 Box summation (seed and fallback)
Let:
docs/EXTERNAL_TEST_DATA.md
-1
View File
@@ -131,7 +131,6 @@ the table below; the repositories themselves are cited in
| [7RJI](https://www.rcsb.org/structure/7RJI) | IRRMC [10.18430/M37RJI](https://doi.org/10.18430/M37RJI) | LNLS W01B-MX2 | 1.71 | H 3 2 | 83.0 83.0 124.8 90.0 90.0 120.0 | PILATUS 2M | BthTX-II variant b, from Bothrops jararacussu venom, complexed with stearic acid |
| [7T5T](https://www.rcsb.org/structure/7T5T) | SBGrid [10.15785/sbgrid/864](https://doi.org/10.15785/sbgrid/864) | SSRL BL9-2 | 1.35 | P 42 21 2 | 95.3 95.3 104.9 90.0 90.0 90.0 | PILATUS 6M | Structure of Thauera sp. K11 CapP |
| [7TCD](https://www.rcsb.org/structure/7TCD) | IRRMC [10.18430/m37tcd](https://doi.org/10.18430/m37tcd) | SLS X06SA | 1.70 | C 1 2 1 | 138.5 47.9 78.1 90.0 107.6 90.0 | Dectris Eiger 16M | LOV2-DARPIN fusion: D13 |
| [7UDI](https://www.rcsb.org/structure/7UDI) | Zenodo [10.5281/zenodo.10022358](https://doi.org/10.5281/zenodo.10022358) | CLSI 08B1-1 | 2.24 | P 41 | 66.7 66.7 129.6 90.0 90.0 90.0 | PILATUS3 S 6M | Full-length dimer of DNA-Damage Response Protein C from Deinococcus radiodurans |
| [7YZX](https://www.rcsb.org/structure/7YZX) | IRRMC [10.18430/M37YZX](https://doi.org/10.18430/M37YZX) | Diamond I24 | 1.90 | P 63 2 2 | 169.4 169.4 141.8 90.0 90.0 120.0 | PILATUS3 6M | ScpA from Streptococcus pyogenes, D783A mutant. |
| [8A1A](https://www.rcsb.org/structure/8A1A) | IRRMC [10.18430/M38A1A](https://doi.org/10.18430/M38A1A) | SLS X06SA | 2.05 | P 65 | 191.9 191.9 122.4 90.0 90.0 120.0 | Dectris Eiger 16M | Structure of a leucinostatin derivative determined by host lattice display : L1F11V1 construct |
| [8AGQ](https://www.rcsb.org/structure/8AGQ) | IRRMC [10.18430/M38AGQ](https://doi.org/10.18430/M38AGQ) | SLS X06DA | 1.09 | C 1 2 1 | 89.9 55.4 54.8 90.0 113.5 90.0 | PILATUS 2MF | Crystal structure of anthocyanin-related GSTF8 from Populus trichocarpa in complex with (-)-catechin and glutathione |
image_analysis/bragg_integration/BraggIntegrationEngine.cpp
+14 -1
View File
@@ -96,6 +96,19 @@ BraggIntegrationEngine::BraggIntegrationEngine(const DiffractionExperiment &expe
stencil.bw_sigma = static_cast<float>(bw_sigma);
stencil.k_sigma = settings.GetStencilKSigma();
stencil.max_grow = bragg_engine::MAX_STENCIL_GROW_OVER_R3 * r3;
// The measured spot footprint (SpotFootprint.h). It moves the ring only where a spot outgrows the
// r1 disk, so a pattern of compact spots integrates exactly as without it.
static_assert(SpotFootprint::MAX_BINS <= BRAGG_FOOTPRINT_MAX_BINS);
stencil.r1 = settings.GetR1();
const auto &fp = settings.GetFootprint();
if (!empirical && !fp.empty()) {
stencil.fp_n = static_cast<int>(fp.sigma_rad.size());
stencil.fp_bin_px = fp.bin_px;
for (int i = 0; i < stencil.fp_n; ++i) {
stencil.fp_sigma_rad[i] = fp.sigma_rad[i];
stencil.fp_sigma_tan[i] = fp.sigma_tan[i];
}
}
// Robust background ring, one estimator or the other (see BraggIntegrationSettings): a high-side
// sigma-clip (rugnux --background-clip, the default) or, when the clip is switched off, a
@@ -137,7 +150,7 @@ BraggIntegrationEngine::BraggIntegrationEngine(const DiffractionExperiment &expe
r_max = std::hypot(std::max<double>(beam_x, static_cast<double>(xpixel) - beam_x),
std::max<double>(beam_y, static_cast<double>(ypixel) - beam_y));
bkg_radial_built = bkg_radial || bkg_radial_auto;
const float grow_max = bkg_radial_built ? BraggStencilGrow_px(static_cast<float>(r_max), stencil)
const float grow_max = bkg_radial_built ? BraggStencilMaxGrow_px(static_cast<float>(r_max), stencil)
: 0.0f;
n_kern = static_cast<int>(std::lround(grow_max)) + 1;
// Every row must fit: the last one is built at grow = n_kern - 1, which rounding can put just
image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp
+21 -5
View File
@@ -224,7 +224,7 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
for (int x = x0; x <= x1; ++x) {
const auto d = BraggStencilDistances(st, x - r.predicted_x, y - r.predicted_y);
const int32_t px = img[y * W + x];
if (d.signal < r1_sq) {
if (BraggStencilInSignal(st, x - r.predicted_x, y - r.predicted_y, d.signal, r1_sq)) {
// A pixel a nearer neighbour owns carries that neighbour's flux, so in Exclude
// mode it leaves the disk entirely - the sum, the pixel count the background is
// subtracted with, and the all-or-nothing validity gate alike. The box sum only
@@ -519,11 +519,12 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
int Rf = R;
const std::vector<double> *Pvec = &shell_P[sh];
const BraggStencil st = MakeBraggStencil(predicted[i].predicted_x, predicted[i].predicted_y, stencil);
if (!empirical) {
const double rx = predicted[i].predicted_x - beam_x, ry = predicted[i].predicted_y - beam_y;
const double Rpx = std::hypot(rx, ry);
const double tan2t = Rpx / F_px;
const double s2t = shell_sigma2[sh].tan;
double s2t = shell_sigma2[sh].tan;
double s2r = s2t, ux = 1.0, uy = 0.0;
bool elong = false;
if (use_ellipse) {
@@ -539,10 +540,18 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
elong = true;
}
}
// A spot that outgrows the r1 disk has widths the disk cannot measure - the moments above
// saturate near r1^2/4 - so the measured footprint sets them instead (SpotFootprint.h).
if (st.fp_s2t > 0.0f) {
ux = st.ux; uy = st.uy;
s2t = std::max<double>(s2t, st.fp_s2t);
s2r = std::max<double>(s2r, st.fp_s2r);
elong = true;
}
// Build the Gaussian per reflection, centred on the sub-pixel predicted position and (when
// needed) radially elongated, on a grid grown to hold the streak.
// needed) elongated, on a grid grown to hold the wider of its two axes.
const double fx = predicted[i].predicted_x - rh.cx, fy = predicted[i].predicted_y - rh.cy;
Rf = elong ? std::min(3 * R, static_cast<int>(std::ceil(r2 + 2.0 * std::sqrt(s2r)))) : R;
Rf = elong ? std::min(3 * R, static_cast<int>(std::ceil(r2 + 2.0 * std::sqrt(std::max(s2r, s2t))))) : R;
// The grid's image rows, asked for now (as in pass A): building the profile below takes
// long enough for them to arrive before the fit reads them.
const int gx0 = std::max(0, rh.cx - Rf), gx1 = std::min(W - 1, rh.cx + Rf);
@@ -582,7 +591,14 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
if (Pp <= 0.0) continue;
const int x = rh.cx + dx, y = rh.cy + dy;
if (x < 0 || y < 0 || x >= W || y >= H) continue;
const bool in_disk = dx * dx + dy * dy < r1_sq;
// The region the summation seed was taken over (pass A), so the guard below
// compares the two over the same pixels.
const bool in_disk = st.fp_s2t > 0.0f
? BraggStencilInSignal(st, x - predicted[i].predicted_x, y - predicted[i].predicted_y,
(x - predicted[i].predicted_x) * (x - predicted[i].predicted_x)
+ (y - predicted[i].predicted_y) * (y - predicted[i].predicted_y),
r1_sq)
: dx * dx + dy * dy < r1_sq;
p_grid += Pp;
p_peak = std::max(p_peak, Pp);
if (in_disk) m_all += Pp;
image_analysis/bragg_integration/BraggIntegrationEngineGPU.cu
+18 -6
View File
@@ -220,7 +220,7 @@ __global__ void boxsum(const float *px_x, const float *px_y, const float *dd,
// The window is the bounding box of the outer ellipse, so nearly half of it is neither the
// signal disk nor the background ring. Reading the pixel only once it is known to be wanted
// keeps those slots from fetching a cache line for nothing.
if (d.signal < p.r1_sq) {
if (BraggStencilInSignal(st, (float) x - cx, (float) y - cy, d.signal, p.r1_sq)) {
// A pixel a nearer neighbour owns carries that neighbour's flux; see the CPU engine.
++l_nd;
if (p.overlap) {
@@ -625,6 +625,7 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x
if (!ok_a[i]) { if (threadIdx.x == 0) ok_o[i] = 0; return; }
const int cx = cx_a[i], cy = cy_a[i];
const BraggStencil st = MakeBraggStencil(px_x[i], px_y[i], p.stencil);
// As in learn_profile: one thread per block, not one per thread. Two double-precision divisions
// on a card whose double throughput is a sixty-fourth of its single is worth doing once.
__shared__ int s_sh;
@@ -642,7 +643,7 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x
__syncthreads();
} else {
const int si = use_shell ? sh : N_SHELL; // N_SHELL is the global slot
const float s2t = sigma2_t[si];
float s2t = sigma2_t[si];
const float rx = px_x[i] - p.beam_x, ry = px_y[i] - p.beam_y;
const float Rpx = sqrtf(rx * rx + ry * ry);
const float tan2t = Rpx / p.F_px;
@@ -655,7 +656,14 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x
const float radial_extra = fmaxf(sigma2_r[si] - s2t, sbw * sbw + p.c_radial * tan2t * tan2t);
if (Rpx > 1e-6f && radial_extra > 0.25f) { ux = rx / Rpx; uy = ry / Rpx; s2r = s2t + radial_extra; elong = true; }
}
const int Rf = elong ? min(3 * p.R, (int) ceilf(p.r2 + 2.0f * sqrtf(s2r))) : p.R;
// A spot that outgrows the r1 disk takes the measured footprint's widths (see the CPU engine).
if (st.fp_s2t > 0.0f) {
ux = st.ux; uy = st.uy;
s2t = fmaxf(s2t, st.fp_s2t);
s2r = fmaxf(s2r, st.fp_s2r);
elong = true;
}
const int Rf = elong ? min(3 * p.R, (int) ceilf(p.r2 + 2.0f * sqrtf(fmaxf(s2r, s2t)))) : p.R;
const int Gf = 2 * Rf + 1;
if (threadIdx.x == 0) { s_Rf = Rf; s_Gf = Gf; }
__syncthreads();
@@ -697,7 +705,11 @@ __global__ void fit(const int32_t *img, const uint32_t *owner, const float *px_x
const int dx = k % Gf - Rf, dy = k / Gf - Rf;
const int x = cx + dx, y = cy + dy;
if (x < 0 || y < 0 || x >= p.W || y >= p.H) continue;
const bool in_disk = (float) (dx * dx + dy * dy) < p.r1_sq;
// The region the summation seed was taken over (boxsum), as on the CPU.
const float ddx = (float) x - px_x[i], ddy = (float) y - px_y[i];
const bool in_disk = st.fp_s2t > 0.0f
? BraggStencilInSignal(st, ddx, ddy, ddx * ddx + ddy * ddy, p.r1_sq)
: (float) (dx * dx + dy * dy) < p.r1_sq;
l_pgrid += Pp;
l_ppeak = fmaxf(l_ppeak, Pp);
if (in_disk) l_mall += Pp;
@@ -819,11 +831,11 @@ BraggIntegrationEngineGPU::BraggIntegrationEngineGPU(const DiffractionExperiment
// Shared radial window of the background curve in boxsum. The stencil spans r0 +- the radial
// semi-axis of the outer ellipse, widest at the far corner of the detector, and the window
// covers twice that so a reflection anywhere on the detector fits.
rad_w = 2 * (static_cast<int>(std::ceil(r3 + BraggStencilGrow_px(static_cast<float>(r_max), stencil))) + 1) + 1;
rad_w = 2 * (static_cast<int>(std::ceil(r3 + BraggStencilMaxGrow_px(static_cast<float>(r_max), stencil))) + 1) + 1;
// The pixels one reflection can mark: the box mark_mask walks, at the widest aperture on the
// detector. Run() weighs that against the frame to decide how to clear the mask afterwards.
const int mark_half = static_cast<int>(std::ceil(r2 + BraggStencilGrow_px(static_cast<float>(r_max), stencil))) + 1;
const int mark_half = static_cast<int>(std::ceil(r2 + BraggStencilMaxGrow_px(static_cast<float>(r_max), stencil))) + 1;
mask_box_px = static_cast<size_t>(2 * mark_half + 1) * static_cast<size_t>(2 * mark_half + 1);
boxsum_shared_bytes = static_cast<size_t>(rad_w) * (sizeof(unsigned long long) + sizeof(int));
image_analysis/bragg_integration/BraggStencil.h
+96 -7
View File
@@ -65,13 +65,27 @@
#define BRAGG_STENCIL_HD inline
#endif
// The measured spot footprint (SpotFootprint.h): the radial and tangential standard deviations of a
// spot [px], tabulated against distance from the beam centre in bins of fp_bin_px. Held here, by value,
// because both engines build every stencil from these parameters - the GPU gets the table with the
// kernel arguments and needs no buffer of its own.
constexpr int BRAGG_FOOTPRINT_MAX_BINS = 16;
// A spot reaches about this many of its standard deviations: the background ring starts there and the
// profile is fitted out to it.
constexpr float BRAGG_FOOTPRINT_NSIGMA = 3.0f;
// Fixed per-experiment inputs to the stencil law (mirrors BraggIntegrationEngine's members).
struct BraggStencilParams {
float beam_x = 0.0f, beam_y = 0.0f;
float r1 = 4.0f;
float r2 = 6.0f, r3 = 10.0f;
float bw_sigma = 0.0f; // radial streak per pixel of radius (bandwidth sigma, dimensionless)
float k_sigma = 0.0f; // radial sigmas to push the ring out by; 0 = the old circular stencil
float max_grow = 0.0f; // hard cap on the radial growth [px]; <= 0 means uncapped
int fp_n = 0; // bins of the measured footprint; 0 = none measured
float fp_bin_px = 0.0f;
float fp_sigma_rad[BRAGG_FOOTPRINT_MAX_BINS] = {};
float fp_sigma_tan[BRAGG_FOOTPRINT_MAX_BINS] = {};
};
// One reflection's stencil, in its own radial/tangential frame.
@@ -79,7 +93,15 @@ struct BraggStencil {
float ux = 1.0f, uy = 0.0f; // unit vector beam -> reflection
float r0 = 0.0f; // distance from the beam centre [px]
float grow = 0.0f; // radial growth of the ring [px] (0 = circular)
float grow_tan = 0.0f; // tangential growth of the ring [px] (footprint only)
float q_in = 0.0f, q_out = 0.0f; // radial shrink coefficients (0 = circular)
float qt_in = 0.0f, qt_out = 0.0f; // tangential shrink coefficients (0 = circular)
// The measured footprint where the spot outgrows the r1 disk (variances [px^2]), else 0: the
// profile fit takes its Gaussian at least this wide.
float fp_s2r = 0.0f, fp_s2t = 0.0f;
// 1 / (BRAGG_FOOTPRINT_NSIGMA * sigma)^2 along and across the radius: the footprint ellipse a wide
// spot is SUMMED over, beside the r1 disk (BraggStencilInSignal).
float core_r = 0.0f, core_t = 0.0f;
float ex_in = 0.0f, ey_in = 0.0f; // axis-aligned half-extent of the inner (r2) ellipse
float ex_out = 0.0f, ey_out = 0.0f; // axis-aligned half-extent of the outer (r3) ellipse
};
@@ -94,6 +116,38 @@ BRAGG_STENCIL_HD float BraggStencilGrow_px(float Rpx, const BraggStencilParams &
return (p.max_grow > 0.0f && grow > p.max_grow) ? p.max_grow : grow;
}
// The footprint at Rpx: linear between bin centres, flat beyond the first and the last.
BRAGG_STENCIL_HD void BraggFootprintAt(float Rpx, const BraggStencilParams &p, float &s_rad, float &s_tan) {
const float t = Rpx / p.fp_bin_px - 0.5f;
int i = (int) floorf(t);
float f = t - (float) i;
if (i < 0) { i = 0; f = 0.0f; }
if (i >= p.fp_n - 1) { i = p.fp_n - 1; f = 0.0f; }
const int j = i + 1 < p.fp_n ? i + 1 : i;
s_rad = p.fp_sigma_rad[i] + f * (p.fp_sigma_rad[j] - p.fp_sigma_rad[i]);
s_tan = p.fp_sigma_tan[i] + f * (p.fp_sigma_tan[j] - p.fp_sigma_tan[i]);
}
// Growth of the ring beyond r2 that the footprint asks for along one axis: none while the spot fits
// the r1 disk - there the disk resolves the spot and the stencil stays what it always was - and
// otherwise out to BRAGG_FOOTPRINT_NSIGMA of it, capped as the bandwidth growth is.
BRAGG_STENCIL_HD float BraggFootprintGrow_px(float sigma, const BraggStencilParams &p) {
const float g = BRAGG_FOOTPRINT_NSIGMA * sigma - p.r2;
if (!(g > 0.0f)) return 0.0f;
return (p.max_grow > 0.0f && g > p.max_grow) ? p.max_grow : g;
}
// The largest growth any reflection on a detector reaching r_max can get, radially - what the
// buffers that hold a ring are sized by.
BRAGG_STENCIL_HD float BraggStencilMaxGrow_px(float r_max, const BraggStencilParams &p) {
float g = BraggStencilGrow_px(r_max, p);
for (int i = 0; i < p.fp_n; ++i) {
const float s = fmaxf(p.fp_sigma_rad[i], p.fp_sigma_tan[i]);
if (BRAGG_FOOTPRINT_NSIGMA * s > p.r1) g = fmaxf(g, BraggFootprintGrow_px(s, p));
}
return g;
}
BRAGG_STENCIL_HD BraggStencil MakeBraggStencil(float px_x, float px_y, const BraggStencilParams &p) {
BraggStencil s;
const float rx = px_x - p.beam_x, ry = px_y - p.beam_y;
@@ -105,24 +159,44 @@ BRAGG_STENCIL_HD BraggStencil MakeBraggStencil(float px_x, float px_y, const Bra
}
s.grow = BraggStencilGrow_px(r0, p);
const float grow = s.grow;
if (grow == 0.0f) { // the common case, and the only one the online path can reach
// A spot wider than the r1 disk can resolve keeps its background ring clear of itself: the ring
// starts BRAGG_FOOTPRINT_NSIGMA of the measured footprint out, radially and tangentially apart.
if (p.fp_n > 0) {
float s_rad, s_tan;
BraggFootprintAt(r0, p, s_rad, s_tan);
if (BRAGG_FOOTPRINT_NSIGMA * fmaxf(s_rad, s_tan) > p.r1) {
s.fp_s2r = s_rad * s_rad;
s.fp_s2t = s_tan * s_tan;
s.core_r = 1.0f / (BRAGG_FOOTPRINT_NSIGMA * BRAGG_FOOTPRINT_NSIGMA * s.fp_s2r);
s.core_t = 1.0f / (BRAGG_FOOTPRINT_NSIGMA * BRAGG_FOOTPRINT_NSIGMA * s.fp_s2t);
s.grow = fmaxf(s.grow, BraggFootprintGrow_px(s_rad, p));
s.grow_tan = BraggFootprintGrow_px(s_tan, p);
}
}
const float grow = s.grow, grow_t = s.grow_tan;
if (grow == 0.0f && grow_t == 0.0f) { // the common case, and the only one the online path can reach
s.ex_in = p.r2; s.ey_in = p.r2;
s.ex_out = p.r3; s.ey_out = p.r3;
return s;
}
const float a_in = p.r2 + grow, a_out = p.r3 + grow;
const float a_in = p.r2 + grow, a_out = p.r3 + grow; // semi-axes along the radius
const float b_in = p.r2 + grow_t, b_out = p.r3 + grow_t; // and across it
const float si = p.r2 / a_in, so = p.r3 / a_out;
s.q_in = 1.0f - si * si;
s.q_out = 1.0f - so * so;
if (grow_t > 0.0f) {
const float ti = p.r2 / b_in, to = p.r3 / b_out;
s.qt_in = 1.0f - ti * ti;
s.qt_out = 1.0f - to * to;
}
// Axis-aligned half-extents of an ellipse with semi-axes (a along u, b across it).
const float ux2 = s.ux * s.ux, uy2 = s.uy * s.uy;
s.ex_in = sqrtf(a_in * a_in * ux2 + p.r2 * p.r2 * uy2);
s.ey_in = sqrtf(a_in * a_in * uy2 + p.r2 * p.r2 * ux2);
s.ex_out = sqrtf(a_out * a_out * ux2 + p.r3 * p.r3 * uy2);
s.ey_out = sqrtf(a_out * a_out * uy2 + p.r3 * p.r3 * ux2);
s.ex_in = sqrtf(a_in * a_in * ux2 + b_in * b_in * uy2);
s.ey_in = sqrtf(a_in * a_in * uy2 + b_in * b_in * ux2);
s.ex_out = sqrtf(a_out * a_out * ux2 + b_out * b_out * uy2);
s.ey_out = sqrtf(a_out * a_out * uy2 + b_out * b_out * ux2);
return s;
}
@@ -183,5 +257,20 @@ BRAGG_STENCIL_HD BraggStencilDist BraggStencilDistances(const BraggStencil &s, f
const float rad2 = d.rad * d.rad;
d.inner = d.signal - s.q_in * rad2;
d.outer = d.signal - s.q_out * rad2;
if (s.qt_in != 0.0f) { // grown tangentially as well (the footprint); tan^2 = d2 - rad^2
const float tan2 = d.signal - rad2;
d.inner -= s.qt_in * tan2;
d.outer -= s.qt_out * tan2;
}
return d;
}
// The signal region: the r1 disk, and for a spot that outgrows it the whole footprint ellipse too, so
// the summation - the seed of the profile fit, and what the fit falls back to - holds the spot rather
// than its core. Without a footprint this is the plain disk test, bit for bit.
BRAGG_STENCIL_HD bool BraggStencilInSignal(const BraggStencil &s, float ddx, float ddy, float d2, float r1_sq) {
if (d2 < r1_sq) return true;
if (!(s.fp_s2t > 0.0f)) return false;
const float rad = ddx * s.ux + ddy * s.uy, tn = -ddx * s.uy + ddy * s.ux;
return rad * rad * s.core_r + tn * tn * s.core_t < 1.0f;
}
image_analysis/bragg_integration/CMakeLists.txt
+2
View File
@@ -3,6 +3,8 @@ ADD_LIBRARY(JFJochBraggIntegration STATIC
BraggIntegrationEngine.h
BraggIntegrationEngineCPU.cpp
BraggIntegrationEngineCPU.h
SpotFootprint.cpp
SpotFootprint.h
Regression.h
CalcISigma.cpp
CalcISigma.h
image_analysis/bragg_integration/SpotFootprint.cpp
+126
View File
@@ -0,0 +1,126 @@
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include "SpotFootprint.h"
#include <algorithm>
#include <cmath>
namespace {
// The window around a spot is BRAGG_FOOTPRINT_NSIGMA-like: three of its standard deviations, at least
// a few pixels and at most this many, which is wider than any spot the integrator could hold.
constexpr float WINDOW_NSIGMA = 3.0f;
constexpr float WINDOW_MIN_PX = 3.0f;
constexpr float WINDOW_MAX_PX = 24.0f;
// The background is the median of an elliptical annulus between these multiples of the window.
constexpr float BKG_INNER = 1.5f;
constexpr float BKG_OUTER = 2.2f;
constexpr int ITERATIONS = 8;
inline bool valid(int32_t v) { return v != INT32_MIN && v != INT32_MAX; }
float median_of(std::vector<float> &v) {
const size_t m = v.size() / 2;
std::nth_element(v.begin(), v.begin() + static_cast<std::ptrdiff_t>(m), v.end());
return v[m];
}
} // namespace
void MeasureFootprintSpots(const int32_t *img, int width, int height, float beam_x, float beam_y,
const std::vector<float> &x, const std::vector<float> &y,
std::vector<FootprintSpot> &out) {
const int half = static_cast<int>(std::ceil(BKG_OUTER * WINDOW_MAX_PX)) + 1;
std::vector<float> ring;
for (size_t s = 0; s < x.size(); ++s) {
const float rx = x[s] - beam_x, ry = y[s] - beam_y;
const float r = std::sqrt(rx * rx + ry * ry);
if (!(r > 1.0f)) continue;
const float ux = rx / r, uy = ry / r;
const int ix = static_cast<int>(std::lround(x[s])), iy = static_cast<int>(std::lround(y[s]));
if (ix - half < 0 || iy - half < 0 || ix + half >= width || iy + half >= height) continue;
// Start from a compact spot at the prediction; each round re-centres on the signal and takes
// the window to three of the widths just measured.
float cx = x[s], cy = y[s];
float s2r = 1.0f, s2t = 1.0f;
bool ok = true;
for (int it = 0; it < ITERATIONS && ok; ++it) {
const float wr = std::clamp(WINDOW_NSIGMA * std::sqrt(s2r), WINDOW_MIN_PX, WINDOW_MAX_PX);
const float wt = std::clamp(WINDOW_NSIGMA * std::sqrt(s2t), WINDOW_MIN_PX, WINDOW_MAX_PX);
const float reach = BKG_OUTER * std::max(wr, wt);
const int x0 = static_cast<int>(std::floor(cx - reach)), x1 = static_cast<int>(std::ceil(cx + reach));
const int y0 = static_cast<int>(std::floor(cy - reach)), y1 = static_cast<int>(std::ceil(cy + reach));
if (x0 < 0 || y0 < 0 || x1 >= width || y1 >= height) { ok = false; break; }
ring.clear();
for (int py = y0; py <= y1; ++py)
for (int px = x0; px <= x1; ++px) {
const float dx = px - cx, dy = py - cy;
const float rad = dx * ux + dy * uy, tn = -dx * uy + dy * ux;
const float e = rad * rad / (wr * wr) + tn * tn / (wt * wt);
const int32_t v = img[static_cast<size_t>(py) * width + px];
if (e >= BKG_INNER * BKG_INNER && e < BKG_OUTER * BKG_OUTER && valid(v))
ring.push_back(static_cast<float>(v));
}
if (ring.size() < 10) { ok = false; break; }
const double bkg = median_of(ring);
double w = 0.0, mr = 0.0, mt = 0.0, m2r = 0.0, m2t = 0.0;
for (int py = y0; py <= y1 && ok; ++py)
for (int px = x0; px <= x1; ++px) {
const float dx = px - cx, dy = py - cy;
const float rad = dx * ux + dy * uy, tn = -dx * uy + dy * ux;
if (rad * rad / (wr * wr) + tn * tn / (wt * wt) >= 1.0f) continue;
const int32_t v = img[static_cast<size_t>(py) * width + px];
if (!valid(v)) { ok = false; break; }
const double net = static_cast<double>(v) - bkg;
w += net;
mr += net * rad;
mt += net * tn;
m2r += net * rad * rad;
m2t += net * tn * tn;
}
if (!ok || !(w > 0.0)) { ok = false; break; }
const double cr = mr / w, ct = mt / w;
s2r = static_cast<float>(std::max(0.25, m2r / w - cr * cr));
s2t = static_cast<float>(std::max(0.25, m2t / w - ct * ct));
cx += static_cast<float>(cr * ux - ct * uy);
cy += static_cast<float>(cr * uy + ct * ux);
}
if (ok)
out.push_back({r, std::sqrt(s2r), std::sqrt(s2t)});
}
}
SpotFootprint FootprintFromSpots(const std::vector<FootprintSpot> &spots, float r_max) {
SpotFootprint fp;
if (!(r_max > 0.0f)) return fp;
const float bin = r_max / FOOTPRINT_BINS;
std::vector<std::vector<float>> rad(FOOTPRINT_BINS), tan(FOOTPRINT_BINS);
for (const auto &s : spots) {
const int b = std::clamp(static_cast<int>(s.r_px / bin), 0, FOOTPRINT_BINS - 1);
rad[b].push_back(s.sigma_rad);
tan[b].push_back(s.sigma_tan);
}
std::vector<int> filled;
std::vector<float> mr(FOOTPRINT_BINS, 0.0f), mt(FOOTPRINT_BINS, 0.0f);
for (int b = 0; b < FOOTPRINT_BINS; ++b)
if (static_cast<int>(rad[b].size()) >= FOOTPRINT_MIN_SPOTS_PER_BIN) {
mr[b] = median_of(rad[b]);
mt[b] = median_of(tan[b]);
filled.push_back(b);
}
if (filled.empty()) return fp;
fp.bin_px = bin;
for (int b = 0; b < FOOTPRINT_BINS; ++b) {
// The nearest bin that has enough spots; the inner one on a tie.
int best = filled.front();
for (int f : filled)
if (std::abs(f - b) < std::abs(best - b)) best = f;
fp.sigma_rad.push_back(mr[best]);
fp.sigma_tan.push_back(mt[best]);
}
return fp;
}
image_analysis/bragg_integration/SpotFootprint.h
+53
View File
@@ -0,0 +1,53 @@
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
// =============================================================================
// SpotFootprint - how far a spot really reaches, measured on the data
// =============================================================================
//
// The integrator's r1 disk and r2..r3 background ring are fixed in pixels, chosen from spots near the
// beam. Away from it a spot can grow several times wider: radially from the sensor's parallax and the
// obliquity of the incidence, tangentially from the crystal's own spread (an azimuthal mosaic spread
// rotates the diffracted beam about the incident one, smearing the spot along the ring). Measured on
// small-molecule data at 20 keV, the standard deviation goes from 1 px near the beam to 5 px at the
// edge, so the fixed disk holds a quarter of the flux there and the background ring a third of it -
// an intensity loss that grows with resolution and that no scaling can see, because symmetry mates
// share their resolution.
//
// The widths the integrator learns itself cannot follow: they are second moments taken inside the r1
// disk, so they saturate near r1^2/4. Here each strong spot is measured with a window that follows
// the spot instead - three of its own standard deviations, iterated - separately along the radius and
// across it, and the widths are tabulated against the distance from the beam centre. The table goes
// to the integrator through BraggIntegrationSettings; where it says a spot outgrows the r1 disk, the
// background ring is moved clear of the spot and the profile is fitted at the measured width
// (BraggStencil.h). Compact spots leave the integration exactly as it was.
// =============================================================================
#include <cstdint>
#include <vector>
#include "../../common/BraggIntegrationSettings.h"
// One spot's measured widths, and its distance from the beam centre [px].
struct FootprintSpot {
float r_px = 0.0f;
float sigma_rad = 0.0f;
float sigma_tan = 0.0f;
};
// Measure the spots at the given (sub-pixel) positions on one preprocessed frame (INT32_MIN masked,
// INT32_MAX saturated). A spot whose window holds an unreadable pixel, or no signal above its
// background, is left out. Results are appended in the order of the positions.
void MeasureFootprintSpots(const int32_t *img, int width, int height, float beam_x, float beam_y,
const std::vector<float> &x, const std::vector<float> &y,
std::vector<FootprintSpot> &out);
// Tabulate the spots: the median widths in bins of distance from the beam centre out to r_max. A bin
// holding too few spots takes the nearest bin that has enough; with no such bin the table is empty.
SpotFootprint FootprintFromSpots(const std::vector<FootprintSpot> &spots, float r_max);
// Spots per distance bin a table needs before it is believed.
constexpr int FOOTPRINT_MIN_SPOTS_PER_BIN = 20;
constexpr int FOOTPRINT_BINS = 12;
image_analysis/indexing/PostIndexingRefinement.cpp
+16 -4
View File
@@ -82,7 +82,6 @@ namespace {
const unsigned nspots = spots.rows();
const unsigned ncells = scores.rows();
VectorX<bool> below{nspots};
MatrixX3<bool> sel{nspots, 3u};
Mx3 resid{nspots, 3u};
Mx3 miller{nspots, 3u};
M3 cell;
@@ -111,9 +110,22 @@ namespace {
break;
threshold *= cifssr.threshold_contraction;
sel.colwise() = below;
HouseholderQR<Mx3> qr{sel.select(spots, .0f)};
cell = qr.solve(sel.select(miller, .0f));
// Least squares spots * cell = miller over the spots below the threshold, by the normal
// equations summed in spot order in double. A QR over all the spots reduces its column
// norms in SIMD packets of the target's width, so its cell - and which candidate a
// borderline first pass indexed with - depended on -march; these sums do not.
Matrix3d ata = Matrix3d::Zero();
Matrix3d atb = Matrix3d::Zero();
for (unsigned i = 0; i < nspots; i++) {
if (!below[i])
continue;
for (int r = 0; r < 3; r++)
for (int c = 0; c < 3; c++) {
ata(r, c) += double(spots(i, r)) * double(spots(i, c));
atb(r, c) += double(spots(i, r)) * double(miller(i, c));
}
}
cell = (ata.inverse() * atb).cast<float>();
}
resid = CalculateResiduals(spots, cell);
image_analysis/scale_merge/RotationScaleMerge.cpp
+59 -14
View File
@@ -36,6 +36,16 @@ namespace {
// These mirror the per-image ScaleOnTheFly / Merge rocking-curve physics verbatim so this flat
// implementation is numerically identical - see the comments there for the details.
constexpr size_t MIN_REFLECTIONS = 20; // per-frame scale needs at least this many
// Per-frame scaling of the PARTIALS needs a frame to carry many rocking events: within one rocking
// curve a change of scale and an error of the partiality model are the same thing, and only events
// caught at different points of their curves tell them apart. On a fine-sliced sparse sweep the fit
// takes one for the other - G swung 0.23..1.2 with a 180 deg period on a small-molecule sweep whose
// own frame scale varies 0.79..0.99 (XDS), and the merge carried an hkl-dependent bias that the
// equivalents cannot see but a refinement against the known structure does (R1 0.096 -> 0.062
// without it). Below this the per-frame scale comes from the fulls alone, as XDS takes it. Measured
// on the battery: the small-molecule sweeps hold 2.6 to 23 events per frame, the protein sets 84 to
// 900 - and on the proteins the fulls-only scale leaves model R-free unchanged (+-0.002).
constexpr double MIN_EVENTS_PER_FRAME_SCALE_PARTIALS = 50.0;
constexpr int64_t MIN_REFLECTIONS_FOR_IMAGE_CC = 20; // below this a frame's CC means nothing
// The per-frame scaling loop has settled when the scales moved less than this between two
// iterations, as the rms of |log(G_new/G_old)| over the frames fitted in both.
@@ -591,7 +601,7 @@ void RotationScaleMerge::Ingest() {
// (see the header). Take its answer now and hand them back. The dump path keeps the full Obs
// array: the CPU combine is what writes the dump.
if (resident_ingest) {
rocking_event_frames_at_ingest = RockingEventFrames();
rocking_event_frames_at_ingest = RockingEventFrames(&rocking_events_at_ingest);
partials_released = true;
}
#endif
@@ -1407,17 +1417,34 @@ void RotationScaleMerge::ReduceScalingGroupMeans(int n_groups, const std::vector
});
}
int RotationScaleMerge::PoolHalfWidth(const std::vector<int32_t> &fcount, int f) {
const int n = static_cast<int>(fcount.size());
int64_t total = fcount[f];
int h = 0;
while (total < static_cast<int64_t>(MIN_REFLECTIONS) && h < n) {
++h;
if (f - h >= 0) total += fcount[f - h];
if (f + h < n) total += fcount[f + h];
}
return h;
}
template <class O>
void RotationScaleMerge::FitPerFrameG(const std::vector<O> &obs, const std::vector<int32_t> &fstart,
const std::vector<int32_t> &fcount,
const std::vector<double> &group_mean_in,
bool unity, std::vector<double> &g) {
std::vector<uint8_t> scaled(fstart.size(), 0);
ParallelFor(static_cast<int>(fstart.size()), nthreads, [&](int f) {
const int n_fr = static_cast<int>(fstart.size());
ParallelFor(n_fr, nthreads, [&](int f) {
// A full is a whole rocking event filed under one frame, so on a sparse sweep a frame holds a
// handful of them - too few to fit a scale on, and the frame was left unscaled, which on such a
// sweep was every frame. Those frames are fitted over the nearest frames on either side that
// together hold enough; a frame with enough of its own is fitted on its own, as before.
const int h = unity && pool_sparse_fulls ? PoolHalfWidth(fcount, f) : 0;
std::vector<ScaleObs> so;
so.reserve(fcount[f]);
const int lo = fstart[f], hi = fstart[f] + fcount[f];
for (int i = lo; i < hi; ++i) {
for (int j = std::max(0, f - h); j <= std::min(n_fr - 1, f + h); ++j)
for (int i = fstart[j]; i < fstart[j] + fcount[j]; ++i) {
const auto &o = obs[i];
if (o.group < 0) continue;
if (o.on_ice) continue;
@@ -2456,24 +2483,26 @@ namespace {
// image order, bridged by the same max_frame_gap - taken as the median over the run's events. It is
// the finest stretch of the sweep this file is allowed to speak about: the partials of one event are
// welded into a single full, so two frames closer together than this are not separable observations.
int RotationScaleMerge::RockingEventFrames() const {
if (partials_released)
int RotationScaleMerge::RockingEventFrames(int64_t *n_events) const {
if (partials_released) {
if (n_events) *n_events = rocking_events_at_ingest;
return rocking_event_frames_at_ingest;
}
if (resident_ingest) // Ingest's own call, before the ingest arrays are handed back
return RockingEventFramesOver(
[&](int i) { return ingest_rock_ok[i] != 0; },
[&](int i) { return ingest_image_number[i]; });
[&](int i) { return ingest_image_number[i]; }, n_events);
return RockingEventFramesOver(
[&](int i) {
const Obs &o = partials[i];
return std::isfinite(o.corr) && o.corr > 0.0f && std::isfinite(o.I)
&& std::isfinite(o.sigma) && o.sigma > 0.0f;
},
[&](int i) { return partials[i].image_number; });
[&](int i) { return partials[i].image_number; }, n_events);
}
template <class UsableFn, class ImgFn>
int RotationScaleMerge::RockingEventFramesOver(UsableFn usable, ImgFn img) const {
int RotationScaleMerge::RockingEventFramesOver(UsableFn usable, ImgFn img, int64_t *n_events) const {
// Blocks of runs on all threads, each counting into a histogram of its own. The counts are
// integers, so the blocks add up to the serial histogram whatever the split.
constexpr int RUNS_PER_BLOCK = 16384;
@@ -2508,6 +2537,7 @@ int RotationScaleMerge::RockingEventFramesOver(UsableFn usable, ImgFn img) const
hist[w] += h[w];
n_event += h[w];
}
if (n_events) *n_events = n_event;
int acc = 0;
for (int w = 1; w <= n_frames + 1; ++w)
if ((acc += hist[w]) * 2 >= n_event)
@@ -5616,7 +5646,7 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st
int smooth_window = 1;
if (smooth_g_deg > 0.0 && osc_deg > 1e-6) {
if (rocking_event_frames_at_start < 0)
rocking_event_frames_at_start = RockingEventFrames();
rocking_event_frames_at_start = RockingEventFrames(&rocking_events_at_start);
smooth_window = std::max(static_cast<int>(std::lround(smooth_g_deg / osc_deg)),
SMOOTH_G_MIN_ROCKING_EVENTS * rocking_event_frames_at_start);
if (smooth_window % 2 == 0) ++smooth_window;
@@ -5624,9 +5654,20 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st
"curves span {} frames)", smooth_window, smooth_window * osc_deg, smooth_g_deg,
rocking_event_frames_at_start);
}
// Without partial scaling (MIN_EVENTS_PER_FRAME_SCALE_PARTIALS) every frame keeps G = 1 here and
// its scale comes from the fulls alone (scale-fulls below).
if (rocking_event_frames_at_start < 0)
rocking_event_frames_at_start = RockingEventFrames(&rocking_events_at_start);
const double events_per_frame = n_frames > 0 ? static_cast<double>(rocking_events_at_start) / n_frames : 0.0;
const bool scale_partials = events_per_frame >= MIN_EVENTS_PER_FRAME_SCALE_PARTIALS;
pool_sparse_fulls = !scale_partials;
if (!scale_partials)
logger.Info("Per-frame scale from the fulls alone: {:.1f} rocking events per frame, under the {:.0f} "
"a partial's scale needs to stay apart from the partiality model", events_per_frame,
MIN_EVENTS_PER_FRAME_SCALE_PARTIALS);
ScalingLoopOutcome partial_loop;
#ifdef JFJOCH_USE_CUDA
if (gpu_active_) {
if (gpu_active_ && scale_partials) {
// The scaling loop runs on the GPU one iteration per call, and corr stays RESIDENT across
// scaling -> smooth-G -> CC -> combine (and across passes, exactly as the old host round-trip
// did). Only the per-frame G/scaled come back after each iteration, for the gauge pin and the
@@ -5647,7 +5688,11 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st
scaled_on_gpu = true;
}
#endif
if (!scaled_on_gpu) {
if (!scale_partials) {
std::fill(g_partial.begin(), g_partial.end(), 1.0);
frame_scaled_scratch.assign(n_frames, 1);
partial_loop.converged = true;
} else if (!scaled_on_gpu) {
const PartialLoopKey key{x.GetSpaceGroupOrP1().xhm(), merge_friedel, d_min_limit, d_max_limit,
min_partiality, smooth_window, scaling_iter};
const auto memo = std::find_if(partial_loop_memos.begin(), partial_loop_memos.end(),
@@ -5937,7 +5982,7 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st
std::vector<uint8_t> scaled_dev(n_frames);
fulls_loop = RunScalingLoop("fulls", g_full, f_count, smooth_window,
/*release_restraint=*/true, [&] {
gpu_->ScaleFulls(1, min_partiality);
gpu_->ScaleFulls(1, min_partiality, pool_sparse_fulls);
gpu_->GetG(g_dev.data(), scaled_dev.data());
for (int f = 0; f < n_frames; ++f)
if (scaled_dev[f]) g_full[f] = g_dev[f];
image_analysis/scale_merge/RotationScaleMerge.h
+12 -2
View File
@@ -43,6 +43,10 @@
// Stills use the per-image ScaleOnTheFly (fixed partiality) instead.
class RotationScaleMerge {
public:
// How many frames on either side a frame's fulls scale is fitted over (used without partial
// scaling): none when the frame holds MIN_REFLECTIONS fulls itself, else the fewest that bring the
// pooled count there.
static int PoolHalfWidth(const std::vector<int32_t> &fcount, int f);
struct Result {
std::vector<MergedReflection> merged;
MergeStatistics statistics;
@@ -219,6 +223,9 @@ private:
// this is how far they may go before giving up and saying so.
int scaling_iter = 100;
bool scale_fulls = true;
// Without partial scaling the fulls carry the whole per-frame scale, and a sparse frame is fitted
// over its neighbours (PoolHalfWidth).
bool pool_sparse_fulls = false;
bool refine_decay_b = false; // per-time-block Debye-Waller decay correction (radiation damage)
int absorption_iter = 0; // >0: fit a goniometer-frame absorption surface over this many iterations
int modulation_iter = 0; // >0: fit a detector-plane modulation (flat-field) surface, this many iterations
@@ -275,6 +282,7 @@ private:
// full-size copy of the partials, gigabytes on a fine-sliced long axis - straight back.
bool partials_released = false;
int rocking_event_frames_at_ingest = 0;
int64_t rocking_events_at_ingest = 0;
std::vector<int32_t> frame_start, frame_count; // CSR ranges of `partials` per frame
std::vector<uint8_t> frame_cell_ok; // per-frame cell-consistency mask (1 = kept)
std::vector<uint8_t> finite_ok; // per-obs AcceptReflection finiteness (immutable; 1 = kept)
@@ -305,6 +313,7 @@ private:
// RockingEventFrames at the start of a Run, where corr has just been restored to corr_ingested - so
// the same for every Run on the same ingest; -1 until the first Run takes it.
int rocking_event_frames_at_start = -1;
int64_t rocking_events_at_start = 0; // the events that walk counted
// Raw-hkl ordering, built ONCE by Ingest and reused: `perm` lists partial indices sorted by
// (raw h,k,l, image_number); each distinct raw hkl is a contiguous run [rawrun_start, +count) of it.
@@ -626,9 +635,10 @@ private:
// The frames one rocking event spans, as the combine cuts them: the median over the run's events.
// Walked on the partials, because the combine also runs on the GPU and a full keeps only the frame
// of its peak partial.
[[nodiscard]] int RockingEventFrames() const;
// n_events (if given) receives how many rocking events the walk counted.
[[nodiscard]] int RockingEventFrames(int64_t *n_events = nullptr) const;
template <class UsableFn, class ImgFn>
[[nodiscard]] int RockingEventFramesOver(UsableFn usable, ImgFn img) const;
[[nodiscard]] int RockingEventFramesOver(UsableFn usable, ImgFn img, int64_t *n_events) const;
// Per-batch delta-CC1/2 on the corrected fulls: measure what keeping each batch of the sweep costs
// the merged intensities, convict the batches that cost significantly, slide the conviction's edges
// onto the frames that carry it, and turn the result into the disposition ledger. Fills
image_analysis/scale_merge/RotationScaleMergeGPU.cu
+32 -16
View File
@@ -122,16 +122,31 @@ namespace {
const int32_t *__restrict__ frame_count,
const float *__restrict__ I, const double *__restrict__ inv_sigma,
const float *__restrict__ sco_coeff, const uint8_t *__restrict__ sco_ok,
const int32_t *__restrict__ perm,
const int32_t *__restrict__ perm, bool pool,
double *__restrict__ g, uint8_t *__restrict__ scaled) {
const int f = blockIdx.x;
if (f >= n_frames) return;
const int lo = frame_start[f], hi = frame_start[f] + frame_count[f];
// pool (the fulls): a frame holding fewer than MIN_REFLECTIONS is fitted over the fewest frames
// on either side that bring the count there - RotationScaleMerge::PoolHalfWidth, same rule.
__shared__ int s_h;
if (threadIdx.x == 0) {
long total = frame_count[f];
int h = 0;
while (pool && total < long(MIN_REFLECTIONS) && h < n_frames) {
++h;
if (f - h >= 0) total += frame_count[f - h];
if (f + h < n_frames) total += frame_count[f + h];
}
s_h = h;
}
__syncthreads();
const int j_lo = max(0, f - s_h), j_hi = min(n_frames - 1, f + s_h);
__shared__ double sh[BLK];
long cnt_local = 0;
for (int i = lo + threadIdx.x; i < hi; i += blockDim.x)
if (sco_ok[perm ? perm[i] : i]) ++cnt_local;
for (int j = j_lo; j <= j_hi; ++j)
for (int i = frame_start[j] + threadIdx.x; i < frame_start[j] + frame_count[j]; i += blockDim.x)
if (sco_ok[perm ? perm[i] : i]) ++cnt_local;
const double cnt = BlockReduceSum(double(cnt_local), sh);
__shared__ double s_cnt;
if (threadIdx.x == 0) s_cnt = cnt;
@@ -139,15 +154,16 @@ namespace {
if (s_cnt < MIN_REFLECTIONS) return; // leave g[f]/scaled[f] as-is
double num = 0.0, den = 0.0;
for (int i = lo + threadIdx.x; i < hi; i += blockDim.x) {
const int a = perm ? perm[i] : i;
if (!sco_ok[a]) continue;
const double coeff = sco_coeff[a];
const double w = inv_sigma[a];
const double w2 = w * w;
num += w2 * coeff * double(I[a]);
den += w2 * coeff * coeff;
}
for (int j = j_lo; j <= j_hi; ++j)
for (int i = frame_start[j] + threadIdx.x; i < frame_start[j] + frame_count[j]; i += blockDim.x) {
const int a = perm ? perm[i] : i;
if (!sco_ok[a]) continue;
const double coeff = sco_coeff[a];
const double w = inv_sigma[a];
const double w2 = w * w;
num += w2 * coeff * double(I[a]);
den += w2 * coeff * coeff;
}
const double tnum = BlockReduceSum(num, sh); __syncthreads();
const double tden = BlockReduceSum(den, sh);
if (threadIdx.x == 0) {
@@ -1080,7 +1096,7 @@ void RotationScaleMergeGPU::ScalePartials(int iters, double min_partiality, bool
d.sco_coeff.get(), d.sco_ok.get());
CudaCheck(cudaGetLastError(), "PrepScaleObsKernel launch");
FitPerFrameGKernel<<<d.n_frames, BLK, 0, impl_->s()>>>(d.n_frames, d.frame_start.get(), d.frame_count.get(),
d.I.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get(), nullptr, d.g.get(), d.scaled.get());
d.I.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get(), nullptr, /*pool=*/false, d.g.get(), d.scaled.get());
CudaCheck(cudaGetLastError(), "FitPerFrameGKernel launch");
UpdateCorrKernel<<<upd_blocks, BLK, 0, impl_->s()>>>(d.n_obs, d.frame.get(), d.prescaling_corr.get(), d.partiality.get(),
d.g.get(), d.scaled.get(), d.corr.get());
@@ -1496,7 +1512,7 @@ void RotationScaleMergeGPU::ResetFullsScale() {
CudaCheck(cudaStreamSynchronize(impl_->s()), "reset fulls scale sync");
}
void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality) {
void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality, bool pool) {
DeviceGuard guard(impl_->device, impl_->available);
auto &d = *impl_;
const int nf = d.n_fulls;
@@ -1522,7 +1538,7 @@ void RotationScaleMergeGPU::ScaleFulls(int iters, double min_partiality) {
CudaCheck(cudaGetLastError(), "PrepScaleObsKernel launch");
FitPerFrameGKernel<<<d.n_frames, BLK, 0, impl_->s()>>>(d.n_frames,
d.f_frame_start.get(), d.f_frame_count.get(), d.f_I.get(), d.f_inv_sigma.get(),
d.f_sco_coeff.get(), d.f_sco_ok.get(), d.f_frame_perm.get(), d.g.get(), d.scaled.get());
d.f_sco_coeff.get(), d.f_sco_ok.get(), d.f_frame_perm.get(), pool, d.g.get(), d.scaled.get());
CudaCheck(cudaGetLastError(), "FitPerFrameGKernel launch");
UpdateCorrKernel<<<obs_blocks, BLK, 0, impl_->s()>>>(nf, d.f_frame.get(), d.f_rlp.get(), d.f_partiality.get(),
d.g.get(), d.scaled.get(), d.f_corr.get());
image_analysis/scale_merge/RotationScaleMergeGPU.h
+3 -1
View File
@@ -184,7 +184,9 @@ public:
// Run `iters` of the Unity scaling loop on the resident fulls (reduce group means -> per-frame LS G
// -> update corr), in place on the fulls' working corr. Requires SetFullsFrameCSR + SetFullsGroups
// and ResetFullsScale.
void ScaleFulls(int iters, double min_partiality);
// pool: a frame with fewer than MIN_REFLECTIONS fulls is fitted over its neighbours
// (RotationScaleMerge::PoolHalfWidth).
void ScaleFulls(int iters, double min_partiality, bool pool);
// The fulls' counterpart of SmoothCorr: f_corr[i] *= ratio[f_frame[i]] where apply[f].
void SmoothFullsCorr(const uint8_t *apply, const double *ratio);
image_analysis/scale_merge/SearchSpaceGroup.cpp
+9 -5
View File
@@ -1900,9 +1900,14 @@ SearchSpaceGroupResult SearchSpaceGroup(
// the point group is one only a non-reference setting carries (Stage A's second pass), since
// otherwise there is no candidate at all and the group would be lost after being found. These are
// alternative namings at the same order, so nothing here can promote the point group; what they
// add is a screw or a centering on the axis the data show it on. Two refusals bound them: a
// setting whose axes the cell does not have is not offered, and one predicting exactly the
// absences a candidate already offered predicts is the same hypothesis under another name.
// add is a screw or a centering on the axis the data show it on. One refusal bounds them: a setting
// predicting exactly the absences a candidate already offered predicts is the same hypothesis
// under another name. The cell is NOT asked again here: a setting of the chosen rotation set holds
// the very rotations the reference candidates hold, so it fits the cell exactly as well as they
// do - and they are offered without asking. Asking only the non-reference ones turned a free-
// refined cell a few tenths of a degree off 90 into a refusal of every setting but the reference
// one, leaving the group with the measured screws on a and c (P 2_1 2 2_1) unoffered and P2_12_12_1
// the only candidate that covered both.
if (opt.cell.has_value() &&
(opt.enumerate_all_settings || (opt.enumerate_all_rotation_sets && sg_cands.empty()))) {
std::vector<std::vector<char>> signatures;
@@ -1910,8 +1915,7 @@ SearchSpaceGroupResult SearchSpaceGroup(
signatures.push_back(AbsenceSignature(*c));
for (const auto& sg : gemmi::spacegroup_tables::main) {
if (!sg.is_sohncke() || sg.is_reference_setting() ||
RotationSetOf(sg) != best_pg->rotation_set ||
!CellHostsRotations(*opt.cell, sg.operations()))
RotationSetOf(sg) != best_pg->rotation_set)
continue;
auto sig = AbsenceSignature(sg);
if (std::find(signatures.begin(), signatures.end(), sig) != signatures.end())
image_analysis/scale_merge/TwinningAnalysis.cpp
+10
View File
@@ -596,6 +596,10 @@ namespace {
result.control.evidence_nats - result.control.n * result.control_excess_per_reflection;
for (auto& z : result.zones)
z.calibrated_evidence_nats = z.evidence_nats - z.n * result.control_excess_per_reflection;
constexpr double PERFECT_TWIN_ACENTRIC_MEAN_ABS_E2_MINUS_1 = 0.541;
result.zones_ambiguous = result.control.n > 0
&& result.control.mean_abs_e2_minus_1 + 3.0 * result.control.standard_error
< PERFECT_TWIN_ACENTRIC_MEAN_ABS_E2_MINUS_1;
}
}
@@ -862,5 +866,11 @@ std::string TwinImmuneZonesToText(const TwinImmuneZoneResult& result) {
for (const auto& z : result.zones)
row(z);
row(result.control);
if (result.zones_ambiguous)
os << " The control reads more compressed than a perfect twin's acentric reflections (0.541), which\n"
" no twin fraction can do: something else - overlapping spots, neighbour correlation - averages\n"
" each reflection with unrelated ones, and it compresses the zones as well. Genuine symmetry then\n"
" reads acentric in its zones too, so here they do not separate a twinned subgroup from the\n"
" adopted group, and no verdict is read from them either way.\n";
return os.str();
}
image_analysis/scale_merge/TwinningAnalysis.h
+9
View File
@@ -143,6 +143,15 @@ struct TwinImmuneZoneResult {
// nats were the normalisation's, and the verdict rescued a twin law. Calibrated, that zone reads
// -115 nats; the zones of genuine promotions keep +140 nats (a 90 deg tetragonal sweep) to +2000.
double control_excess_per_reflection = 0.0;
// The control reads more compressed than a PERFECT twin's acentric population (<|E^2-1|> 0.541, by
// more than three standard errors). Twinning cannot do that at any fraction, so something else
// averages each reflection with unrelated ones - overlapping spots of a long cell, neighbour
// correlation - and it compresses the zones as much as the control: genuine symmetry then reads
// acentric in its zones. Measured on the open battery: a 622 crystal with its control at 0.528
// read its genuine 2-folds at -950 to -2640 nats, and a twinned 4/m crystal promoted to 4/mmm, its
// control at 0.454, read its added operators at -226 to -616. The zones are reported, but no
// verdict is read from them either way.
bool zones_ambiguous = false;
double anisotropy_delta_b_A2 = 0.0; // the anisotropy taken out of the intensities before normalising
double d_max_A = 0.0; // resolution range of the shells read
double d_min_A = 0.0;
rugnux/ResultReport.cpp
+11
View File
@@ -1664,6 +1664,17 @@ ReportDocument BuildReportDocument(const std::string &output_prefix,
Warn(doc, PathologyCode::TWINNING, fmt::format(
"Twinning is indicated ({}) - refinement against the merged data needs a twin law",
TwinningVerdictLine(tw)));
else if (abnormal && tw.laue_class_was_chosen_by_promotion && tw.mean_abs_l < 0.375)
// Below a perfect twin's 0.375 the L-test says nothing about the twin fraction, but it does
// say the intensities are at least as twin-like as a perfect twin's - and a perfect twin of
// a subgroup, its law absorbed by the promotion, predicts exactly the data the promoted
// group does. The promotion is then unconfirmed, however clean its operators look.
Warn(doc, PathologyCode::TWINNING, fmt::format(
"The point group was promoted although the intensities read at least as twin-like as a "
"perfect twin's (<|L|> = {:.3f}, below 0.375): a twin of a subgroup whose law the promotion "
"absorbed predicts the same data, so the adopted group is not confirmed - refine in the "
"subgroup with that twin law too (p_P1.mtz carries the same observations)",
tw.mean_abs_l));
}
// ---- radiation damage
rugnux/Rugnux.cpp
+176 -49
View File
@@ -7,6 +7,8 @@
#include "WriteModel.h"
#include "SpindleCuspLoss.h"
#include "SpotWidth.h"
#include "../image_analysis/bragg_integration/SpotFootprint.h"
#include "../image_analysis/bragg_integration/BraggStencil.h" // BRAGG_FOOTPRINT_NSIGMA
#include "HotPixels.h"
#include "../image_analysis/SensorAbsorption.h"
#include "DiagnosticOutput.h"
@@ -833,6 +835,7 @@ std::optional<CalibrationResult> Rugnux::RebinAndRefit(const CalibrationResult &
return best;
}
void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, RugnuxObserver *observer) {
Logger logger("Rugnux");
@@ -975,7 +978,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
const auto find_spots = [&](PreScanWorker &w, CompressedImage &image, int image_idx,
bool for_beam_center, std::vector<BeamCenterSpot> &out,
bool for_width, std::vector<spot_width::FluxCurve> &curves,
std::vector<float> &spot_q) {
std::vector<float> &spot_q, std::vector<FootprintSpot> &footprint) {
try {
// As in the image loops: a frame the device cannot decode goes to the host decoder.
ImageStatistics stats;
@@ -1010,10 +1013,23 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
}
}
// The width goes first: it wants the whole spot list, to know which spots are isolated.
if (for_width)
if (for_width) {
MeasureSpotFluxCurves(*w.preprocessed, static_cast<int>(prescan_x.GetXPixelsNumConv()),
static_cast<int>(prescan_x.GetYPixelsNumConv()),
prescan_x.GetDiffractionGeometry(), spots, curves);
// The footprint is measured on every spot, not only the isolated ones the width takes:
// it is the far, wide spots it is for (SpotFootprint.h).
std::vector<float> fx, fy;
for (const auto &spot : spots) {
const Coord c = spot.RawCoord();
fx.push_back(c.x);
fy.push_back(c.y);
}
const auto &g = prescan_x.GetDiffractionGeometry();
MeasureFootprintSpots(w.preprocessed->data(), static_cast<int>(prescan_x.GetXPixelsNumConv()),
static_cast<int>(prescan_x.GetYPixelsNumConv()),
g.GetBeamX_pxl(), g.GetBeamY_pxl(), fx, fy, footprint);
}
if (!for_beam_center)
return;
// The strongest of a crowded frame: the symmetry is over-determined either way, and the
@@ -1037,6 +1053,8 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
// The spots the bandwidth is measured on: the width's, and more where those are too few
// (spot_width::BANDWIDTH_POOL_SPOTS).
std::vector<spot_width::FluxCurve> shape_curves;
// Every spot's measured widths, for the spot footprint (SpotFootprint.h).
std::vector<FootprintSpot> footprint_spots;
// Images the width ended up being measured on, for the log: the tiers below stop as soon as the
// answer has settled, so this is a property of the crystal and worth reporting.
size_t width_images = 0;
@@ -1102,6 +1120,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
std::vector<std::vector<BeamCenterSpot>> spots_of(ordinals.size());
std::vector<std::vector<spot_width::FluxCurve>> curves_of(ordinals.size());
std::vector<std::vector<float>> spot_q_of(ordinals.size());
std::vector<std::vector<FootprintSpot>> footprint_of(ordinals.size());
// A frame joins the pool only if it could be read, as it did when this was a serial loop.
std::vector<char> spot_read(ordinals.size(), 0);
// Frames read for the bandwidth alone, past the tier the width settled on. Their spots stay
@@ -1185,7 +1204,7 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
if (for_beam_center) spot_read[i] = 1;
if (for_shape && !for_beam_center) shape_only[i] = 1;
find_spots(w, w.raw_image.image, image_idx, for_beam_center, spots_of[i],
for_width || for_shape, curves_of[i], spot_q_of[i]);
for_width || for_shape, curves_of[i], spot_q_of[i], footprint_of[i]);
}
}));
for (auto &f : futures) f.get();
@@ -1216,6 +1235,9 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
for (size_t i = 0; i < ordinals.size(); i++)
if (!shape_only[i])
prescan_spot_q.insert(prescan_spot_q.end(), spot_q_of[i].begin(), spot_q_of[i].end());
// In sample order, so the table does not depend on how the workers interleaved.
for (const auto &f : footprint_of)
footprint_spots.insert(footprint_spots.end(), f.begin(), f.end());
// Frame numbering follows the sample order, exactly as the serial read did.
for (size_t i = 0; i < ordinals.size(); i++) {
@@ -1278,6 +1300,43 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
"integration radii r1={:.1f} r2={:.1f} r3={:.2f}", *r80, spot_width::D_REF_A,
width_curves.size(), width_images, r1, r2, r3);
}
// How far the spots reach away from the beam, where the radius chosen above at 5 A no
// longer holds them (SpotFootprint.h). It changes the integration only where a spot
// outgrows the r1 disk.
const auto &g = experiment_.GetDiffractionGeometry();
const float bx = g.GetBeamX_pxl(), by = g.GetBeamY_pxl();
const float W = static_cast<float>(experiment_.GetXPixelsNum());
const float H = static_cast<float>(experiment_.GetYPixelsNum());
const SpotFootprint fp = FootprintFromSpots(footprint_spots,
std::hypot(std::max(bx, W - bx), std::max(by, H - by)));
// It acts only where a spot outgrows the r1 disk (BraggStencil.h), so a pattern of compact
// spots is left on exactly the settings - and the passes - it had without it.
const float r1_now = experiment_.GetBraggIntegrationSettings().GetR1();
bool outgrows = false;
for (size_t b = 0; b < fp.sigma_rad.size(); ++b)
outgrows |= BRAGG_FOOTPRINT_NSIGMA * std::max(fp.sigma_rad[b], fp.sigma_tan[b]) > r1_now;
if (fp.empty()) {
logger.Info("Spot footprint: too few spots to tabulate ({})", footprint_spots.size());
} else if (!outgrows) {
logger.Info("Spot footprint from {} spots: every spot fits the r1={:.1f} disk", footprint_spots.size(),
r1_now);
} else {
std::string table;
for (size_t b = 0; b < fp.sigma_rad.size(); ++b)
table += fmt::format(" {:.0f}:{:.1f}/{:.1f}", (b + 0.5f) * fp.bin_px,
fp.sigma_rad[b], fp.sigma_tan[b]);
logger.Info("Spot footprint from {} spots, sigma radial/tangential [px] by distance from "
"the beam [px]:{}", footprint_spots.size(), table);
// On the adaptive side, like the radius: the geometry pre-pass integrates without it,
// and a canonical pass whose wider rings the neighbours starve falls back to the
// settings before it (bragg_before_adaptive_, see RunAllPasses).
BraggIntegrationSettings bis = experiment_.GetBraggIntegrationSettings();
if (!bragg_before_adaptive_)
bragg_before_adaptive_ = bis;
bis.Footprint(fp);
experiment_.ImportBraggIntegrationSettings(bis);
}
}
// The X-ray bandwidth, read off the shapes of the same spots (spot_width::EstimateBandwidth). A
@@ -1605,8 +1664,9 @@ void Rugnux::PreScan(int start_image, int images_to_process, int frame_count, Ru
// The extra frames go to the beam centre alone; the width and the powder
// rings have already been measured on the projection's own sample.
std::vector<float> unused_q;
std::vector<FootprintSpot> unused_fp;
find_spots(w, w.raw_image.image, image_idx, true, extra_spots[i],
false, width_curves, unused_q);
false, width_curves, unused_q, unused_fp);
}
}
}));
@@ -2103,7 +2163,9 @@ namespace {
zone.standard_error, z.control.mean_abs_e2_minus_1, z.control_excess_per_reflection,
zone.evidence_nats, zone.calibrated_evidence_nats, z.d_max_A, z.d_min_A,
z.tncs_normalised ? ", normalised per pseudo-translation class" : "",
verdict < 0 ? "acentric, the added operators are a twin law or a pseudo-symmetry"
verdict < 0 && z.zones_ambiguous
? "acentric, but the control is compressed beyond a perfect twin, so genuine symmetry would read so too"
: verdict < 0 ? "acentric, the added operators are a twin law or a pseudo-symmetry"
: verdict > 0 ? "centric, the added operators are real" : "undecided");
// Recorded only where the zone DECIDED something. An undecided read decides nothing - the
// rule stays as it was - and recording it had the report announce "a twin gate refused this
@@ -3097,6 +3159,13 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
// This pass is the answer whatever it measures - the guard has had its one chance -
// so it must not conclude from its own search merge that it is going to be re-run.
quality_guard_pass1_.reset();
// What the guard preferred is pass 1 - its lattice, orientation and axis at this very
// geometry - so that is what is re-integrated, not a fresh de-novo index. A de-novo
// re-index here is a new hypothesis nobody judged: measured, at the same centre where
// pass 1 indexed 56/60 validation frames it found 5/60, took the opposite axis sign and
// shipped a merge with R_meas in the hundreds of percent.
if (prepass_result_)
force_rotation_result_ = *prepass_result_;
rerun_if_starved_ = true;
auto redo = RunPipeline(observer, /*write_output=*/true, /*geometry_prepass=*/false);
rerun_if_starved_ = false;
@@ -3105,7 +3174,9 @@ ProcessResult Rugnux::RunAllPasses(RugnuxObserver *observer) {
redo.pass_count = pass2.pass_count + 1;
redo.pass_decision = fmt::format(
"header geometry re-adopted: the post-refined pass was worse ({})", worse);
// Still forced: the re-run at the fixed radius is the same pass.
rerun_if_starved(redo, redo.pass_decision);
force_rotation_result_.reset();
return redo;
}
if (pass2.pass_decision.empty())
@@ -5475,55 +5546,83 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
// frame in six, several times below the weakest crystal that still merges. It is checked AFTER
// the long-axis rescue, so a metric that rescue recovers is never rejected on its pre-rescue
// score.
if (best.score < static_cast<int>(validation.size()) / 6) {
// The second (refined-geometry) pass re-indexes DE NOVO, and the post-refined geometry -
// fitted to the lattice pass 1 already found - can tip the blind FFT onto an axis harmonic
// of a long cell that indexes too few frames to clear this floor (measured: a tripled
// ~100 A c-axis at 7/60). Pass 1 found and integrated the true cell at the header geometry,
// so prefer its whole result rather than abort the run on a re-index only the refined
// geometry made fail. This is the floor-side companion of the supercell-collapse guard
// below: that one only reaches a pass-2 lattice that DID clear the floor, whereas here the
// harmonic fell BELOW it and the throw would run before the guard could ever see it.
if (!geometry_prepass && prepass_result_.has_value()) {
const auto &pc = prepass_result_->search_result.conventional.GetUnitCell();
// The second (refined-geometry) pass re-indexes DE NOVO, and the post-refined geometry - fitted
// to the lattice pass 1 already found - can tip the blind FFT, or the long-axis rescue above,
// onto another lattice: an axis harmonic of a long cell (measured: a tripled ~100 A c-axis at
// 7/60), or a related centred cell that indexes fewer frames than pass 1's lattice does at
// this very geometry (measured: a C-centred 303 A cell at 41/60 against pass 1's triclinic
// cell at 60/60). Pass 1's lattice is therefore carried here as a hypothesis of its own and
// the two are judged on the same validation frames: where the re-index found a different
// lattice and indexes fewer frames than pass 1's, or too few to be this crystal's lattice at
// all, pass 1's whole result is integrated instead. The same lattice found again is kept as
// the re-index refined it at this geometry. This is the floor-side companion of the
// supercell-collapse guard in RunAllPasses, which only sees a pass-2 lattice that is LARGER.
if (!geometry_prepass && prepass_result_.has_value()) {
RotationIndexerResult pass1_lattice = *prepass_result_;
// Pass 1's result carries pass 1's goniometer; forcing it whole would put the
// uncorrected angles back (the same repair as the supercell re-run in RunAllPasses).
if (prepass_rotation_scale_ && pass1_lattice.axis)
pass1_lattice.axis = ScaleRotation(*pass1_lattice.axis, *prepass_rotation_scale_);
// Pass 1's lattice was FITTED at pass 1's detector distance, and this pass
// integrates at the post-refined one. Re-scoring it here is not the same as
// re-fitting it: a real-space cell is measured against the distance the spots were
// seen at, so carrying it across a distance change scales the whole cell by the
// ratio of the two. Measured on a 545 A axis over 225 -> 226 mm: 0.4 %, exactly
// 225/226, shipped as the run's answer. Scale it to the distance it will be used
// at; the orientation is unaffected.
const float d_fit = prepass_result_->geom.GetDetectorDistance_mm();
const float d_use = experiment_.GetDetectorDistance_mm();
const bool rescaled = d_fit > 0 && d_use > 0 && d_fit != d_use;
if (rescaled) {
const float f = d_use / d_fit;
const auto &l = pass1_lattice.lattice;
pass1_lattice.lattice = CrystalLattice(l.Vec0() * f, l.Vec1() * f, l.Vec2() * f);
}
const CrystalLattice pass1_primitive = pass1_lattice.lattice
.ToPrimitive(pass1_lattice.search_result.centering).NiggliReduce();
const double pass1_vol = std::abs(pass1_primitive.CalcVolume());
// The same lattice whatever setting or class it is held in - a centred cell and the
// triclinic primitive of the same points are one lattice, and the re-index's symmetric
// setting must not lose to its own unconstrained primitive on a frame count: the Niggli-
// reduced primitive edges agree within 2 % (the battery's lattice-identity test).
const auto reduced_edges = [](const CrystalLattice &l) {
const auto uc = l.GetUnitCell();
std::array<double, 3> e = {uc.a, uc.b, uc.c};
std::sort(e.begin(), e.end());
return e;
};
const auto e1 = reduced_edges(pass1_primitive);
const auto e2 = reduced_edges(best.result->lattice
.ToPrimitive(best.result->search_result.centering).NiggliReduce());
bool same_lattice = true;
for (int i = 0; i < 3; i++)
same_lattice = same_lattice && std::abs(e2[i] - e1[i]) <= 0.02 * e1[i];
const int floor = static_cast<int>(validation.size()) / 6;
const int pass1_score = (best.score < floor || !same_lattice)
? count_indexed(*indexer, pass1_lattice) : -1;
if (best.score < floor || pass1_score > best.score) {
const auto &pc = pass1_lattice.search_result.conventional.GetUnitCell();
const auto &bc = best.result->search_result.conventional.GetUnitCell();
logger.Warning("Two-pass: the refined-geometry re-index indexes only {}/{} validation "
"frames ({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f}) - too "
"few for this crystal's lattice; integrating with pass-1's lattice "
"instead ({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f})",
logger.Warning("Two-pass: the refined-geometry re-index indexes {}/{} validation frames "
"({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} {:.2f}), pass-1's lattice "
"{}/{} at this geometry ({}-centred {}, {:.2f} {:.2f} {:.2f} {:.2f} {:.2f} "
"{:.2f}) - integrating with pass-1's lattice",
best.score, static_cast<int>(validation.size()),
best.result->search_result.centering,
gemmi::crystal_system_str(best.result->search_result.system),
bc.a, bc.b, bc.c, bc.alpha, bc.beta, bc.gamma,
prepass_result_->search_result.centering,
gemmi::crystal_system_str(prepass_result_->search_result.system),
pass1_score, static_cast<int>(validation.size()),
pass1_lattice.search_result.centering,
gemmi::crystal_system_str(pass1_lattice.search_result.system),
pc.a, pc.b, pc.c, pc.alpha, pc.beta, pc.gamma);
best.result = *prepass_result_;
// Pass 1's result carries pass 1's goniometer; forcing it whole would put the
// uncorrected angles back (the same repair as the supercell re-run in RunAllPasses).
if (prepass_rotation_scale_ && best.result->axis)
best.result->axis = ScaleRotation(*best.result->axis, *prepass_rotation_scale_);
// Pass 1's lattice was FITTED at pass 1's detector distance, and this pass
// integrates at the post-refined one. Re-scoring it here is not the same as
// re-fitting it: a real-space cell is measured against the distance the spots were
// seen at, so carrying it across a distance change scales the whole cell by the
// ratio of the two. Measured on a 545 A axis over 225 -> 226 mm: 0.4 %, exactly
// 225/226, shipped as the run's answer. Scale it to the distance it will be used
// at; the orientation is unaffected.
const float d_fit = prepass_result_->geom.GetDetectorDistance_mm();
const float d_use = experiment_.GetDetectorDistance_mm();
if (d_fit > 0 && d_use > 0 && d_fit != d_use) {
const float f = d_use / d_fit;
const auto &l = best.result->lattice;
best.result->lattice = CrystalLattice(l.Vec0() * f, l.Vec1() * f, l.Vec2() * f);
if (rescaled)
logger.Info("Two-pass: pass-1's cell was fitted at {:.3f} mm and is used at "
"{:.3f} mm - scaled by {:.6f} to the distance it is integrated at",
d_fit, d_use, f);
}
best.score = count_indexed(*indexer, *best.result); // re-score at the refined geometry
best.vol = std::abs(best.result->lattice
.ToPrimitive(best.result->search_result.centering).CalcVolume());
best.name = "pass-1 lattice (refined-geometry re-index too sparse)";
d_fit, d_use, d_use / d_fit);
best.result = std::move(pass1_lattice);
best.score = pass1_score;
best.vol = pass1_vol;
best.name = "pass-1 lattice (refined-geometry re-index found less)";
}
}
@@ -7062,11 +7161,39 @@ ProcessResult Rugnux::RunPipeline(RugnuxObserver *observer, bool write_output, b
[](const auto &a, const auto &b) { return a.first > b.first; }); // low -> high res
const int bins = std::clamp(static_cast<int>(rs.size() / 100), 1, 40);
const size_t per = (rs.size() + bins - 1) / static_cast<size_t>(bins);
for (size_t b = 0; b * per < rs.size(); ++b) {
const size_t lo = b * per, hi = std::min(rs.size(), lo + per);
// The shell means, then their best NON-INCREASING fit (pool adjacent violators, weighted
// by shell size). The cut is where the FIT drops under 1, not where the first shell
// does: on a merge whose <I/sigma> is flat near 1 - its ISa has collapsed - the first
// single shell to dip under 1 is a coin toss that moved the cut from 0.85 to 1.19 A
// between two compiler flags on one crystal, and the coarser cut left its glide zones
// too few absences to be judged. The fit moves by as much as its input does.
std::vector<double> mean, weight;
for (size_t lo = 0; lo < rs.size(); lo += per) {
const size_t hi = std::min(rs.size(), lo + per);
double sum = 0.0;
for (size_t j = lo; j < hi; ++j) sum += rs[j].second;
if (sum / static_cast<double>(hi - lo) < 1.0) {
mean.push_back(sum / static_cast<double>(hi - lo));
weight.push_back(static_cast<double>(hi - lo));
}
std::vector<double> block_mean, block_weight;
std::vector<size_t> block_size;
for (size_t b = 0; b < mean.size(); ++b) {
block_mean.push_back(mean[b]); block_weight.push_back(weight[b]); block_size.push_back(1);
while (block_mean.size() > 1 && block_mean[block_mean.size() - 2] < block_mean.back()) {
const size_t k = block_mean.size() - 1;
const double w = block_weight[k - 1] + block_weight[k];
block_mean[k - 1] = (block_mean[k - 1] * block_weight[k - 1] + block_mean[k] * block_weight[k]) / w;
block_weight[k - 1] = w;
block_size[k - 1] += block_size[k];
block_mean.pop_back(); block_weight.pop_back(); block_size.pop_back();
}
}
std::vector<double> fit;
for (size_t k = 0; k < block_mean.size(); ++k)
fit.insert(fit.end(), block_size[k], block_mean[k]);
for (size_t b = 0; b < fit.size(); ++b) {
const size_t lo = b * per;
if (fit[b] < 1.0) {
// Cut the noise-dominated high-res shells. When even the lowest-res shell fails,
// keep that shell alone rather than abandoning the cut. The bound is absolute
// while the merged I/sigma it tests saturates at the merge's own ISa (times the
tests/BraggIntegrationEngineGPUTest.cpp
+30 -2
View File
@@ -146,10 +146,16 @@ double CompareCpuVsGpu(IntegratorMode mode, std::optional<float> bandwidth_fwhm,
float stencil_k = 0.0f,
float r1 = 0.0f, float r2 = 0.0f, float r3 = 0.0f,
OverlapMode overlap = OverlapMode::Off, float companion_dx = 0.0f,
bool clip_spots = false, float odd_partiality = 1.0f) {
const DiffractionExperiment experiment =
bool clip_spots = false, float odd_partiality = 1.0f,
const SpotFootprint *footprint = nullptr) {
DiffractionExperiment experiment =
MakeExperiment(mode, bandwidth_fwhm, clip_nsigma, radial, DetJF(2), stencil_k, r1, r2, r3,
overlap);
if (footprint) {
BraggIntegrationSettings settings = experiment.GetBraggIntegrationSettings();
settings.Footprint(*footprint);
experiment.ImportBraggIntegrationSettings(settings);
}
const size_t width = experiment.GetXPixelsNum();
const size_t height = experiment.GetYPixelsNum();
const size_t npixel = experiment.GetPixelsNum();
@@ -254,6 +260,28 @@ TEST_CASE("BraggIntegrationEngineGPU_MatchesCPU") {
CompareCpuVsGpu(IntegratorMode::ProfileGaussian, 0.04f, 0.0f, false, 120, 4.0f, 6.0f, 8.0f, 12.0f);
}
SECTION("ProfileEmpirical") { CompareCpuVsGpu(IntegratorMode::ProfileEmpirical, std::nullopt); }
// A measured footprint wider than the r1 disk (SpotFootprint.h): the summation ellipse, the ring
// grown along and across the radius and the footprint-width Gaussian are all reflection-dependent
// geometry the two engines have to classify alike - spaced so the rings stay clear, and crowded
// so the grown neighbour mask and the ring gate come into play.
SpotFootprint fp;
fp.bin_px = 100.0f;
for (int b = 0; b < 8; ++b) {
fp.sigma_rad.push_back(1.5f + 0.2f * b);
fp.sigma_tan.push_back(1.8f + 0.3f * b);
}
SECTION("ProfileGaussian footprint") {
CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, false, 120, 0.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, false, 1.0f, &fp);
}
SECTION("ProfileGaussian footprint crowded exclude") {
CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, false, 40, 0.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Exclude, 0.0f, false, 1.0f, &fp);
}
SECTION("ProfileGaussian footprint radial") {
CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, true, 120, 0.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, false, 1.0f, &fp);
}
// Overlap treatment: companions 4 px apart put each reflection's centre inside its neighbour's
// signal disk, so the owner map, the excluded pixels and the profile fraction the two modes act on
// all have to come out the same in both engines - the ownership atomic in particular is settled by
tests/BraggStencilTest.cpp
+67
View File
@@ -158,3 +158,70 @@ TEST_CASE("BraggStencil_KernelIndexInRange", "[Integration][portable]") {
}
}
}
namespace {
// A footprint growing linearly from sigma 1 px at the beam to `edge` px at 800 px, radial and tangential
// alike unless told otherwise.
BraggStencilParams FootprintParams(float edge_rad, float edge_tan) {
BraggStencilParams p = Params(0.0f, 0.0f);
p.r1 = 4.0f;
p.fp_n = 8;
p.fp_bin_px = 100.0f;
for (int i = 0; i < p.fp_n; ++i) {
const float t = (i + 0.5f) / p.fp_n;
p.fp_sigma_rad[i] = 1.0f + t * (edge_rad - 1.0f);
p.fp_sigma_tan[i] = 1.0f + t * (edge_tan - 1.0f);
}
return p;
}
} // namespace
// Where the footprint fits the r1 disk (3 sigma <= r1) the stencil is the circular one, bit for bit:
// compact spots integrate exactly as without a footprint.
TEST_CASE("BraggStencil_FootprintInsideDiskIsInert", "[Integration]") {
const BraggStencilParams p = FootprintParams(1.3f, 1.3f); // 3 sigma < 4 everywhere
const BraggStencilParams none = Params(0.0f, 0.0f);
for (float py = 0.0f; py < 800.0f; py += 37.0f)
for (float px = 0.0f; px < 800.0f; px += 41.0f) {
const BraggStencil s = MakeBraggStencil(px, py, p);
const BraggStencil n = MakeBraggStencil(px, py, none);
REQUIRE(s.fp_s2r == 0.0f);
REQUIRE(s.grow == 0.0f);
REQUIRE(s.grow_tan == 0.0f);
for (int dy = -12; dy <= 12; ++dy)
for (int dx = -12; dx <= 12; ++dx) {
const auto d = BraggStencilDistances(s, static_cast<float>(dx), static_cast<float>(dy));
const auto e = BraggStencilDistances(n, static_cast<float>(dx), static_cast<float>(dy));
REQUIRE(d.inner == e.inner);
REQUIRE(d.outer == e.outer);
}
}
}
// A spot wider than the disk pushes the ring to 3 sigma along each axis separately: a pixel at 3 sigma
// along the radius (or across it) is no longer background, one just beyond the grown ring's inner edge
// is.
TEST_CASE("BraggStencil_FootprintGrowsRingAlongEachAxis", "[Integration]") {
const BraggStencilParams p = FootprintParams(3.0f, 5.0f);
const float px = 400.0f + 700.0f, py = 400.0f; // on +x, 700 px out: radial = x, tangential = y
const BraggStencil s = MakeBraggStencil(px, py, p);
float sr, st;
BraggFootprintAt(700.0f, p, sr, st);
REQUIRE(s.fp_s2r == sr * sr);
REQUIRE(s.fp_s2t == st * st);
REQUIRE_THAT(s.grow, Catch::Matchers::WithinAbs(3.0f * sr - p.r2, 1e-4));
REQUIRE_THAT(s.grow_tan, Catch::Matchers::WithinAbs(3.0f * st - p.r2, 1e-4));
const float ar = p.r2 + s.grow, at = p.r2 + s.grow_tan;
// Just inside the inner ellipse along each axis: not background.
REQUIRE(BraggStencilDistances(s, 0.98f * ar, 0.0f).inner < p.r2 * p.r2);
REQUIRE(BraggStencilDistances(s, 0.0f, 0.98f * at).inner < p.r2 * p.r2);
// Just outside: background ring.
REQUIRE(BraggStencilDistances(s, 1.02f * ar, 0.0f).inner >= p.r2 * p.r2);
REQUIRE(BraggStencilDistances(s, 0.0f, 1.02f * at).inner >= p.r2 * p.r2);
// The bounding box holds the outer ellipse.
REQUIRE(s.ex_out >= p.r3 + s.grow - 1e-3f);
REQUIRE(s.ey_out >= p.r3 + s.grow_tan - 1e-3f);
REQUIRE(BraggStencilMaxGrow_px(1000.0f, p) >= std::max(s.grow, s.grow_tan));
}
tests/CMakeLists.txt
+2
View File
@@ -17,6 +17,7 @@ ADD_EXECUTABLE(jfjoch_portable_test EXCLUDE_FROM_ALL
AzimuthalIntegrationTest.cpp
CalcBraggPredictionTest.cpp
BraggStencilTest.cpp
SpotFootprintTest.cpp
BraggIntegrationEngineCompressedImageTest.cpp
BraggIntegrationEngineCPUTest.cpp
WriteReflectionsTest.cpp
@@ -79,6 +80,7 @@ ADD_EXECUTABLE(jfjoch_test
BraggIntegrationEngineCompressedImageTest.cpp
BraggIntegrationEngineCPUTest.cpp
BraggStencilTest.cpp
SpotFootprintTest.cpp
LossyFilterTest.cpp
ImageBufferTest.cpp
PixelMaskTest.cpp
tests/RotationScaleWalkTest.cpp
+13
View File
@@ -5,6 +5,7 @@
#include <catch2/catch_all.hpp>
#include "../rugnux/Rugnux.h"
#include "../image_analysis/scale_merge/RotationScaleMerge.h"
namespace {
// A synthetic sweep whose stage turned `true_scale` times the stored angles. Scored at a scale k,
@@ -92,3 +93,15 @@ TEST_CASE("WalkRotationScale_StoredAnglesStand", "[RotationScale]") {
CHECK(sweep.index_calls == 0);
}
}
TEST_CASE("PoolHalfWidth_FitsSparseFramesOverTheirNeighbours", "[RotationScale]") {
// 20 fulls of its own: fitted on its own.
CHECK(RotationScaleMerge::PoolHalfWidth({0, 20, 0}, 1) == 0);
// 5 per frame: two frames either side bring 25 >= 20.
const std::vector<int32_t> five(11, 5);
CHECK(RotationScaleMerge::PoolHalfWidth(five, 5) == 2);
// At the end of the sweep the window grows on the one side there is.
CHECK(RotationScaleMerge::PoolHalfWidth(five, 0) == 3);
// A sweep that never holds enough stops at its length.
CHECK(RotationScaleMerge::PoolHalfWidth({1, 1, 1}, 1) == 3);
}
tests/SearchSpaceGroupTest.cpp
+11
View File
@@ -444,6 +444,17 @@ TEST_CASE("SearchSpaceGroup names an orthorhombic screw pair on the axes it lies
REQUIRE(result.best_space_group.has_value());
CHECK(result.best_space_group->xhm() == "P 2 21 21");
}
// A cell refined free after integration is a few tenths of a degree off 90. The reference
// candidates hold the same rotations and are offered on it, so the other settings must be too.
SECTION("a free cell a few tenths off 90 still offers it") {
opt.cell = gemmi::UnitCell(40.0, 50.0, 60.0, 90.32, 90.19, 90.19);
opt.enumerate_all_settings = true;
const auto result = SearchSpaceGroup(merged, opt);
INFO(SearchSpaceGroupResultToText(result));
REQUIRE(result.best_space_group.has_value());
CHECK(result.best_space_group->xhm() == "P 2 21 21");
}
}
// A screw on a row the sweep never recorded is not a group the data refused, it is a question
tests/SpotFootprintTest.cpp
+68
View File
@@ -0,0 +1,68 @@
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>
#include <cmath>
#include <vector>
#include "../image_analysis/bragg_integration/SpotFootprint.h"
// Spots drawn as Gaussians elongated along and across the radius are measured back at their widths,
// whatever their azimuth, and tabulated by distance from the beam.
TEST_CASE("SpotFootprint_MeasuresRadialAndTangentialWidths", "[Integration][portable]") {
const int W = 1200, H = 1200;
const float bx = 600.0f, by = 600.0f;
std::vector<int32_t> img(static_cast<size_t>(W) * H, 10); // flat background
std::vector<float> xs, ys;
// Width grows with distance: sigma_rad = 1 + r/200, sigma_tan = 1 + r/100.
for (int k = 0; k < 400; ++k) {
const float r = 60.0f + 480.0f * static_cast<float>(k % 20) / 20.0f;
const float phi = 0.61f * static_cast<float>(k);
const float x = bx + r * std::cos(phi), y = by + r * std::sin(phi);
const float ux = std::cos(phi), uy = std::sin(phi);
const float sr = 1.0f + r / 200.0f, st = 1.0f + r / 100.0f;
for (int py = static_cast<int>(y) - 30; py <= static_cast<int>(y) + 30; ++py)
for (int px = static_cast<int>(x) - 30; px <= static_cast<int>(x) + 30; ++px) {
if (px < 0 || py < 0 || px >= W || py >= H) continue;
const float dx = px - x, dy = py - y;
const float rad = dx * ux + dy * uy, tn = -dx * uy + dy * ux;
img[static_cast<size_t>(py) * W + px] += static_cast<int32_t>(
std::lround(2000.0f * std::exp(-rad * rad / (2 * sr * sr) - tn * tn / (2 * st * st))));
}
xs.push_back(x);
ys.push_back(y);
}
// Overlapping spots are not what this checks: keep those far from every other one.
std::vector<float> kx, ky;
for (size_t i = 0; i < xs.size(); ++i) {
bool alone = true;
for (size_t j = 0; j < xs.size(); ++j)
if (i != j && std::hypot(xs[i] - xs[j], ys[i] - ys[j]) < 45.0f) alone = false;
if (alone) { kx.push_back(xs[i]); ky.push_back(ys[i]); }
}
REQUIRE(kx.size() > 40);
std::vector<FootprintSpot> spots;
MeasureFootprintSpots(img.data(), W, H, bx, by, kx, ky, spots);
REQUIRE(spots.size() > 30);
for (const auto &s : spots) {
CHECK_THAT(s.sigma_rad, Catch::Matchers::WithinRel(1.0f + s.r_px / 200.0f, 0.12f));
CHECK_THAT(s.sigma_tan, Catch::Matchers::WithinRel(1.0f + s.r_px / 100.0f, 0.12f));
}
}
TEST_CASE("SpotFootprint_TableFillsSparseBinsFromNeighbours", "[Integration][portable]") {
std::vector<FootprintSpot> spots;
for (int i = 0; i < FOOTPRINT_MIN_SPOTS_PER_BIN; ++i) {
spots.push_back({50.0f, 1.0f, 1.5f}); // bin 0
spots.push_back({1150.0f, 3.0f, 4.0f}); // last bin
}
const SpotFootprint fp = FootprintFromSpots(spots, 1200.0f);
REQUIRE(fp.sigma_rad.size() == static_cast<size_t>(FOOTPRINT_BINS));
REQUIRE(fp.sigma_rad.front() == 1.0f);
REQUIRE(fp.sigma_tan.back() == 4.0f);
REQUIRE(fp.sigma_rad[2] == 1.0f); // nearer the first bin
REQUIRE(fp.sigma_rad[FOOTPRINT_BINS - 3] == 3.0f); // nearer the last
REQUIRE(FootprintFromSpots({}, 1200.0f).empty());
}
tools/battery/open.json
-1
View File
@@ -198,7 +198,6 @@
{"id": "8v2t", "input": "8v2t/data/MJ504_2_3_00001.cbf", "wavelength": 1.1, "ref": {"sg": "P 42 21 2", "sgno": 94, "cell": [60.906, 60.906, 92.728, 90.0, 90.0, 90.0], "dmin": 1.402}, "tags": ["cbf", "tetragonal"]},
{"id": "5jk4", "input": "5jk4/dioxyhipeg/dioxyhipeg_1_001.img", "wavelength": 0.933, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [37.729, 77.856, 56.287, 90.0, 102.08, 90.0], "dmin": 1.1}, "tags": ["smv", "monoclinic"], "pinned": true},
{"id": "6cs9", "input": "6cs9/568/HBD2D7_2_1_001.img", "wavelength": 0.9537, "ref": {"sg": "P 1 21 1", "sgno": 4, "cell": [32.871, 25.538, 40.17, 90.0, 98.64, 90.0], "dmin": 1.85}, "tags": ["smv", "monoclinic"]},
{"id": "7udi", "input": "7udi/data/MJ7121_0001.cbf", "wavelength": 0.9795, "ref": {"sg": "P 41", "sgno": 76, "cell": [66.698, 66.698, 129.581, 90.0, 90.0, 90.0], "dmin": 2.24}, "tags": ["cbf", "tetragonal"]},
{"id": "9lxl", "input": "9lxl/6/1_T10Y-FFp-PNPI_6_master.h5", "wavelength": 0.97919, "ref": {"sg": "P 41 21 2", "sgno": 92, "cell": [76.757, 76.757, 225.338, 90.0, 90.0, 90.0], "dmin": 2.191}, "tags": ["h5", "tetragonal"]},
{"id": "9s02", "input": "9s02/PYCR1-D11_1_master.h5", "wavelength": 0.97625, "ref": {"sg": "P 21 21 2", "sgno": 18, "cell": [163.674, 88.042, 116.731, 90.0, 90.0, 90.0], "dmin": 1.65}, "tags": ["h5", "orthorhombic"]},
{"id": "6gvk", "input": "6gvk/212_3_0001.cbf", "wavelength": 0.97915, "ref": {"sg": "C 1 2 1", "sgno": 5, "cell": [105.58, 59.52, 42.4, 90.0, 113.5, 90.0], "dmin": 1.55}, "tags": ["cbf", "monoclinic"], "pinned": true}