The gain of an input in detector units (SMV, marCCD, XTC, the XFEL formats) is measured once, before anything reads the frames (RugnuxMeasureGain: photon-peak spacing and background variance per unit mean), or given with --gain, and handed to the integration and the spot finding by one function (ApplyRugnuxPixelGain). The rotation pre-scan no longer measures it, and the stills reader no longer converts frames of a detector whose photons are not resolved: SACLA MPCCD, Rayonix and CCD frames are integrated in ADU at the gain, rotation and stills alike. Where a photon count was assumed, it is now taken at the gain (bit-identical at gain 1): - integration: the background clip's one-photon floor, the sigma floor (one photon), the profile fit's background floor (0.01 photon), and a still's overload kept as a lower bound, which was left in ADU while every other reflection left the integrator in photons - the cause of the earlier loss of the engine route on clipped CCD stills (merged low resolution inflated by the gain, the rocking-width fit's bandwidth collapsing); CPU and CUDA, Metal through the shared CPU code; - the adaptive spot threshold (Poisson arm and its one-photon floor), CPU and GPU; - the pre-scan spot-width measurement (strong-spot SNR, the ring clip) behind the stills bandwidth and the rotation signal radius; - the hot/dead pixel tests, CPU and GPU; the empty-aperture ring clip. BackgroundPixelGain is gone: one pixel_gain. A detector whose background resolves single photons (photon peaks: JUNGFRAU in keV, AGIPD) is a photon counter and its frames are still counted in whole photons as read: analysed in keV at the gain instead, a photon-counting still set lost 0.008 R-free (fewer spots, a noisier beam-centre trial) - the rounding is the detector's own discrimination. --gain sets the value, the background still says which kind. Validation, against rc178: photon-counting and g<=1 rotation p.hkl md5-identical (lyso, thau, cytc, myob, insu, a small-molecule set, 5reo, 6cee, 6i3j, 2xfw, 6s1u, 8v4j); CCD g>1 at or within noise of rc178 (5uth R_meas 0.113 -> 0.108). Stills R-free (0-cycle REFMAC): MPCCD sets 0.184 -> 0.185 and 0.199 -> 0.196, Rayonix identical, photon-resolving sets back on their rc178 numbers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu (cherry picked from commit da8dbfc8147354b63e375f2d9521dffb80938b6d)
113 lines
6.3 KiB
C++
113 lines
6.3 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
// Per-resolution-ring detection-threshold math shared by the CPU adaptive spot finder
|
|
// (AdaptiveSpotFinderCPU) and its GPU fused counterpart (AdaptiveSpotFinderGPU). Both engines reduce
|
|
// every pixel into resolution rings, take a robust per-ring background (mean, sigma), and turn it into
|
|
// a strong-pixel threshold with the SAME formula - so keeping that formula in one place is what makes
|
|
// the GPU engine reproduce the CPU one. These are plain host functions (the threshold is computed on
|
|
// the host in both engines, once per frame, over the small per-ring arrays).
|
|
|
|
#include <algorithm>
|
|
#include <cmath>
|
|
#include <cstdint>
|
|
|
|
namespace adaptive_threshold {
|
|
|
|
// Number of background pixels a ring needs before its own statistics are trusted; sparser rings
|
|
// (detector corners, heavily masked, innermost) fall back to the whole-frame background.
|
|
constexpr int64_t MIN_RING_PIXELS = 40;
|
|
|
|
// Detector-level excess-noise floor (photons). Near-zero-background rings scatter MORE than pure
|
|
// Poisson (charge sharing / read noise / occasional spurious low counts), so a per-ring sigma alone
|
|
// collapses toward zero on empty high-resolution rings and the threshold would flood. READ is a
|
|
// photon-scale constant (the same for every dataset -- NOT the per-dataset knob), so the operating
|
|
// point still self-calibrates through mean and sigma while staying physical where the background
|
|
// vanishes.
|
|
constexpr float READ = 1.0f;
|
|
|
|
// Inverse standard-normal CDF (Acklam's rational approximation, ~1e-9 accuracy). Only called once
|
|
// per frame, so accuracy over speed.
|
|
inline double NormalQuantile(double p) {
|
|
if (p <= 0.0) return -40.0;
|
|
if (p >= 1.0) return 40.0;
|
|
static const double a[] = {-3.969683028665376e+01, 2.209460984245205e+02, -2.759285104469687e+02,
|
|
1.383577518672690e+02, -3.066479806614716e+01, 2.506628277459239e+00};
|
|
static const double b[] = {-5.447609879822406e+01, 1.615858368580409e+02, -1.556989798598866e+02,
|
|
6.680131188771972e+01, -1.328068155288572e+01};
|
|
static const double c[] = {-7.784894002430293e-03, -3.223964580411365e-01, -2.400758277161838e+00,
|
|
-2.549732539343734e+00, 4.374664141464968e+00, 2.938163982698783e+00};
|
|
static const double d[] = {7.784695709041462e-03, 3.224671290700398e-01, 2.445134137142996e+00,
|
|
3.754408661907416e+00};
|
|
const double plow = 0.02425, phigh = 1.0 - 0.02425;
|
|
if (p < plow) {
|
|
double q = std::sqrt(-2.0 * std::log(p));
|
|
return (((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) /
|
|
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1.0);
|
|
} else if (p <= phigh) {
|
|
double q = p - 0.5, r = q*q;
|
|
return (((((a[0]*r+a[1])*r+a[2])*r+a[3])*r+a[4])*r+a[5])*q /
|
|
(((((b[0]*r+b[1])*r+b[2])*r+b[3])*r+b[4])*r+1.0);
|
|
} else {
|
|
double q = std::sqrt(-2.0 * std::log(1.0 - p));
|
|
return -(((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) /
|
|
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1.0);
|
|
}
|
|
}
|
|
|
|
// Smallest integer count whose Poisson(mu) upper tail P(X >= k) <= p. This is the correct
|
|
// significance floor while the background is countable (it carries the sqrt(mu) shot-noise
|
|
// implicitly, so a bright low-resolution ring gets a high threshold). It DEGENERATES at mu -> 0
|
|
// (a single photon on a zero background is "significant"), which is why it is max'd with a
|
|
// read-noise-floored Gaussian arm by the caller.
|
|
//
|
|
// Past the summation limit the quantile is taken from the Cornish-Fisher expansion
|
|
// (Cornish and Fisher (1938) Rev. Int. Stat. Inst. 5, 307-320), whose skewness
|
|
// term (z^2-1)/6 is what a plain mu + z*sqrt(mu) leaves out. At the 4-6 sigma this operating point
|
|
// works at, that term is 3-6 counts, so the Gaussian form alone stood BELOW the true Poisson
|
|
// quantile - and it did so with a step at the switch, since below it the exact quantile was used.
|
|
// Cornish-Fisher is within one count of the exact value at every mu, so the two arms now join
|
|
// smoothly.
|
|
//
|
|
// How much this is worth depends on which arm of RingThreshold wins, and on measured data it is
|
|
// often neither: where the ring background is over-dispersed (a clipped ring sigma that still
|
|
// carries the ring's own azimuthal structure, 1.2-4.9x sqrt(mu) on a strongly diffracting rotation
|
|
// set) the Gaussian arm is the larger of the two on every ring above mu = 50 and this correction
|
|
// changes no threshold at all. It is the right value to return regardless: a caller that ever sees
|
|
// the Poisson arm win up there would otherwise get a bar that jumps at mu = 50.
|
|
inline float PoissonThreshold(double mu, double p, double z) {
|
|
// The summation below needs k up to about mu + z*sqrt(mu), and exp(-mu) has to stay normal.
|
|
constexpr double SUM_LIMIT = 200.0;
|
|
if (mu > SUM_LIMIT)
|
|
return static_cast<float>(mu + z * std::sqrt(mu) + (z * z - 1.0) / 6.0);
|
|
if (mu < 1e-6) mu = 1e-6;
|
|
const double target = 1.0 - p;
|
|
double pmf = std::exp(-mu);
|
|
double cdf = pmf;
|
|
int k = 0;
|
|
while (cdf < target && k < 1000) {
|
|
++k;
|
|
pmf *= mu / k;
|
|
cdf += pmf;
|
|
}
|
|
return static_cast<float>(k + 1);
|
|
}
|
|
|
|
// A ring's threshold is background mean + z sigmas, computed two ways and max'd: Poisson significance
|
|
// (correct where the background is countable) floored by a read-noise-aware Gaussian arm (which alone
|
|
// survives mean -> 0, where Poisson degenerates to "one photon is significant" and would flood the
|
|
// empty high-resolution rings). p, z are the frame-wide operating point (p = E / N_pixels). mean and
|
|
// sigma are in the frame's own units, gain of them to a photon (SpotFindingSettings::adu_per_photon):
|
|
// the Poisson arm counts photons and the floor is a photon, so both are taken in photons and handed
|
|
// back in the frame's units. A gain of 1 is exactly the photon-counting threshold.
|
|
inline float RingThreshold(float mean, float sigma, double p, float z, float gain) {
|
|
const float read = READ * gain;
|
|
const float gauss = mean + z * std::sqrt(sigma * sigma + read * read);
|
|
const float poisson = gain * PoissonThreshold(static_cast<double>(mean / gain), p, static_cast<double>(z));
|
|
return std::max(gauss, poisson);
|
|
}
|
|
|
|
} // namespace adaptive_threshold
|