Files
Jungfraujoch/image_analysis/spot_finding/AdaptiveThreshold.h
T
leonarski_f 711f902aa7 One gain mechanism: frames analysed in their own units, the gain applied where a photon count is assumed
The gain of an input in detector units (SMV, marCCD, XTC, the XFEL formats) is measured once, before
anything reads the frames (RugnuxMeasureGain: photon-peak spacing and background variance per unit
mean), or given with --gain, and handed to the integration and the spot finding by one function
(ApplyRugnuxPixelGain). The rotation pre-scan no longer measures it, and the stills reader no longer
converts frames of a detector whose photons are not resolved: SACLA MPCCD, Rayonix and CCD frames are
integrated in ADU at the gain, rotation and stills alike.

Where a photon count was assumed, it is now taken at the gain (bit-identical at gain 1):
- integration: the background clip's one-photon floor, the sigma floor (one photon), the profile fit's
  background floor (0.01 photon), and a still's overload kept as a lower bound, which was left in ADU
  while every other reflection left the integrator in photons - the cause of the earlier loss of the
  engine route on clipped CCD stills (merged low resolution inflated by the gain, the rocking-width
  fit's bandwidth collapsing); CPU and CUDA, Metal through the shared CPU code;
- the adaptive spot threshold (Poisson arm and its one-photon floor), CPU and GPU;
- the pre-scan spot-width measurement (strong-spot SNR, the ring clip) behind the stills bandwidth and
  the rotation signal radius;
- the hot/dead pixel tests, CPU and GPU; the empty-aperture ring clip.
BackgroundPixelGain is gone: one pixel_gain.

A detector whose background resolves single photons (photon peaks: JUNGFRAU in keV, AGIPD) is a photon
counter and its frames are still counted in whole photons as read: analysed in keV at the gain instead,
a photon-counting still set lost 0.008 R-free (fewer spots, a noisier beam-centre trial) - the rounding
is the detector's own discrimination. --gain sets the value, the background still says which kind.

Validation, against rc178: photon-counting and g<=1 rotation p.hkl md5-identical (lyso, thau, cytc,
myob, insu, a small-molecule set, 5reo, 6cee, 6i3j, 2xfw, 6s1u, 8v4j); CCD g>1 at or within noise of rc178 (5uth
R_meas 0.113 -> 0.108). Stills R-free (0-cycle REFMAC): MPCCD sets 0.184 -> 0.185 and 0.199 -> 0.196,
Rayonix identical, photon-resolving sets back on their rc178 numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
(cherry picked from commit da8dbfc8147354b63e375f2d9521dffb80938b6d)
2026-10-23 17:56:51 +02:00

113 lines
6.3 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
// Per-resolution-ring detection-threshold math shared by the CPU adaptive spot finder
// (AdaptiveSpotFinderCPU) and its GPU fused counterpart (AdaptiveSpotFinderGPU). Both engines reduce
// every pixel into resolution rings, take a robust per-ring background (mean, sigma), and turn it into
// a strong-pixel threshold with the SAME formula - so keeping that formula in one place is what makes
// the GPU engine reproduce the CPU one. These are plain host functions (the threshold is computed on
// the host in both engines, once per frame, over the small per-ring arrays).
#include <algorithm>
#include <cmath>
#include <cstdint>
namespace adaptive_threshold {
// Number of background pixels a ring needs before its own statistics are trusted; sparser rings
// (detector corners, heavily masked, innermost) fall back to the whole-frame background.
constexpr int64_t MIN_RING_PIXELS = 40;
// Detector-level excess-noise floor (photons). Near-zero-background rings scatter MORE than pure
// Poisson (charge sharing / read noise / occasional spurious low counts), so a per-ring sigma alone
// collapses toward zero on empty high-resolution rings and the threshold would flood. READ is a
// photon-scale constant (the same for every dataset -- NOT the per-dataset knob), so the operating
// point still self-calibrates through mean and sigma while staying physical where the background
// vanishes.
constexpr float READ = 1.0f;
// Inverse standard-normal CDF (Acklam's rational approximation, ~1e-9 accuracy). Only called once
// per frame, so accuracy over speed.
inline double NormalQuantile(double p) {
if (p <= 0.0) return -40.0;
if (p >= 1.0) return 40.0;
static const double a[] = {-3.969683028665376e+01, 2.209460984245205e+02, -2.759285104469687e+02,
1.383577518672690e+02, -3.066479806614716e+01, 2.506628277459239e+00};
static const double b[] = {-5.447609879822406e+01, 1.615858368580409e+02, -1.556989798598866e+02,
6.680131188771972e+01, -1.328068155288572e+01};
static const double c[] = {-7.784894002430293e-03, -3.223964580411365e-01, -2.400758277161838e+00,
-2.549732539343734e+00, 4.374664141464968e+00, 2.938163982698783e+00};
static const double d[] = {7.784695709041462e-03, 3.224671290700398e-01, 2.445134137142996e+00,
3.754408661907416e+00};
const double plow = 0.02425, phigh = 1.0 - 0.02425;
if (p < plow) {
double q = std::sqrt(-2.0 * std::log(p));
return (((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) /
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1.0);
} else if (p <= phigh) {
double q = p - 0.5, r = q*q;
return (((((a[0]*r+a[1])*r+a[2])*r+a[3])*r+a[4])*r+a[5])*q /
(((((b[0]*r+b[1])*r+b[2])*r+b[3])*r+b[4])*r+1.0);
} else {
double q = std::sqrt(-2.0 * std::log(1.0 - p));
return -(((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) /
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1.0);
}
}
// Smallest integer count whose Poisson(mu) upper tail P(X >= k) <= p. This is the correct
// significance floor while the background is countable (it carries the sqrt(mu) shot-noise
// implicitly, so a bright low-resolution ring gets a high threshold). It DEGENERATES at mu -> 0
// (a single photon on a zero background is "significant"), which is why it is max'd with a
// read-noise-floored Gaussian arm by the caller.
//
// Past the summation limit the quantile is taken from the Cornish-Fisher expansion
// (Cornish and Fisher (1938) Rev. Int. Stat. Inst. 5, 307-320), whose skewness
// term (z^2-1)/6 is what a plain mu + z*sqrt(mu) leaves out. At the 4-6 sigma this operating point
// works at, that term is 3-6 counts, so the Gaussian form alone stood BELOW the true Poisson
// quantile - and it did so with a step at the switch, since below it the exact quantile was used.
// Cornish-Fisher is within one count of the exact value at every mu, so the two arms now join
// smoothly.
//
// How much this is worth depends on which arm of RingThreshold wins, and on measured data it is
// often neither: where the ring background is over-dispersed (a clipped ring sigma that still
// carries the ring's own azimuthal structure, 1.2-4.9x sqrt(mu) on a strongly diffracting rotation
// set) the Gaussian arm is the larger of the two on every ring above mu = 50 and this correction
// changes no threshold at all. It is the right value to return regardless: a caller that ever sees
// the Poisson arm win up there would otherwise get a bar that jumps at mu = 50.
inline float PoissonThreshold(double mu, double p, double z) {
// The summation below needs k up to about mu + z*sqrt(mu), and exp(-mu) has to stay normal.
constexpr double SUM_LIMIT = 200.0;
if (mu > SUM_LIMIT)
return static_cast<float>(mu + z * std::sqrt(mu) + (z * z - 1.0) / 6.0);
if (mu < 1e-6) mu = 1e-6;
const double target = 1.0 - p;
double pmf = std::exp(-mu);
double cdf = pmf;
int k = 0;
while (cdf < target && k < 1000) {
++k;
pmf *= mu / k;
cdf += pmf;
}
return static_cast<float>(k + 1);
}
// A ring's threshold is background mean + z sigmas, computed two ways and max'd: Poisson significance
// (correct where the background is countable) floored by a read-noise-aware Gaussian arm (which alone
// survives mean -> 0, where Poisson degenerates to "one photon is significant" and would flood the
// empty high-resolution rings). p, z are the frame-wide operating point (p = E / N_pixels). mean and
// sigma are in the frame's own units, gain of them to a photon (SpotFindingSettings::adu_per_photon):
// the Poisson arm counts photons and the floor is a photon, so both are taken in photons and handed
// back in the frame's units. A gain of 1 is exactly the photon-counting threshold.
inline float RingThreshold(float mean, float sigma, double p, float z, float gain) {
const float read = READ * gain;
const float gauss = mean + z * std::sqrt(sigma * sigma + read * read);
const float poisson = gain * PoissonThreshold(static_cast<double>(mean / gain), p, static_cast<double>(z));
return std::max(gauss, poisson);
}
} // namespace adaptive_threshold