Bragg integration: elongate the background ring per reflection

The signal disk and the r2..r3 background ring were fixed pixel circles, identical for every
reflection at every resolution. A reflection is not round: a finite bandwidth streaks it radially by
bw_sigma*Rpx, so at high resolution the ring sits within 1.3-2.2 sigma of the reflection's own
profile and measures its tails as background.

--integration-stencil <k> makes the RING an ellipse, elongated along the beam->reflection direction
by k times that streak, capped at 2*r3. The tangential half-widths stay r2 and r3, and the r1 signal
disk stays a circle: r1 drives the all-or-nothing n_inner_valid == n_inner gate, so growing it
rejects any reflection carrying one bad pixel along a long streak, and the flux a circular r1 loses
is a function of resolution alone, which the per-shell scale absorbs.

The geometry lives in one shared header compiled by both the host compiler and nvcc, so the seven
pixel-classification sites - the CPU mask/main/clip loops and the GPU mark_mask/main/trim/clip
kernels - cannot drift apart. Rather than evaluate an ellipse, each pixel's squared distance has its
radial part scaled down, d2 - q*rad^2 against r2^2/r3^2 with q = 1 - (r/(r+grow))^2, so grow = 0
gives q = 0 and both tests collapse onto d2 exactly in floating point.

The width is the bandwidth streak alone, not the profile's full radial variance, which also carries
the sensor parallax and weak-spot capture terms. Deriving the growth from those was implemented
first and measured on the rotation battery: at k=1 it took Thau_9's high-shell CC1/2 from 75.8 to
27.9 and Benas_3's from 14.1 to 6.0, against cytC_10 +1.2 and lyso_ref flat. On a monochromatic beam
they are the only terms there are, and C_CAPTURE is 64% of them. Keeping only the streak also makes
the option exactly inert without a bandwidth, rather than merely small.

Default 0. Measured on broadband rotation data with the bandwidth set to its spectroscopic value,
matched resolution limits: high-shell CC1/2 30.6 -> 46.4 at k=4, and better in EVERY shell in both
CC1/2 and R_meas (top shell R_meas 194.7% -> 138.7%), with completeness, multiplicity and space
group unchanged and 28 of 98833 unique reflections lost. Anomalous peak height over 18 sites
+0.107 +- 0.039 sigma (p = 0.013). The full 38-crystal rotation battery is unchanged to every
reported digit, base against k=3.

Two consequences of an elongated ring are handled rather than inherited. The neighbour exclusion
marks the inner ELLIPSE in each neighbour's own frame, or an elongated neighbour leaks its tails
into this reflection's ring. And the radial-background curvature kernel becomes a small table
indexed by the growth, because its azimuthal average makes one kernel serve every reflection only
while their stencils are identical; the GPU's radial window, previously a fixed 32 bins, is now
sized on the host from the widest ring on the detector.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-10 15:19:28 +02:00
co-authored by Claude Opus 5
parent 52ea727650
commit 61d24db59f
16 changed files with 656 additions and 120 deletions
+1
View File
@@ -3,6 +3,7 @@
### 1.0.0-rc.161
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.
* rugnux: The background ring can be **elongated radially per reflection** for broadband data (`--integration-stencil <k>`, default 0 = the fixed circular ring), by `k` times the beam's radial streak; the `r1` signal box stays circular.
* rugnux: New **beam-stop shadow detection**, **on by default** (`--detect-beam-stop[=N|off]`), finds the beam stop and its holder in a projection of N images (default 60) and adds them to the pixel mask as bit 9, which is cleared at the start of every run.
* Viewer: the detected beam-stop shadow is drawn in coral, with a "Show beam stop" switch in the side panel.
* rugnux: Scaling and merging now apply a **low-resolution limit** of 50 Å (`--scaling-low-resolution <num>`, 0 removes it).
+21 -4
View File
@@ -565,7 +565,24 @@ Pixels are classified by their squared distance $r^2=(x-x_p)^2+(y-y_p)^2$:
- **signal region:** $r^2 < r_1^2$,
- **background annulus:** $r_2^2 \le r^2 < r_3^2$.
Invalid pixels (masked/bad/saturated) are excluded from both sums. In addition, pixels lying inside the signal disk ($r<r_2$) of any *other* predicted reflection are removed from this reflection's background annulus, so a neighbouring spot cannot leak into the background estimate.
Invalid pixels (masked/bad/saturated) are excluded from both sums. In addition, pixels lying inside the signal disk ($r<r_2$) of any *other* predicted reflection are removed from this reflection's background annulus, so a neighbouring spot cannot leak into the background estimate. (Both the annulus and that exclusion become ellipses when the option below is used; with it off, which is the default, they are the circles just described.)
**Radially elongated background ring (opt-in, `--integration-stencil <k>`, default 0).** The three radii above are fixed pixel counts, identical for every reflection at every resolution. A reflection is not round, though: a finite bandwidth streaks it radially by $\sigma_\mathrm{bw}=\text{bandwidth}\cdot R_\mathrm{px}$. On a radially smeared spot the fixed $6\ldots10$ px ring therefore sits only $\approx1.3$$2.2$ radial $\sigma$ from the centre — on the reflection's own tails, which it then measures as background.
With $k>0$ the **background ring becomes an ellipse**, elongated along the beam→reflection direction by $k\sigma_\mathrm{bw}$. The **radial** semi-axes become $r_2+k\sigma_\mathrm{bw}$ and $r_3+k\sigma_\mathrm{bw}$; the **tangential** half-widths stay $r_2$ and $r_3$; and the growth is capped at $2r_3$, which bounds what a mis-declared bandwidth can do to the bounding box. Pixels are then classified as
- **signal region:** $r^2 < r_1^2$ — a circle, unchanged,
- **background ring:** $r^2-q_\mathrm{in}\rho^2 \ge r_2^2$ **and** $r^2-q_\mathrm{out}\rho^2 < r_3^2$,
where $\rho$ is the pixel's radial offset (its projection on the beam→reflection direction), $g=\min(k\sigma_\mathrm{bw},\,2r_3)$ is the capped growth, and $q=1-\big(r/(r+g)\big)^2$ for the boundary concerned. Written this way $k=0$ gives $q=0$ and both tests collapse onto $r^2$ **exactly in floating point**, so the default classifies every pixel exactly as the circular stencil did. The neighbour exclusion above follows: each neighbour's **inner ellipse**, taken in that neighbour's own radial frame, is what is masked out of this reflection's ring.
The width is the bandwidth streak alone, and deliberately **not** the profile's full radial variance of §9.3, which also carries the sensor parallax and weak-spot capture terms. Those two are the only terms there are on a monochromatic beam, and widening the ring by them was measured on the rotation battery: it neither helped the crystals with clean high-resolution shells nor left the weak ones alone. The bandwidth streak, by contrast, is a measured elongation of the recorded spot — principal axis along the radius to within a couple of degrees, and azimuth-independent. Keeping only it also makes the option exactly inert on a monochromatic beam, where $\sigma_\mathrm{bw}$ is zero.
One caveat on "exactly": the *classification* is bit-identical at $k=0$, and so is every intensity on the default path. The radial-background curve of §9.2 is a separate matter — its per-pixel radial offset used to be computed in double precision on the CPU and is now computed in single, as it always was on the GPU, so with `--background-radial` on a pixel can change radial bin. That brings the two engines together rather than apart, but it does mean the CPU is not bit-compatible with earlier builds on that one path.
Growing the ring also grows the neighbour exclusion, so on a crowded pattern fewer background pixels survive; a reflection left with too few is rejected outright. On the data this was measured on the loss is under 0.1% of reflections, but it is not structurally zero.
Only the ring moves. The signal disk $r_1$ stays circular, deliberately: it sets $n_S$, it sets $\mathrm{var}(\hat b)$, and it drives the all-or-nothing “every signal pixel valid” acceptance gate (§9.2), so growing it would reject any reflection carrying a single bad pixel anywhere along a long streak. What a circular $r_1$ loses is flux, and that loss is a function of resolution alone, which the per-shell scale absorbs.
### 9.2 Box summation (seed and fallback)
@@ -580,7 +597,7 @@ $
\hat{b} = \frac{B}{n_B},\qquad
\hat{I} = S - n_S \hat{b},
$
with a Poisson-like uncertainty $\sigma(\hat{I})=\max\!\big(1,\ r_\sigma\hat{I},\ \sqrt{S + n_S^2\,\mathrm{var}(\hat{b})}\big)$, i.e. $\sqrt{S}$ floored both at 1 and at a small fraction $r_\sigma$ of the intensity. The second term under the root is the **uncertainty of the background estimate itself**: $\hat b$ is measured from a finite number of ring pixels, $\mathrm{var}(\hat b)=\hat b/n_B$, and it is subtracted $n_S$ times over, so it enters squared. Omitting it understates $\sigma$ by $\sqrt{1+n_S/n_B}$ — 1.109 with the shipped stencil — uniformly, on every reflection of every dataset. The same term is carried into the profile fit (§9.3), where it adds $(\sum wP/\sum P^2/v)^2\,\mathrm{var}(\hat b)$; $n_B$ is the count of pixels behind the *final* background value, so a clip or trim that discards ring pixels raises it. A reflection is accepted as “observed” only if all signal pixels were valid and $n_B$ exceeds a minimum. This box sum is the classical estimator; it is used directly with `--integrator boxsum`, and otherwise seeds the profile fit below.
with a Poisson-like uncertainty $\sigma(\hat{I})=\max\!\big(1,\ r_\sigma\hat{I},\ \sqrt{S + n_S^2\,\mathrm{var}(\hat{b})}\big)$, i.e. $\sqrt{S}$ floored both at 1 and at a small fraction $r_\sigma$ of the intensity. The second term under the root is the **uncertainty of the background estimate itself**: $\hat b$ is measured from a finite number of ring pixels, $\mathrm{var}(\hat b)=\hat b/n_B$, and it is subtracted $n_S$ times over, so it enters squared. Omitting it understates $\sigma$ by $\sqrt{1+n_S/n_B}$ — 1.109 with the shipped circular stencil; with an elongated ring $n_B$ grows with resolution, so the factor is no longer one number for a run — uniformly, on every reflection of every dataset. The same term is carried into the profile fit (§9.3), where it adds $(\sum wP/\sum P^2/v)^2\,\mathrm{var}(\hat b)$; $n_B$ is the count of pixels behind the *final* background value, so a clip or trim that discards ring pixels raises it. A reflection is accepted as “observed” only if all signal pixels were valid and $n_B$ exceeds a minimum. This box sum is the classical estimator; it is used directly with `--integrator boxsum`, and otherwise seeds the profile fit below.
**High-side clipped background (default on).** Because $\hat{I}=S-n_S\hat{b}$ is a small difference of large numbers for weak reflections, a per-pixel background bias $\delta\hat{b}$ becomes a *fractional* intensity bias $\approx n_S\,\delta\hat{b}/\hat{I}$ that grows as $\hat{I}$ shrinks — worst at the resolution edge. A plain ring mean reads high there, because neighbour-spot wings that survive the signal-disk mask, tails and zingers are one-sided (positive) contaminants. The ring mean is therefore made robust: pixels above $\hat{b}+n\sqrt{\hat{b}}$ are rejected and the mean recomputed, with $n=4$ (`--background-clip`; $n=0$ disables), lowered by `rugnux` to $n=3$ on broadband (non-zero bandwidth: pink-beam / DMM) data, where a bandwidth-streaked high-resolution spot leaks into the ring more readily. That is only a default — the flag sets $n$ whatever the bandwidth is. A clean Poisson ring is essentially unchanged by the cut (measured false-rejection rate 0.040.39 % at $4\sigma$), while a 40-pixel neighbour core at $+100$ counts shifts the estimate by $+0.009$ ct/px.
@@ -588,11 +605,11 @@ The clip cuts only the high tail, which matters: the **symmetric** trimmed mean
Both estimators are computed in the shared background pass, but only the trim reaches plain box summation: the high-side clip is skipped for `--integrator boxsum`, which therefore uses the plain ring mean unless `--background-trim` is given.
**Radial background correction (opt-in).** A ring mean estimates the background *under* the signal disk correctly only if the background is flat there. The signal disk and the ring are concentric, so for a background that is **linear** in position $\langle B\rangle_\mathrm{ring}=\langle B\rangle_\mathrm{disk}$ identically — a plane or gradient fit buys exactly nothing. The leading error is the **curvature** of the radial background, which is negligible on a smooth background but reaches tens of counts on a single reflection sitting on a sharp powder ring. Because every reflection uses the same stencil, that error is a fixed kernel over radial offset,
**Radial background correction (opt-in).** A ring mean estimates the background *under* the signal disk correctly only if the background is flat there. The signal disk and the ring are concentric, so for a background that is **linear** in position $\langle B\rangle_\mathrm{ring}=\langle B\rangle_\mathrm{disk}$ identically — a plane or gradient fit buys exactly nothing. The leading error is the **curvature** of the radial background, which is negligible on a smooth background but reaches tens of counts on a single reflection sitting on a sharp powder ring. That error is a kernel over radial offset,
$
\delta \hat b \;=\; \textstyle\sum_k \kappa_k\, \bar B(r_0+k),
$
with $\kappa$ obtained once by azimuth-averaging the stencil and $\bar B(r)$ the image's own radial background curve. Applying it costs one short dot product per reflection and no extra pixel reads; correcting the background *scalar* means the box sum, the profile fit and the variance all pick it up. The curve is accumulated from the same annulus pixels the background pass already reads (a pixel's radius is the reflection's radius plus the pixel's projection on the beam→reflection direction, so no per-pixel square root is needed) and specifically from the **clipped** pixels, or it would carry neighbour tails and zingers — which is why the correction is inert under `--integrator boxsum`, that path having no clip pass.
with $\kappa$ the annulus-minus-disk histogram of the stencil over radial offset, averaged over azimuth, and $\bar B(r)$ the image's own radial background curve. With the fixed circular stencil ($k=0$, §9.1) that single kernel serves every reflection. An elongated ring does not: its radial-offset histogram depends on how far that particular reflection's ring was grown, so $\kappa$ becomes a small table of kernels, indexed by the growth rounded to whole pixels. The azimuthal average survives the change unaltered, because the stencil is rebuilt in the reflection's own frame at each azimuth and so stays radially aligned: what is averaged over is the sub-pixel phase of the detector grid against the radius, which is what genuinely differs between reflections. Applying it costs one short dot product per reflection and no extra pixel reads; correcting the background *scalar* means the box sum, the profile fit and the variance all pick it up. The curve is accumulated from the same annulus pixels the background pass already reads (a pixel's radius is the reflection's radius plus the pixel's projection on the beam→reflection direction, so no per-pixel square root is needed) and specifically from the **clipped** pixels, or it would carry neighbour tails and zingers — which is why the correction is inert under `--integrator boxsum`, that path having no clip pass.
The model is a function of **radius alone**, so it is applied only where that is true of the background. `--background-radial` takes `on`, `off` or `auto`. It is **off by default**; under `auto` each image's peak-excluded ice score (§3.3) is taken after spot detection and before integration, and the correction is applied to that image when the score reaches the same `--ice-min-score` gate. Smooth powder ice *is* a radial feature and is corrected; ice made of discrete crystallite spots — which the profile channel is blind to and the spot channel catches — leaves no smooth ring to model, and correcting it makes matters worse. Measured against a fixed atomic model, comparing ice bands with resolution-matched decoy bands carrying no ice: on a crystal with pure smooth ice the correction removes **43 % of the bands' excess amplitude**, and the improvement is **7× larger inside the bands than outside**, which is its stated mechanism; on a crystal whose ice is textured the same correction *increased* the excess amplitude by half; on a clean crystal it is inert to four decimal places. Auto engages only where a peak-excluded score exists (adaptive spot finding, §3.2) — a plain azimuthal profile carries the Bragg peaks and cannot support an absolute threshold, so without one auto leaves the correction off.
+2 -1
View File
@@ -345,8 +345,9 @@ Integration:
| --- | --- |
| `--integrator <txt>` | Spot integrator: `gaussian` (profile-fit, default) \| `empirical` \| `boxsum` (classical fallback) |
| `--integration-radius <r>` | Signal-box radius `r1`, or `r1,r2,r3` (px). One value ⇒ `r2=r1+2`, `r3=r1+4` |
| `--integration-stencil <k>` | Push the `r2..r3` background ring out by `k` times the beam's radial streak `bandwidth·R_px`, per reflection (default `0` = the fixed circular ring). A fixed ring otherwise ends up on a streaked reflection's own tails at high resolution and measures them as background. Only the ring moves, and only radially — the `r1` signal box stays a circle — and the growth is capped at `2·r3`. The neighbour exclusion grows with it, so on a crowded pattern a few reflections can be left with too little background and dropped. Needs `--bandwidth`: on a monochromatic beam the streak is zero and this does nothing |
| `--background-clip <n>` | Monochromatic (rotation + still): high-side clip of the background ring at `mean + n·√mean` (default 4; 0 = off). The default background estimator — it rejects neighbour cores and zingers without the symmetric trim's Poisson skew bias. Broadband data always clip, at 3σ; ignored by `--integrator boxsum` |
| `--background-trim <f>` | Use the old symmetric trimmed mean for the background ring instead of the clip, 0≤f<0.5 (`0.10` was the former default). Switches `--background-clip` off. A symmetric trim is biased low on Poisson data and adds ~5 counts to every partial, so this is for back compatibility only; `0` = plain ring mean |
| `--background-trim <f>` | Use the old symmetric trimmed mean for the background ring instead of the clip, 0≤f<0.5 (`0.10` was the former default). Switches `--background-clip` off. A symmetric trim is biased low on Poisson data and adds ~5 counts to every partial, so this is for back compatibility only; `0` = plain ring mean. Rings holding more than 512 pixels fall back to the plain mean (the GPU sorts the ring in shared memory and the CPU now matches it), which the default radii never reach but wide ones do |
| `--background-radial[=on\|off\|auto]` | Correct the background ring for the **curvature** of the radial background (default **off**). Disk and ring are concentric, so a background linear in position cancels between them and only curvature survives — which on a smooth ice ring reaches +26 counts on a single reflection. `auto` applies it per image where that image's ice score shows a *smooth* powder ring, since the model is a function of radius alone: on ice made of discrete crystallite spots there is no smooth ring and the correction makes the bias worse. Ignored by `--integrator boxsum` (no clip pass to take the curve from) |
| `--integration-high-resolution <num>` | High-resolution limit for prediction and integration. Omitted (or 0) means integration extends as far as the detector reaches — which is what the predictor can place on the detector anyway, since it rejects reflections that miss it. Set a value to integrate less than the detector offers |
| `--max-hkl <n>` | Predict reflections with \|h\|,\|k\|,\|l\| ≤ `n` (max 511). By default this is derived per crystal from the refined cell as `ceil(max(a,b,c)/d_min) + 1`, which is the exact bound: the predictor keeps only \|q\| ≤ 1/d_min and `h = a·q`, so no reflection can lie outside it and no candidate inside it is wasted on a shorter axis. Set it only to override that |