Both were opt-in adaptive-spot refinements that did not help. Soft per-spot
weighting was index-rate neutral across the battery (re-ranking only bites when
spots exceed the max-spot cap, which weak serial data does not reach). The
local-SNR gate was neutral on index rate and degraded merged CC1/2 on flooded
XFEL data. Drops the flags, ApplyWeights/FilterByLocalSNR, the per-spot weight
field, and the by-weight FilterSpotsByCount branch (now strongest-first only).
--adaptive-spots itself is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drops --persistence-spots and AdaptiveSpotFinderCPU::RunPersistence (the 0-D
topological-persistence detector added in 5a33b0743). It was a research variant
that never beat the hard-threshold adaptive detector on a CC1/2 basis and is a
GPU dead-end (global candidate sort + union-find), so it is not a production
path. The hard-threshold --adaptive-spots detector is unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two opt-in tools for weak serial-stills tuning; both default-off, so the
default pipeline is bit-identical (verified: a serial-stills reference run
reproduces HEAD's 7.85% indexing rate exactly).
--local-snr <sigma> (AdaptiveSpotFinderCPU::FilterByLocalSNR): after the loose
per-ring adaptive threshold builds connected-component spots, drop any spot that
does not stand this many sigmas above its OWN LOCAL background (robust median/MAD
of a square annulus), not just the azimuthal ring mean. On structured-background
(XFEL) frames the ring mean underestimates the local diffuse level in some
sectors, so the ring threshold floods; a real Bragg peak still stands many local
sigmas proud. Validated on XFEL stills to separate real peaks from flood at the
pixel level (real median local-SNR ~70 vs flood ~2.6; SNR>=5 keeps ~99.8% of
real peaks, ~14% of flood). GPU-portable (a per-spot local reduction). NOTE: on
the current serial-stills battery it is index-rate/CC1/2 neutral -- the flood that
survives as CC clusters overlaps weak-real spots, and only lattice-fit separates
those -- but it is the correct tool for genuinely floody data (ice/jet/loosened
detector) and the right substrate for the online FPGA path.
--min-indexed-fraction <f>: exposes the previously hardcoded 0.20 minimum
indexed-spot fraction (AnalyzeIndexing) as a per-run setting. Lowering it admits
weaker/sparser crystals; on flooded XFEL data the extra lattices are spurious
(pair with --min-image-cc to gate them), on clean synchrotron data there are no
marginal frames so it is a no-op -- useful as a gating-experiment primitive.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add --soft-weight (implies --adaptive-spots): give every detected spot a
continuous quality weight in (0,1] and keep the highest-weight spots rather than
the brightest, so a deliberately loose detector self-cleans -- bright ice / salt
/ jet blobs and single-pixel noise no longer evict faint clean Bragg spots from
the max-spots cut.
The weight is a product of dimensionless gates (AdaptiveSpotFinderCPU::ApplyWeights,
computed against the per-ring background the adaptive finder already builds): a
logistic ramp in the spot's SNR and a soft size band (rises from one pixel,
plateaus, falls for oversized ice/salt/streak blobs). It carries on
DiffractionSpot -> SpotToSave and is consumed by FilterSpotsByCount, which ranks
by {non-ice, weight, intensity} when requested and by intensity otherwise, so the
classic and FPGA paths are unchanged.
Honest result: on the serial-stills battery this is index-rate-NEUTRAL. The
weighted ranking only changes the outcome when the spot count exceeds the
max-spots cap and the weight disagrees with intensity in a way that affects
indexing; the adaptive detectors already produce clean spot lists and the weak
sets sit under the cap, so re-ranking is a wash there (and a wash, not a
regression, on the one set that floods). Its intended benefit -- robustness to
ice/jet-contaminated frames and to a loosened detector -- is not exercised by
this battery; kept opt-in as the substrate for that.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add --persistence-spots, a second parameter-free detector alongside --adaptive-spots.
Instead of a hard per-ring threshold it builds the noise-normalised image
z = (I - ring_mean) / sqrt(ring_sigma^2 + read^2) (same per-ring background as the
hard variant) and scores every intensity maximum by its 0-D topological persistence:
sweeping the height from high to low, each maximum is born and, when its basin meets
a taller one at a saddle, dies with persistence = birth - saddle, in sigma. A lone
noise spike merges into the background almost immediately (persistence ~1 sigma); a
real peak stands many sigma proud. Emitting maxima whose persistence clears the same
z(E) significance bar needs no photon threshold and no min-pix, and it deblends
touching peaks (each keeps its own maximum). Implemented with the same union-find
idiom as the connected-component labeller.
On serial stills this auto-adapts with no per-dataset tuning like --adaptive-spots,
finding fewer but cleaner (deblended) spots; the hard-threshold variant remains more
sensitive on the very weakest data. Both share the per-ring background and read-noise
floor. comp_of is allocated lazily so the default and hard-adaptive paths pay nothing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The offline CPU spot finder marks a pixel strong when it clears a fixed photon
count AND a local-window SNR. The fixed photon floor forces per-dataset tuning:
its sweet spot tracks the background level (weak sets want a low threshold,
strong or high-background sets a high one) and the usable window is narrow, so
users hand-tune --spot-threshold/--spot-sigma per dataset.
Add an opt-in --adaptive-spots mode (AdaptiveSpotFinderCPU) that replaces the
fixed floor with a per-resolution-ring threshold derived from each image's own
noise. Per ring it computes a peak-excluded background mean and sigma (one plain
pass + two sigma-clip passes over the assembled photon image, binned by the
azimuthal-integration ring index) and sets
thr = max( PoissonTail(mean, p), mean + z * sqrt(sigma^2 + read^2) )
with p = false_pixels_per_frame / n_pixels the single portable knob (default
100) and z = Phi^-1(1 - p). The Poisson arm is the correct significance where
the background is countable (it carries the sqrt(mean) shot noise, so a bright
low-resolution ring gets a high threshold); the read-noise-floored Gaussian arm
keeps the threshold physical where the background vanishes (empty high-resolution
rings), without which those rings flood. read is a detector-level constant, not
a per-dataset knob. Both arms are needed: Poisson alone floods near-zero
background, Gaussian alone drops the shot-noise term and under-thresholds bright
rings.
One --adaptive-spots setting then adapts across a wide range of serial datasets
with no per-dataset threshold, matching or beating hand-tuned thresholds and the
peakfinder8/xgandalf reference on both weak large-cell and strong serial data,
with equal merged R-free.
The finder runs on the CPU (offline/viewer path) and reads the host image, which
the GPU pipeline already keeps in sync, so it works in either build. The default
(non-adaptive) path and the online/FPGA path are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leonarski_fandClaude Opus 4.8