The per-image ice score was read off the PLAIN azimuthal profile. That profile is a
per-ring mean, so a few strong Bragg reflections landing in a ring's q bin lift it
exactly as ice would. Measured over 37 rotation crystals, that did not merely add
noise - it INVERTED the metric: the two highest-scoring crystals had no ice at all
(4.23 and 4.06), while a clean control read 1.57. A decoy null - the identical
statistic evaluated at q positions where hexagonal ice cannot be - reaches 1.51 at its
99th percentile and 2.70 at its maximum, so that metric cannot support any absolute
threshold whatsoever.
The adaptive spot finder already computes the right input for its own threshold: a
sigma-clipped per-resolution-ring background, in the same bins. A powder ring is
azimuthally smooth and survives the clip; Bragg peaks do not. On the clipped profile
the clean population tightens to 1.00-1.22 and the crystals with confirmed ice sit at
2.08-2.37, against a decoy null that never exceeds 1.29.
That channel is blind to one thing: ice in large crystallites diffracts as DISCRETE
spots and leaves the radial profile flat. So a second channel counts found spots on the
rings against the same q width of ice-free flanks beside them. The two barely overlap -
the smooth-ice crystals read 2.1-2.4 / ~1.0 and the textured ones ~1.1 / 3.8-17.6,
while a clean crystal reads 1.04 on both.
Both are then used as a GATE (--ice-min-score 1.5, --ice-min-spot-ratio 2.0, both
calibrated on the battery, 0 disables): the eleven fixed hexagonal bands cover 16-26 %
of the unique reflections at typical resolutions whether or not the crystal has ice, so
flagging, the exclusion from the scale fit and the merge-time CC1/2 ring mask are now
all skipped when neither channel sees any. The gate is applied in the full pipeline and
in --scale, which reads the stored per-image values back out of the _process.h5.
Also fixes the merge-time mask's control: the shoulder now excludes reflections that
are themselves on an ice ring. The rings are not evenly spaced - 1.947/1.916/1.882 A
sit 0.05-0.06 apart in q - so for those three the [w,3w) shoulder landed squarely on
the neighbours and the test compared ice against ice. Measured, that is the only thing
this changes: it removes firings on those three rings and leaves every other firing's
CC pair identical to three decimals.
And the online ice half-width, which was 0.02 in the API against 0.03 offline, so the
same data got a narrower band online than the measured ~0.06 ring FWHM justifies.
Battery (37 rotation crystals, against the previous behaviour): space groups 34/37 in
both and NO crystal's space group changes; 6 crystals gain unique reflections, 1 loses.
Best of them gains 7082 unique reflections with R_meas 16.0 -> 14.3, CC1/2 95.9 -> 97.3
and ISa 13.7 -> 19.0; another goes R_meas 54.9 -> 42.9, CC1/2 84.0 -> 90.4, ISa
3.9 -> 5.5; a third reaches CC1/2 99.4 from 95.7 at an unchanged reflection count. The
one crystal that loses reflections improves on both R_meas and CC1/2.
Not done here: the ScanResult/API/plot-type/frontend/viewer layers for the new
spot_count_ice_control (they need the OpenAPI regeneration). Message, CBOR, HDF5
write/read and the receiver plots are.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mapping took its width and height from the CONVERTED geometry
unconditionally, while pixel_to_bin is sized per mode: converted when the
geometry is transformed, raw module layout when it is not. In raw mode
the two disagreed - 2068x2162 reported against a 1024x4096 map on a JF4M.
Only the adaptive spot finders read those dimensions, and they read them
for exactly the thing that breaks: the CPU finder derives npix = w*h and
then indexes the image, pixel_to_bin and the resolution mask with it, so
it walked ~277k pixels past the end of all three; the GPU finder stays in
bounds but decodes the strong-pixel bit index with the wrong row stride
and reports spots at wrong coordinates. Nothing combines raw geometry
with adaptive detection today, so this was latent rather than live.
Take them from GetXPixelsNum()/GetYPixelsNum(), which already follow the
geometry mode. The converted path is unchanged - it is the same number
there - and every internal use is inside SetupConvGeom, which only runs
when the geometry is transformed.
Covered by two tests: the mapping's dimensions must match pixel_to_bin in
both modes, and the CPU adaptive finder must return a spot planted on a
raw-geometry image at that raw pixel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Clear() reallocated sum and count but left sum2 as the constructor had
sized it. Every caller reuses one profile across the images of a dataset
(rugnux keeps one per worker, the viewer one per view), so the squares
kept accumulating while the means restarted at zero: the per-image
standard deviation written to /entry/azint and shown in the plots was
meaningless from the second image on, and grew without bound over a run.
The size mismatch was the sharper edge. Clearing to a mapping with a
different bin count left sum2 shorter than sum, and GetStd() and
operator+= then read past its end - reachable in the viewer by opening a
dataset with a wider q range than the one before it.
Both are covered by tests: the same frame twice with a Clear() in
between has to give the same standard deviation, and a profile cleared
to a wider mapping has to report the right value in a bin that only the
wider mapping has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.
* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin
Reviewed-on: #58
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.
* jfjoch_broker: Azimuthal integration mapping is generated with parallel computations, significantly reducing setup times
* frontend: Fix selection of FFTW in indexing settings
Reviewed-on: #51
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.
* Multiple small bug fixes scattered across the whole code base. (detected with GPT-5.4)
* jfjoch_viewer: Improve image render performance
Reviewed-on: #44
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
This is an UNSTABLE release.
* jfjoch_viewer: Minor polishing of new functionality
* jfjoch_broker: User NaN for empty azimuthal bins
Reviewed-on: #12
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
leonarski_fandClaude Opus 5