leonarski_fandClaude Opus 5.5
beb97b62d3
Unmasked persistent hot pixels are integrated into whichever reflection's box they fall in; under
rotation one pixel collects a different reflection on every frame that reaches it, and the merge
carries intensities hundreds to thousands of times their shell mean on one or two observations.
HotPixelFinder (rugnux/HotPixels.{h,cpp}) reads the pre-scan sample a second time, once the
beam-stop projection has measured where the background puts the beam, and on each frame calls a
pixel lit when it exceeds its 2 px iso-2theta ring level (max of the ring and 1/16-sector medians)
by 3.3 sigma (sqrt(level) or the ring's MAD) + 2. A pixel lit on at least max(k1, kB) frames is
persistent: k1 = 1 + ceil((osc + 5 deg)/|zeta| / frame spacing) is more than one reflection can
light, kB the binomial bound (0.01 family-wise over the detector) from the ring's own lit rate.
A persistent pixel is masked, as the new PixelMask bit 10, only if it stands alone (component of
persistent pixels <= 2), reads on average >= 10x its ring and its mean excess is above the Poisson
bound; pixels holding the error value on most frames are masked with them. Counting sensors (thickness > 0)
and rotation data only; a CCD is left alone. One log line reports the counts.
Drawing the rings about the file's centre, as a first version did, masked pixels along the
background fall-off on a sweep whose file centre is 171 px from the background's and cost it 14%
ISa; about the measured centre that sweep is within 1%. Masking the detector's outermost row and
column unconditionally was tried and dropped: the persistence test already catches the hot pixels
there, and the whole lines bought nothing measurable.
Numbers below are from the looser first criterion (no isolation / 10x gate), against rc173-final
on the same base: merged reflections > 30x their shell mean gone on the
sets with proven hot pixels (7brr 21 -> 0, 9ih9 15 -> 0, 8xte 10 -> 0, 6z8o worst 1706x -> 40x);
6z8o CC1/2 0.50 -> 0.995, ISa 8.6 -> 12.8, CC to model 0.82 -> 0.90; 8xte ISa 8.0 -> 10.8; 6u7g
ISa 9.8 -> 12.0; controls (lyso_x06da_ref, marCCD) unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
2.3 KiB
Pixel mask
Mask format
Jungfraujoch generally follows the NXmx format for the pixel mask. The pixel mask is a 32-bit unsigned integer array of the same size as the image. The conditions for masking a pixel are encoded by setting a particular bit to one. This makes it possible to record the reason why a pixel is included in the mask, and several reasons can be recorded for one pixel at the same time.
Bit values are set as follows:
Bit 0 - gap (pixel with no sensor)
Bit 1 - error pixel (for PSI JUNGFRAU: pixel doesn't set proper gain during pedestal, for DECTRIS: pixel is part of detector pixel mask)
Bit 4 - noisy pixel (for PSI JUNGFRAU: pixel pedestal G0 RMS is over threshold, for DECTRIS: pixel was flagged with signal during dark data collection at initialization)
Bit 8 - user defined mask
Bit 9 - beam stop shadow (found by rugnux --detect-beam-stop, on by default; see Rugnux).
Unlike the other bits this one belongs to the run that found it, not to the dataset: Rugnux clears it
at the start of every run, so a mask read back from a file that carries one starts clear. The user
mask (bit 8) is left alone.
Bit 10 - defective pixel found on the run's own frames (rotation data from a counting sensor; see CPU data analysis §1.6): lit above its resolution ring on more frames than one reflection or chance explains, or holding the detector's error value on most frames.
Bit 30 - module edge (only for PSI systems)
Bit 31 - chip edge interpolated pixel (multipixel)
Custom user mask
Jungfraujoch allows a custom user mask to be uploaded. This happens in two steps. First create the mask in TIFF format:
import numpy as np
import tifffile as tiff
# Create an array matching a 2068 x 2164 (width x height) image: 2164 rows, 2068 columns
array = np.zeros((2164, 2068), dtype=np.uint32)
# Mark the pixel at column 400, row 300 with the value 1
array[300, 400] = 1
# Save the array as a TIFF file
tiff.imwrite('mask.tiff', array)
Pixels with non-zero value in the TIFF file will be marked as belonging to the user mask (bit 8).
Then upload the mask to Jungfraujoch server:
curl -v http://<jfjoch_broker http address>/config/user_mask.tiff -XPUT --data-binary @mask.tiff