leonarski_fandClaude Opus 5.5
beb97b62d3
Unmasked persistent hot pixels are integrated into whichever reflection's box they fall in; under
rotation one pixel collects a different reflection on every frame that reaches it, and the merge
carries intensities hundreds to thousands of times their shell mean on one or two observations.
HotPixelFinder (rugnux/HotPixels.{h,cpp}) reads the pre-scan sample a second time, once the
beam-stop projection has measured where the background puts the beam, and on each frame calls a
pixel lit when it exceeds its 2 px iso-2theta ring level (max of the ring and 1/16-sector medians)
by 3.3 sigma (sqrt(level) or the ring's MAD) + 2. A pixel lit on at least max(k1, kB) frames is
persistent: k1 = 1 + ceil((osc + 5 deg)/|zeta| / frame spacing) is more than one reflection can
light, kB the binomial bound (0.01 family-wise over the detector) from the ring's own lit rate.
A persistent pixel is masked, as the new PixelMask bit 10, only if it stands alone (component of
persistent pixels <= 2), reads on average >= 10x its ring and its mean excess is above the Poisson
bound; pixels holding the error value on most frames are masked with them. Counting sensors (thickness > 0)
and rotation data only; a CCD is left alone. One log line reports the counts.
Drawing the rings about the file's centre, as a first version did, masked pixels along the
background fall-off on a sweep whose file centre is 171 px from the background's and cost it 14%
ISa; about the measured centre that sweep is within 1%. Masking the detector's outermost row and
column unconditionally was tried and dropped: the persistence test already catches the hot pixels
there, and the whole lines bought nothing measurable.
Numbers below are from the looser first criterion (no isolation / 10x gate), against rc173-final
on the same base: merged reflections > 30x their shell mean gone on the
sets with proven hot pixels (7brr 21 -> 0, 9ih9 15 -> 0, 8xte 10 -> 0, 6z8o worst 1706x -> 40x);
6z8o CC1/2 0.50 -> 0.995, ISa 8.6 -> 12.8, CC to model 0.82 -> 0.90; 8xte ISa 8.0 -> 10.8; 6u7g
ISa 9.8 -> 12.0; controls (lyso_x06da_ref, marCCD) unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
77 lines
3.7 KiB
C++
77 lines
3.7 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include "CompressedImage.h"
|
|
#include "DetectorSetup.h"
|
|
#include "DiffractionExperiment.h"
|
|
#include "../jungfrau/JFCalibration.h"
|
|
|
|
struct PixelMaskStatistics {
|
|
uint32_t user_mask;
|
|
uint32_t noisy_pixel;
|
|
uint32_t error_pixel;
|
|
uint32_t chip_gap_pixel;
|
|
uint32_t total_masked;
|
|
uint32_t module_gap_pixel;
|
|
};
|
|
|
|
class PixelMask {
|
|
std::vector<uint32_t> mask;
|
|
std::vector<uint32_t> raw_mask;
|
|
// One byte per pixel, 1 where the pixel is masked at all - the form the GPU image preprocessor
|
|
// uploads - and the checksum the shared device-table cache keys on (CudaSharedTables.h). Both are
|
|
// pure functions of `mask`, and an analysis engine is built per worker per pass, so they are
|
|
// derived here once instead of in every one of those engines.
|
|
std::vector<uint8_t> binary_mask;
|
|
uint64_t binary_mask_checksum = 0;
|
|
|
|
uint32_t LoadMask(const std::vector<uint32_t>& mask, uint8_t bit);
|
|
// Everything that follows from the mask, recomputed wherever the mask changes.
|
|
void UpdateDerived(const DiffractionExperiment& experiment);
|
|
void UpdateBinaryMask();
|
|
|
|
void CalcEdgePixels_i(const DiffractionExperiment& experiment);
|
|
|
|
public:
|
|
// NXmx bits
|
|
constexpr static const uint8_t ModuleGapPixelBit = 0;
|
|
constexpr static const uint8_t ErrorPixelBit = 1;
|
|
constexpr static const uint8_t NoisyPixelBit = 4;
|
|
constexpr static const uint8_t UserMaskedPixelBit = 8;
|
|
constexpr static const uint8_t BeamStopPixelBit = 9;
|
|
// Found bad on the run's own frames (rugnux pre-scan, HotPixelFinder): lit above its ring on too
|
|
// many frames, or holding the error value on most of them.
|
|
constexpr static const uint8_t HotPixelBit = 10;
|
|
constexpr static const uint8_t ChipGapPixelBit = 31;
|
|
constexpr static const uint8_t ModuleEdgePixelBit = 30;
|
|
|
|
PixelMask();
|
|
explicit PixelMask(size_t width, size_t height);
|
|
explicit PixelMask(const DiffractionExperiment& experiment);
|
|
explicit PixelMask(const std::vector<uint32_t> &mask);
|
|
|
|
void CalcEdgePixels(const DiffractionExperiment& experiment);
|
|
void LoadUserMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
void LoadUserMask(const DiffractionExperiment& experiment, const CompressedImage& image);
|
|
void LoadBeamStopMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
// The beam-stop shadow belongs to the run that found it, not to the dataset, so a mask read back
|
|
// from a file that carries one starts clear. The user mask (bit 8) is deliberately left alone.
|
|
void ClearBeamStopMask(const DiffractionExperiment& experiment);
|
|
void LoadHotPixelMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
void LoadDECTRISBadPixelMask(const std::vector<uint32_t>& mask);
|
|
void LoadDarkBadPixelMask(const DiffractionExperiment& experiment, const std::vector<uint32_t>& mask);
|
|
void LoadDetectorBadPixelMask(const DiffractionExperiment& experiment, const JFCalibration *calib);
|
|
|
|
[[nodiscard]] const std::vector<uint32_t> &GetMaskRaw() const;
|
|
[[nodiscard]] const std::vector<uint32_t> &GetMask(const DiffractionExperiment& experiment) const;
|
|
[[nodiscard]] const std::vector<uint32_t> &GetMask() const;
|
|
[[nodiscard]] const std::vector<uint8_t> &GetBinaryMask() const;
|
|
[[nodiscard]] uint64_t GetBinaryMaskChecksum() const;
|
|
|
|
[[nodiscard]] std::vector<uint32_t> GetUserMask(const DiffractionExperiment& experiment) const;
|
|
[[nodiscard]] std::vector<uint32_t> GetUserMask() const;
|
|
[[nodiscard]] PixelMaskStatistics GetStatistics() const;
|
|
};
|