leonarski_fandClaude Opus 5.5
e4cb3e1b09
Foreign stills files state their saturation level wrongly or not at all: an MPCCD Cheetah file is
dark-subtracted per pixel, so each pixel clips at the ADC limit less its own dark, and its
CrystFEL geometry leaves max_adu commented out - the run reported "Overloads: 0" while the inner
panels saturated in most frames.
saturation_ceiling (image_preprocessing/SaturationCeiling) histograms pixel values on a log
scale (64 bins per octave from 256) and returns the lower edge of the lowest bin holding over
twice what any of the 8 bins below it held (and at least 50 values): a bright tail only falls, a
clipped one piles up below where it stops, as a band where the level differs per pixel.
DetectSaturation runs it on 40 frames spread over every stills run and logs it beside the file's
limit; for any input but a Jungfraujoch-written file (ProcessConfig::detect_saturation) a level
below the file's replaces it. The CrystFEL/CXI reader, where the geometry states no max_adu, now
takes the stored integer type's maximum (an int16 Cheetah file clips at 32767).
The geometry-file-free CXI path (previous commit) is limited to square frames: a tiled detector's
Cheetah slab (CSPAD, AGIPD, MPCCD, ...) read as one flat panel would place every pixel wrongly
without a word, and those slabs are not square.
Measured:
MPCCD stills, 1339 frames: pile-up from 1218 photons (by hand 12000 ADU ~ 1200 ph); 5.5% of
reflections coarser than 4 A flagged; lowest-shell CC1/2 0.545 -> 0.674, overall 0.741 ->
0.793; same dimple refinement against the deposited model R-free 0.2586 -> 0.2556
single-panel Rayonix CXI, 1316 frames: no pile-up (20 pixels at the int16 limit in 40 frames);
the int16 limit flags 123 reflections; CC1/2 0.737 -> 0.787, rugnux R-free 0.382 -> 0.358
JUNGFRAU stills (Jungfraujoch files: lysozyme and two private sets): no pile-up below the stated
limit, results bit-identical
Cost 0.5 s per run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
53 lines
2.0 KiB
C++
53 lines
2.0 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include <catch2/catch_all.hpp>
|
|
#include <cmath>
|
|
#include <random>
|
|
|
|
#include "../image_analysis/image_preprocessing/SaturationCeiling.h"
|
|
|
|
// A bright tail, P(> v) ~ 1/v, as the cores of Bragg peaks give.
|
|
static saturation_ceiling::Histogram Tail(int64_t clip_from, int64_t clip_to) {
|
|
saturation_ceiling::Histogram h{};
|
|
std::mt19937 rng(1);
|
|
std::uniform_real_distribution<double> u(0.0, 1.0);
|
|
for (int i = 0; i < 2000000; i++) {
|
|
auto v = static_cast<int64_t>(std::min(1e9, 100.0 / (1.0 - u(rng)))); // P(> v) ~ 1/v above 100
|
|
if (v >= clip_from) {
|
|
// Clipped at a ceiling of its own, somewhere in [clip_from, clip_to].
|
|
v = clip_from + static_cast<int64_t>(u(rng) * static_cast<double>(clip_to - clip_from));
|
|
}
|
|
saturation_ceiling::Add(h, v);
|
|
}
|
|
return h;
|
|
}
|
|
|
|
TEST_CASE("SaturationCeiling_NoClipping", "[portable]") {
|
|
CHECK(!saturation_ceiling::Ceiling(Tail(INT32_MAX, INT32_MAX)).has_value());
|
|
}
|
|
|
|
TEST_CASE("SaturationCeiling_OneLevel", "[portable]") {
|
|
const auto c = saturation_ceiling::Ceiling(Tail(12000, 12000));
|
|
REQUIRE(c.has_value());
|
|
CHECK(*c <= 12000);
|
|
CHECK(*c > 12000 * 0.98);
|
|
}
|
|
|
|
TEST_CASE("SaturationCeiling_BandPerPixel", "[portable]") {
|
|
// Each pixel stops at its own level - the ADC's limit less that pixel's dark.
|
|
const auto c = saturation_ceiling::Ceiling(Tail(12400, 15600));
|
|
REQUIRE(c.has_value());
|
|
CHECK(*c <= 12400);
|
|
CHECK(*c > 12400 * 0.98);
|
|
}
|
|
|
|
TEST_CASE("SaturationCeiling_MaskedPixelsLeftOut", "[portable]") {
|
|
saturation_ceiling::Histogram h{};
|
|
const std::vector<int32_t> values(1000, 5000);
|
|
saturation_ceiling::Add(h, values, std::vector<uint32_t>(1000, 1));
|
|
CHECK(!saturation_ceiling::Ceiling(h).has_value());
|
|
saturation_ceiling::Add(h, values, {});
|
|
CHECK(saturation_ceiling::Ceiling(h).has_value());
|
|
}
|