image_preprocessing: treat the type-max marker as invalid for unsigned pixels

The error/invalid pixel marker (the pixel type's extreme value, e.g. 0xFFFFFFFF for
EIGER uint32) was only mapped to the internal error value for SIGNED types - the check
was gated on std::is_signed<T>. For unsigned EIGER data a 0xFFFFFFFF pixel therefore
fell through to the saturation test and, exceeding the (HDF5) saturation_value, was
silently classified as saturated rather than invalid.

Both are excluded from integration, so results are unchanged where such pixels are also
in the static pixel mask (as in the JUNGFRAU test data - Thau_9 and lyso are bit-for-bit
the same), but the classification/statistics were wrong and a stray 0xFFFFFFFF outside
the static mask would be mis-accounted.

Test the error marker before saturation and drop the is_signed gate in both the CPU and
GPU preprocessors: the HDF5 saturation_value stays the saturation threshold, the pixel
type's maximum is the invalid marker (priority masked > error > saturated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-10 14:17:56 +02:00
co-authored by Claude Opus 4.8
parent f0dc61ef58
commit fd7e2ae6f9
2 changed files with 14 additions and 11 deletions
@@ -46,13 +46,15 @@ ImageStatistics ImagePreprocessorCPU::Analyze(ImagePreprocessorBuffer &processed
if (mask_1bit[i] != 0) {
processed_image[i] = INT32_MIN;
++ret.masked_pixel_count;
} else if (image[i] == err_pixel_val) {
// Error/invalid marker = the pixel type's extreme value (0xFFFFFFFF for EIGER uint32).
// Tested before saturation, since for unsigned types the marker also exceeds sat_pixel_val
// (which is clipped above to the HDF5 saturation_value).
processed_image[i] = INT32_MIN;
++ret.error_pixel_count;
} else if (image[i] >= sat_pixel_val) {
processed_image[i] = INT32_MAX;
++ret.saturated_pixel_count;
} else if (std::is_signed<T>::value && (image[i] == err_pixel_val)) {
// Error pixels are possible only for signed types
processed_image[i] = INT32_MIN;
++ret.error_pixel_count;
} else {
processed_image[i] = static_cast<int32_t>(image[i]);
if (image[i] > ret.max_value)
@@ -35,27 +35,28 @@ __global__ void preprocess_kernel(
long long local_max = INT64_MIN;
long long local_min = INT64_MAX;
constexpr bool is_signed = std::is_signed<T>::value;
for (int i = blockIdx.x * blockDim.x + threadIdx.x;
i < npixels;
i += blockDim.x * gridDim.x) {
T v = input[i];
bool is_masked = mask[i];
bool is_sat = v >= saturation_limit;
bool is_err = is_signed && (v == err_value);
// Error/invalid marker = the pixel type's extreme value (0xFFFFFFFF for EIGER uint32); tested
// before saturation, since for unsigned types the marker also exceeds saturation_limit (which is
// clipped to the HDF5 saturation_value). Priority: masked > error > saturated.
bool is_err = (v == err_value);
bool is_sat = !is_err && (v >= saturation_limit);
bool valid = !(is_masked || is_sat || is_err);
// Output
output[i] =
is_masked ? INT32_MIN : is_sat ? INT32_MAX : is_err ? INT32_MIN : (int32_t) v;
is_masked ? INT32_MIN : is_err ? INT32_MIN : is_sat ? INT32_MAX : (int32_t) v;
// Counters
local_masked += is_masked;
local_saturated += (!is_masked && is_sat);
local_error += (!is_masked && !is_sat && is_err);
local_error += (!is_masked && is_err);
local_saturated += (!is_masked && !is_err && is_sat);
// Min/max only for valid
if (valid) {