A saturated pixel in a spot means the brightest part of the reflection was not measured. The integration used to drop the peak frame's partial (its peak pixel is unreadable) and keep the flanks, so the combine extrapolated the event from its tails by the partiality model: on a strongly diffracting small-molecule crystal the strongest low-order reflections read 2-3x low and were the largest SHELXL misfits. XDS drops such a reflection (OVERLOAD); so does rugnux now. - Integration (CPU + GPU engines): a reflection is `overloaded` when a signal-disk pixel is saturated, or unreadable on this frame but not in the run's pixel mask - EIGER/PILATUS write their error value for a pixel they could not count, which the preprocessor turns into a masked pixel like a gap's. The engines now receive the PixelMask to tell the two apart (an earlier attempt that re-classified the marker as saturation in the preprocessor broke a dataset whose gaps are not in the file's mask). An overloaded reflection is kept with its box sum, unfitted, only so its event can be recognised. - Rotation combine (CPU + GPU): an event with any overloaded partial is dropped whole; counted in the log and the report (OBSERVATIONS_REJECTED_OVERLOAD=). The unmerged MTZ export drops it too. - Everything else that reads reflections leaves an overloaded one out: AcceptReflection (stills merge, per-image scaling), the post-refinement gather, the axial-row sums. - Capture uncertainty: the merge rebuilds each full's variance at the reflection's mean (counting_variance / ModelSigma) and dropped the capture term the combine had put into sigma, so a full extrapolated from part of its rocking curve merged at the weight of a whole one. Fulls now carry it (Obs::capture) and the rebuilt variance adds (capture * <I>)^2, host and device. SHELXL R1 on rugnux's own integration (harness), median fix -> this: citric acid .0648 -> .0420 (XDS .051; 221 events dropped, EXTI 1.02 -> 0.29), HEPES .0396 -> .0381 (184), aspirin 20 keV .0387 -> .0385 (6), aspirin 25 keV .0376 -> .0375 (5); metformin/nidppe/dnba/lalanine/cytidine no overloads, unchanged. YAG .116 -> .128 (87 dropped; its scale loop does not settle either way). Proteins and private subset: see the branch report. Tests: BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped (new), BraggIntegrationEngineGPU_MatchesCPU (overloaded flag compared), AcceptReflection_ResolutionLimits, [write_reflections], [large]. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
80 lines
3.9 KiB
C++
80 lines
3.9 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include "BraggIntegrationEngine.h"
|
|
#include "../../common/PixelMask.h"
|
|
|
|
class CompressedImage;
|
|
|
|
// Plain-C++ reference/fallback engine: a faithful serial re-expression of BraggIntegrate2D (box
|
|
// sum) and ProfileIntegrate2D (Kabsch profile fit) reading the preprocessed int32 image. Also the
|
|
// numeric oracle the CUDA engine is checked against.
|
|
class BraggIntegrationEngineCPU : public BraggIntegrationEngine {
|
|
// Core integrator, templated on a pixel sampler so it reads either the preprocessed int32 buffer
|
|
// or a raw CompressedImage of any pixel type - both presented per-pixel in the INT32_MIN(masked)/
|
|
// INT32_MAX(saturated) convention - without ever materialising a second full-image copy.
|
|
// Full-frame scratch of RunImpl: the reflection mask and the signal-region owner map. A call writes
|
|
// only around its reflections, so these keep only the 16x16-pixel tiles written since the last
|
|
// Clear(): a frame-sized array was mostly never read, yet over a sweep every page of it got
|
|
// touched, in every worker. Reading a tile nothing wrote gives `empty`.
|
|
template <class T>
|
|
class TiledFrame {
|
|
static constexpr int TILE = 16;
|
|
int tiles_x;
|
|
T empty;
|
|
std::vector<int32_t> tile_start; // per tile: where it starts in `pixels`, -1 = not written
|
|
std::vector<int32_t> written; // the tiles written, for Clear()
|
|
std::vector<T> pixels;
|
|
public:
|
|
TiledFrame(int width, int height, T empty)
|
|
: tiles_x((width + TILE - 1) / TILE), empty(empty),
|
|
tile_start(static_cast<size_t>(tiles_x) * ((height + TILE - 1) / TILE), -1) {}
|
|
T Get(int x, int y) const {
|
|
const int32_t start = tile_start[(y / TILE) * tiles_x + x / TILE];
|
|
return start < 0 ? empty : pixels[start + (y % TILE) * TILE + x % TILE];
|
|
}
|
|
T &At(int x, int y) {
|
|
const int t = (y / TILE) * tiles_x + x / TILE;
|
|
if (tile_start[t] < 0) {
|
|
tile_start[t] = static_cast<int32_t>(pixels.size());
|
|
pixels.resize(pixels.size() + TILE * TILE, empty);
|
|
written.push_back(t);
|
|
}
|
|
return pixels[tile_start[t] + (y % TILE) * TILE + x % TILE];
|
|
}
|
|
void Clear() {
|
|
for (int t : written)
|
|
tile_start[t] = -1;
|
|
written.clear();
|
|
pixels.clear();
|
|
}
|
|
};
|
|
TiledFrame<uint8_t> refl_mask;
|
|
TiledFrame<uint32_t> owner;
|
|
// The run's pixel mask, packed 32 pixels to a word (PixelMask::GetPackedMask). An unreadable pixel
|
|
// it does not explain was unreadable on this frame only - an overload (see Reflection::overloaded).
|
|
std::vector<uint32_t> static_mask;
|
|
|
|
template <class Sampler>
|
|
std::vector<Reflection> RunImpl(const Sampler &img, const std::vector<Reflection> &predicted,
|
|
size_t npredicted, int64_t image_number);
|
|
|
|
public:
|
|
BraggIntegrationEngineCPU(const DiffractionExperiment &experiment, const PixelMask &mask);
|
|
|
|
using BraggIntegrationEngine::Run; // keep the preprocessed-buffer overload visible
|
|
|
|
std::vector<Reflection> Run(const ImagePreprocessorBuffer &image,
|
|
const std::vector<Reflection> &predicted, size_t npredicted,
|
|
int64_t image_number) override;
|
|
|
|
// FPGA workflow: integrate straight off the assembled detector image, reading only the pixels
|
|
// inside each reflection disk (no whole-image conversion - the FPGA host cannot afford one at its
|
|
// frame rate). Masked pixels carry the type minimum and saturated the type maximum.
|
|
std::vector<Reflection> Run(const CompressedImage &image,
|
|
const std::vector<Reflection> &predicted, size_t npredicted,
|
|
int64_t image_number);
|
|
};
|