Files
Jungfraujoch/tests/BraggIntegrationEngineCPUTest.cpp
leonarski_fandClaude Opus 5.5 37a8c8e24e Rotation merge: drop rocking events with an overloaded pixel; capture uncertainty in the merge variance
A saturated pixel in a spot means the brightest part of the reflection was
not measured. The integration used to drop the peak frame's partial (its
peak pixel is unreadable) and keep the flanks, so the combine extrapolated
the event from its tails by the partiality model: on a strongly
diffracting small-molecule crystal the strongest low-order reflections
read 2-3x low and were the largest SHELXL misfits. XDS drops such a
reflection (OVERLOAD); so does rugnux now.

- Integration (CPU + GPU engines): a reflection is `overloaded` when a
  signal-disk pixel is saturated, or unreadable on this frame but not in
  the run's pixel mask - EIGER/PILATUS write their error value for a
  pixel they could not count, which the preprocessor turns into a masked
  pixel like a gap's. The engines now receive the PixelMask to tell the
  two apart (an earlier attempt that re-classified the marker as
  saturation in the preprocessor broke a dataset whose gaps are not in
  the file's mask). An overloaded reflection is kept with its box sum,
  unfitted, only so its event can be recognised.
- Rotation combine (CPU + GPU): an event with any overloaded partial is
  dropped whole; counted in the log and the report
  (OBSERVATIONS_REJECTED_OVERLOAD=). The unmerged MTZ export drops it too.
- Everything else that reads reflections leaves an overloaded one out:
  AcceptReflection (stills merge, per-image scaling), the post-refinement
  gather, the axial-row sums.
- Capture uncertainty: the merge rebuilds each full's variance at the
  reflection's mean (counting_variance / ModelSigma) and dropped the
  capture term the combine had put into sigma, so a full extrapolated
  from part of its rocking curve merged at the weight of a whole one.
  Fulls now carry it (Obs::capture) and the rebuilt variance adds
  (capture * <I>)^2, host and device.

SHELXL R1 on rugnux's own integration (harness), median fix -> this:
citric acid .0648 -> .0420 (XDS .051; 221 events dropped, EXTI 1.02 -> 0.29),
HEPES .0396 -> .0381 (184), aspirin 20 keV .0387 -> .0385 (6),
aspirin 25 keV .0376 -> .0375 (5); metformin/nidppe/dnba/lalanine/cytidine
no overloads, unchanged. YAG .116 -> .128 (87 dropped; its scale loop does
not settle either way). Proteins and private subset: see the branch report.
Tests: BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped (new),
BraggIntegrationEngineGPU_MatchesCPU (overloaded flag compared),
AcceptReflection_ResolutionLimits, [write_reflections], [large].

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
2026-10-04 21:01:40 +02:00

116 lines
5.4 KiB
C++

// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#include <catch2/catch_all.hpp>
#include <cmath>
#include <cstdint>
#include <vector>
#include "../common/BraggIntegrationSettings.h"
#include "../common/DetectorSetup.h"
#include "../common/DiffractionExperiment.h"
#include "../common/Reflection.h"
#include "../image_analysis/bragg_integration/BraggIntegrationEngineCPU.h"
#include "../image_analysis/image_preprocessing/ImagePreprocessorBuffer.h"
// One spot on a flat background, inside a grid of predictions 8 px apart that put no flux on the
// frame - the tails of reflections recorded on the frames either side of a finely sliced one. Their
// r2 regions cover every background ring, so whether the spot keeps a clean ring depends only on
// whether those predictions are allowed to mask it.
TEST_CASE("BraggIntegrationEngineCPU_NeighbourMaskFollowsPartiality", "[Integration][portable]") {
DiffractionExperiment experiment(DetJF(2));
experiment.DetectorDistance_mm(100.0f).IncidentEnergy_keV(WVL_1A_IN_KEV).BeamX_pxl(400.0f).BeamY_pxl(400.0f);
experiment.ImportBraggIntegrationSettings(BraggIntegrationSettings());
const size_t width = experiment.GetXPixelsNum(), npixel = experiment.GetPixelsNum();
const float cx = 600.3f, cy = 300.2f, amp = 800.0f, sigma = 1.3f;
ImagePreprocessorBuffer image(npixel);
for (size_t i = 0; i < npixel; ++i)
image[i] = 12;
for (int dy = -6; dy <= 6; ++dy)
for (int dx = -6; dx <= 6; ++dx) {
const int x = static_cast<int>(std::lround(cx)) + dx, y = static_cast<int>(std::lround(cy)) + dy;
const float ex = x - cx, ey = y - cy;
image[y * width + x] += static_cast<int32_t>(std::lround(amp * std::exp(-(ex * ex + ey * ey) / (2 * sigma * sigma))));
}
auto run = [&](float neighbour_partiality, BraggIntegrationCounts &counts) {
std::vector<Reflection> predicted;
for (int gy = -4; gy <= 4; ++gy)
for (int gx = -4; gx <= 4; ++gx) {
Reflection r{};
r.h = gx; r.k = gy; r.l = 1;
r.predicted_x = cx + 8.0f * gx;
r.predicted_y = cy + 8.0f * gy;
r.d = 2.0f;
r.prescaling_corr = 1.0f;
r.partiality = (gx == 0 && gy == 0) ? 0.5f : neighbour_partiality;
predicted.push_back(r);
}
BraggIntegrationEngineCPU engine(experiment, PixelMask(experiment));
const auto out = engine.Run(image, predicted, predicted.size(), 0);
counts = engine.Counts();
std::vector<Reflection> spot;
for (const auto &r : out)
if (r.h == 0 && r.k == 0)
spot.push_back(r);
return spot;
};
// Tails: no reflection is dropped, and the spot is measured against the clean flat background -
// while the density the widened-radius guard reads still sees every prediction.
BraggIntegrationCounts tails;
const auto spot = run(0.01f, tails);
CHECK(tails.bkg_starved == 0);
CHECK(tails.bkg_starved_by_neighbour > 0);
REQUIRE(spot.size() == 1);
CHECK(spot[0].bkg == Catch::Approx(12.0f));
CHECK(spot[0].I == Catch::Approx(6.2831853 * sigma * sigma * amp).epsilon(0.05));
// The same predictions with their flux on this frame do take the rings, the spot's among them, and
// it is the neighbours, not the detector, that starve them - the same rings the density counted.
BraggIntegrationCounts on_frame;
CHECK(run(1.0f, on_frame).empty());
CHECK(on_frame.bkg_starved > 0);
CHECK(on_frame.bkg_starved_by_neighbour == on_frame.bkg_starved);
CHECK(on_frame.bkg_starved_by_neighbour == tails.bkg_starved_by_neighbour);
}
// A saturated pixel (INT32_MAX, as the preprocessor marks it) at the peak of a spot: the reflection is
// not measured but kept, flagged overloaded, so that a rotation merge can drop its whole rocking event.
// The same spot without the saturated pixel is an ordinary measurement.
TEST_CASE("BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped", "[Integration][portable]") {
DiffractionExperiment experiment(DetJF(2));
experiment.DetectorDistance_mm(100.0f).IncidentEnergy_keV(WVL_1A_IN_KEV).BeamX_pxl(400.0f).BeamY_pxl(400.0f);
experiment.ImportBraggIntegrationSettings(BraggIntegrationSettings());
const size_t width = experiment.GetXPixelsNum(), npixel = experiment.GetPixelsNum();
const int cx = 600, cy = 300;
ImagePreprocessorBuffer image(npixel);
for (size_t i = 0; i < npixel; ++i)
image[i] = 12;
for (int dy = -6; dy <= 6; ++dy)
for (int dx = -6; dx <= 6; ++dx)
image[(cy + dy) * width + cx + dx] += static_cast<int32_t>(std::lround(800.0 * std::exp(-(dx * dx + dy * dy) / (2 * 1.3 * 1.3))));
Reflection r{};
r.h = 1; r.k = 2; r.l = 3;
r.predicted_x = cx; r.predicted_y = cy;
r.d = 2.0f;
r.prescaling_corr = 1.0f;
r.partiality = 1.0f;
const std::vector<Reflection> predicted{r};
BraggIntegrationEngineCPU engine(experiment, PixelMask(experiment));
const auto clean = engine.Run(image, predicted, 1, 0);
REQUIRE(clean.size() == 1);
CHECK_FALSE(clean[0].overloaded);
image[cy * width + cx] = INT32_MAX;
const auto saturated = engine.Run(image, predicted, 1, 0);
REQUIRE(saturated.size() == 1);
CHECK(saturated[0].overloaded);
CHECK(saturated[0].clipped);
}