mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases 68 Wiki Activity

Bragg integration: keep a reflection that lost a wing, not one that lost its peak

Browse Source 
MINPK asks how MUCH of the expected profile is readable. It does not ask WHERE, and
the two are not the same question. The renormalisation argument the rescue rests on -
a fit over a subset of a normalised profile is unbiased - needs the pixels to go
missing for reasons unrelated to the reflection. A gap, a mask or the edge of the
sensor is such a reason: the loss is set by the detector, and the fit renormalises
over what is left. A pixel invalidated BY THE FLUX IT SAW is not: it goes missing
because the reflection was bright, and it is the peak.

Measured on the combined fulls, against the mean of the complete observations of the
same reflection, in the innermost resolution shell of the high-multiplicity control
and of a weaker crystal: a rescued reflection whose unreadable pixel sits within a
pixel of the predicted centre reads |I - <I>|/I of 0.50 and 0.53, against 0.073 and
0.212 for a complete observation - 6.8x and 2.5x - and carries several times the mean
intensity of its shell. On the control that is 0.21% of the shell's observations
supplying 1.77% of the R_meas numerator; on the weaker crystal 0.52% supplying 6.82%.
Rescues that lost only rim pixels are unremarkable by the same measure, 1.19x and
0.88x. Dropping the peak-losers alone takes the shell's R_meas from 7.440% back to
7.315% (unrescued: 7.307%) and from 22.03% to 21.06% (unrescued: 21.28%) - which is
the whole of the low-resolution R_meas the rescue cost, and on the second crystal
rather more.

Raw frames say what they are. The pattern is a dead-centre invalid pixel with 5878,
9875 and 27583 counts around it: the detector's per-frame invalid marker on the
brightest reflections. MINPK cannot catch them because it cuts on profile MASS, and
the peak of a broad spot is a few percent of the mass.

So a second condition, in the loop that already measures the readable fraction: no
unreadable pixel may carry more than 0.9 of the profile's own peak value. A fraction
of the peak rather than a radius in pixels because the peak is as wide as the spot -
for a Gaussian the cut is at sqrt(-2 ln f) sigma, 0.46 sigma here, which is the peak
pixel alone where sigma is 0.8 px and the crest of the ridge where the profile is a
bandwidth streak. Swept against the alternatives on two crystals: a fixed radius
needs 1.0-1.5 px to do the same work and costs 3-9x more observations for it, and
0.5 px does not reach the peak of a sub-pixel-offset prediction at all; tightening
the fraction to 0.5 or 0.2 buys nothing beyond 0.9 and costs 7x more.

Six crystals, three detectors, against the rescue as it stands: the rule keeps
99.86-99.96% of the recovered observations and returns R_meas to its unrescued value
or below (4.6 -> 4.5%, 6.7 -> 6.6%, 25.1 -> 25.0%), R_meas in the innermost shell
likewise (2.7 -> 2.6%, 5.9 -> 5.3% against 5.4% unrescued, 16.5 -> 16.4%), <I/sigma>
up or level everywhere, and every unique reflection the rescue won is kept. Raising
--overlap-minpk to 0.90 instead reaches the same place on two of them and short of it
on the third, while discarding 0.8% of the recovered observations rather than 0.05%.
An elongated pink-beam profile on a 9M detector and an EIGER2 16M dataset are both
untouched at 99.9%, so the crest protection does not over-reject a streak.

One crystal is not improved: a dataset whose error model rugnux declines to fit for
want of strong reflections, whose <I/sigma> is <= 0 in eight of its ten shells and
whose R_meas is undefined in as many. There the rule costs about 3% of <I/sigma> in
the one shell that has signal, reproducibly, on top of the 9% the rescue itself costs
there - while its overall R_meas moves 1.5 points on nothing but the thread count.

The parity test gains four sections. Unreadable pixels were only ever punched into
empty sky, so neither the rescue nor this rule had any CPU/GPU coverage at all;
they now go into the signal disks - the peak of every fifth reflection, ~1.1 sigma
out of every seventh, the disk edge of every eleventh - for both profile modes, a
box sum and an elongated stencil, with a check that the clipping actually costs
reflections so the coverage cannot go quietly vacuous.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Full 38-crystal rotation battery against the rescue without this rule, both on the
same base:

  ISa         better 19 / worse  4,   +0.73
  CC1/2       better  3 / worse  1,   +1.3
  R_meas_lo   better  4 / worse  3,   -0.3
  space groups unchanged

for 17 770 observations, 0.09 % of the run total and under 2 % of what the rescue
had won. The two crystals whose peak-loss population was measured beforehand
land on their predicted values: a tetragonal reference goes R_meas_lo 2.7 ->
2.6 % and ISa 27.11 -> 27.42, a cubic insulin 5.9 -> 5.3 % and 20.34 -> 20.65.

One crystal pays: a cubic case with 2381 unique reflections goes R_meas 8.8 ->
9.6 % and ISa 4.08 -> 3.49. It is the crystal in the battery with the fewest
uniques, so its rescued population is small and its shell statistics are coarse,
but the loss is real and not noise in the R_meas. The R_meas sum over the battery
reads +1.3, of which +3.2 is one crystal whose R_meas moves 1.5 points on thread
count alone; without it the sum is negative.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-08-12 03:41:21 +02:00
co-authored by Claude Opus 5
parent cafc312681
commit e4ce573075
6 changed files with 105 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/BraggIntegrationEngineGPUTest.cpp
+57 -3
View File
@@ -41,7 +41,12 @@ Reflection MakeReflection(float x, float y, float d, int hkl) {
// companion_dx > 0 puts a second spot that many pixels beside every grid spot, so their r1 signal
// disks share pixels while the background rings still see clean sky - which is what a dense pattern
// actually looks like (crowded along one reciprocal axis, sparse across it).
Scene BuildScene(size_t width, size_t height, int spacing = 60, float companion_dx = 0.0f) {
// clip_spots punches unreadable pixels into the spots themselves rather than into empty sky: the
// centre of every 5th, a mid-profile pixel of every 7th and a disk-edge pixel of every 11th. That is
// the MINPK rescue's own case - a reflection kept and fitted over the pixels it has - and with it the
// peak-loss rule, which has to fire on the same reflections in both engines.
Scene BuildScene(size_t width, size_t height, int spacing = 60, float companion_dx = 0.0f,
bool clip_spots = false) {
Scene s;
s.width = width;
s.height = height;
@@ -91,6 +96,19 @@ Scene BuildScene(size_t width, size_t height, int spacing = 60, float companion_
const size_t idx = (static_cast<size_t>(k) * 2654435761u) % s.image.size();
s.image[idx] = (k % 2) ? INT32_MIN : INT32_MAX;
}
if (clip_spots)
for (size_t n = 0; n < s.predicted.size(); ++n) {
int dx = 0, dy = 0;
if (n % 5 == 0) { dx = 0; dy = 0; } // the peak itself: the rule must reject
else if (n % 7 == 0) { dx = 1; dy = 1; } // ~1.1 sigma out: near the rule's boundary
else if (n % 11 == 0) { dx = 3; dy = -2; } // disk edge: MINPK keeps it, the rule does not fire
else continue;
const int x = static_cast<int>(std::lround(s.predicted[n].predicted_x)) + dx;
const int y = static_cast<int>(std::lround(s.predicted[n].predicted_y)) + dy;
if (x < 0 || y < 0 || x >= static_cast<int>(width) || y >= static_cast<int>(height)) continue;
s.image[y * width + x] = (n % 2) ? INT32_MAX : INT32_MIN;
}
return s;
}
@@ -126,7 +144,8 @@ void CompareCpuVsGpu(IntegratorMode mode, std::optional<float> bandwidth_fwhm,
float clip_nsigma = 4.0f, bool radial = false, int spacing = 60,
float stencil_k = 0.0f,
float r1 = 0.0f, float r2 = 0.0f, float r3 = 0.0f,
OverlapMode overlap = OverlapMode::Off, float companion_dx = 0.0f) {
OverlapMode overlap = OverlapMode::Off, float companion_dx = 0.0f,
bool clip_spots = false) {
const DiffractionExperiment experiment =
MakeExperiment(mode, bandwidth_fwhm, clip_nsigma, radial, DetJF(2), stencil_k, r1, r2, r3,
overlap);
@@ -135,7 +154,7 @@ void CompareCpuVsGpu(IntegratorMode mode, std::optional<float> bandwidth_fwhm,
const size_t npixel = experiment.GetPixelsNum();
REQUIRE(npixel == width * height);
const Scene scene = BuildScene(width, height, spacing, companion_dx);
const Scene scene = BuildScene(width, height, spacing, companion_dx, clip_spots);
REQUIRE(scene.image.size() == npixel);
REQUIRE(scene.predicted.size() > 60);
@@ -161,6 +180,18 @@ void CompareCpuVsGpu(IntegratorMode mode, std::optional<float> bandwidth_fwhm,
// atomic summation of the learned profile, so compare up to a small tolerance.
REQUIRE(out_gpu.size() == out_cpu.size());
REQUIRE(out_cpu.size() > 40);
if (clip_spots) {
// Guard against the coverage going vacuous: the punched pixels have to actually cost some
// reflections, or the two engines are being compared on a case neither of them meets.
const Scene clean_scene = BuildScene(width, height, spacing, companion_dx, false);
ImagePreprocessorBuffer clean_image(npixel);
for (size_t i = 0; i < npixel; ++i)
clean_image[i] = clean_scene.image[i];
BraggIntegrationEngineCPU clean_cpu(experiment);
const auto out_clean = clean_cpu.Run(clean_image, clean_scene.predicted,
clean_scene.predicted.size(), 5);
CHECK(out_cpu.size() < out_clean.size());
}
for (size_t i = 0; i < out_cpu.size(); ++i) {
INFO("mode " << static_cast<int>(mode) << " reflection " << i << " hkl " << out_cpu[i].h);
CHECK(out_gpu[i].h == out_cpu[i].h);
@@ -239,6 +270,29 @@ TEST_CASE("BraggIntegrationEngineGPU_MatchesCPU") {
0.0f, 0.0f, 0.0f, OverlapMode::Exclude);
}
SECTION("ProfileGaussian mono trim") { CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 0.0f); }
// Unreadable pixels inside the signal disks themselves: the MINPK rescue keeps the reflection and
// fits it over what is left, and the peak-loss rule throws back the ones that lost the profile's
// maximum. Both decisions are per-reflection cuts on a reduction over the profile grid, computed
// independently in the two engines (serial max vs an atomicMax on the float bit pattern), so they
// have to reject exactly the same reflections - a mismatch shows up as a size mismatch here.
SECTION("ProfileGaussian clipped disks") {
CompareCpuVsGpu(IntegratorMode::ProfileGaussian, std::nullopt, 4.0f, false, 60, 0.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, true);
}
SECTION("ProfileEmpirical clipped disks") {
CompareCpuVsGpu(IntegratorMode::ProfileEmpirical, std::nullopt, 4.0f, false, 60, 0.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, true);
}
// The same, with an elongated profile: the peak is then a ridge, so the fraction-of-peak test has
// to protect a crest rather than one pixel, and the grid it reduces over is reflection-dependent.
SECTION("ProfileGaussian clipped disks stencil") {
CompareCpuVsGpu(IntegratorMode::ProfileGaussian, 0.005f, 4.0f, false, 120, 3.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, true);
}
SECTION("BoxSum clipped disks") {
CompareCpuVsGpu(IntegratorMode::BoxSum, std::nullopt, 4.0f, false, 60, 0.0f,
0.0f, 0.0f, 0.0f, OverlapMode::Off, 0.0f, true);
}
// The radial background curvature correction is computed independently in the two engines
// (host loop vs radial_correct kernel), so it needs its own parity coverage.
SECTION("BoxSum radial") { CompareCpuVsGpu(IntegratorMode::BoxSum, std::nullopt, 4.0f, true); }