mx/Jungfraujoch
3
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 81 Wiki Activity

Rotation merge: drop rocking events with an overloaded pixel; capture uncertainty in the merge variance

Browse Source 
A saturated pixel in a spot means the brightest part of the reflection was
not measured. The integration used to drop the peak frame's partial (its
peak pixel is unreadable) and keep the flanks, so the combine extrapolated
the event from its tails by the partiality model: on a strongly
diffracting small-molecule crystal the strongest low-order reflections
read 2-3x low and were the largest SHELXL misfits. XDS drops such a
reflection (OVERLOAD); so does rugnux now.

- Integration (CPU + GPU engines): a reflection is `overloaded` when a
  signal-disk pixel is saturated, or unreadable on this frame but not in
  the run's pixel mask - EIGER/PILATUS write their error value for a
  pixel they could not count, which the preprocessor turns into a masked
  pixel like a gap's. The engines now receive the PixelMask to tell the
  two apart (an earlier attempt that re-classified the marker as
  saturation in the preprocessor broke a dataset whose gaps are not in
  the file's mask). An overloaded reflection is kept with its box sum,
  unfitted, only so its event can be recognised.
- Rotation combine (CPU + GPU): an event with any overloaded partial is
  dropped whole; counted in the log and the report
  (OBSERVATIONS_REJECTED_OVERLOAD=). The unmerged MTZ export drops it too.
- Everything else that reads reflections leaves an overloaded one out:
  AcceptReflection (stills merge, per-image scaling), the post-refinement
  gather, the axial-row sums.
- Capture uncertainty: the merge rebuilds each full's variance at the
  reflection's mean (counting_variance / ModelSigma) and dropped the
  capture term the combine had put into sigma, so a full extrapolated
  from part of its rocking curve merged at the weight of a whole one.
  Fulls now carry it (Obs::capture) and the rebuilt variance adds
  (capture * <I>)^2, host and device.

SHELXL R1 on rugnux's own integration (harness), median fix -> this:
citric acid .0648 -> .0420 (XDS .051; 221 events dropped, EXTI 1.02 -> 0.29),
HEPES .0396 -> .0381 (184), aspirin 20 keV .0387 -> .0385 (6),
aspirin 25 keV .0376 -> .0375 (5); metformin/nidppe/dnba/lalanine/cytidine
no overloads, unchanged. YAG .116 -> .128 (87 dropped; its scale loop does
not settle either way). Proteins and private subset: see the branch report.
Tests: BraggIntegrationEngineCPU_SaturatedPeakIsFlaggedNotDropped (new),
BraggIntegrationEngineGPU_MatchesCPU (overloaded flag compared),
AcceptReflection_ResolutionLimits, [write_reflections], [large].

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5K8jvPPbmCrbqnWkddTuB
This commit is contained in:
leonarski_f
2026-10-04 21:01:40 +02:00
co-authored by Claude Opus 5.5
parent d061e52ba1
commit 37a8c8e24e
25 changed files with 252 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
image_analysis/bragg_integration/BraggIntegrationEngineCPU.cpp
+27 -4
View File
@@ -62,8 +62,10 @@ struct ImageSampler {
} // namespace
BraggIntegrationEngineCPU::BraggIntegrationEngineCPU(const DiffractionExperiment &experiment)
BraggIntegrationEngineCPU::BraggIntegrationEngineCPU(const DiffractionExperiment &experiment,
const PixelMask &mask)
: BraggIntegrationEngine(experiment),
static_mask(mask.GetPackedMask()),
refl_mask(static_cast<int>(xpixel), static_cast<int>(ypixel), 0),
owner(static_cast<int>(xpixel), static_cast<int>(ypixel), BRAGG_OWNER_NONE) {}
@@ -158,6 +160,14 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
int cx = 0, cy = 0, shell = -1;
bool ok = false, strong = false, has_obs = false;
bool full = false; // every pixel of the signal disk was readable
bool overloaded = false; // a pixel of the signal disk was saturated
};
// A saturated pixel, or one unreadable on this frame alone - the overload marker of a detector
// that writes its error value for a pixel it could not count (EIGER), which the preprocessor turns
// into a masked pixel like a gap's. The run's mask holds the gaps, so whatever is unreadable beyond
// it was lost to the flux.
auto static_masked = [&](size_t idx) {
return idx / 32 < static_mask.size() && ((static_mask[idx / 32] >> (idx % 32)) & 1U);
};
std::vector<Rough> rough(npredicted);
double inv_d2_min = std::numeric_limits<double>::max(), inv_d2_max = 0.0;
@@ -235,6 +245,9 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
else if (exclude) continue;
}
++n_inner;
if (px == INT32_MAX
|| (px == INT32_MIN && !static_masked(static_cast<size_t>(y) * W + x)))
out.overloaded = true;
if (!valid(px)) continue;
I_sum += px;
I_sum_x += static_cast<int64_t>(x) * px;
@@ -269,7 +282,7 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
// reflection is dropped whole - the one thing the stencil geometry does to the DATA rather
// than to a measurement. Counted here because it is the only direct evidence of a radius that has
// outgrown the pattern it is integrating (BraggIntegrationCounts).
const bool keep_partial = full || mode != IntegratorMode::BoxSum;
const bool keep_partial = full || mode != IntegratorMode::BoxSum || out.overloaded;
if (keep_partial && n_bkg <= 5) ++counts.bkg_starved;
// Would every predicted neighbour, the tails included, leave the ring starved that the
// detector alone would not? That is the pattern's density, what the guard on a widened
@@ -411,7 +424,7 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
if (overlap == OverlapMode::Reject && rh.n_own < overlap_min_peak * rh.n_disk) continue;
results[i] = {static_cast<float>(rh.I), static_cast<float>(rh.sigma), static_cast<float>(rh.bkg),
static_cast<float>(rh.obs_x), static_cast<float>(rh.obs_y),
static_cast<float>(rh.var_bkg), true, rh.has_obs};
static_cast<float>(rh.var_bkg), true, rh.has_obs, rh.overloaded};
}
return Finalize(predicted, npredicted, results, image_number);
}
@@ -515,6 +528,16 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
for (size_t i = 0; i < npredicted; ++i) {
const auto &rh = rough[i];
if (!rh.ok) continue;
// An overloaded reflection is not measured by any fit - its brightest pixels are what is
// missing. It is passed on with its box sum and the flag, for the merge to drop its rocking
// event (Reflection::overloaded); the missing-peak cut below would otherwise drop just this
// frame and leave the event to be extrapolated from its flanks.
if (rh.overloaded) {
results[i] = {static_cast<float>(rh.I), static_cast<float>(rh.sigma), static_cast<float>(rh.bkg),
static_cast<float>(rh.obs_x), static_cast<float>(rh.obs_y),
static_cast<float>(rh.var_bkg), true, false, true};
continue;
}
const int sh = rh.shell < 0 ? 0 : rh.shell;
int Rf = R;
@@ -668,7 +691,7 @@ std::vector<Reflection> BraggIntegrationEngineCPU::RunImpl(const Sampler &img,
results[i] = {static_cast<float>(I), static_cast<float>(sigma),
static_cast<float>(rh.bkg),
static_cast<float>(rh.obs_x), static_cast<float>(rh.obs_y),
static_cast<float>(var_bkg), true, rh.has_obs};
static_cast<float>(var_bkg), true, rh.has_obs, rh.overloaded};
}
return Finalize(predicted, npredicted, results, image_number);