CPU spot finder: the vertical window slide vectorises, output unchanged

SlideVertical took a missing row as a null pointer and read `in ? in[col] : INT32_MIN` inside the
loop. A load under a condition cannot be if-converted, so GCC 13 (-march=x86-64-v3) vectorised only
the rarely taken variant and ran the one every interior row takes - both rows present - as a scalar
loop with branches. A missing row is now a row of INT32_MIN, which the loop already counts as
nothing, so both rows are always read and the loop vectorises (AVX2, alias-versioned). Same integer
sums in the same columns, so the same strong pixels.

--no-gpu, perf (load 23-35, provisional): SlideVertical cycles -47% / instructions -71% on
myoglobin (7.20e11 -> 3.84e11 cycles), -43% / -71% on YAG, -53% / -71% on cytochrome C; the
detection pass as a whole (FlagRow + SlideVertical + DetectPass) -17%, -16%, -26%. The pass is now
bound by the first read of each image row and its ring bins (FlagRow, IPC ~0.4), so total cycles
move by only 1-3%. p.hkl and MTZ data identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
This commit is contained in:
2026-10-15 17:42:35 +02:00
co-authored by Claude Opus 5.5
parent 873880bba1
commit 96388a389f
@@ -45,14 +45,15 @@ bool StrongInWindow(int64_t pxl_val, int64_t sum, int64_t sum2, int64_t valid,
// pixel is above SNR threshold
}
// Adds row `in` to the vertical sums and takes row `out` out of them; nullptr is no row. A bad or
// saturated pixel adds 0. Written without branches, and as a function of its own so that the loop
// vectorises whatever it is inlined into; integer sums, so the same totals as pixel by pixel.
// Adds row `in` to the vertical sums and takes row `out` out of them; a row of INT32_MIN is no row. A
// bad or saturated pixel adds 0. Written without branches - both rows are always read, so no load is
// conditional - and as a function of its own, so that the loop vectorises whatever it is inlined into;
// integer sums, so the same totals as pixel by pixel.
void SlideVertical(const int32_t *in, const int32_t *out, int32_t width,
int64_t *sum, int64_t *sum2, uint16_t *valid) {
for (int32_t col = 0; col < width; col++) {
const int64_t a = in ? in[col] : INT32_MIN;
const int64_t r = out ? out[col] : INT32_MIN;
const int64_t a = in[col];
const int64_t r = out[col];
const bool a_ok = a != INT32_MAX && a != INT32_MIN;
const bool r_ok = r != INT32_MAX && r != INT32_MIN;
const int64_t a_v = a_ok ? a : 0;
@@ -190,9 +191,10 @@ void ImageSpotFinderCPU::DetectPass(const ImagePreprocessorBuffer &image,
int64_t *sv = sum_vert.data();
int64_t *sv2 = sum2_vert.data();
uint16_t *vv = valid_vert.data();
const std::vector<int32_t> no_row(width, INT32_MIN);
auto slide_vert = [&](int line_in, int line_out) {
SlideVertical(line_in >= 0 ? img + static_cast<size_t>(line_in) * w : nullptr,
line_out >= 0 ? img + static_cast<size_t>(line_out) * w : nullptr, w, sv, sv2, vv);
SlideVertical(line_in >= 0 ? img + static_cast<size_t>(line_in) * w : no_row.data(),
line_out >= 0 ? img + static_cast<size_t>(line_out) * w : no_row.data(), w, sv, sv2, vv);
if (!prev_strong)
return;
// sign = +1 takes out what line_in added, -1 puts back what line_out took out.