From 65eb86ab66d3c58183990c1900f39fe7d234e697 Mon Sep 17 00:00:00 2001
From: jungfrau <jungfrau@mx-aare-test.psi.ch>
Date: Mon, 17 Aug 2026 18:05:19 -0400
Subject: [PATCH] Size the post-refine gather by its reflections, not by its
 frames

Both passes of the gather are chunked over OUTCOMES, one per frame, but their cost is
the tens of thousands of reflections inside each. ThreadsForWork was being asked about
the frames: a rotation dataset has on the order of a thousand of them against a floor
of 32768 items per thread, so it answered 1, and the pass that walks every reflection
in the run ran on a single thread on every dataset in the test set.

RotationScaleMerge::Ingest does the identical thing and asks with the observation
count, which is what makes this an inconsistency rather than a tuning choice. The
reflection total is already to hand a few lines up.

The split is unchanged - only the worker count moves - so the output is identical.
Measured on the heaviest crystal: the phase drops 33 %, from 8.6 s to 5.5 s, and its
mean occupancy goes from 4.1 threads to 12.3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
 image_analysis/geom_refinement/PostRefine.cpp | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/image_analysis/geom_refinement/PostRefine.cpp b/image_analysis/geom_refinement/PostRefine.cpp
index 21948c46..25f23ef1 100644
--- a/image_analysis/geom_refinement/PostRefine.cpp
+++ b/image_analysis/geom_refinement/PostRefine.cpp
@@ -136,8 +136,14 @@ PostRefineResult PostRefineRotationGeometry(const std::vector<IntegrationOutcome
         // reflections copies the whole thing every time it doubles - several gigabytes of pure
         // copying - and the counts are cheap to take. Each outcome then owns a slice, so the fill
         // runs on all threads and lands in the order the serial loop produced.
+        // Both passes below are chunked over OUTCOMES but their cost is in the reflections inside them,
+        // tens of thousands per frame, so the worker count has to come from the reflection count. Asked
+        // about the outcomes instead, ThreadsForWork sees a frame count against its 32768-per-thread
+        // floor and answers 1 - which ran the whole gather on one thread on every dataset in the test
+        // set. RotationScaleMerge::Ingest already asks the same question the right way round.
+        const size_t gather_threads = ThreadsForWork(n_integrated, nthreads);
         std::vector<size_t> pts_offset(n_out + 1, 0);
-        ParallelChunks(n_out, ThreadsForWork(static_cast<size_t>(n_out), nthreads), [&](int lo, int hi) {
+        ParallelChunks(n_out, gather_threads, [&](int lo, int hi) {
             for (int o = lo; o < hi; o++) {
                 size_t keep = 0;
                 for (const auto &r : outcomes[o].reflections)
@@ -151,7 +157,7 @@ PostRefineResult PostRefineRotationGeometry(const std::vector<IntegrationOutcome
             pts_offset[o + 1] += pts_offset[o];
 
         std::vector<Partial> pts(pts_offset[n_out]);
-        ParallelChunks(n_out, ThreadsForWork(static_cast<size_t>(n_out), nthreads), [&](int lo, int hi) {
+        ParallelChunks(n_out, gather_threads, [&](int lo, int hi) {
             for (int o = lo; o < hi; o++) {
                 size_t at = pts_offset[o];
                 for (const auto &r : outcomes[o].reflections) {