mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 68 Wiki Activity

Size the post-refine gather by its reflections, not by its frames

Browse Source 
Both passes of the gather are chunked over OUTCOMES, one per frame, but their cost is
the tens of thousands of reflections inside each. ThreadsForWork was being asked about
the frames: a rotation dataset has on the order of a thousand of them against a floor
of 32768 items per thread, so it answered 1, and the pass that walks every reflection
in the run ran on a single thread on every dataset in the test set.

RotationScaleMerge::Ingest does the identical thing and asks with the observation
count, which is what makes this an inconsistency rather than a tuning choice. The
reflection total is already to hand a few lines up.

The split is unchanged - only the worker count moves - so the output is identical.
Measured on the heaviest crystal: the phase drops 33 %, from 8.6 s to 5.5 s, and its
mean occupancy goes from 4.1 threads to 12.3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jungfrau
2026-08-17 18:05:19 -04:00
co-authored by Claude Opus 5
parent 2308bbad8c
commit 65eb86ab66
1 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
image_analysis/geom_refinement/PostRefine.cpp
+8 -2
View File
@@ -136,8 +136,14 @@ PostRefineResult PostRefineRotationGeometry(const std::vector<IntegrationOutcome
// reflections copies the whole thing every time it doubles - several gigabytes of pure
// copying - and the counts are cheap to take. Each outcome then owns a slice, so the fill
// runs on all threads and lands in the order the serial loop produced.
// Both passes below are chunked over OUTCOMES but their cost is in the reflections inside them,
// tens of thousands per frame, so the worker count has to come from the reflection count. Asked
// about the outcomes instead, ThreadsForWork sees a frame count against its 32768-per-thread
// floor and answers 1 - which ran the whole gather on one thread on every dataset in the test
// set. RotationScaleMerge::Ingest already asks the same question the right way round.
const size_t gather_threads = ThreadsForWork(n_integrated, nthreads);
std::vector<size_t> pts_offset(n_out + 1, 0);
ParallelChunks(n_out, ThreadsForWork(static_cast<size_t>(n_out), nthreads), [&](int lo, int hi) {
ParallelChunks(n_out, gather_threads, [&](int lo, int hi) {
for (int o = lo; o < hi; o++) {
size_t keep = 0;
for (const auto &r : outcomes[o].reflections)
@@ -151,7 +157,7 @@ PostRefineResult PostRefineRotationGeometry(const std::vector<IntegrationOutcome
pts_offset[o + 1] += pts_offset[o];
std::vector<Partial> pts(pts_offset[n_out]);
ParallelChunks(n_out, ThreadsForWork(static_cast<size_t>(n_out), nthreads), [&](int lo, int hi) {
ParallelChunks(n_out, gather_threads, [&](int lo, int hi) {
for (int o = lo; o < hi; o++) {
size_t at = pts_offset[o];
for (const auto &r : outcomes[o].reflections) {