mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 67 Wiki Activity

rugnux: make the geometry-refinement sample deterministic, and spread it over the run

Browse Source 
The stills first pass drew frames from a shared cursor and stopped when a shared counter
reached its target, which got two things wrong at once. The cursor walked the equally
spaced sample in ascending order, so stopping early read only its leading PREFIX - the
beam centre, distance and cell were fitted to the beginning of the run, not across it,
and the comment claiming otherwise was wrong. And where the stop landed depended on how
the workers happened to interleave, so the set of frames varied run to run: on the same
data at -N 32 and -N 8 the pass examined 483 and 457 frames and refined the detector
distance to 168.0481 and 168.0530 mm.

The sample is now cut into a fixed number of interleaved stripes, each stopping once it
has contributed its share. Every stripe spans the whole run, so an early stop no longer
biases the fit, and a stripe is processed identically whichever worker claims it - so
what gets examined depends only on the data, not on timing and not on -N. The same three
runs now give 451 frames examined and 168.0452 mm, identically.

The bundle selection was order-dependent too: frames are collected in worker-completion
order and sorted by spot count with a non-stable sort, so equally strong frames swapped
places between runs. They carry their image ordinal now and it breaks the tie.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-08-03 15:19:11 +02:00
co-authored by Claude Opus 5
parent 5f8d37cdab
commit 212fbf9bab
2 changed files with 29 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
image_analysis/geom_refinement/GeometryRefiner.h
+2
View File
@@ -27,6 +27,8 @@ struct GeomRefineSpot {
struct GeomRefineFrame {
CrystalLattice lattice; // per-frame indexed lattice (real space), for the orientation seed
std::vector<GeomRefineSpot> spots; // indexed spots on this frame
int32_t ordinal = 0; // image this came from; ties are broken on it, so the
// selected bundle does not depend on collection order
};
struct GeometryRefinerSettings {
rugnux/Rugnux.cpp
+27 -8
View File
@@ -126,13 +126,22 @@ void Rugnux::RefineStillsGeometry(int start_image, int end_image, int images_to_
// draining the whole budget. The budget is sized for a low-hit-rate dataset (refine_frames * 50, at
// least 8000), so on data that indexes well it meant re-indexing the ENTIRE run to keep 200 frames -
// 99% of this pass was sampling. Four times the bundle leaves the "strongest N" selection a real
// pool to choose from; the frames are equally spaced, so an early stop still spans the run.
// pool to choose from.
const int strong_target = refine_frames * 4;
// The sample is cut into a FIXED number of interleaved stripes - stripe s is sample positions
// s, s+STRIPES, s+2*STRIPES, ... - and each stripe stops once it has contributed its share of the
// strong frames. Two things follow, both of which the previous shared-cursor-with-a-shared-stop
// arrangement got wrong. Every stripe is spread over the WHOLE run, so stopping early no longer
// means fitting the geometry to the beginning of it. And which frames get examined depends only
// on the data: it is not a race between the workers, and it does not change with -N, because a
// stripe is processed identically whichever worker happens to claim it.
constexpr int STRIPES = 32;
const int stripe_strong_target = std::max(1, (strong_target + STRIPES - 1) / STRIPES);
std::vector<GeomRefineFrame> frames;
std::mutex frames_mutex;
std::atomic<int> next_i = 0;
std::atomic<int> strong_found = 0;
std::atomic<int> next_stripe = 0;
std::atomic<int> examined = 0;
auto worker = [&]() {
@@ -141,9 +150,11 @@ void Rugnux::RefineStillsGeometry(int start_image, int end_image, int images_to_
/*enable_fused_adaptive_gpu=*/true);
AzimuthalIntegrationProfile profile(mapping);
while (!cancelled_ && strong_found.load(std::memory_order_relaxed) < strong_target) {
const int idx = next_i.fetch_add(1);
if (idx >= static_cast<int>(sample.size())) break;
for (int stripe = next_stripe.fetch_add(1); stripe < STRIPES && !cancelled_;
stripe = next_stripe.fetch_add(1)) {
int stripe_strong = 0;
for (int idx = stripe; idx < static_cast<int>(sample.size()) && !cancelled_
&& stripe_strong < stripe_strong_target; idx += STRIPES) {
examined.fetch_add(1, std::memory_order_relaxed);
const int ordinal = sample[idx];
const int image_idx = start_image + ordinal * config_.stride;
@@ -174,17 +185,19 @@ void Rugnux::RefineStillsGeometry(int start_image, int end_image, int images_to_
GeomRefineFrame f;
f.lattice = *msg.indexing_lattice;
f.ordinal = ordinal;
for (const auto &s : msg.spots)
if (s.indexed && s.lattice == 0)
f.spots.push_back(GeomRefineSpot{s.x, s.y,
static_cast<int32_t>(s.h), static_cast<int32_t>(s.k), static_cast<int32_t>(s.l)});
if (f.spots.size() >= 6) {
if (static_cast<int>(f.spots.size()) >= MIN_STRONG_SPOTS)
strong_found.fetch_add(1, std::memory_order_relaxed);
stripe_strong++;
const std::unique_lock ul(frames_mutex);
frames.push_back(std::move(f));
}
}
}
};
std::vector<std::future<void>> futures;
@@ -203,8 +216,14 @@ void Rugnux::RefineStillsGeometry(int start_image, int end_image, int images_to_
for (auto &f : frames)
if (static_cast<int>(f.spots.size()) >= MIN_STRONG_SPOTS)
strong.push_back(std::move(f));
// frames is in the order the workers happened to finish, so the spot count alone does not order
// it: without the ordinal tie-break, equally strong frames would swap places between runs and a
// different bundle would be refined.
std::sort(strong.begin(), strong.end(),
[](const GeomRefineFrame &a, const GeomRefineFrame &b) { return a.spots.size() > b.spots.size(); });
[](const GeomRefineFrame &a, const GeomRefineFrame &b) {
if (a.spots.size() != b.spots.size()) return a.spots.size() > b.spots.size();
return a.ordinal < b.ordinal;
});
if (static_cast<int>(strong.size()) > refine_frames)
strong.resize(refine_frames);