stills geometry sample and trial spot finding run as many workers as the image pass

The geometry refinement's sample (up to 10000 frames: spot finding + indexing on the card, the same
per-frame work as the image pass) and the stills geometry trial's 96-frame spot finding ran -N workers
(32 here), each with its own GPU engines. The image pass caps the workers that drive a card at 16 per
card, fewer where their engines would not fit half the card (GPUWorkersPerCard), because past that a
card stops taking queue work and more workers make the loop slower. The cap is now one function,
ImageWorkers(), used by all three loops. On a 16.9 Mpx JUNGFRAU 16M the 32 sample workers held ~13 GB
of a 16 GB card's engines (24 B/px each) where the cap allows 19.

Which frames the sample examines does not depend on the worker count (fixed stripes), and each trial
frame is found on its own: outputs identical (JF16M int16 15 runs: same 5174 frames sampled, same
module placement, geometry and merge).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CeEGgpKdfkCTJb3YNMJcu
This commit is contained in:
leonarski_fandClaude Opus 5.5 committed 2026-10-21 17:59:58 +02:00
1 parent 96322a8ec9
commit b87b015253
3 files changed
+26 -18

No files matched your search

+3
View File
@@ -1439,6 +1439,9 @@ class Rugnux {
// How many workers with GPU engines of their own - at most `most` - one card takes: decided from
// the card's total memory and the detector, never from what is free (RugnuxImagePass.cpp).
[[nodiscard]] int GPUWorkersPerCard(int most) const;
// How many workers a loop analysing whole frames on the cards runs - the image pass, and the stills
// samples ahead of it (RugnuxImagePass.cpp).
[[nodiscard]] int ImageWorkers() const;
// The FFT indexers of a first pass's pool: two, for its two schemes at once; one for a probe pass on a
// card too small for it beside the main line, which then indexes the schemes one after the other.
// An FFT indexer holds 0.3-0.6 GB of the card, and which one indexes a frame does not change what it
+19 -15
View File
@@ -95,6 +95,21 @@ int Rugnux::GPUWorkersPerCard(int most) const {
return static_cast<int>(std::clamp<int64_t>(fit, 1, most));
}
// Every worker submits its own kernels to a card, and a card runs out of room to accept them long before it
// runs out of work: measured on two GPUs, the image loop's own time falls to sixteen workers and then rises
// again, so forty-eight is slower than sixteen on a 16 Mpx set. Sixteen is where that measurement put the
// minimum, and a later one on a single card agrees - eight leaves the loop waiting on the queue rather than
// the card. The cap is per card because that is what the queue depth belongs to, and it only applies when -N
// was left at its default - an explicit -N is a deliberate instruction and is obeyed, which is what a
// previous attempt at this got wrong. A worker also holds GPU memory, so a small card gets fewer of them
// (GPUWorkersPerCard).
int Rugnux::ImageWorkers() const {
const int gpus = get_gpu_count();
if (!config_.nthreads_auto || gpus == 0)
return config_.nthreads;
return std::min(config_.nthreads, GPUWorkersPerCard(16) * gpus);
}
void Rugnux::ImagePass(PipelineLocals &p) {
auto &logger = p.logger;
auto &result = p.result;
@@ -374,22 +389,11 @@ void Rugnux::ImagePass(PipelineLocals &p) {
std::function<void()> worker = per_image_analysis ? std::function<void()>(full_worker)
: std::function<void()>(azint_worker);
// How many workers the loop actually wants. Every one of them submits its own kernels to a card,
// and a card runs out of room to accept them long before it runs out of work: measured on two
// GPUs, the loop's own time falls to sixteen workers and then rises again, so forty-eight is
// slower than sixteen on a 16 Mpx set. Sixteen is where that measurement put the minimum, and a
// later one on a single card agrees - eight leaves the loop waiting on the queue rather than the
// card. The cap is per card because that is what the queue depth belongs to, and it only applies
// when -N was left at its default - an explicit -N is a deliberate instruction and is obeyed,
// which is what a previous attempt at this got wrong.
// Only the full-analysis worker drives a card; the azimuthal one preprocesses and integrates on
// the CPU and wants every thread it can have, so the cap must not reach it.
// A worker also holds GPU memory, so a small card gets fewer of them (GPUWorkersPerCard).
// How many workers the loop actually wants (ImageWorkers). Only the full-analysis worker drives a
// card; the azimuthal one preprocesses and integrates on the CPU and wants every thread it can have,
// so the cap must not reach it.
const int gpus = get_gpu_count();
const int workers_per_card = gpus > 0 ? GPUWorkersPerCard(16) : 16;
const int image_workers = (per_image_analysis && config_.nthreads_auto && gpus > 0)
? std::min(config_.nthreads, workers_per_card * gpus)
: config_.nthreads;
const int image_workers = per_image_analysis ? ImageWorkers() : config_.nthreads;
if (image_workers < config_.nthreads)
logger.Info("Per-image loop: {} of {} threads ({} GPUs) - past what a card can take queue "
"work from, more workers make it slower, not faster; pass -N to override",
+4 -3
View File
@@ -234,10 +234,11 @@ void Rugnux::RefineStillsGeometry(int start_image, int end_image, int images_to_
}
};
// As many workers as the image pass: each drives the card the same way, and how many there are does not
// change which frames are examined (the stripes above).
TimingMark sampling_mark(logger, "geometry refinement sample");
std::vector<std::future<void>> futures;
futures.reserve(config_.nthreads);
for (int i = 0; i < config_.nthreads; ++i)
for (int i = 0; i < ImageWorkers(); ++i)
futures.push_back(std::async(std::launch::async, worker));
for (auto &fut : futures)
fut.get();
@@ -459,7 +460,7 @@ std::vector<DataMessage> Rugnux::FindStillsSampleSpots(const std::vector<int> &o
}
};
std::vector<std::future<void>> futures;
for (int i = 0; i < config_.nthreads; ++i)
for (int i = 0; i < ImageWorkers(); ++i)
futures.push_back(std::async(std::launch::async, worker));
for (auto &f : futures)
f.get();