jfjoch_process: spread per-image GPU work across all visible GPUs

The offline worker threads built MXAnalysisWithoutFPGA without selecting a CUDA
device, so all per-image preprocessing/spot-finding/azimuthal integration ran on
GPU 0 (only the indexer pool was distributed). Add pin_gpu() to CUDAWrapper - a
process-wide round-robin counter (counter++ % get_gpu_count(), no thread id, no-op
without a GPU, honours CUDA_VISIBLE_DEVICES) - and call it once per worker before
building the analysis resources so their CUDA streams/engines land on distinct
devices.

Also add NUMA_GPU_REVIEW.md: a working note mapping ImageBuffer/NUMAHWPolicy/GPU
dispatch with goals and a staged plan (multi-broker GPU isolation via
CUDA_VISIBLE_DEVICES, dropping libnuma, reassessing NUMA pinning for the FPGA path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-17 15:29:52 +02:00
co-authored by Claude Opus 4.8
parent 1056acc3a6
commit 02fa15c2b9
5 changed files with 136 additions and 0 deletions
+6
View File
@@ -25,6 +25,7 @@
#include "../common/print_license.h"
#include "../image_analysis/MXAnalysisWithoutFPGA.h"
#include "../image_analysis/indexing/IndexerFactory.h"
#include "../common/CUDAWrapper.h"
#include "../writer/FileWriter.h"
#include "../image_analysis/IndexAndRefine.h"
#include "../receiver/JFJochReceiverPlots.h"
@@ -847,6 +848,11 @@ int main(int argc, char **argv) {
std::atomic<int> finished_count = 0;
auto worker = [&](int thread_id) {
// Spread the per-image GPU pipeline across all visible GPUs (round-robin).
// Must run before building the analysis resources so their CUDA streams/engines are
// created on this thread's device.
pin_gpu();
// Thread-local analysis resources
MXAnalysisWithoutFPGA analysis(experiment, mapping, pixel_mask, indexer);