jfjoch_process: spread per-image GPU work across all visible GPUs
The offline worker threads built MXAnalysisWithoutFPGA without selecting a CUDA device, so all per-image preprocessing/spot-finding/azimuthal integration ran on GPU 0 (only the indexer pool was distributed). Add pin_gpu() to CUDAWrapper - a process-wide round-robin counter (counter++ % get_gpu_count(), no thread id, no-op without a GPU, honours CUDA_VISIBLE_DEVICES) - and call it once per worker before building the analysis resources so their CUDA streams/engines land on distinct devices. Also add NUMA_GPU_REVIEW.md: a working note mapping ImageBuffer/NUMAHWPolicy/GPU dispatch with goals and a staged plan (multi-broker GPU isolation via CUDA_VISIBLE_DEVICES, dropping libnuma, reassessing NUMA pinning for the FPGA path). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -25,6 +25,7 @@
|
||||
#include "../common/print_license.h"
|
||||
#include "../image_analysis/MXAnalysisWithoutFPGA.h"
|
||||
#include "../image_analysis/indexing/IndexerFactory.h"
|
||||
#include "../common/CUDAWrapper.h"
|
||||
#include "../writer/FileWriter.h"
|
||||
#include "../image_analysis/IndexAndRefine.h"
|
||||
#include "../receiver/JFJochReceiverPlots.h"
|
||||
@@ -847,6 +848,11 @@ int main(int argc, char **argv) {
|
||||
std::atomic<int> finished_count = 0;
|
||||
|
||||
auto worker = [&](int thread_id) {
|
||||
// Spread the per-image GPU pipeline across all visible GPUs (round-robin).
|
||||
// Must run before building the analysis resources so their CUDA streams/engines are
|
||||
// created on this thread's device.
|
||||
pin_gpu();
|
||||
|
||||
// Thread-local analysis resources
|
||||
MXAnalysisWithoutFPGA analysis(experiment, mapping, pixel_mask, indexer);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user