Files
Jungfraujoch/image_analysis/indexing/IndexerThreadPool.h
T
leonarski_f cc925b2668
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m30s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m13s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m45s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m13s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m51s
Build Packages / build:rpm (rocky8) (push) Successful in 8m29s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 9m31s
Build Packages / build:rpm (rocky9) (push) Successful in 9m42s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m47s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m23s
Build Packages / Generate python client (push) Successful in 19s
Build Packages / Build documentation (push) Successful in 38s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 6m18s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m4s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m35s
Build Packages / DIALS test (push) Successful in 10m39s
Build Packages / Unit tests (push) Successful in 1h24m58s
Remove NUMAHWPolicy and the libnuma dependency
NUMA CPU/memory pinning is no longer worthwhile: the FPGA DMA buffers are
placed device-local by the kernel (dma_alloc_coherent), the big RAM ring
buffer is random-access (first-touch handles placement), and GPU work is
already spread across all visible devices. So drop the pinning entirely
and with it libnuma.

- Delete NUMAHWPolicy; the only concern worth keeping - GPU selection -
  is done directly via pin_gpu() (round-robin over visible GPUs) in the
  indexer pool and the Lite analysis threads. CPU-only threads
  (FPGA acquire/pedestal/summation/frame-transform) no longer bind
  anything.
- Drop get_gpu_numa_node() (sysfs lookup) - only SelectGPUAndItsNUMA
  used it.
- numa_policy broker setting is deprecated and ignored (kept in the API
  for backward compatibility; warns once on startup).
- Remove NUMA_LIBRARY / numa.h / numaif.h detection from CMake.
- Docs: drop the NUMA dependency, remove the numa_policy config example,
  and document running multiple brokers on disjoint GPUs via
  CUDA_VISIBLE_DEVICES.
- Remove NUMA_GPU_REVIEW.md (the planning note; this work is now done).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 20:25:36 +02:00

62 lines
1.7 KiB
C++

// SPDX-FileCopyrightText: 2025 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <thread>
#include <mutex>
#include <condition_variable>
#include <queue>
#include <functional>
#include <future>
#include <vector>
#include <optional>
#include <memory>
#include <latch>
#include "../common/JFJochMessages.h"
#include "../common/DiffractionSpot.h"
#include "../common/DiffractionExperiment.h"
#include "Indexer.h"
class IndexerThread {
struct TaskInput {
const DiffractionExperiment &experiment;
const std::vector<Coord> &recip;
};
bool stop = false;
enum class TaskState {STARTING, IDLE, READY, RUNNING, COMPLETED, ERROR} state = TaskState::STARTING;
std::mutex m;
std::condition_variable c_running;
std::condition_variable c_start;
std::condition_variable c_done;
std::unique_ptr<IndexerResult> result = nullptr;
std::unique_ptr<TaskInput> task_input = nullptr;
std::thread worker_thread;
void Worker(const IndexingSettings& settings, int threadid);
public:
IndexerThread(const IndexingSettings& settings, int threadid);
~IndexerThread();
std::unique_ptr<IndexerResult> Run(const DiffractionExperiment &experiment, const std::vector<Coord> &recip);
void Finalize();
};
class IndexerThreadPool {
std::mutex m;
std::condition_variable c;
std::vector<uint8_t> worker_busy;
size_t worker_free_count;
std::vector<std::unique_ptr<IndexerThread>> tasks;
const int64_t viable_cell_min_spots;
const bool blocking;
int GetFreeWorker();
public:
IndexerThreadPool(const IndexingSettings& settings);
IndexerResult Run(const DiffractionExperiment& experiment, const std::vector<Coord>& recip);
};