leonarski_fandClaude Opus 5.5
fe053aff0f
Exact: P4 p.hkl identical to the base on 39 noise-subset sets; concurrent arms p.hkl md5 and MTZ data identical to P4 on 12 sets at default -N and -N 8 (incl. 5 sets where the arms disagree). Side arm on card 2 with 3+ cards, else on the last card when 2 x observations x 256 B fits its total memory (decided up front). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
99 lines
4.7 KiB
C++
99 lines
4.7 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <chrono>
|
|
#include <cstddef>
|
|
#include <cstdint>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
int32_t get_gpu_count();
|
|
|
|
// Names of the visible GPUs, in device order and one entry per device, so repeated cards repeat.
|
|
// Empty without CUDA and on a machine with no device, which is also what get_gpu_count() == 0 says.
|
|
std::vector<std::string> get_gpu_names();
|
|
|
|
// The same list collapsed for a person: "4x NVIDIA A100-SXM4-80GB", or several such groups separated
|
|
// by ", " on a mixed machine. Empty when no GPU is visible.
|
|
std::string get_gpu_description();
|
|
|
|
// The device's total memory in bytes - not what is free at the moment. 0 without CUDA, or when the
|
|
// device cannot be queried.
|
|
size_t get_gpu_total_memory(int32_t dev_id);
|
|
|
|
void set_gpu(int32_t dev_id);
|
|
|
|
// Pin the calling thread to the next GPU in round-robin order, using a process-wide counter
|
|
// (counter++ % get_gpu_count()). Call once per thread; no thread id needed. No-op when no GPU
|
|
// is visible. Honours CUDA_VISIBLE_DEVICES via get_gpu_count().
|
|
void pin_gpu();
|
|
|
|
// From here on, pin_gpu() also keeps the calling thread on the CPUs of the NUMA node its GPU is
|
|
// attached to (ThreadAffinity.h) - on a machine with one node, or without CUDA, nothing changes.
|
|
// Off unless a program asks for it.
|
|
void enable_gpu_numa_binding();
|
|
|
|
// pin_gpu() onto a given device: set_gpu(dev_id), and the NUMA binding above where it is enabled.
|
|
// For a worker that takes its card by index rather than round-robin.
|
|
void pin_gpu(int32_t dev_id);
|
|
|
|
// pin_gpu(dev_id) for as long as the object lives; then the calling thread gets back the device and
|
|
// the CPUs it had before. For a thread that drives a card only for a while - the merging engine, whose
|
|
// calls come from a thread that goes on to other work - where a pin for the rest of its life would
|
|
// carry the card's node onto everything that thread does, and starts, afterwards.
|
|
class ScopedGpuPin {
|
|
public:
|
|
explicit ScopedGpuPin(int32_t dev_id);
|
|
~ScopedGpuPin();
|
|
ScopedGpuPin(const ScopedGpuPin &) = delete;
|
|
ScopedGpuPin &operator=(const ScopedGpuPin &) = delete;
|
|
private:
|
|
int32_t prev_device = -1; // -1: nothing to give back
|
|
std::vector<int> prev_cpus; // empty: the CPUs were not changed
|
|
};
|
|
|
|
// Have every GPU's host threads BLOCK in a synchronisation (cudaDeviceScheduleBlockingSync) instead
|
|
// of spinning on a core until the device finishes. Must be called before anything creates a CUDA
|
|
// context; it creates none itself. No-op without CUDA.
|
|
void set_gpu_blocking_sync();
|
|
|
|
// Create the CUDA context of every device from first_device on, one thread per device, and return
|
|
// when they all exist - for a caller that runs it beside other work, so that the first use of a card
|
|
// does not wait for its context. No-op without CUDA.
|
|
void create_gpu_contexts(int32_t first_device);
|
|
|
|
// Destroy every device's CUDA context, one thread per device, for a program about to exit with no
|
|
// CUDA work left: at exit the runtime destroys them one after another, a fifth of a second per card.
|
|
// No-op without CUDA.
|
|
void release_gpu_contexts();
|
|
|
|
// Drop the error CUDA has recorded for the calling thread. Call it where a CUDA failure has been
|
|
// HANDLED - a device route that fell back to the host, an indexing attempt whose failure was turned
|
|
// into a result - because the error otherwise stays as the thread's last error and the next
|
|
// cuda_err(cudaGetLastError()) after some later kernel launch reports it, over work that went fine.
|
|
// A sticky error (an illegal access, say) is not cleared by this, and nothing here pretends it is:
|
|
// the context is gone in that case and every later call fails on its own. No-op without CUDA.
|
|
void cuda_clear_error();
|
|
|
|
// Throw if the CUDA context is lost - a sticky error (an illegal access, say) that every later call
|
|
// on this device will fail with. For a device route about to fall back to the host: there is no
|
|
// fallback from that, and falling back only moves the failure somewhere less clear. A handled,
|
|
// non-sticky failure passes. No-op without CUDA.
|
|
void cuda_throw_if_context_lost();
|
|
|
|
// GPU work that runs beside the main line of a process and gives all of its device memory back when it
|
|
// ends: a probe pass run on a copy of the run while the run itself scales and merges. Hold one for as
|
|
// long as that work runs. Build-independent - it only counts.
|
|
class GPUWorkBeside {
|
|
public:
|
|
GPUWorkBeside();
|
|
~GPUWorkBeside();
|
|
GPUWorkBeside(const GPUWorkBeside &) = delete;
|
|
GPUWorkBeside &operator=(const GPUWorkBeside &) = delete;
|
|
};
|
|
|
|
// Wait until no GPUWorkBeside is held, for at most `timeout`. True once none is - at once if none was.
|
|
bool wait_for_gpu_work_beside(std::chrono::seconds timeout);
|