leonarski_fandClaude Opus 5.5
4b1d7feebd
- set_gpu_blocking_sync(): every device is put in cudaDeviceScheduleBlockingSync before its context exists, so a host thread waiting on the GPU sleeps instead of spinning on a core. On a 16M rotation run a fifth of all CPU time was that spinning; wall time unchanged within noise. Called first thing in rugnux. - enable_gpu_numa_binding(): from then on pin_gpu() (and the new pin_gpu(dev), used by the first-pass spot workers that take a card by index) also keeps the thread on the CPUs of the NUMA node the card hangs off. The node and its CPUs come from /sys (no libnuma), intersected with the process's own mask; Linux only, and nothing happens on a machine with a single node. rugnux turns it on; the broker does not. - A thread inherits its creator's affinity, so the shared ParallelFor pool would run every later pass on one socket if a pinned worker created it: its threads now reset to the mask the process started with (common/ThreadAffinity). Byte-identical output. The NUMA part is a no-op on the single-node test box and still has to be measured on a two-socket machine. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C
47 lines
2.3 KiB
C++
47 lines
2.3 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <cstdint>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
int32_t get_gpu_count();
|
|
|
|
// Names of the visible GPUs, in device order and one entry per device, so repeated cards repeat.
|
|
// Empty without CUDA and on a machine with no device, which is also what get_gpu_count() == 0 says.
|
|
std::vector<std::string> get_gpu_names();
|
|
|
|
// The same list collapsed for a person: "4x NVIDIA A100-SXM4-80GB", or several such groups separated
|
|
// by ", " on a mixed machine. Empty when no GPU is visible.
|
|
std::string get_gpu_description();
|
|
void set_gpu(int32_t dev_id);
|
|
|
|
// Pin the calling thread to the next GPU in round-robin order, using a process-wide counter
|
|
// (counter++ % get_gpu_count()). Call once per thread; no thread id needed. No-op when no GPU
|
|
// is visible. Honours CUDA_VISIBLE_DEVICES via get_gpu_count().
|
|
void pin_gpu();
|
|
|
|
// From here on, pin_gpu() also keeps the calling thread on the CPUs of the NUMA node its GPU is
|
|
// attached to (ThreadAffinity.h) - on a machine with one node, or without CUDA, nothing changes.
|
|
// Off unless a program asks for it.
|
|
void enable_gpu_numa_binding();
|
|
|
|
// pin_gpu() onto a given device: set_gpu(dev_id), and the NUMA binding above where it is enabled.
|
|
// For a worker that takes its card by index rather than round-robin.
|
|
void pin_gpu(int32_t dev_id);
|
|
|
|
// Have every GPU's host threads BLOCK in a synchronisation (cudaDeviceScheduleBlockingSync) instead
|
|
// of spinning on a core until the device finishes. Must be called before anything creates a CUDA
|
|
// context; a device that already has one keeps the flags it was created with. No-op without CUDA.
|
|
void set_gpu_blocking_sync();
|
|
|
|
// Drop the error CUDA has recorded for the calling thread. Call it where a CUDA failure has been
|
|
// HANDLED - a device route that fell back to the host, an indexing attempt whose failure was turned
|
|
// into a result - because the error otherwise stays as the thread's last error and the next
|
|
// cuda_err(cudaGetLastError()) after some later kernel launch reports it, over work that went fine.
|
|
// A sticky error (an illegal access, say) is not cleared by this, and nothing here pretends it is:
|
|
// the context is gone in that case and every later call fails on its own. No-op without CUDA.
|
|
void cuda_clear_error();
|