leonarski_fandClaude Opus 5.5
943d3fc655
set_gpu_blocking_sync() called cudaSetDevice() on every card, and since CUDA 12 that creates the card's primary context: ~0.18 s per card on the workstation (measured, RTX 5080), one after another, before WALL_TIME starts. On the 4-card server that is most of the 1.73 s per run spent outside WALL_TIME (vs 0.37 s on one card). - The blocking-sync flag is now set with the driver's cuDevicePrimaryCtxSetFlags, taken through cudaGetDriverEntryPointByVersion (no link against libcuda, so a CUDA build still starts without a driver). It creates no context; contexts created later carry the flag (cudaGetDeviceFlags 0xc, as before; user CPU time unchanged, so no spinning). - Card 0's context is created by the existing cuFFT warm-up thread, beside the file open and pre-scan; cards 1..N-1 get theirs on a new background thread per card (create_gpu_contexts), as the pre-scan uses card 0 only. - At exit the contexts are destroyed by one thread per card (release_gpu_contexts, cudaDeviceReset) after the run has destroyed everything that used them, instead of serially by the runtime's exit handler. Microbenchmark: an explicit reset costs the same as the implicit teardown on one card (175 ms), so no change on one card. Which card does what is unchanged; only when contexts are made/destroyed. Measured on the 1-card workstation (load 7-16, 2 repeats x 2 sets plus 2 earlier pairs on myob): process start -> first log line median 0.39 -> 0.18 s; last log line -> exit 0.18 -> 0.20 s (one context either way); elapsed - WALL_TIME median 0.53 -> 0.37 s; WALL_TIME unchanged (context creation now overlaps the input read). The multi-card gain (3 serial context creations at start, 4 serial destroys at exit) cannot be measured here. p.hkl md5 identical on myob_x10sa, cytc_x10sa, 2xfw, 6zqy -N 8; all runs exit 0. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
87 lines
2.0 KiB
C++
87 lines
2.0 KiB
C++
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include "CUDAWrapper.h"
|
|
|
|
#include <condition_variable>
|
|
#include <mutex>
|
|
|
|
// Build-independent: the CUDA build gets get_gpu_names() from CUDAWrapper.cu, the CPU-only build from
|
|
// the stub below, and this collapses whichever list came back. Four identical cards read better as
|
|
// "4x <name>" than as the same name four times, and a mixed machine keeps one group per model.
|
|
std::string get_gpu_description() {
|
|
const auto names = get_gpu_names();
|
|
|
|
std::string out;
|
|
for (size_t i = 0; i < names.size();) {
|
|
size_t n = 1;
|
|
while (i + n < names.size() && names[i + n] == names[i])
|
|
n++;
|
|
if (!out.empty())
|
|
out += ", ";
|
|
if (n > 1)
|
|
out += std::to_string(n) + "x ";
|
|
out += names[i];
|
|
i += n;
|
|
}
|
|
return out;
|
|
}
|
|
|
|
namespace {
|
|
std::mutex gpu_work_beside_mutex;
|
|
std::condition_variable gpu_work_beside_done;
|
|
int gpu_work_beside = 0;
|
|
}
|
|
|
|
GPUWorkBeside::GPUWorkBeside() {
|
|
std::lock_guard lock(gpu_work_beside_mutex);
|
|
++gpu_work_beside;
|
|
}
|
|
|
|
GPUWorkBeside::~GPUWorkBeside() {
|
|
{
|
|
std::lock_guard lock(gpu_work_beside_mutex);
|
|
--gpu_work_beside;
|
|
}
|
|
gpu_work_beside_done.notify_all();
|
|
}
|
|
|
|
bool wait_for_gpu_work_beside(std::chrono::seconds timeout) {
|
|
std::unique_lock lock(gpu_work_beside_mutex);
|
|
return gpu_work_beside_done.wait_for(lock, timeout, [] { return gpu_work_beside == 0; });
|
|
}
|
|
|
|
#ifndef JFJOCH_USE_CUDA
|
|
|
|
int32_t get_gpu_count() {
|
|
return 0;
|
|
}
|
|
|
|
std::vector<std::string> get_gpu_names() {
|
|
return {};
|
|
}
|
|
|
|
void set_gpu(int32_t dev_id) {}
|
|
|
|
void pin_gpu() {}
|
|
|
|
void pin_gpu(int32_t dev_id) {}
|
|
|
|
ScopedGpuPin::ScopedGpuPin(int32_t) {}
|
|
|
|
ScopedGpuPin::~ScopedGpuPin() {}
|
|
|
|
void enable_gpu_numa_binding() {}
|
|
|
|
void set_gpu_blocking_sync() {}
|
|
|
|
void create_gpu_contexts(int32_t) {}
|
|
|
|
void release_gpu_contexts() {}
|
|
|
|
void cuda_clear_error() {}
|
|
|
|
void cuda_throw_if_context_lost() {}
|
|
|
|
#endif
|