leonarski_fandClaude Opus 5
e4dce1d778
The results report said what the run determined but nothing about how it was produced, so a report read next to a lost shell history could not be reproduced or compared. Four keys in the header block, all new, none renamed - a consumer that greps for what it needs is unaffected, so REPORT_VERSION does not move: COMMAND_LINE= rugnux -o myrun --model model.pdb dataset_master.h5 WALL_TIME= 262.41 GPU_COUNT= 4 GPU= 4x NVIDIA A100-SXM4-80GB The command line is argv as one shell-ready line; an argument that would not survive being pasted back into a shell is single-quoted, so a file prefix with a space comes back as the one argument it was. It replaces the raw argv echo at the top of the run, which had no quoting at all. WALL_TIME is the whole invocation, timed from the top of the CLI. It is deliberately not result.total_time_s, which starts inside Rugnux::Run and so counts neither opening the file nor setting up the analysis - and which --mode scale never sets at all, having no ProcessResult of its own. It is printed on stdout as well, next to the processing time it is slightly larger than. The GPUs are the reason rugnux is fast, and until now nothing said whether any were being used. get_gpu_names() reports them per device and get_gpu_description() collapses repeats, so four identical cards read as one line rather than the same name four times and a mixed machine keeps one group per model. Both have a CPU-only implementation, so the JFJOCH_USE_CUDA=OFF build reports GPU_COUNT= 0 rather than failing to link. The same line is printed at startup, before the run rather than after it: a machine that turns out to have no GPU - a driver mismatch, a CUDA_VISIBLE_DEVICES left over from another job - is worth knowing about while there is still time to stop, not once the run has taken an order of magnitude longer than it should. GPU_COUNT= 0 is written with no GPU= line beside it, because the absence is the statement. The header comment claiming timing is deliberately absent from the report is now wrong and says so: rates and per-image costs stay on stdout, the total does not. Verified on the rotation test dataset in both build configurations, with and without CUDA_VISIBLE_DEVICES, in --mode mx and --mode scale. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016WmryXe8ASbNi632sUMfsa
64 lines
1.9 KiB
Plaintext
64 lines
1.9 KiB
Plaintext
// SPDX-FileCopyrightText: 2024 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#include <fstream>
|
|
#include <atomic>
|
|
|
|
#include "CUDAWrapper.h"
|
|
#include "JFJochException.h"
|
|
|
|
inline void cuda_err(cudaError_t val) {
|
|
if (val != cudaSuccess)
|
|
throw JFJochException(JFJochExceptionCategory::GPUCUDAError, cudaGetErrorString(val));
|
|
}
|
|
|
|
int32_t get_gpu_count() {
|
|
int device_count;
|
|
cudaError_t val = cudaGetDeviceCount(&device_count);
|
|
switch (val) {
|
|
case cudaSuccess:
|
|
return device_count;
|
|
case cudaErrorNoDevice:
|
|
case cudaErrorInsufficientDriver:
|
|
return 0;
|
|
default:
|
|
throw JFJochException(JFJochExceptionCategory::GPUCUDAError, cudaGetErrorString(val));
|
|
}
|
|
|
|
}
|
|
|
|
std::vector<std::string> get_gpu_names() {
|
|
std::vector<std::string> names;
|
|
const int32_t count = get_gpu_count();
|
|
names.reserve(count);
|
|
for (int32_t i = 0; i < count; i++) {
|
|
cudaDeviceProp prop{};
|
|
// A device that cannot be queried still exists and still gets work, so it is listed - just
|
|
// without a name. Losing the whole list over one unreadable device would be worse.
|
|
if (cudaGetDeviceProperties(&prop, i) == cudaSuccess)
|
|
names.emplace_back(prop.name);
|
|
else
|
|
names.emplace_back("unknown GPU");
|
|
}
|
|
return names;
|
|
}
|
|
|
|
void set_gpu(int32_t dev_id) {
|
|
auto dev_count = get_gpu_count();
|
|
|
|
// Ignore if no GPU present
|
|
if (dev_count > 0) {
|
|
if ((dev_id < 0) || (dev_id >= dev_count))
|
|
throw JFJochException(JFJochExceptionCategory::InputParameterInvalid, "Device ID cannot be negative");
|
|
|
|
cuda_err(cudaSetDevice(dev_id));
|
|
}
|
|
}
|
|
|
|
void pin_gpu() {
|
|
static std::atomic<uint32_t> counter{0};
|
|
auto dev_count = get_gpu_count();
|
|
if (dev_count > 0)
|
|
set_gpu(counter.fetch_add(1) % dev_count);
|
|
}
|