leonarski_fandClaude Opus 5.5
672e182d6a
A pooled CudaDevicePtr frees on the thread's allocation stream, not on the engine's stream, and the
pool may hand the memory to another engine - or, past its release threshold, unmap it - as soon as
that free is reached, which on an idle allocation stream is at once. An engine destroyed with work
still queued (FFTIndexerGPU after SearchCap's last DirectionsChanged upload, a spot finder between
DetectAt and Extract, a shadow accumulator after a pending fold, any engine on an exception path)
thus had kernels or copies writing memory that was someone else's or no longer mapped. Now:
- CudaStream synchronises before cudaStreamDestroy (destructor and move-assignment), which covers
engines whose own stream is declared after their buffers (FFTIndexerGPU, the gather buffer);
- every engine holding pooled buffers and a stream (shared or own, declared before the buffers)
synchronises it in its destructor; BraggIntegrationEngineGPU also before EnsureCapacity
reallocates, where a Run that threw leaves work queued.
A GPU failure that is handled no longer hides a lost context: ShadowFinder, BeamCenterFFT, the
rigid-body pool and model validation call cuda_throw_if_context_lost() before cuda_clear_error(),
as the device-decode fallbacks already did; RotationScaleMergeGPU's Alloc does so before waiting up
to ten minutes for GPU work beside it and then reporting a lost device as out of memory; and a
failed cudaMalloc says why. BeamCenterFFT logs the failure it used to drop silently, the
speculative geometry probe logs the exception it swallowed (its GPU fault was otherwise reported by
the merge beside it, under the merge's name), and RotationScaleMergeGPU's DeviceGuard no longer
throws from its destructor.
Only synchronisation and error paths change: p.hkl md5 and the MTZ data (gemmi) are identical to
the b530c2d full battery on myob_x10sa, cytc_x10sa, 8a1a, 9gdj, 11if, kdp_x10sa_20keV and 6z9g.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVmAWnzCmRKAXVUCdc4iNi
40 lines
1.6 KiB
C++
40 lines
1.6 KiB
C++
// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute <filip.leonarski@psi.ch>
|
|
// SPDX-License-Identifier: GPL-3.0-only
|
|
|
|
#pragma once
|
|
|
|
#include <memory>
|
|
|
|
#include "ROIIntegration.h"
|
|
#include "../indexing/CUDAMemHelpers.h"
|
|
|
|
class ROIIntegrationGPU : public ROIIntegration {
|
|
std::shared_ptr<CudaStream> stream;
|
|
int threads;
|
|
int blocks;
|
|
size_t shared_needed;
|
|
|
|
CudaDevicePtr<uint16_t> gpu_roi_map;
|
|
// 64-bit sums are accumulated as unsigned long long (two's-complement bit
|
|
// pattern) because CUDA atomicAdd has no signed 64-bit overload.
|
|
CudaDevicePtr<unsigned long long> gpu_sum;
|
|
CudaDevicePtr<unsigned long long> gpu_sum2;
|
|
CudaDevicePtr<unsigned long long> gpu_pixels;
|
|
CudaDevicePtr<unsigned long long> gpu_x_weighted;
|
|
CudaDevicePtr<unsigned long long> gpu_y_weighted;
|
|
CudaDevicePtr<int> gpu_max;
|
|
|
|
std::vector<unsigned long long> host_sum;
|
|
std::vector<unsigned long long> host_sum2;
|
|
std::vector<unsigned long long> host_pixels;
|
|
std::vector<unsigned long long> host_x_weighted;
|
|
std::vector<unsigned long long> host_y_weighted;
|
|
std::vector<int> host_max;
|
|
std::vector<int> max_init; // INT_MIN seed copied into gpu_max each frame
|
|
public:
|
|
ROIIntegrationGPU(const DiffractionExperiment &experiment, std::shared_ptr<CudaStream> stream);
|
|
// Waits for the stream: its pooled buffers are freed next (see CudaDevicePtr).
|
|
~ROIIntegrationGPU() override { if (stream) cudaStreamSynchronize(*stream); }
|
|
void Run(const ImagePreprocessorBuffer &image, std::map<std::string, ROIMessage> &out) override;
|
|
};
|