mx/Jungfraujoch
2
1
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases 67 Wiki Activity

CUDA: the engines' setup copies belong on the engine's stream

Browse Source 
Making the worker streams non-blocking removed the implicit ordering that the
constructors were still relying on. Each engine uploads its static inputs - the
pixel mask, the pixel-to-bin map, the corrections, the ROI map - with a blocking
NULL-stream cudaMemcpy, and then reads them from kernels on its own stream. A
pageable host-to-device cudaMemcpy returns once the source has been staged, with
the DMA still in flight, and a non-blocking stream no longer waits for the NULL
stream. The failure mode is a silently unapplied mask or a stale mapping, not a
crash, so it would not have announced itself.

Put them on the stream the engine already owns, and synchronise once at the end of
the constructor - that is required for the preprocessor, whose source is a local
vector, and leaves the others settled rather than in flight for the cost of one
one-time sync. The GPU spot-finder test uploaded its image the same way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
leonarski_f
2026-07-31 11:51:22 +02:00
co-authored by Claude Opus 5
parent a2adc4e021
commit bf866a0d4c
5 changed files with 39 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/AdaptiveSpotFinderGPUTest.cpp
+12
View File
@@ -87,6 +87,9 @@ TEST_CASE("AdaptiveSpotFinderGPU_SpotFindingParity", "[AdaptiveSpotFinderGPU]")
FillTestImage(buffer, x);
REQUIRE(cudaMemcpy(buffer.getGPUBuffer(), buffer.getBuffer().data(),
x.GetPixelsNum() * sizeof(int32_t), cudaMemcpyHostToDevice) == cudaSuccess);
// The engines run on non-blocking streams, which do not wait for this NULL-stream copy: a pageable
// H2D cudaMemcpy returns once the source is staged, with the DMA still in flight.
REQUIRE(cudaDeviceSynchronize() == cudaSuccess);
std::vector<bool> res_mask(x.GetPixelsNum(), false);
const SpotFindingSettings settings = AdaptiveSettings();
@@ -120,6 +123,9 @@ TEST_CASE("AdaptiveSpotFinderGPU_AzimuthalIntegration", "[AdaptiveSpotFinderGPU]
FillTestImage(buffer, x);
REQUIRE(cudaMemcpy(buffer.getGPUBuffer(), buffer.getBuffer().data(),
x.GetPixelsNum() * sizeof(int32_t), cudaMemcpyHostToDevice) == cudaSuccess);
// The engines run on non-blocking streams, which do not wait for this NULL-stream copy: a pageable
// H2D cudaMemcpy returns once the source is staged, with the DMA still in flight.
REQUIRE(cudaDeviceSynchronize() == cudaSuccess);
std::vector<bool> res_mask(x.GetPixelsNum(), false);
const SpotFindingSettings settings = AdaptiveSettings();
@@ -165,6 +171,9 @@ TEST_CASE("AdaptiveSpotFinderGPU_RunToRunReproducible", "[AdaptiveSpotFinderGPU]
FillTestImage(buffer, x);
REQUIRE(cudaMemcpy(buffer.getGPUBuffer(), buffer.getBuffer().data(),
x.GetPixelsNum() * sizeof(int32_t), cudaMemcpyHostToDevice) == cudaSuccess);
// The engines run on non-blocking streams, which do not wait for this NULL-stream copy: a pageable
// H2D cudaMemcpy returns once the source is staged, with the DMA still in flight.
REQUIRE(cudaDeviceSynchronize() == cudaSuccess);
std::vector<bool> res_mask(x.GetPixelsNum(), false);
const SpotFindingSettings settings = AdaptiveSettings();
@@ -195,6 +204,9 @@ TEST_CASE("AdaptiveSpotFinderGPU_Speed", "[AdaptiveSpotFinderGPU][.benchmark]")
FillTestImage(buffer, x);
REQUIRE(cudaMemcpy(buffer.getGPUBuffer(), buffer.getBuffer().data(),
x.GetPixelsNum() * sizeof(int32_t), cudaMemcpyHostToDevice) == cudaSuccess);
// The engines run on non-blocking streams, which do not wait for this NULL-stream copy: a pageable
// H2D cudaMemcpy returns once the source is staged, with the DMA still in flight.
REQUIRE(cudaDeviceSynchronize() == cudaSuccess);
std::vector<bool> res_mask(x.GetPixelsNum(), false);
const SpotFindingSettings settings = AdaptiveSettings();