GPU: kernels read host buffers in place on unified-memory systems

gpu_direct_host_read() (CUDAWrapper) is true where the device accesses pageable
memory through the host's page tables (cudaDevAttrPageableMemoryAccess and
...UsesHostPageTables: GB10, GH200, GB200), asked once per device;
JFJOCH_FORCE_DIRECT_HOST_READ forces it on for testing with page-locked inputs.
Where it holds:
- BSLZ4DecoderGPU: the LZ4/fused kernels and the tail copy read the reader's
  compressed chunk directly, no pinned staging memcpy and no H2D copy;
- ImagePreprocessorGPU: Analyze and the UINT16_WIDENED path hand the host
  buffer to preprocess_kernel (if aligned to the pixel type), and nothing is
  page-locked;
- RotationScaleMergeGPU::IngestUpload: IngestExtractKernel reads the pinned
  stage in place, without device staging buffers; the stage is reused only
  after the kernel has run.
Discrete cards take the unchanged path. Every caller synchronises its stream
before the host buffer goes away. DirectHostReadGPUTest checks the three
sites against the CPU on page-locked inputs, in either mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-14 21:49:37 +02:00
co-authored by Claude Opus 5.5
parent fe4db26716
commit e3b33591ca
10 changed files with 263 additions and 29 deletions
@@ -66,6 +66,9 @@ class BSLZ4DecoderGPU {
// uploads pageable memory through small staging buffers of its own, a piece at a time - at a few
// GB/s on a host whose memory is far from the card. Copied here, the upload is one DMA at the link rate.
CudaHostPtr<uint8_t> host_compressed;
// The card reads the chunk where the reader left it (gpu_direct_host_read()); neither buffer above
// is then used.
bool direct_host_read = false;
CudaDevicePtr<uint8_t> gpu_shuffled; // LZ4 output, still bitshuffled
CudaDevicePtr<BSLZ4BlockDesc> gpu_desc;
CudaHostPtr<BSLZ4BlockDesc> host_desc; // pinned, so the descriptor upload is truly async