ClusterFinderCUDA mixed three things: which kernel runs and what device
state it needs, the per-stream buffers, and the batching pipeline. A
second algorithm meant a second copy of all of it. The pieces are now
separate and the driver is templated on the algorithm.
cuda::FixedWindow (clusterfinder_algo.cuh) the fixed-size window
search: DeviceState pointer bundle, launch(), and the pedestal
transforms. Stateless; every pointer is supplied by the caller,
so external code can drive the kernel on its own memory and
stream without touching the driver.
StreamContext<Algo> one stream's buffers, RAII, non-movable.
ClusterFinderCUDA<Algo> the pipeline, unchanged in behaviour.
no_2x2_cluster moves to Cluster.hpp so the kernel and algorithm headers
no longer drag in ClusterFinder.hpp. The Python factory gains
algorithm="fixed_window"; memory stays internal.
Kernel SASS identical to before. Positions identical against
ClusterFinderFrozen for 3x3, 5x5, 9x9 (stream and graph finders). New
Catch2 test drives FixedWindow::launch on raw cudaMalloc buffers and
requires exact agreement with the driver.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
collect() cached queue emptiness across sweeps, so a worker's last
write could land after the final sweep but before shutdown was
observed, stranding that frame's clusters. Read the stop flags before
sweeping instead: once workers are joined, a sweep that moves nothing
proves the queues are drained.
Found via ClusterFinderCUDA.ipynb's drift plot, which jumped by one
frame's worth of clusters at the very end.
Doxygen never read *.cuh and needs EXTENSION_MAPPING to parse it; add
both, plus C++/Python pages and AARE_CUDA build docs.
Delete ClusterFinderCUDA_old.hpp (declared the same class, polluting
the API page), utils/batch.hpp, and the deck. All unreferenced.
TODO:
- Frozen + CUDAGraph: bound in Python, headers not installed. Ship or drop?
- test_ClusterFinderCUDA.py guarded on _cuda_available()
- AARE_CUDA_ARCHITECTURES=native needs CMake 3.24, project min is 3.15
- make docs somewhere with sphinx+breathe; new .rst files never rendered
- fmt nvcc warnings: gone at -std=c++20 or with -isystem
Host-side pipeline work ported from the benchmark branch. The kernel is
unchanged, so cluster results are identical.
- collect_view()/BatchView: read clusters in place from the pinned D2H
buffer instead of copying one ClusterVector per frame. The view is
released back to the finder, so keep nothing that borrows it.
- reserve_output_slots(), chunk_size_for(): pre-pin the output slots and
size a batch to fit one.
- find_cluster_views_batched_iter(): submits chunk i+1 before collecting
chunk i.
- time_kernels, default off. avg_kernel_time_ms() returns NaN unless
enabled, and always for the Graph variant.
- n_streams defaults to 4 everywhere.
ClusterFinderCUDA.ipynb cut to CPU MT vs batched CUDA plus a cluster-set
diff, and it now reports peak clusters/frame.
Removed src/ClusterFinderCUDA{,_old}.test.cu and src/Makefile (early C++
benchmarks, never in the CMake build), the perf and Frozen-vs-CUDA
notebooks, and helper.py.
- Eliminate the ~200–300 µs inter-batch idle gap by allowing two batches
to be in-flight simultaneously:
- submit_batch() enqueues H2D+kernel+D2H without blocking
- collect() syncs via cudaEventSynchronize (not
cudaStreamSynchronize) so a queued second batch runs uninterrupted.
- Two ping-pong output slots (NUM_SLOTS=2) with per-slot pinned buffers
and cudaEventDisableTiming sync events.
- find_clusters_batched() keeps its direct implementation.
* Measured: 0.026 -> 0.022 ms/frame (~18%).
- Device pedestal arrays (mean/sum/sum2) are now float instead of
double: halves global-memory bandwidth for pedestal reads/writes and
eliminates FP64 arithmetic in the kernel (3.3x kernel speedup,
15µs -> 4.6µs).
- Replace the per-cluster push_back loop in the D2H drain with a
single resize()+memcpy().
Rework the multi-stream pipeline to eliminate per-frame sync barriers and
fix the D2H staging architecture.
Sync reduction:
- Replace one cudaStreamSynchronize per frame with one per stream per batch,
cutting synchronisation calls from O(n_frames x n_streams) to O(n_streams)
- Introduce a unified per-frame D2H output layout [uint32_t count | clusters[max]]
stored in a single class-level lazy-allocated pinned pool (h_output_pinned),
replacing the per-stream separate cluster/count device buffers
- Move CUDA event pool from per-stream fixed-size to per-frame-slot lazy-allocated,
enabling correct kernel timing across any batch size
Pinned H2D without CPU-side copy:
- Add register_input_buffer(ptr, bytes) / unregister_input_buffer() wrapping
cudaHostRegister so callers can pin their existing batch buffer once; all
find_clusters_batched() slices then transfer at DMA speed (~22 GB/s) instead
of ~15 GB/s for pageable, with no extra memcpy or WC-memory penalty
Result (RTX 4090, 400x400 uint16, 3x3 clusters, batch=2000, 5 streams):
Before: ~34 µs/frame -> After: ~28 µs/frame (−18 %)
- Use per-stream pinned host staging buffers for truly async CUDA transfers.
- Avoid reserving full device capacity per result frame.
- Reduce kernel work by delaying cluster payload construction.
- Use squared comparisons and removing per-pixel sqrtf() ops.
- Stencil arithmetic and shared memory use float (COMPUTE_TYPE alias).
- Pedestal accumulation stays double to preserve variance accuracy.
Notes:
- On RTX 4090, FP32 throughput is ~64× higher than FP64, so moving
stencil math to float improves performance.
- Using float also avoids shared memory bank conflicts: stride-18 maps
to distinct banks for 32-bit values, but caused conflicts with 64-bit.
- Add bind_ClusterFinderCUDA.hpp with pybind11 bindings for
ClusterFinderCUDA
- Build CUDA bindings as separate _aare_cuda.so to avoid
segfaults from mixing nvcc and gcc compiled code in the
same shared object
- Re-export CUDA classes onto _aare in __init__.py so user
code uses `from aare import ClusterFinderCUDA` regardless
of which .so hosts the class
- Factory in ClusterFinder.py selects backend; RuntimeError
if GPU requested on CPU-only build
- Update python/CMakeLists.txt: _aare_cuda module gated
behind AARE_CUDA and AARE_PYTHON_BINDINGS
- Add validation notebook: ~20x speedup vs sequential ClusterFinder
kferjaoui