mirror of
https://github.com/slsdetectorgroup/aare.git
synced 2026-09-02 23:20:43 +02:00
docs: Performance study
This commit is contained in:
kferjaoui
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1 +0,0 @@
|
||||
,ferjao_k,pc-moench-04.psi.ch,11.08.2026 11:46,file:///home/ferjao_k/.config/libreoffice/4;
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -1,467 +0,0 @@
|
||||
# ClusterFinderCUDA — Benchmark Results, opt1 → opt6
|
||||
|
||||
Consolidated, verified performance numbers for the CUDA cluster-finder optimization
|
||||
ladder. Every number below is traced to the code, notebook cell, or profiler report
|
||||
that produced it, and each is tagged **quotable** or **not quotable** with the reason.
|
||||
|
||||
Companion documents:
|
||||
- `docs/pedestal_precision_f32_cancellation.md` — why the naive f32 pedestal failed and how B1 fixes it
|
||||
- `docs/cuda_optimization_recap.md`, `docs/optimization_summary.md` — earlier narrative/roadmap notes
|
||||
- `docs/ClusterFinderCUDA_optimizations.pptx` — the deck these numbers feed
|
||||
|
||||
---
|
||||
|
||||
## 1. Environment
|
||||
|
||||
| item | value |
|
||||
|---|---|
|
||||
| GPU | NVIDIA GeForce RTX 4090 (Ada, sm_89), 24 GB, driver 595.71.05 |
|
||||
| GPU clocks | idle 210 MHz → boost 3120 MHz; **persistence mode disabled** |
|
||||
| FP64 rate | 1/64 of FP32 on this part (relevant to opt6) |
|
||||
| CPU | AMD Ryzen 9 7950X, 16 cores / 32 threads |
|
||||
| RAM | 125 GiB, **no swap** |
|
||||
| CUDA | 12.4 (nvcc V12.4.131) |
|
||||
| Profiler | Nsight Systems 2024.5.1 (`/opt/nvidia/nsight-systems/2024.5.1`) |
|
||||
| Host | `pc-moench-04` |
|
||||
| Branch | `bench/opt2-pipeline` (off `feature/cuda_clusterfinder` @ `ce256dd`) |
|
||||
|
||||
**Dataset** — MOENCH, MAX IV beamtime, Cu fluorescence:
|
||||
|
||||
```
|
||||
/mnt/sls_det_storage/moench_data/2603_MaxIVBeamtime/2026032408/process/xrf/
|
||||
Cu_factor_10_data_master_0.json (100 000 frames, 400×400 uint16)
|
||||
Cu_factor_10_pedestal_master_0.json (1 000 frames used for pedestal training)
|
||||
```
|
||||
|
||||
Frame size 400×400×2 B = 320 000 B (312.5 KiB). All finders trained on the
|
||||
**same 1 000 pedestal frames**;
|
||||
`n_sigma = 5` throughout. Data pre-loaded into RAM with `read_n()` so file I/O is
|
||||
outside every timing loop.
|
||||
|
||||
---
|
||||
|
||||
## 2. The optimization ladder
|
||||
|
||||
| step | what changed | how it is measured |
|
||||
|---|---|---|
|
||||
| **baseline** | `ClusterFinderMT`, 48 threads | `ClusterFinderMT(..., n_threads=48)` + `ClusterCollector` |
|
||||
| **opt1** | first CUDA port: 1 stream, one launch per frame, no batching | `ClusterFinderCUDAOpt2(..., n_streams=1)` + `find_clusters()` per frame |
|
||||
| **opt2** | multi-stream scaffolding + host-side batching (bulk memcpy) | `ClusterFinderCUDAOpt2(..., n_streams=4)` + `find_clusters_batched()`, batch 2000 |
|
||||
| **opt3** | pipeline rework: remove per-round sync barriers, fixed-size D2H | `ClusterFinderCUDA(..., n_streams=4)` + `find_clusters_batched()`, **no pinning** |
|
||||
| **opt4** | DMA-speed transfers via pinned host input | opt3 + `register_input_buffer(data)` |
|
||||
| **opt5** | CUDA Graphs (pre-recorded H2D→kernel→D2H per stream) | `ClusterFinderCUDAGraph(...)` + pinned input |
|
||||
| **opt6** | **first kernel optimization**: f32 device pedestal + variance rewrite (B1) | rebuild with `DEVICE_PED_TYPE = float` |
|
||||
|
||||
opt1–opt5 are **pipeline/host-side**; opt6 is the first change to the **kernel itself**.
|
||||
|
||||
### Code behind each step
|
||||
|
||||
| step | primary source |
|
||||
|---|---|
|
||||
| opt1, opt2 | [`include/aare/ClusterFinderCUDAOpt2.hpp`](../include/aare/ClusterFinderCUDAOpt2.hpp), [`include/aare/clusterfinder_kernel_opt2.cuh`](../include/aare/clusterfinder_kernel_opt2.cuh) — snapshot of commit `88e0e8d` (pre-refactor pipeline), namespace `aare::device_opt2` |
|
||||
| opt3, opt4, opt6 | [`include/aare/ClusterFinderCUDA.hpp`](../include/aare/ClusterFinderCUDA.hpp), [`include/aare/clusterfinder_kernel.cuh`](../include/aare/clusterfinder_kernel.cuh) |
|
||||
| opt5 | [`include/aare/ClusterFinderCUDA_graph.hpp`](../include/aare/ClusterFinderCUDA_graph.hpp) |
|
||||
| bindings | [`python/src/bind_ClusterFinderCUDAOpt2.hpp`](../python/src/bind_ClusterFinderCUDAOpt2.hpp), [`python/src/cuda_bindings.cu`](../python/src/cuda_bindings.cu) |
|
||||
| factories | [`python/aare/ClusterFinder.py`](../python/aare/ClusterFinder.py) |
|
||||
|
||||
Relevant history: `3ed773e` (multi-stream+batched) → `ac96d1f` (mixed precision) →
|
||||
`88e0e8d` (**opt1/opt2 snapshot**) → `6a12e3d` (pipeline refactor = opt3) →
|
||||
`4c66802` (FP32 pedestal, introduced the tail) → `5922c73` (async API) →
|
||||
`1bf317f` (local-max fix) → `a42d71c` (graphs = opt5).
|
||||
|
||||
### Precision configuration
|
||||
|
||||
Two type aliases in [`clusterfinder_kernel.cuh:16-17`](../include/aare/clusterfinder_kernel.cuh#L16-L17):
|
||||
|
||||
```cpp
|
||||
using COMPUTE_TYPE = float; // stencil arithmetic — float in ALL builds below
|
||||
using DEVICE_PED_TYPE = double; // device pedestal — the opt6 knob (double → float)
|
||||
```
|
||||
|
||||
- **"f64 build"** in this document = `COMPUTE_TYPE=float`, `DEVICE_PED_TYPE=double` (mixed precision).
|
||||
- **"f32 build"** (opt6) = `COMPUTE_TYPE=float`, `DEVICE_PED_TYPE=float` (100 % f32).
|
||||
|
||||
> ⚠️ `ClusterFinderCUDAOpt2` is templated on `PEDESTAL_TYPE`, which its binding pins
|
||||
> to `double` ([`bind_ClusterFinderCUDAOpt2.hpp:14`](../python/src/bind_ClusterFinderCUDAOpt2.hpp#L14)).
|
||||
> **opt1 and opt2 are therefore unaffected by the opt6 flip** — their numbers are
|
||||
> identical in both campaigns by construction. Only opt3/opt4/opt5 respond to opt6.
|
||||
|
||||
---
|
||||
|
||||
## 3. Methodology — what is measurable and what is not
|
||||
|
||||
Three measurement artifacts were identified and controlled. **All three matter for
|
||||
how numbers may be quoted on a slide.**
|
||||
|
||||
### 3.1 First-run soft page faults (dominant, ~1.5–4 s per cell)
|
||||
|
||||
Each timed cell materializes ~233 M clusters ≈ **10 GB of host result heap**. On the
|
||||
first execution in a process, every 4 kB page must be faulted in and zeroed by the OS
|
||||
(millions of minor faults). Measured cost on this machine: **0.7 µs/fault**.
|
||||
|
||||
Instrumentation added to all seven timed notebook cells:
|
||||
|
||||
```python
|
||||
import resource
|
||||
def _faults():
|
||||
r = resource.getrusage(resource.RUSAGE_SELF)
|
||||
return r.ru_minflt, r.ru_majflt
|
||||
```
|
||||
bracketing the timed region, printing `minor faults: N (~N×0.7µs est.)`.
|
||||
|
||||
**Protocol: quote the run where the fault counter has plateaued** (< ~200 k). Cold
|
||||
numbers are inflated by 25–45 %. Verified by direct correlation (§9).
|
||||
|
||||
### 3.2 CUDA-event `kernel_ms` inflates under multi-stream saturation
|
||||
|
||||
`avg_kernel_time_ms()` uses a CUDA event pair on the kernel's own stream. It measures
|
||||
**elapsed time on that stream's timeline**, which includes queue-wait when other
|
||||
streams are competing for SMs. Consequences:
|
||||
|
||||
- In transfer-paced regimes: `event ≈ true kernel + ~5–7 µs` launch gap → usable.
|
||||
- In kernel-saturated regimes: **inflated up to 3.5×** → **not quotable**.
|
||||
- Symptom of saturation: the derived `PCIe + overhead = wall/N − kernel_ms` **goes
|
||||
negative** (kernels overlap, so wall/frame < kernel/frame).
|
||||
|
||||
Ground truth requires Nsight Systems (§8).
|
||||
|
||||
### 3.3 Profiler distorts wall clock
|
||||
|
||||
Under `nsys`, wall time per frame is ~4× the unprofiled value (API tracing overhead).
|
||||
**Take per-operation GPU times from nsys; take wall times from unprofiled runs.** The
|
||||
standalone probe script also pays the full first-touch fault tax inside its single
|
||||
timed call (fresh process, no warm pass), so **its wall times are not throughput
|
||||
numbers** either.
|
||||
|
||||
### 3.4 Other controls
|
||||
|
||||
- GPU clock ramp (210 MHz → 3.1 GHz) is real but < 0.1 % of a multi-second run; the
|
||||
invariance of `kernel_ms` across loaded/idle runs confirms it is not a factor.
|
||||
- The `cf_cuda_v1` notebook cell builds a histogram **inside** its timed loop
|
||||
(~4.5 s over 100 k frames). It is a reference row, **not part of the arc**.
|
||||
- `ClusterFinderMT` cannot restart after `stop()`; the CPU baseline is therefore
|
||||
always a first-pass number and carries its own ~1.8 s of allocator faults.
|
||||
|
||||
---
|
||||
|
||||
## 4. Campaign A — 3×3 pipeline arc (f64 pedestal) ★ headline
|
||||
|
||||
**Config**: `cluster_size=(3,3)`, `N=100 000`, `n_frames_pd=1000`, `n_sigma=5`,
|
||||
`BATCH_SIZE=2000`, `n_streams=4`, `max_clusters_per_frame=3000`.
|
||||
**Source**: [`python/tests/ClusterFinderCUDA_perf.ipynb`](../python/tests/ClusterFinderCUDA_perf.ipynb),
|
||||
warm pass (steady state, faults plateaued).
|
||||
|
||||
| step | variant | wall [s] | FPS | µs/frame | kernel [µs/fr] | host ovhd [µs/fr] | **vs CPU** | step gain | minor faults |
|
||||
|---|---|--:|--:|--:|--:|--:|--:|--:|--:|
|
||||
| baseline | CPU MT (48 threads) | 21.00 | 4 761 | 210.0 | — | — | 1.00× | — | 2.63 M |
|
||||
| **opt1** | 1 stream, per-frame | 6.68 | 14 968 | 66.8 | 23 | 44 | **3.14×** | 3.14× | 1 |
|
||||
| **opt2** | 4 streams + batching | 4.32 | 23 134 | 43.2 | 22 | 21 | **4.86×** | 1.55× | 64 931 |
|
||||
| **opt3** | pipeline rework, no pin | 3.76 | 26 588 | 37.6 | 24 | 14 | **5.58×** | 1.15× | 59 814 |
|
||||
| **opt4** | + pinned input (DMA) | 2.72 | 36 810 | 27.2 | 24 | 3 | **7.73×** | 1.38× | 1 027 |
|
||||
| **opt5** | + CUDA Graph | 2.53 | 39 472 | 25.3 | 25 * | — | **8.29×** | 1.07× | 1 032 |
|
||||
|
||||
\* graph reports kernel+PCIe+overhead combined; the kernel is not timed separately.
|
||||
|
||||
Reference row (not in the arc): current finder, per-frame, histogram inside the timed
|
||||
loop — 11.35 s / 8 813 FPS / 1.85×.
|
||||
|
||||
**Status: quotable.** All CUDA rows at steady state; CPU baseline is first-pass by
|
||||
necessity (see §3.4) — if a fault-corrected CPU (≈19.2 s) is preferred, all speedups
|
||||
shrink by ~9 % (e.g. opt5 8.29× → 7.57×). Be consistent and state the choice.
|
||||
|
||||
### The story in one row — host overhead
|
||||
|
||||
```
|
||||
opt1 → opt2 → opt3 → opt4 host overhead per frame
|
||||
44 → 21 → 14 → 3 µs (kernel fixed at ~23 µs)
|
||||
```
|
||||
|
||||
opt4's 3 µs means H2D/kernel/D2H are essentially fully overlapped; opt5 then trims
|
||||
residual launch overhead. **After opt4 the 3×3 pipeline is host/PCIe-bound, not
|
||||
kernel-bound** — which is exactly why opt6 does nothing here (§5).
|
||||
|
||||
---
|
||||
|
||||
## 5. Campaign B — opt6 at 3×3 (100 % f32)
|
||||
|
||||
Same config as Campaign A; rebuilt with `DEVICE_PED_TYPE = float`.
|
||||
**Source**: same notebook, warm pass.
|
||||
|
||||
| step | wall f64 → f32 [s] | FPS f64 → f32 | kernel f64 → f32 [µs] | Δ wall |
|
||||
|---|--:|--:|--:|--:|
|
||||
| CPU baseline | 21.00 → 21.45 | 4 761 → 4 662 | — | (run-to-run) |
|
||||
| v1 per-frame (serialized) | 11.35 → 10.57 | 8 813 → 9 465 | **24 → 13** | **−7 %** ✓ |
|
||||
| opt3 (no pin) | 3.76 → 3.88 | 26 588 → 25 798 | **24 → 14** | +3 % (noise) |
|
||||
| opt4 (pinned) | 2.72 → 2.83 | 36 810 → 35 372 | **24 → 16** | +4 % (noise) |
|
||||
| opt5 (graph) | 2.53 → 2.53 | 39 472 → 39 503 | — | **0 %** |
|
||||
| opt1 / opt2 | 6.68 → 6.73 / 4.32 → 4.88 | — | 23 / 22 unchanged | n/a — f64-pinned class |
|
||||
|
||||
**Result: at 3×3, opt6 halves the kernel and buys nothing end-to-end.** The kernel
|
||||
(13–16 µs) sits *below* the ~25–27 µs/frame transfer+host floor, so it is entirely
|
||||
hidden by stream overlap. The control is the serialized `v1` path, where the kernel
|
||||
*cannot* hide: there the saving does appear (−7 % wall ≈ the 8–10 µs/frame kernel gain).
|
||||
|
||||
The f32 opt2 row (4.88 s) still carried 822 k faults and is **not quotable**; it is
|
||||
also irrelevant, being the f64-pedestal class.
|
||||
|
||||
**Status: quotable** (opt3/opt4/opt5/v1 rows).
|
||||
|
||||
---
|
||||
|
||||
## 6. Campaign C — opt6 at 9×9 (where the kernel *is* the bottleneck) ★
|
||||
|
||||
**Config change**: `cluster_size=(9,9)`, `N=20 000`, `n_streams=8`,
|
||||
`max_clusters_per_frame=1500`, everything else unchanged.
|
||||
Cap 1500 chosen so the CUDA finders record **all** clusters the CPU finds.
|
||||
**Source**: same notebook, warm runs (faults ≤ 102).
|
||||
|
||||
| build | path | wall [s] | FPS | µs/frame | kernel [µs/fr] | derived ovhd | vs CPU | clusters/frame |
|
||||
|---|---|--:|--:|--:|--:|--:|--:|--:|
|
||||
| **f64 ped** | opt4 batched+pin | 1.540 | 12 988 | 77.0 | 171 † | **−94** † | 6.01× | 1422.13 |
|
||||
| **f64 ped** | opt5 graph | 1.482 | 13 491 | 74.1 | 74 * | — | 6.24× | 1422.13 |
|
||||
| **f32 ped** | opt4 batched+pin | 1.423 | 14 057 | 71.2 | 33 | +38 | 6.61× | 1422.33 |
|
||||
| **f32 ped** | opt5 graph | 1.372 | 14 580 | 68.6 | 69 * | — | 6.86× | 1422.33 |
|
||||
|
||||
† **Not quotable** — event-timer inflation under 8-stream saturation; the negative
|
||||
overhead is the tell-tale. True kernel time is 43 µs (§8). \* combined metric.
|
||||
|
||||
CPU baselines (derived from the printed speedup ratios): f64 run ≈ 9.25 s (2 162 FPS),
|
||||
f32 run ≈ 9.41 s (2 126 FPS).
|
||||
|
||||
**Result: at 9×9, opt6 is worth ~8 % end-to-end** (1.540 → 1.423 s batched;
|
||||
1.482 → 1.372 s graph) **and flips the regime**: the f64 build is kernel-bound
|
||||
(negative derived overhead), the f32 build is transfer-bound (+38 µs).
|
||||
|
||||
**Status: wall/FPS/counts quotable; kernel column must come from §8.**
|
||||
|
||||
---
|
||||
|
||||
## 7. The rule that unifies Campaigns B and C
|
||||
|
||||
Per-frame GPU operation profile at 9×9 (nsys, §8):
|
||||
|
||||
```
|
||||
f64: kernel 43 µs | D2H 19 µs | H2D 13 µs → kernel is the tallest bar
|
||||
f32: kernel 26 µs | D2H 20 µs | H2D 13 µs → near-balanced
|
||||
3×3: kernel 13–24 µs, below the ~27 µs host/PCIe floor → always hidden
|
||||
```
|
||||
|
||||
> **f32 buys exactly the distance between the kernel bar and the next-tallest bar.**
|
||||
> Nothing at 3×3 (kernel already hidden); ~8 % at 9×9/f64 (kernel on the critical
|
||||
> path); more for larger windows, faster links, or fewer streams.
|
||||
|
||||
---
|
||||
|
||||
## 8. Nsight Systems ground truth (9×9, cap 1500)
|
||||
|
||||
**Script**: [`python/tests/nsys_kernel_probe.py`](../python/tests/nsys_kernel_probe.py)
|
||||
(also in the session scratchpad). Trains 1 000 pedestal frames, runs one batched pass
|
||||
over 2 000 frames, prints wall + event `kernel_ms`.
|
||||
|
||||
```bash
|
||||
nsys profile --trace=cuda --sample=none --cpuctxsw=none -o probe_s1 \
|
||||
python nsys_kernel_probe.py 1 2000 # 1 stream → kernels serialized
|
||||
nsys profile --trace=cuda --sample=none --cpuctxsw=none -o probe_s8 \
|
||||
python nsys_kernel_probe.py 8 2000 # 8 streams → deck config
|
||||
nsys stats --report cuda_gpu_kern_sum --report cuda_gpu_mem_time_sum probe_s1.nsys-rep
|
||||
```
|
||||
|
||||
### Kernel — `aare::device::find_clusters_in_single_frame<Cluster<int,9,9,uint16_t>>`
|
||||
|
||||
| build | streams | instances | **avg** | median | min | max | σ |
|
||||
|---|--:|--:|--:|--:|--:|--:|--:|
|
||||
| f64 ped | 1 | 2 000 | **43 011 ns** | 42 945 | 40 929 | 45 632 | 580 |
|
||||
| f64 ped | 8 | 2 000 | 46 714 ns | 46 880 | 41 632 | 61 441 | 1 988 |
|
||||
| f32 ped | 1 | 2 000 | **25 602 ns** | 25 569 | 24 737 | 27 073 | 411 |
|
||||
| f32 ped | 8 | 2 000 | 25 840 ns | 25 728 | 24 864 | 30 624 | 578 |
|
||||
|
||||
> ### **opt6 kernel result: 43.0 µs → 25.6 µs = −40 %** (9×9, exclusive time)
|
||||
|
||||
### Memory operations (1-stream, uncontended)
|
||||
|
||||
| op | f64 | f32 | payload | effective BW (decimal) |
|
||||
|---|--:|--:|---|--:|
|
||||
| D2H (clusters) | 19 424 ns | 19 845 ns | 1500 × 328 B = 492 000 B (480.5 KiB) | **25.3 GB/s** (23.6 GiB/s) |
|
||||
| H2D (frame) | 13 210 ns | 13 451 ns | 400×400×2 B = 320 000 B (312.5 KiB) | **24.2 GB/s** (22.6 GiB/s) |
|
||||
| memset | 359 ns | 365 ns | — | — |
|
||||
|
||||
The H2D payload is **one frame**; the D2H payload is one frame's cluster buffer
|
||||
(`max_clusters_per_frame` × `sizeof(Cluster<int,9,9,uint16_t>)`, transferred at
|
||||
fixed size regardless of how many clusters were actually found).
|
||||
|
||||
PCIe 4.0 ×16 theoretical is 31.5 GB/s (16 GT/s × 16 lanes × 128b/130b), so H2D
|
||||
reaches **77% of theoretical** — the signature of a real DMA path. Pageable
|
||||
transfers, which the driver stages through a hidden pinned buffer, run ~15 GB/s.
|
||||
|
||||
Transfers are identical between builds and at full DMA speed — confirming pinning
|
||||
(opt4) is doing its job and that only the kernel changed.
|
||||
|
||||
### Cross-validation of the event timer
|
||||
|
||||
| build | streams | nsys avg | event `kernel_ms` | offset |
|
||||
|---|--:|--:|--:|--:|
|
||||
| f64 | 1 | 43.0 µs | 0.050 ms | +7.0 µs (launch gap) |
|
||||
| f64 | 8 | 46.7 µs | 0.149 ms | **+102 µs (queue-wait)** |
|
||||
| f32 | 1 | 25.6 µs | 0.033 ms | +7.4 µs |
|
||||
| f32 | 8 | 25.8 µs | 0.033 ms | +7.2 µs |
|
||||
|
||||
Independently reproduced unprofiled by the user (f32 build, `N=20000`): event
|
||||
`kernel_ms` = **0.030–0.032** warm (= 25.6 µs + ~5–6 µs gap); 0.047 and 0.184 on
|
||||
cold-clock first invocations.
|
||||
|
||||
Note the 8-stream instance time stretches only +9 % (f64) / +1 % (f32): **one 9×9
|
||||
kernel nearly fills the GPU, so streams queue rather than co-execute.** Multi-streaming
|
||||
at 9×9 buys transfer overlap, not kernel concurrency.
|
||||
|
||||
**Reports retained** (openable in `nsys-ui` for timeline figures):
|
||||
`probe_s1.nsys-rep`, `probe_s8.nsys-rep` (f64), `probe_f32_s1.nsys-rep`,
|
||||
`probe_f32_s8.nsys-rep` (f32).
|
||||
|
||||
---
|
||||
|
||||
## 9. Supporting study — the page-fault artifact
|
||||
|
||||
### Synthetic isolation
|
||||
|
||||
Allocating ~8 GB in ClusterVector-sized chunks (90 000 × 93 kB), touching every page,
|
||||
freeing, repeating in one process:
|
||||
|
||||
| run | wall | minor faults |
|
||||
|---|--:|--:|
|
||||
| 1 (cold heap) | 2.05 s | 2 046 594 |
|
||||
| 2 (warm heap) | 0.07 s | 2 232 |
|
||||
| 3 (warm heap) | 0.07 s | 2 016 |
|
||||
|
||||
**30× faster, 1000× fewer faults**, same allocations — glibc retains the arenas.
|
||||
|
||||
### In situ (opt2 cell, three consecutive executions)
|
||||
|
||||
| run | wall [s] | FPS | minor faults |
|
||||
|---|--:|--:|--:|
|
||||
| 1 | 6.110 | 16 366 | 2 625 948 |
|
||||
| 2 | 4.872 | 20 526 | 729 866 |
|
||||
| 3 | 4.452 | 22 459 | 185 885 |
|
||||
|
||||
Correlation Δwall vs Δfaults:
|
||||
|
||||
| interval | Δwall | Δfaults | implied cost |
|
||||
|---|--:|--:|--:|
|
||||
| 1 → 2 | 1.238 s | 1 895 667 | **0.65 µs/fault** |
|
||||
| 2 → 3 | 0.420 s | 543 981 | **0.77 µs/fault** |
|
||||
| 1 → 3 | 1.658 s | 2 439 648 | **0.68 µs/fault** |
|
||||
|
||||
Reconstruction of run 1 from run 3:
|
||||
`4.452 s + 2 439 648 × 0.68 µs = 6.111 s` vs **measured 6.110 s** (1 ms error over 6 s).
|
||||
|
||||
Kernel time was constant (0.022 ms) across all three — the GPU is not involved.
|
||||
**Conclusion: the entire first-run penalty is OS page population of the host result
|
||||
heap.** Persistence mode / clock ramp are not responsible.
|
||||
|
||||
---
|
||||
|
||||
## 10. Correctness (held constant across the whole arc)
|
||||
|
||||
### 3×3, N = 100 000
|
||||
|
||||
| finder | clusters | /frame | diff vs CPU |
|
||||
|---|--:|--:|--:|
|
||||
| CPU MT | 233 085 343 | 2330.85 | — |
|
||||
| opt1 | 233 094 770 | 2330.95 | 0.0040 % |
|
||||
| opt2 | 233 093 553 | 2330.94 | 0.0035 % |
|
||||
| opt3 / opt4 / opt5 (f64) | 233 093 484 | 2330.93 | 0.0035 % |
|
||||
| opt3 / opt4 / opt5 (**f32**) | 233 093 554 – 233 094 465 | 2330.94 | 0.0039 % |
|
||||
|
||||
The residual ~0.004 % is the known per-frame vs per-pixel pedestal-update difference
|
||||
(analysed in `ClusterFinderFrozen_vs_CUDA.ipynb`), **not** a precision effect.
|
||||
|
||||
**f64 vs f32 builds differ by ~70 clusters out of 233 M (3 × 10⁻⁷).**
|
||||
|
||||
### 9×9, N = 20 000
|
||||
|
||||
| build | clusters | /frame |
|
||||
|---|--:|--:|
|
||||
| f64 | 28 442 582 | 1422.13 |
|
||||
| f32 | 28 446 667 | 1422.33 |
|
||||
|
||||
### Two fixes that made this possible
|
||||
|
||||
**(a) B1 — per-pixel offset accumulation** (`docs/pedestal_precision_f32_cancellation.md`).
|
||||
Before: naive f32 pedestal produced **+28.06 % clusters and an unphysical high-energy
|
||||
tail** — catastrophic cancellation in `var = sum2/n − mean²` (both terms ≈ 2.17 × 10⁷,
|
||||
variance ≈ 2025) drove quiet pixels to `rms=0`, so they fired every frame. Fix: freeze
|
||||
a per-pixel baseline `X0 ≈ round(mean)` at t=0 and accumulate centered `Y = X − X0`.
|
||||
After: full-f32 matches f64 to **3 × 10⁻⁷**. **opt6 is only shippable because of B1.**
|
||||
|
||||
**(b) Test3 local-max gate backported** into
|
||||
[`clusterfinder_kernel_opt2.cuh`](../include/aare/clusterfinder_kernel_opt2.cuh)
|
||||
so opt1/opt2 stop over-counting extended charge-shared events:
|
||||
|
||||
| | before | after |
|
||||
|---|--:|--:|
|
||||
| opt1 | 233 940 268 (2339.40/fr) | 233 094 770 (2330.95/fr) |
|
||||
| opt2 | 233 931 015 (2339.31/fr) | 233 093 553 (2330.94/fr) |
|
||||
|
||||
−0.36 %, now matching CPU. Kernel time unchanged (the gate is an early `return`).
|
||||
|
||||
---
|
||||
|
||||
## 11. Numbers that must NOT be used
|
||||
|
||||
| number | why |
|
||||
|---|---|
|
||||
| 9×9 **cap = 1000** runs (f64: 1.277 s / 15 657 FPS / 7.37×; f32: 1.087 s / 18 395 FPS / 8.53×) | cluster cap saturated → **exactly** 1000.00/frame, ~30 % of clusters truncated; CPU found ~1422/frame. Timing story survives but counts and speedups are dishonest. Superseded by cap 1500 (§6). |
|
||||
| `kernel_ms` under multi-stream saturation (f64 9×9: 0.171 / 0.192; 8-stream first-launch 0.184) | event-timer queue-wait inflation up to 3.5× (§3.2). Use nsys §8. |
|
||||
| Negative "PCIe + overhead" (−0.094, −0.128, −0.007) | arithmetic artifact of `wall/N − inflated kernel`. Useful only as a *kernel-bound indicator*, never as a transfer cost. |
|
||||
| Any wall time from `nsys_kernel_probe.py` (5.3–6.6 s for 20 k frames) | fresh process → full first-touch fault tax inside the timed call; single mega-batch; profiler overhead when traced. |
|
||||
| Cold-pass notebook numbers (opt2 6.18 s, opt3 5.29 s, batched 4.45 s, graph 4.02 s) | 2.1–2.6 M unresolved page faults each (§9). |
|
||||
| `cf_cuda_v1` cell as an arc data point | builds a histogram inside the timed loop (~4.5 s / 100 k frames). Valid only as the *serialized-path* control in §5. |
|
||||
|
||||
---
|
||||
|
||||
## 12. Reproduction index
|
||||
|
||||
| artifact | path | role |
|
||||
|---|---|---|
|
||||
| main benchmark notebook | `python/tests/ClusterFinderCUDA_perf.ipynb` | all wall/FPS/count numbers; 7 instrumented timed cells |
|
||||
| nsys probe | `python/tests/nsys_kernel_probe.py` | exclusive kernel + memcpy times |
|
||||
| correctness notebook | `python/tests/ClusterFinderFrozen_vs_CUDA.ipynb` | CPU↔CUDA agreement analysis |
|
||||
| precision study | `docs/pedestal_precision_f32_cancellation.md` | B1 derivation |
|
||||
| opt1/opt2 class | `include/aare/ClusterFinderCUDAOpt2.hpp` | pre-refactor pipeline snapshot |
|
||||
| opt1/opt2 kernel | `include/aare/clusterfinder_kernel_opt2.cuh` | `88e0e8d` kernel + backported Test3 gate |
|
||||
| opt3/opt4/opt6 | `include/aare/ClusterFinderCUDA.hpp`, `include/aare/clusterfinder_kernel.cuh` | current pipeline; precision knob at lines 16–17 |
|
||||
| opt5 | `include/aare/ClusterFinderCUDA_graph.hpp` | graph-based finder |
|
||||
| deck | `docs/ClusterFinderCUDA_optimizations.pptx` | target |
|
||||
|
||||
### Protocol to reproduce a clean number
|
||||
|
||||
1. Idle machine. Optionally `sudo nvidia-smi -pm 1` (kills ~1–3 s of per-process
|
||||
driver init; does **not** affect the fault artifact).
|
||||
2. Restart the Jupyter kernel; run the notebook top to bottom (cold pass).
|
||||
3. **Re-run each timed CUDA cell** until its `minor faults` line plateaus
|
||||
(< ~200 k; usually 2–3 executions). Quote that run.
|
||||
4. The CPU baseline cannot be re-run (`ClusterFinderMT.stop()` is terminal) —
|
||||
re-run the *Build finders* + pedestal cells first if a warm CPU number is needed.
|
||||
5. For kernel times: use `nsys` at `n_streams=1`, never the event timer under load.
|
||||
|
||||
---
|
||||
|
||||
## 13. Slide-ready takeaways
|
||||
|
||||
1. **The pipeline arc (opt1 → opt5) is monotonic**: 3.14× → 4.86× → 5.58× → 7.73× →
|
||||
**8.29×** over a 48-thread CPU baseline, at identical correctness (0.004 %).
|
||||
2. **The optimization story is host overhead collapsing**: 44 → 21 → 14 → **3 µs/frame**
|
||||
against a fixed ~23 µs kernel. By opt4 the GPU is fed almost perfectly.
|
||||
3. **opt6 (f32 pedestal) is a kernel win, not always a throughput win**: kernel
|
||||
**43.0 → 25.6 µs (−40 %, nsys-verified)**, but end-to-end **0 % at 3×3** and
|
||||
**−8 % wall at 9×9**. The bottleneck is workload-dependent.
|
||||
4. **The unifying rule**: f32 buys the distance between the kernel bar and the
|
||||
next-tallest bar in the per-frame GPU profile.
|
||||
5. **opt6 is only correct because of the B1 variance rewrite** — the naive f32 pedestal
|
||||
gave +28 % clusters and an unphysical tail; the centered accumulation restores
|
||||
agreement with f64 to 3 × 10⁻⁷.
|
||||
6. **Measurement discipline was necessary to get here**: soft page faults (0.7 µs each,
|
||||
up to 4 s per run) and CUDA-event queue-wait inflation (up to 3.5×) both had to be
|
||||
identified and controlled before any number was trustworthy.
|
||||
7. **Where the next win is**: after opt4 the 3×3 pipeline is PCIe/host-bound. Further
|
||||
kernel work has no payoff at that window size; the next target is transfer volume
|
||||
(on-GPU reduction, keeping results on device) rather than kernel speed.
|
||||
@@ -1,744 +0,0 @@
|
||||
"""Rebuild docs/ClusterFinderCUDA_optimizations.pptx — opt1..opt6, in the deck's
|
||||
own design language (extracted from the original file)."""
|
||||
from pptx import Presentation
|
||||
from pptx.util import Inches as In, Pt, Emu
|
||||
from pptx.dml.color import RGBColor
|
||||
from pptx.enum.text import PP_ALIGN, MSO_ANCHOR
|
||||
from pptx.enum.shapes import MSO_SHAPE
|
||||
from lxml import etree
|
||||
from pathlib import Path
|
||||
from PIL import Image
|
||||
|
||||
FIGS = Path(__file__).parent / "figs"
|
||||
OUT = Path("/home/ferjao_k/aare/docs/ClusterFinderCUDA_optimizations.pptx")
|
||||
|
||||
# ---------------------------------------------------------------- design tokens
|
||||
BG = RGBColor(0x0B, 0x10, 0x18)
|
||||
PANEL = RGBColor(0x12, 0x1A, 0x28)
|
||||
CODEBG = RGBColor(0x0E, 0x14, 0x20)
|
||||
RULE = RGBColor(0x1E, 0x28, 0x36)
|
||||
ACCENT = RGBColor(0x1E, 0x90, 0xC2)
|
||||
AMBER = RGBColor(0xE8, 0xB2, 0x5C)
|
||||
PALE = RGBColor(0xE7, 0xED, 0xF4)
|
||||
TEXT2 = RGBColor(0xA5, 0xB2, 0xC4)
|
||||
MUTED = RGBColor(0x6B, 0x7A, 0x90)
|
||||
|
||||
UI, MONO = "Segoe UI", "Consolas"
|
||||
W, H = 13.333, 7.5
|
||||
M = 0.7 # left margin
|
||||
COL = 7.9 # left column width
|
||||
RAIL_X, RAIL_W = 9.2, 3.5 # right rail
|
||||
|
||||
prs = Presentation()
|
||||
prs.slide_width, prs.slide_height = In(W), In(H)
|
||||
BLANK = prs.slide_layouts[6]
|
||||
N_SLIDES = 19
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- helpers
|
||||
def new_slide():
|
||||
s = prs.slides.add_slide(BLANK)
|
||||
bg = etree.SubElement(s._element, "{http://schemas.openxmlformats.org/presentationml/2006/main}bg")
|
||||
pr = etree.SubElement(bg, "{http://schemas.openxmlformats.org/presentationml/2006/main}bgPr")
|
||||
fill = etree.SubElement(pr, "{http://schemas.openxmlformats.org/drawingml/2006/main}solidFill")
|
||||
clr = etree.SubElement(fill, "{http://schemas.openxmlformats.org/drawingml/2006/main}srgbClr")
|
||||
clr.set("val", "0B1018")
|
||||
etree.SubElement(pr, "{http://schemas.openxmlformats.org/drawingml/2006/main}effectLst")
|
||||
s._element.insert(0, bg)
|
||||
return s
|
||||
|
||||
|
||||
def rect(s, x, y, w, h, color, shape=MSO_SHAPE.RECTANGLE):
|
||||
sh = s.shapes.add_shape(shape, In(x), In(y), In(w), In(h))
|
||||
sh.fill.solid(); sh.fill.fore_color.rgb = color
|
||||
sh.line.fill.background(); sh.shadow.inherit = False
|
||||
return sh
|
||||
|
||||
|
||||
def tb(s, x, y, w, h, anchor=MSO_ANCHOR.TOP):
|
||||
box = s.shapes.add_textbox(In(x), In(y), In(w), In(h))
|
||||
tf = box.text_frame
|
||||
tf.word_wrap = True
|
||||
tf.margin_left = tf.margin_right = tf.margin_top = tf.margin_bottom = 0
|
||||
tf.vertical_anchor = anchor
|
||||
return tf
|
||||
|
||||
|
||||
def para(tf, first=False, space_after=0, space_before=0, line=None, align=None):
|
||||
p = tf.paragraphs[0] if first else tf.add_paragraph()
|
||||
p.space_after = Pt(space_after); p.space_before = Pt(space_before)
|
||||
if line: p.line_spacing = line
|
||||
if align: p.alignment = align
|
||||
return p
|
||||
|
||||
|
||||
def run(p, text, size=11, color=TEXT2, font=UI, bold=False, italic=False, spc=None):
|
||||
r = p.add_run(); r.text = text
|
||||
f = r.font
|
||||
f.name, f.size, f.bold, f.italic = font, Pt(size), bold, italic
|
||||
f.color.rgb = color
|
||||
if spc is not None:
|
||||
r.font._rPr.set("spc", str(int(spc * 100)))
|
||||
return r
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ chrome
|
||||
def chrome(s, idx, eyebrow, title, title_size=27):
|
||||
rect(s, M, 0.60, 0.35, 0.035, ACCENT)
|
||||
tf = tb(s, 1.17, 0.50, 10.33, 0.32)
|
||||
run(para(tf, True), eyebrow.upper(), 9, MUTED, bold=True, spc=1.6)
|
||||
|
||||
tf = tb(s, M, 0.86, 11.9, 1.0)
|
||||
run(para(tf, True, line=1.05), title, title_size, PALE, bold=True)
|
||||
|
||||
# progress bar
|
||||
span, n = 11.0, N_SLIDES
|
||||
pitch = span / n; wseg = pitch * 0.90
|
||||
for i in range(n):
|
||||
rect(s, M + i * pitch, 7.28, wseg, 0.045, ACCENT if i <= idx - 1 else RULE)
|
||||
tf = tb(s, 12.0, 7.14, 0.9, 0.3)
|
||||
run(para(tf, True, align=PP_ALIGN.RIGHT), f"{idx} / {n}", 8.5, MUTED)
|
||||
|
||||
|
||||
def bullets(s, x, y, w, items, size=11, gap=7):
|
||||
tf = tb(s, x, y, w, 0.3)
|
||||
for i, it in enumerate(items):
|
||||
color, txt = (it if isinstance(it, tuple) else (TEXT2, it))
|
||||
p = para(tf, i == 0, space_after=gap, line=1.25)
|
||||
run(p, "• ", size, MUTED)
|
||||
# inline emphasis with **...**
|
||||
for j, part in enumerate(txt.split("**")):
|
||||
if part:
|
||||
run(p, part, size, PALE if j % 2 else color, bold=bool(j % 2))
|
||||
return tf
|
||||
|
||||
|
||||
def code(s, x, y, w, lines, size=8.5, title=None):
|
||||
lh = 0.148
|
||||
h = 0.24 + len(lines) * lh + (0.22 if title else 0)
|
||||
rect(s, x, y, w, h, CODEBG, MSO_SHAPE.ROUNDED_RECTANGLE)
|
||||
ty = y + 0.12
|
||||
if title:
|
||||
tf = tb(s, x + 0.18, ty, w - 0.36, 0.2)
|
||||
run(para(tf, True), title, 7.5, MUTED, bold=True, spc=1.2)
|
||||
ty += 0.22
|
||||
tf = tb(s, x + 0.18, ty, w - 0.36, h - 0.24)
|
||||
for i, ln in enumerate(lines):
|
||||
p = para(tf, i == 0, line=1.12)
|
||||
if ln.strip().startswith(("//", "#")):
|
||||
run(p, ln, size, MUTED, MONO)
|
||||
continue
|
||||
for j, part in enumerate(ln.split("«")):
|
||||
for k, seg in enumerate(part.split("»")):
|
||||
if not seg: continue
|
||||
hi = (j > 0 and k == 0)
|
||||
run(p, seg, size, ACCENT if hi else TEXT2, MONO, bold=hi)
|
||||
return h
|
||||
|
||||
|
||||
def callout(s, x, y, w, text, h=0.78, color=ACCENT, size=10.5):
|
||||
rect(s, x + 0.045, y, w - 0.045, h, PANEL)
|
||||
rect(s, x, y, 0.045, h, color)
|
||||
tf = tb(s, x + 0.28, y + 0.10, w - 0.5, h - 0.2, MSO_ANCHOR.MIDDLE)
|
||||
p = para(tf, True, line=1.2)
|
||||
for j, part in enumerate(text.split("**")):
|
||||
if part:
|
||||
run(p, part, size, PALE if j % 2 else TEXT2, bold=bool(j % 2))
|
||||
|
||||
|
||||
def rail(s, items, y0=2.0, divider=True):
|
||||
if divider:
|
||||
rect(s, 8.95, 2.0, 0.012, 4.55, RULE)
|
||||
y = y0
|
||||
for it in items:
|
||||
kind = it[0]
|
||||
if kind == "label":
|
||||
tf = tb(s, RAIL_X, y, RAIL_W, 0.26)
|
||||
run(para(tf, True), it[1].upper(), 8.5, MUTED, bold=True, spc=1.4)
|
||||
y += 0.28
|
||||
elif kind == "stat":
|
||||
_, lab, val, col = it
|
||||
tf = tb(s, RAIL_X, y, RAIL_W, 0.24)
|
||||
run(para(tf, True), lab.upper(), 8.5, MUTED, spc=1.2)
|
||||
tf = tb(s, RAIL_X, y + 0.24, RAIL_W, 0.6)
|
||||
run(para(tf, True), val, 26, col, bold=True)
|
||||
y += 0.98
|
||||
elif kind == "row":
|
||||
_, lab, val, col = it
|
||||
tf = tb(s, RAIL_X, y, RAIL_W, 0.24)
|
||||
run(para(tf, True), lab.upper(), 8.5, MUTED, spc=1.2)
|
||||
tf = tb(s, RAIL_X, y + 0.22, RAIL_W, 0.3)
|
||||
run(para(tf, True), val, 13, col, bold=True)
|
||||
y += 0.66
|
||||
elif kind == "note":
|
||||
tf = tb(s, RAIL_X, y, RAIL_W, 0.9)
|
||||
run(para(tf, True, line=1.25), it[1], 9, TEXT2)
|
||||
y += 0.30 + 0.17 * (len(it[1]) // 42 + 1)
|
||||
elif kind == "gap":
|
||||
y += it[1]
|
||||
return y
|
||||
|
||||
|
||||
def figure(s, name, x, y, w):
|
||||
p = FIGS / f"{name}.png"
|
||||
iw, ih = Image.open(p).size
|
||||
h = w * ih / iw
|
||||
s.shapes.add_picture(str(p), In(x), In(y), In(w), In(h))
|
||||
return h
|
||||
|
||||
|
||||
def caption(s, x, y, w, text, size=9):
|
||||
tf = tb(s, x, y, w, 0.3)
|
||||
run(para(tf, True, line=1.25), text, size, MUTED)
|
||||
|
||||
|
||||
# =========================================================== 1 · TITLE
|
||||
s = new_slide()
|
||||
rect(s, 0, 0, 0.16, H, ACCENT)
|
||||
tf = tb(s, M + 0.3, 0.85, 11, 0.3)
|
||||
run(para(tf, True), "AARE · PSI HYBRID PIXEL DETECTORS · CUDA CLUSTERFINDER",
|
||||
9.5, MUTED, bold=True, spc=1.8)
|
||||
|
||||
tf = tb(s, M + 0.3, 1.35, 11.4, 1.7)
|
||||
run(para(tf, True, line=1.02), "Feeding the GPU", 46, PALE, bold=True)
|
||||
tf = tb(s, M + 0.3, 2.30, 11.4, 0.8)
|
||||
run(para(tf, True, line=1.05), "Six optimization steps of the CUDA ClusterFinder",
|
||||
22, ACCENT)
|
||||
|
||||
tf = tb(s, M + 0.3, 3.25, 9.6, 0.8)
|
||||
run(para(tf, True, line=1.3),
|
||||
"Five of the six steps never touch the arithmetic. They are about keeping "
|
||||
"a 24 µs kernel supplied with data — and about learning to measure honestly.",
|
||||
12.5, TEXT2)
|
||||
|
||||
stats = [("×8.29", "VS 48-THREAD CPU", ACCENT), ("39,472", "FRAMES / SECOND", PALE),
|
||||
("25.3 µs", "PER FRAME, END TO END", PALE), ("0.004%", "CLUSTER-COUNT DRIFT", AMBER)]
|
||||
for i, (v, l, c) in enumerate(stats):
|
||||
x = M + 0.3 + i * 2.85
|
||||
rect(s, x, 4.35, 0.035, 0.95, c)
|
||||
tf = tb(s, x + 0.22, 4.35, 2.5, 0.55)
|
||||
run(para(tf, True), v, 30, c, bold=True)
|
||||
tf = tb(s, x + 0.22, 4.98, 2.5, 0.3)
|
||||
run(para(tf, True), l, 8.5, MUTED, spc=1.2)
|
||||
|
||||
rect(s, M + 0.3, 6.05, 11.0, 0.012, RULE)
|
||||
tf = tb(s, M + 0.3, 6.25, 11.4, 0.6)
|
||||
run(para(tf, True, line=1.35),
|
||||
"RTX 4090 (Ada, sm_89) · PCIe 4.0 ×16 · Mönch 400×400 uint16 · 3×3 clusters · "
|
||||
"100 000 frames · Cu fluorescence, MAX IV", 10, MUTED)
|
||||
tf = tb(s, M + 0.3, 6.62, 11.4, 0.3)
|
||||
run(para(tf, True), "Khalil Ferjaoui · Paul Scherrer Institut", 10, TEXT2)
|
||||
|
||||
# =========================================================== 2 · PROBLEM
|
||||
s = new_slide()
|
||||
chrome(s, 2, "The problem & the baseline", "What has to happen to every frame")
|
||||
bullets(s, M, 1.95, COL, [
|
||||
"Per pixel: subtract a **running pedestal** (mean ± rms), keep pixels above "
|
||||
"**nσ · rms**, cut a 3×3 cluster around each local maximum.",
|
||||
"400×400 = 160 k pixels, **312.5 kB per frame**; Cu data yields ~2 330 clusters "
|
||||
"per frame at 3×3.",
|
||||
"The pedestal is **updated by every non-photon pixel**, every frame — so the "
|
||||
"arithmetic and the data movement are coupled.",
|
||||
])
|
||||
code(s, M, 3.55, COL, [
|
||||
"// the whole algorithm, per pixel",
|
||||
"v = frame[i] - pedestal_mean[i]",
|
||||
"rms = sqrt(pedestal_sum2[i]/n - pedestal_mean[i]^2)",
|
||||
"if (v > «nSigma» * rms && v == max(3x3 window)) -> emit cluster",
|
||||
"else -> update pedestal",
|
||||
], title="THE KERNEL IN FIVE LINES")
|
||||
callout(s, M, 5.55, COL,
|
||||
"**Thesis of this talk:** the compute was fast almost immediately. "
|
||||
"Five of six steps are about feeding it.")
|
||||
rail(s, [
|
||||
("label", "Baseline · same data, same threshold"),
|
||||
("gap", 0.15),
|
||||
("stat", "CPU, 1 thread", "1.75 ms", MUTED),
|
||||
("stat", "CPU MT, 48 threads", "210 µs", PALE),
|
||||
("gap", 0.1),
|
||||
("row", "That is the bar", "4 761 frames / s", TEXT2),
|
||||
("gap", 0.25),
|
||||
("note", "Every CUDA number in this deck is measured against the 48-thread "
|
||||
"CPU on the same 100 000 frames."),
|
||||
])
|
||||
|
||||
# =========================================================== 3 · THE LADDER
|
||||
s = new_slide()
|
||||
chrome(s, 3, "Roadmap", "Two acts: feed the GPU, then speed up the kernel")
|
||||
rows = [
|
||||
("opt1", "First CUDA port", "1 stream, one launch per frame", "×3.14", ACCENT),
|
||||
("opt2", "Streams + batching", "4 streams, 2 000-frame batches", "×4.86", ACCENT),
|
||||
("opt3", "Pipeline rework", "sync barriers removed", "×5.58", ACCENT),
|
||||
("opt4", "Pinned memory", "DMA-speed host transfers", "×7.73", ACCENT),
|
||||
("opt5", "CUDA Graphs", "one launch replaces six", "×8.29", ACCENT),
|
||||
("opt6", "FP32 pedestal + variance rewrite", "the first kernel change", "kernel −40%", AMBER),
|
||||
]
|
||||
y = 2.05
|
||||
for i, (tag, name, sub, gain, col) in enumerate(rows):
|
||||
rect(s, M, y, 11.9, 0.72, PANEL if i % 2 == 0 else BG)
|
||||
rect(s, M, y, 0.035, 0.72, col)
|
||||
tf = tb(s, M + 0.28, y + 0.13, 1.0, 0.4)
|
||||
run(para(tf, True), tag, 15, col, bold=True, font=MONO)
|
||||
tf = tb(s, M + 1.45, y + 0.10, 5.0, 0.3)
|
||||
run(para(tf, True), name, 13, PALE, bold=True)
|
||||
tf = tb(s, M + 1.45, y + 0.38, 5.6, 0.3)
|
||||
run(para(tf, True), sub, 10, MUTED)
|
||||
tf = tb(s, 9.4, y + 0.18, 3.1, 0.4)
|
||||
run(para(tf, True, align=PP_ALIGN.RIGHT), gain, 15, col, bold=True)
|
||||
y += 0.78
|
||||
rect(s, M, 2.05, 0.012, 4.68, RULE)
|
||||
caption(s, M, 6.72, 11.9,
|
||||
"opt1–opt5 change only how work is scheduled and moved — the arithmetic is "
|
||||
"byte-identical. opt6 is the first step that changes the kernel itself.")
|
||||
|
||||
# =========================================================== 4 · METHODOLOGY
|
||||
s = new_slide()
|
||||
chrome(s, 4, "Before any number is believed", "Three ways a GPU benchmark lies")
|
||||
items = [
|
||||
("First-touch page faults", AMBER,
|
||||
"Each run materialises ~10 GB of clusters. The first pass faults in ~2.6 M "
|
||||
"pages at 0.7 µs each — up to 4 s of pure OS work inside the timer.",
|
||||
"Fix: re-run until getrusage() minor faults plateau (< 200 k)."),
|
||||
("CUDA-event kernel timing", AMBER,
|
||||
"avg_kernel_time_ms() measures elapsed time on a stream — including waiting "
|
||||
"for other streams. Under 8-stream load it over-reads by up to 3.5×.",
|
||||
"Fix: Nsight Systems per-instance times; 1 stream for exclusive numbers."),
|
||||
("The profiler itself", AMBER,
|
||||
"Under nsys, wall time per frame inflates ~4× from API tracing.",
|
||||
"Fix: GPU op times from nsys, wall times from unprofiled runs."),
|
||||
]
|
||||
x = M
|
||||
for title, col, body, fix in items:
|
||||
rect(s, x, 2.0, 3.83, 3.15, PANEL)
|
||||
rect(s, x, 2.0, 3.83, 0.035, col)
|
||||
tf = tb(s, x + 0.26, 2.28, 3.3, 0.6)
|
||||
run(para(tf, True, line=1.15), title, 13, PALE, bold=True)
|
||||
tf = tb(s, x + 0.26, 3.02, 3.3, 1.5)
|
||||
run(para(tf, True, line=1.3), body, 10, TEXT2)
|
||||
tf = tb(s, x + 0.26, 4.42, 3.3, 0.65)
|
||||
run(para(tf, True, line=1.25), fix, 9.5, ACCENT)
|
||||
x += 4.03
|
||||
code(s, M, 5.4, 11.9, [
|
||||
"# every timed cell in the benchmark notebook is bracketed with:",
|
||||
"mf0 = resource.getrusage(resource.RUSAGE_SELF).ru_minflt",
|
||||
"... t = time.perf_counter() - t0 ...",
|
||||
"print(f'minor faults: {mf1-mf0:,}') # quote the run where this plateaus",
|
||||
], title="THE FAULT PROTOCOL — python/tests/ClusterFinderCUDA_perf.ipynb")
|
||||
callout(s, M, 6.52, 11.2,
|
||||
"Validated: **wall = steady-state + faults × 0.68 µs** reproduced a 6.110 s "
|
||||
"run to within **1 ms**. Kernel time stayed constant throughout — the GPU was never the variable.",
|
||||
h=0.70, size=9.5)
|
||||
|
||||
# =========================================================== 5 · OPT1
|
||||
s = new_slide()
|
||||
chrome(s, 5, "opt1 · the first CUDA port", "One frame, one stream, fully synchronous")
|
||||
bullets(s, M, 1.95, COL, [
|
||||
"Shared-memory tiling with **halo loading** for any cluster size; pedestal "
|
||||
"subtraction fused into the tile load.",
|
||||
"Cluster geometry is a **compile-time template parameter** → the 3×3 stencil "
|
||||
"is fully unrolled.",
|
||||
"One cudaMemcpy in, one kernel, one cudaMemcpy out — **the host blocks "
|
||||
"on every frame**.",
|
||||
])
|
||||
code(s, M, 3.62, COL, [
|
||||
"// one frame at a time — the host waits at every step",
|
||||
"cudaMemcpy(d_frame, h_frame, bytes, cudaMemcpyHostToDevice);",
|
||||
"find_clusters_in_single_frame<ClusterType, FRAME_TYPE>",
|
||||
" <<<grid, block, shmem>>>(d_frame, d_pd_mean, ...);",
|
||||
"cudaMemcpy(h_out, d_out, out_bytes, cudaMemcpyDeviceToHost);",
|
||||
], title="ClusterFinderCUDAOpt2.hpp · find_clusters()")
|
||||
callout(s, M, 5.62, COL,
|
||||
"The stencil was **already fast**: 23 µs of kernel inside a 67 µs frame. "
|
||||
"The other 44 µs is the host standing still.")
|
||||
rail(s, [
|
||||
("label", "opt1 · 3×3 · 100 k frames"),
|
||||
("gap", 0.15),
|
||||
("stat", "End to end", "66.8 µs", PALE),
|
||||
("stat", "vs 48-thread CPU", "×3.14", ACCENT),
|
||||
("gap", 0.05),
|
||||
("row", "Kernel (GPU)", "23 µs", TEXT2),
|
||||
("row", "Host + PCIe", "44 µs", AMBER),
|
||||
("gap", 0.2),
|
||||
("note", "Two thirds of the frame is spent not computing."),
|
||||
])
|
||||
|
||||
# =========================================================== 6 · OPT2
|
||||
s = new_slide()
|
||||
chrome(s, 6, "opt2 · streams and batching", "A CUDA stream is a queue the GPU can overlap")
|
||||
bullets(s, M, 1.95, COL, [
|
||||
"A **stream** is an ordered queue of GPU work. Work in **different** streams may "
|
||||
"overlap — so a copy can run while another stream computes.",
|
||||
"Each stream gets its own **StreamContext**: device frame buffer, output buffer "
|
||||
"and pedestal. Frames are handed out **round-robin**.",
|
||||
"The host now submits **2 000 frames per call** instead of one.",
|
||||
])
|
||||
code(s, M, 3.62, COL, [
|
||||
"struct StreamContext {",
|
||||
" cudaStream_t stream;",
|
||||
" FRAME_TYPE *d_frame; ClusterType *d_clusters;",
|
||||
" PEDESTAL_TYPE *d_pd_mean, *d_pd_sum, *d_pd_sum2;",
|
||||
"};",
|
||||
"auto &sc = v_sc[frame_idx % «n_streams»]; // round-robin",
|
||||
], title="ClusterFinderCUDA.hpp · per-stream state")
|
||||
callout(s, M, 5.72, COL,
|
||||
"**Scaffolding, not yet the payoff.** The streams exist, but the host still "
|
||||
"synchronises after every round — see opt3.")
|
||||
rail(s, [
|
||||
("label", "opt2 · 4 streams · batch 2 000"),
|
||||
("gap", 0.15),
|
||||
("stat", "End to end", "43.2 µs", PALE),
|
||||
("stat", "vs CPU", "×4.86", ACCENT),
|
||||
("gap", 0.05),
|
||||
("row", "Step gain over opt1", "×1.55", ACCENT),
|
||||
("row", "Host + PCIe", "21 µs (was 44)", AMBER),
|
||||
])
|
||||
|
||||
# =========================================================== 7 · OPT3
|
||||
s = new_slide()
|
||||
chrome(s, 7, "opt3 · remove the sync barriers", "Stop draining the GPU between rounds")
|
||||
bullets(s, M, 1.95, 7.4, [
|
||||
"opt2 synchronised **all streams after every round** of n_streams frames. "
|
||||
"The GPU drained to empty each time.",
|
||||
"opt3 submits every frame's H2D → kernel → D2H **asynchronously**, then "
|
||||
"synchronises **once at the end of the batch**.",
|
||||
], size=10.5)
|
||||
figure(s, "fig_streams", M, 3.05, 7.55)
|
||||
code(s, 8.35, 1.95, 4.25, [
|
||||
"// opt2: barrier after every round",
|
||||
"for (round) {",
|
||||
" submit(n_streams frames);",
|
||||
" «cudaDeviceSynchronize»();",
|
||||
"}",
|
||||
"",
|
||||
"// opt3: submit everything, sync once",
|
||||
"for (frame : batch) {",
|
||||
" cudaMemcpyAsync(..., sc.stream);",
|
||||
" kernel<<<..., sc.stream>>>(...);",
|
||||
" cudaMemcpyAsync(..., sc.stream);",
|
||||
"}",
|
||||
"for (sc : streams)",
|
||||
" «cudaStreamSynchronize»(sc.stream);",
|
||||
], size=8, title="THE ONE-LINE IDEA")
|
||||
callout(s, 8.35, 5.05, 4.25,
|
||||
"**37.6 µs/frame · ×5.58**\nHost overhead 21 → **14 µs**", h=0.86, size=11)
|
||||
caption(s, 8.35, 6.15, 4.25,
|
||||
"Each lane is one stream. Removing the barrier lets a stream start its next "
|
||||
"frame while its neighbours are still copying.")
|
||||
|
||||
# =========================================================== 8 · OPT4
|
||||
s = new_slide()
|
||||
chrome(s, 8, "opt4 · pinned (page-locked) memory", "What pinning is, and why the GPU cares")
|
||||
bullets(s, M, 1.95, 12.0, [
|
||||
"Normal host memory is **pageable** — the OS may move or swap it. A DMA engine "
|
||||
"cannot safely read that, so the driver first copies your data into a **hidden "
|
||||
"pinned staging buffer**. Every transfer is copied twice.",
|
||||
"**Pinning** locks the pages in physical RAM. The GPU's DMA engine then reads "
|
||||
"host memory **directly** — no staging copy, and the transfer can be truly asynchronous.",
|
||||
], size=10.5)
|
||||
figure(s, "fig_pinning", M, 3.15, 7.6)
|
||||
code(s, 8.5, 3.15, 4.1, [
|
||||
"// pin the whole dataset once",
|
||||
"«cudaHostRegister»(ptr, bytes,",
|
||||
" cudaHostRegisterDefault);",
|
||||
"",
|
||||
"// ... run the whole campaign ...",
|
||||
"",
|
||||
"«cudaHostUnregister»(ptr);",
|
||||
], size=8, title="ClusterFinderCUDA.hpp")
|
||||
callout(s, 8.5, 4.72, 4.1,
|
||||
"**27.2 µs/frame · ×7.73**\nHost overhead 14 → **3 µs**", h=0.86, size=11)
|
||||
caption(s, 8.5, 5.82, 4.1,
|
||||
"Measured H2D: one 400×400 uint16 frame (312.5 KiB = 320 000 B) in 13.2 µs "
|
||||
"= 24.2 GB/s — 77% of PCIe 4.0 ×16 theoretical, i.e. true DMA speed. "
|
||||
"Pageable staging runs ~15 GB/s.")
|
||||
caption(s, M, 6.62, 7.6,
|
||||
"Caveat: pinned memory is a finite system resource — it cannot be swapped. "
|
||||
"aare exposes a budget helper (print_pinning_budget) before you pin 31 GB.")
|
||||
|
||||
# =========================================================== 9 · OPT5
|
||||
s = new_slide()
|
||||
chrome(s, 9, "opt5 · CUDA Graphs", "Record the pipeline once, replay it with one call")
|
||||
bullets(s, M, 1.95, 12.0, [
|
||||
"Every cudaMemcpyAsync / kernel launch costs the **CPU** a few microseconds of "
|
||||
"driver work — per frame, per operation. At 39 k frames/s that is the budget.",
|
||||
"A **CUDA Graph** captures the whole dependency DAG once. Replaying it is a "
|
||||
"**single** cudaGraphLaunch — the driver already knows every node and edge.",
|
||||
], size=10.5)
|
||||
figure(s, "fig_graphs", M, 3.15, 7.6)
|
||||
code(s, 8.5, 3.15, 4.1, [
|
||||
"// record once, at setup",
|
||||
"cudaStreamBeginCapture(sc.stream, ...);",
|
||||
" submit_h2d_kernel_d2h(sc);",
|
||||
"cudaStreamEndCapture(sc.stream, &sc.graph);",
|
||||
"«cudaGraphInstantiate»(&sc.graphExec, ...);",
|
||||
"",
|
||||
"// per batch — one call",
|
||||
"«cudaGraphLaunch»(sc.graphExec, sc.stream);",
|
||||
], size=8, title="ClusterFinderCUDA_graph.hpp")
|
||||
callout(s, 8.5, 4.88, 4.1,
|
||||
"**25.3 µs/frame · ×8.29**\nBest end-to-end result", h=0.86, size=11)
|
||||
caption(s, M, 6.62, 12.0,
|
||||
"Worth 7% here because opt4 already removed the transfer cost — what remains "
|
||||
"is CPU launch overhead, which is exactly what graphs eliminate. "
|
||||
"Trade-off: shapes are frozen at record time, so the batch geometry must be fixed.")
|
||||
|
||||
# =========================================================== 10 · OPT6 why
|
||||
s = new_slide()
|
||||
chrome(s, 10, "opt6 · FP32 device pedestal", "The first change to the kernel itself")
|
||||
bullets(s, M, 1.95, 7.5, [
|
||||
"~99.9% of pixels take the **pedestal-update** branch, which reads and writes "
|
||||
"mean, sum and sum². In FP64 that is **32 bytes per pixel**; in FP32, 16.",
|
||||
"The kernel is **bandwidth-bound**, so halving that traffic nearly halves the time.",
|
||||
"Second effect: on a GeForce part, **FP64 arithmetic runs at 1/64 of FP32**. "
|
||||
"The pedestal update was paying that tax on every pixel.",
|
||||
], size=10.5)
|
||||
figure(s, "fig_f32_kernel", M, 4.05, 7.5)
|
||||
code(s, 8.5, 1.95, 4.1, [
|
||||
"// clusterfinder_kernel.cuh",
|
||||
"using COMPUTE_TYPE = float;",
|
||||
"using DEVICE_PED_TYPE = «float»;",
|
||||
"// was: double",
|
||||
], size=8.5, title="ONE TYPEDEF")
|
||||
callout(s, 8.5, 3.05, 4.1,
|
||||
"Kernel, 9×9, measured with Nsight Systems\n**43.0 µs → 25.6 µs (−40%)**",
|
||||
h=0.92, size=11)
|
||||
caption(s, 8.5, 4.20, 4.1,
|
||||
"Exclusive per-instance kernel time, 2 000 instances, 1 stream. "
|
||||
"σ = 0.4 µs. Transfers are unchanged, as they must be.")
|
||||
callout(s, 8.5, 5.30, 4.1,
|
||||
"But a faster kernel is **not automatically a faster frame** — and naive FP32 "
|
||||
"is **wrong**. Both on the next slides.", h=1.05, size=10, color=AMBER)
|
||||
|
||||
# =========================================================== 11 · OPT6 trap
|
||||
s = new_slide()
|
||||
chrome(s, 11, "opt6 · the correctness trap", "Why the obvious FP32 pedestal is broken")
|
||||
bullets(s, M, 1.95, 7.9, [
|
||||
"The running variance was computed as **var = E[X²] − mean²**. With a pedestal "
|
||||
"mean of ~4 655 ADU, both terms are ≈ 2.17 × 10⁷ while the answer is ≈ 2 000.",
|
||||
"In FP32 the spacing between representable numbers at 2.17 × 10⁷ is **2 048** — "
|
||||
"larger than the variance itself. This is **catastrophic cancellation**.",
|
||||
], size=10.5)
|
||||
figure(s, "fig_cancellation", M, 3.25, 7.5)
|
||||
rail(s, [
|
||||
("label", "What it looked like"),
|
||||
("gap", 0.15),
|
||||
("stat", "Extra clusters", "+28.06%", AMBER),
|
||||
("gap", 0.05),
|
||||
("note", "Quiet pixels got rms → 0, so their threshold became 0 and they fired "
|
||||
"on every single frame — producing a large unphysical high-energy tail "
|
||||
"in the spectrum."),
|
||||
("gap", 0.35),
|
||||
("row", "Affected pixels", "~1–2% of the sensor", TEXT2),
|
||||
("row", "Written up in", "docs/pedestal_precision_…", MUTED),
|
||||
])
|
||||
|
||||
# =========================================================== 12 · OPT6 fix
|
||||
s = new_slide()
|
||||
chrome(s, 12, "opt6 · the variance rewrite", "Accumulate what is small, not what is large")
|
||||
bullets(s, M, 1.95, COL, [
|
||||
"Freeze a per-pixel baseline **X₀ = round(mean)** once, at the end of pedestal "
|
||||
"training, and never move it again.",
|
||||
"Accumulate the **centred** value Y = X − X₀ instead of X. Now both sums are "
|
||||
"O(rms)-sized — the huge common term is gone before the subtraction.",
|
||||
"The reported pedestal mean is still the full value, **X₀ + sum/n**, so nothing "
|
||||
"downstream changes.",
|
||||
])
|
||||
code(s, M, 4.0, COL, [
|
||||
"// before — both terms ~2.17e7, answer ~2000",
|
||||
"var = sum2/n - mean*mean;",
|
||||
"",
|
||||
"// after — centred on a frozen per-pixel offset X0",
|
||||
"DEVICE_PED_TYPE resid = mean - «d_pd_off»[i]; // ~O(1)",
|
||||
"DEVICE_PED_TYPE var_px = sum2[i]/n - resid*resid; // no cancellation",
|
||||
], title="clusterfinder_kernel.cuh")
|
||||
callout(s, M, 6.05, COL,
|
||||
"Result: the 100% FP32 build now matches the FP64 build to "
|
||||
"**3 × 10⁻⁷** — 70 clusters out of 233 million.")
|
||||
rail(s, [
|
||||
("label", "Why it works"),
|
||||
("gap", 0.15),
|
||||
("note", "Precision is relative. Floats resolve small numbers finely and large "
|
||||
"numbers coarsely — so never let a small answer be the difference of "
|
||||
"two large numbers."),
|
||||
("gap", 0.5),
|
||||
("row", "f32 vs f64 counts", "3 × 10⁻⁷", ACCENT),
|
||||
("row", "vs CPU", "0.0039%", ACCENT),
|
||||
("gap", 0.3),
|
||||
("note", "X₀ must never be updated — the accumulators are defined relative to it."),
|
||||
])
|
||||
|
||||
# =========================================================== 13 · OPT6 when
|
||||
s = new_slide()
|
||||
chrome(s, 13, "opt6 · when does a faster kernel help?", "Only if the kernel was the tallest bar")
|
||||
figure(s, "fig_bottleneck", M, 2.05, 11.9)
|
||||
callout(s, M, 5.35, 11.9,
|
||||
"**f32 buys exactly the distance between the kernel and the next-tallest bar.** "
|
||||
"At 3×3 the kernel already hides inside the transfers, so end-to-end throughput does not move at all. "
|
||||
"At 9×9 the kernel is on the critical path, and the same change is worth 8% of the frame.",
|
||||
h=0.95, size=11)
|
||||
caption(s, M, 6.55, 11.9,
|
||||
"9×9, 20 000 frames, 8 streams, cap 1 500 (all CPU clusters recorded): "
|
||||
"1.540 s → 1.423 s batched, 1.482 s → 1.372 s with graphs. "
|
||||
"The FP64 build shows the classic kernel-bound signature — per-frame wall time shorter than per-frame kernel time, because kernels from different streams queue.")
|
||||
|
||||
# =========================================================== 14 · RESULTS
|
||||
s = new_slide()
|
||||
chrome(s, 14, "Results", "The whole ladder, one dataset, one baseline")
|
||||
figure(s, "fig_arc", M, 1.95, 11.9)
|
||||
callout(s, M, 5.55, 5.85,
|
||||
"**×8.29 over 48 CPU threads** — 21.0 s → 2.53 s for 100 000 frames.", h=0.8)
|
||||
callout(s, 6.75, 5.55, 5.85,
|
||||
"Every step is **monotonic**, and correctness is held constant at **0.004%** throughout.",
|
||||
h=0.8, color=AMBER)
|
||||
caption(s, M, 6.55, 11.9,
|
||||
"3×3 clusters · nσ = 5 · 100 000 frames · batch 2 000 · 4 streams · warm run "
|
||||
"(page faults plateaued) · CPU baseline = ClusterFinderMT with 48 threads.")
|
||||
|
||||
# =========================================================== 15 · WHERE TIME GOES
|
||||
s = new_slide()
|
||||
chrome(s, 15, "Where the time actually went", "The kernel never changed — the overhead collapsed")
|
||||
figure(s, "fig_overhead", M, 2.05, 5.9)
|
||||
bullets(s, 7.0, 2.15, 5.6, [
|
||||
"The GPU kernel is a **flat ~23 µs** across opt1–opt5. Not one of those steps "
|
||||
"made the arithmetic faster.",
|
||||
"What changed is everything around it: **44 → 21 → 14 → 3 µs** of host and PCIe "
|
||||
"time per frame.",
|
||||
"By opt4 the pipeline is **fed almost perfectly** — which is precisely why opt5 "
|
||||
"(launch overhead) is the only lever left, and why opt6 shows nothing at 3×3.",
|
||||
], size=10.5)
|
||||
callout(s, 7.0, 5.15, 5.6,
|
||||
"The bottleneck moved from **the host**, to **PCIe**, and finally — only for "
|
||||
"large cluster windows — to **the kernel**.", h=0.95, size=10.5)
|
||||
caption(s, M, 6.4, 11.9,
|
||||
"Per-frame GPU operation profile at 9×9 (Nsight Systems): kernel 43 µs · "
|
||||
"D2H 19.4 µs · H2D 13.2 µs. At 3×3 the kernel is 13–24 µs against a ~25 µs "
|
||||
"transfer-and-host floor — which is the entire story of this deck in three numbers.")
|
||||
|
||||
# =========================================================== 16 · CORRECTNESS
|
||||
s = new_slide()
|
||||
chrome(s, 16, "Validation", "Same physics out of every variant")
|
||||
figure(s, "fig_correctness", M, 2.0, 7.6)
|
||||
bullets(s, 8.6, 2.05, 4.1, [
|
||||
"233 million clusters over 100 000 frames.",
|
||||
"All CUDA variants agree with the CPU to **0.004%**.",
|
||||
"The residual is **not** precision: it is the CUDA finder updating the pedestal "
|
||||
"**once per frame** vs the CPU's per-pixel update.",
|
||||
], size=10)
|
||||
code(s, M, 4.85, 7.6, [
|
||||
"CPU (ClusterFinderMT) 233 085 343 2330.85 / frame reference",
|
||||
"opt1 .. opt5 (f64 ped) 233 093 484 2330.93 / frame 0.0035 %",
|
||||
"opt6 (f32 ped) 233 094 465 2330.94 / frame 0.0039 %",
|
||||
], size=8.5, title="CLUSTER COUNTS · 3×3 · 100 000 FRAMES")
|
||||
callout(s, 8.6, 4.85, 4.1,
|
||||
"A local-maximum gate had to be **back-ported** into the opt1/opt2 snapshot so "
|
||||
"the whole ladder is compared at identical correctness.", h=1.0, size=10, color=AMBER)
|
||||
caption(s, M, 6.35, 11.9,
|
||||
"Cross-checks: ClusterFinderFrozen (a CPU finder with the CUDA pedestal-update "
|
||||
"timing) isolates that residual; energy spectra overlay within statistics.")
|
||||
|
||||
# =========================================================== 17 · API 1
|
||||
s = new_slide()
|
||||
chrome(s, 17, "For users · Python API", "The fast path in eight lines")
|
||||
code(s, M, 1.95, 7.6, [
|
||||
"from aare import File, ClusterFinderCUDA",
|
||||
"",
|
||||
"cf = ClusterFinderCUDA(image_size=(400, 400), cluster_size=(3, 3),",
|
||||
" n_sigma=5, «n_streams»=4,",
|
||||
" «max_clusters_per_frame»=3000)",
|
||||
"",
|
||||
"for _ in range(1000): # 1. train the pedestal",
|
||||
" cf.push_pedestal_frame(pd.read_frame())",
|
||||
"",
|
||||
"data = f.read_n(100_000) # 2. one contiguous array",
|
||||
"cf.«register_input_buffer»(data) # 3. pin it once",
|
||||
"",
|
||||
"for s in range(0, N, 2000): # 4. batch through it",
|
||||
" clusters = cf.«find_clusters_batched»(data[s:s+2000], first_frame=s)",
|
||||
"",
|
||||
"cf.unregister_input_buffer() # 5. release the pages",
|
||||
], size=9, title="THE RECOMMENDED PATTERN")
|
||||
bullets(s, 8.6, 2.0, 4.1, [
|
||||
"find_clusters_batched returns **one ClusterVector per frame**, in order.",
|
||||
"register_input_buffer is what turns opt3 into opt4 — **one call**.",
|
||||
"Swap in ClusterFinderCUDAGraph for opt5; the API is identical.",
|
||||
], size=10)
|
||||
callout(s, 8.6, 4.55, 4.1,
|
||||
"GIL is released for both find_clusters and find_clusters_batched, so "
|
||||
"reading the next file can overlap with the GPU.", h=1.05, size=10)
|
||||
callout(s, 8.6, 5.80, 4.1,
|
||||
"Pin **once**, outside the loop. Slices of a registered array inherit the pinning.",
|
||||
h=0.85, size=10, color=AMBER)
|
||||
|
||||
# =========================================================== 18 · API 2
|
||||
s = new_slide()
|
||||
chrome(s, 18, "For users · choosing the knobs", "What to set, and what it costs you")
|
||||
hdr = [("Parameter", 1.05), ("What it does", 3.6), ("Guidance", 5.2)]
|
||||
y = 2.0
|
||||
rect(s, M, y, 11.9, 0.4, PANEL)
|
||||
for lab, dx in hdr:
|
||||
tf = tb(s, M + dx - 0.85 if dx > 1.05 else M + 0.28, y + 0.09, 5.0, 0.3)
|
||||
run(para(tf, True), lab.upper(), 9, MUTED, bold=True, spc=1.3)
|
||||
y += 0.44
|
||||
params = [
|
||||
("n_streams", "How many frames are in flight at once.",
|
||||
"4 is right for 3×3. Larger windows saturate the GPU — 8 helps at 9×9."),
|
||||
("max_clusters_per_frame", "Fixed size of the per-frame D2H transfer.",
|
||||
"Must exceed the real maximum or clusters are silently dropped. Too high wastes PCIe."),
|
||||
("batch size", "Frames per find_clusters_batched call.",
|
||||
"2 000 amortises launch overhead without a large pinned footprint."),
|
||||
("cluster_size", "Compile-time stencil geometry.",
|
||||
"3×3 and 9×9 are registered; 9×9 shifts the bottleneck onto the kernel."),
|
||||
("register_input_buffer", "Page-locks the host array for DMA.",
|
||||
"Always, if the data is already in RAM. Check the pinning budget first."),
|
||||
]
|
||||
for i, (p_, what, guide) in enumerate(params):
|
||||
if i % 2 == 0:
|
||||
rect(s, M, y, 11.9, 0.82, PANEL)
|
||||
tf = tb(s, M + 0.28, y + 0.14, 2.6, 0.5)
|
||||
run(para(tf, True, line=1.1), p_, 9.5, ACCENT, font=MONO, bold=True)
|
||||
tf = tb(s, M + 3.0, y + 0.14, 2.9, 0.6)
|
||||
run(para(tf, True, line=1.2), what, 9.5, PALE)
|
||||
tf = tb(s, M + 6.15, y + 0.14, 5.4, 0.6)
|
||||
run(para(tf, True, line=1.2), guide, 9.5, TEXT2)
|
||||
y += 0.80
|
||||
callout(s, M, 6.55, 11.2,
|
||||
"The single most common mistake: leaving **max_clusters_per_frame** too low. "
|
||||
"It does not error — it truncates, and every frame quietly returns the same count.",
|
||||
h=0.66, size=10, color=AMBER)
|
||||
|
||||
# =========================================================== 19 · NEXT
|
||||
s = new_slide()
|
||||
chrome(s, 19, "Where this leaves us", "The bottleneck has moved — twice")
|
||||
cards = [
|
||||
("DONE", ACCENT, "×8.29 over 48 CPU threads",
|
||||
"25.3 µs/frame end to end, at 0.004% cluster agreement. Five pipeline steps "
|
||||
"and one kernel step."),
|
||||
("DONE", ACCENT, "FP32 pedestal, safely",
|
||||
"−40% kernel, and correct — because the variance is now accumulated on a frozen "
|
||||
"per-pixel offset instead of a raw second moment."),
|
||||
("NEXT", AMBER, "Attack the transfers, not the kernel",
|
||||
"At 3×3 the kernel is already hidden. The remaining per-frame cost is PCIe: "
|
||||
"492 kB of clusters out, 312 kB of frame in."),
|
||||
("NEXT", AMBER, "Keep results on the device",
|
||||
"On-GPU reduction, eta/interpolation on device, or compressed cluster formats — "
|
||||
"so the D2H bar stops setting the floor."),
|
||||
]
|
||||
x, y = M, 2.05
|
||||
for i, (tag, col, title, body) in enumerate(cards):
|
||||
cx = M + (i % 2) * 6.05
|
||||
cy = 2.05 + (i // 2) * 2.35
|
||||
rect(s, cx, cy, 5.85, 2.05, PANEL)
|
||||
rect(s, cx, cy, 5.85, 0.035, col)
|
||||
tf = tb(s, cx + 0.3, cy + 0.26, 1.4, 0.26)
|
||||
run(para(tf, True), tag, 8.5, col, bold=True, spc=1.5)
|
||||
tf = tb(s, cx + 0.3, cy + 0.60, 5.2, 0.4)
|
||||
run(para(tf, True, line=1.1), title, 14, PALE, bold=True)
|
||||
tf = tb(s, cx + 0.3, cy + 1.12, 5.2, 0.85)
|
||||
run(para(tf, True, line=1.3), body, 10, TEXT2)
|
||||
callout(s, M, 6.58, 11.2,
|
||||
"Full numbers, methodology and reproduction steps: **docs/benchmark_opt1_opt6_results.md** · "
|
||||
"notebook **python/tests/ClusterFinderCUDA_perf.ipynb** · profiler probe **python/tests/nsys_kernel_probe.py**",
|
||||
h=0.66, size=9.5)
|
||||
|
||||
prs.save(OUT)
|
||||
print(f"saved {OUT} ({len(prs.slides.__iter__.__self__._sldIdLst)} slides)")
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+800
-113
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,236 @@
|
||||
"""Figures for the fused deck — kernel/algorithm/occupancy half.
|
||||
|
||||
Same palette and conventions as make_figs.py (deck palette, dark, tight bbox).
|
||||
Writes into docs/figures/ alongside the optimization figures.
|
||||
|
||||
Occupancy numbers are not hand-computed: they come from
|
||||
cudaOccupancyMaxActiveBlocksPerMultiprocessor + cudaFuncGetAttributes on the
|
||||
real kernel (RTX 4090, sm_89), measured 2026-08-11 — see the table in
|
||||
build_fused_deck.py.
|
||||
"""
|
||||
import sys
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
from matplotlib.colors import LinearSegmentedColormap
|
||||
from matplotlib.patches import Rectangle, FancyArrowPatch
|
||||
from pathlib import Path
|
||||
|
||||
OUT = Path(__file__).resolve().parent.parent / "figures"
|
||||
OUT.mkdir(exist_ok=True)
|
||||
|
||||
BG = "#0B1018"
|
||||
PANEL = "#121A28"
|
||||
RULE = "#1E2836"
|
||||
ACCENT = "#1E90C2"
|
||||
AMBER = "#E8B25C"
|
||||
PALE = "#E7EDF4"
|
||||
TEXT2 = "#A5B2C4"
|
||||
MUTED = "#6B7A90"
|
||||
|
||||
plt.rcParams.update({
|
||||
"font.family": "DejaVu Sans", "font.size": 9,
|
||||
"text.color": PALE, "axes.labelcolor": TEXT2,
|
||||
"xtick.color": TEXT2, "ytick.color": TEXT2,
|
||||
"axes.edgecolor": RULE, "axes.facecolor": "none",
|
||||
"figure.facecolor": BG, "savefig.facecolor": BG,
|
||||
"axes.grid": False, "svg.fonttype": "none",
|
||||
})
|
||||
|
||||
# deck-native sequential map: background → accent → amber → white-hot
|
||||
CMAP = LinearSegmentedColormap.from_list(
|
||||
"deck", ["#080C12", "#10202F", ACCENT, AMBER, "#FFF3DC"])
|
||||
|
||||
|
||||
def save(fig, name):
|
||||
fig.savefig(OUT / f"{name}.png", dpi=220, transparent=False,
|
||||
bbox_inches="tight", pad_inches=0.08)
|
||||
plt.close(fig)
|
||||
print("wrote", name)
|
||||
|
||||
|
||||
def bare(ax, keep=("left", "bottom")):
|
||||
for s in ("top", "right", "left", "bottom"):
|
||||
ax.spines[s].set_visible(s in keep)
|
||||
|
||||
|
||||
# ------------------------------------------------ 1. what a frame looks like
|
||||
def fig_frame():
|
||||
"""Pedestal-subtracted MOENCH frame + a zoom on one real 3×3 cluster."""
|
||||
sys.path.append("/home/ferjao_k/aare/build")
|
||||
from aare import File
|
||||
base = Path("/mnt/sls_det_storage/moench_data/2603_MaxIVBeamtime/"
|
||||
"2026032408/process/xrf/")
|
||||
pd = File(base / "Cu_factor_10_pedestal_master_0.json")
|
||||
ped = np.mean([np.asarray(pd.read_frame(), dtype=np.float64)
|
||||
for _ in range(200)], axis=0)
|
||||
f = File(base / "Cu_factor_10_data_master_0.json")
|
||||
frame = np.asarray(f.read_frame(), dtype=np.float64) - ped
|
||||
|
||||
crop = frame[40:190, 40:190]
|
||||
|
||||
fig = plt.figure(figsize=(7.4, 3.25))
|
||||
ax = fig.add_axes([0.0, 0.02, 0.44, 0.94])
|
||||
im = ax.imshow(crop, cmap=CMAP, vmin=-40, vmax=1200, interpolation="nearest")
|
||||
ax.set_xticks([]); ax.set_yticks([])
|
||||
for s in ax.spines.values():
|
||||
s.set_color(RULE)
|
||||
ax.set_title("one frame, pedestal subtracted · 150×150 crop",
|
||||
color=MUTED, fontsize=8, pad=7)
|
||||
cb = fig.colorbar(im, ax=ax, fraction=0.045, pad=0.02)
|
||||
cb.outline.set_edgecolor(RULE)
|
||||
cb.ax.tick_params(labelsize=7, color=RULE)
|
||||
cb.set_label("ADU above pedestal", color=MUTED, fontsize=7.5)
|
||||
|
||||
# Pick a clean, well-isolated charge-sharing event: a local maximum of
|
||||
# moderate amplitude whose 3×3 core carries the charge and whose
|
||||
# surrounding ring is quiet — i.e. what the algorithm is designed to find.
|
||||
best, win = None, None
|
||||
for r in range(6, frame.shape[0] - 6):
|
||||
for c in range(6, frame.shape[1] - 6):
|
||||
v = frame[r, c]
|
||||
if not (500 < v < 2000):
|
||||
continue
|
||||
w = frame[r - 4:r + 5, c - 4:c + 5]
|
||||
if w.max() > v: # must be the local max
|
||||
continue
|
||||
core = w[3:6, 3:6]
|
||||
ring = np.concatenate([w[:3].ravel(), w[6:].ravel(),
|
||||
w[3:6, :3].ravel(), w[3:6, 6:].ravel()])
|
||||
if ring.max() > 80: # neighbourhood must be quiet
|
||||
continue
|
||||
share = (core.sum() - v) / core.sum() # charge outside the peak
|
||||
if best is None or share > best[0]:
|
||||
best, win = (share, r, c), w
|
||||
if win is None: # fallback: brightest pixel
|
||||
inner = frame[6:-6, 6:-6]
|
||||
r, c = np.unravel_index(np.argmax(inner), inner.shape)
|
||||
win = frame[r + 2:r + 11, c + 2:c + 11]
|
||||
|
||||
ax2 = fig.add_axes([0.60, 0.10, 0.30, 0.78])
|
||||
ax2.imshow(win, cmap=CMAP, vmin=-40, vmax=1200, interpolation="nearest")
|
||||
ax2.set_xticks([]); ax2.set_yticks([])
|
||||
for s in ax2.spines.values():
|
||||
s.set_color(RULE)
|
||||
ax2.add_patch(Rectangle((2.5, 2.5), 3, 3, fill=False, edgecolor=PALE,
|
||||
lw=1.8, zorder=5))
|
||||
for dy in (-1, 0, 1):
|
||||
for dx in (-1, 0, 1):
|
||||
v = win[4 + dy, 4 + dx]
|
||||
ax2.text(4 + dx, 4 + dy, f"{v:.0f}", ha="center", va="center",
|
||||
color=BG if v > 500 else PALE, fontsize=7,
|
||||
fontweight="bold" if dx == 0 and dy == 0 else "normal",
|
||||
zorder=6)
|
||||
ax2.set_title("9×9 zoom on one hit", color=MUTED, fontsize=8, pad=7)
|
||||
ax2.text(4, 9.2, f"3×3 sum = {win[3:6, 3:6].sum():.0f} ADU — one photon.\n"
|
||||
"The peak pixel holds only part of the charge.",
|
||||
ha="center", va="top", color=TEXT2, fontsize=7.5)
|
||||
save(fig, "fig_frame")
|
||||
|
||||
|
||||
# --------------------------------------------- 2. shared-memory tile + halo
|
||||
def fig_tile():
|
||||
B, r = 16, 1 # 16×16 block, 3×3 cluster → 1-px halo
|
||||
n = B + 2 * r
|
||||
fig = plt.figure(figsize=(7.6, 3.1))
|
||||
|
||||
ax = fig.add_axes([0.0, 0.0, 0.44, 1.0])
|
||||
ax.set_aspect("equal"); ax.axis("off")
|
||||
ax.set_xlim(-0.6, n + 0.6); ax.set_ylim(-3.2, n + 1.3)
|
||||
for i in range(n):
|
||||
for j in range(n):
|
||||
halo = i < r or j < r or i >= n - r or j >= n - r
|
||||
ax.add_patch(Rectangle((j, n - 1 - i), 0.92, 0.92,
|
||||
facecolor=RULE if halo else "#17394F",
|
||||
edgecolor="none"))
|
||||
# one thread's 3×3 neighbourhood
|
||||
ti, tj = 6, 5
|
||||
for di in (-1, 0, 1):
|
||||
for dj in (-1, 0, 1):
|
||||
ax.add_patch(Rectangle((tj + r + dj, n - 1 - (ti + r + di)), 0.92,
|
||||
0.92, facecolor=ACCENT, edgecolor="none"))
|
||||
ax.add_patch(Rectangle((tj + r, n - 1 - (ti + r)), 0.92, 0.92,
|
||||
facecolor=AMBER, edgecolor="none"))
|
||||
ax.text(n / 2, n + 0.45, "shared-memory tile · 18 × 18",
|
||||
ha="center", color=MUTED, fontsize=8)
|
||||
for y, c, t in [(-1.05, AMBER, "the thread's own pixel"),
|
||||
(-1.85, ACCENT, "its 3×3 neighbourhood"),
|
||||
(-2.65, RULE, "halo — loaded, never centred on")]:
|
||||
ax.add_patch(Rectangle((0, y), 0.7, 0.36, facecolor=c, edgecolor="none"))
|
||||
ax.text(1.0, y + 0.18, t, va="center", color=TEXT2, fontsize=7.5)
|
||||
|
||||
# right: tile cost vs cluster size
|
||||
ax2 = fig.add_axes([0.575, 0.20, 0.40, 0.62])
|
||||
labels = ["3×3\n18×18", "5×5\n20×20", "7×7\n22×22", "9×9\n24×24"]
|
||||
kb = [(16 + 2 * (k // 2)) ** 2 * 4 / 1024 for k in (3, 5, 7, 9)]
|
||||
ax2.bar(np.arange(4), kb, width=0.55, color=ACCENT, zorder=3)
|
||||
for i, v in enumerate(kb):
|
||||
ax2.text(i, v + 0.12, f"{v:.1f}", ha="center", color=PALE, fontsize=9,
|
||||
fontweight="bold")
|
||||
ax2.axhline(100, color=PALE, lw=1.2, ls="--")
|
||||
ax2.set_xticks(np.arange(4)); ax2.set_xticklabels(labels, color=TEXT2,
|
||||
fontsize=8)
|
||||
ax2.set_ylim(0, 3.4); ax2.set_yticks([])
|
||||
bare(ax2, keep=("bottom",))
|
||||
ax2.set_title("KB of shared memory per 16×16 block (float tile)",
|
||||
color=MUTED, fontsize=8, pad=8)
|
||||
ax2.text(3.55, 3.15, "100 KB available per SM on Ada\n"
|
||||
"— shared memory is never the limit",
|
||||
ha="right", va="top", color=PALE, fontsize=7.5)
|
||||
save(fig, "fig_tile")
|
||||
|
||||
|
||||
# ------------------------------------------------- 3. occupancy / registers
|
||||
def fig_occupancy():
|
||||
fig, (ax, ax2) = plt.subplots(1, 2, figsize=(11.2, 2.95),
|
||||
gridspec_kw={"width_ratios": [1, 1.5]})
|
||||
|
||||
# left — registers set the occupancy, per cluster size
|
||||
occ = [100.0, 33.3]
|
||||
ax.bar([0, 1], occ, width=0.5, color=[ACCENT, AMBER], zorder=3)
|
||||
for i, o in enumerate(occ):
|
||||
ax.text(i, o + 3, f"{o:.0f}%", ha="center", color=PALE, fontsize=12,
|
||||
fontweight="bold")
|
||||
ax.set_xticks([0, 1])
|
||||
ax.set_xticklabels(["3×3 cluster\n38 regs/thread · 6 blocks/SM",
|
||||
"9×9 cluster\n128 regs/thread · 2 blocks/SM"],
|
||||
color=TEXT2, fontsize=8.5)
|
||||
ax.set_ylim(0, 122); ax.set_yticks([])
|
||||
bare(ax, keep=("bottom",))
|
||||
ax.set_title("achieved occupancy, 16×16 block · f32 build", color=MUTED,
|
||||
fontsize=8.5, pad=8)
|
||||
|
||||
# right — block-size sweep, both cluster sizes
|
||||
o3 = [100.0, 100.0, 66.7]
|
||||
o9 = [33.3, 33.3, 0.0]
|
||||
halo3 = [56, 27, 13]
|
||||
blocks = [f"{b}\nhalo +{h}% of the tile"
|
||||
for b, h in zip(["8×8 · 64 threads", "16×16 · 256 threads",
|
||||
"32×32 · 1024 threads"], halo3)]
|
||||
x = np.arange(3); w = 0.34
|
||||
ax2.bar(x - w / 2, o3, width=w, color=ACCENT, zorder=3, label="3×3 cluster")
|
||||
ax2.bar(x + w / 2, o9, width=w, color=AMBER, zorder=3, label="9×9 cluster")
|
||||
for xi, (a, b) in enumerate(zip(o3, o9)):
|
||||
ax2.text(xi - w / 2, a + 3, f"{a:.0f}%", ha="center", color=TEXT2,
|
||||
fontsize=8.5)
|
||||
ax2.text(xi + w / 2, b + 3,
|
||||
("will not launch\n(registers)" if b == 0 else f"{b:.0f}%"),
|
||||
ha="center", va="bottom", color=AMBER if b == 0 else TEXT2,
|
||||
fontsize=8 if b == 0 else 8.5,
|
||||
fontweight="bold" if b == 0 else "normal")
|
||||
ax2.set_xticks(x); ax2.set_xticklabels(blocks, color=TEXT2, fontsize=8.5)
|
||||
ax2.set_ylim(0, 122); ax2.set_yticks([])
|
||||
bare(ax2, keep=("bottom",))
|
||||
ax2.legend(frameon=False, fontsize=8.5, labelcolor=TEXT2, loc="upper right")
|
||||
ax2.set_title("occupancy vs block size (halo overhead quoted for 3×3)",
|
||||
color=MUTED, fontsize=8.5, pad=8)
|
||||
fig.subplots_adjust(bottom=0.26)
|
||||
save(fig, "fig_occupancy")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
fig_tile()
|
||||
fig_occupancy()
|
||||
fig_frame()
|
||||
print("done ->", OUT)
|
||||
@@ -0,0 +1,876 @@
|
||||
{
|
||||
"n_frames": 10000,
|
||||
"totals": {
|
||||
"cpu": 23244602,
|
||||
"frozen": 23244605,
|
||||
"cuda": 23244611
|
||||
},
|
||||
"pairs": {
|
||||
"cpu vs frozen": {
|
||||
"a_only": 8,
|
||||
"b_only": 11
|
||||
},
|
||||
"cpu vs cuda": {
|
||||
"a_only": 8,
|
||||
"b_only": 17
|
||||
},
|
||||
"frozen vs cuda": {
|
||||
"a_only": 0,
|
||||
"b_only": 6
|
||||
}
|
||||
},
|
||||
"extras": [
|
||||
{
|
||||
"frame": 147,
|
||||
"x": 202,
|
||||
"y": 8,
|
||||
"shift": 1
|
||||
},
|
||||
{
|
||||
"frame": 1757,
|
||||
"x": 102,
|
||||
"y": 167,
|
||||
"shift": 1
|
||||
},
|
||||
{
|
||||
"frame": 2204,
|
||||
"x": 397,
|
||||
"y": 178,
|
||||
"shift": 1
|
||||
},
|
||||
{
|
||||
"frame": 2790,
|
||||
"x": 1,
|
||||
"y": 374,
|
||||
"shift": 1
|
||||
},
|
||||
{
|
||||
"frame": 4098,
|
||||
"x": 348,
|
||||
"y": 46,
|
||||
"shift": 1
|
||||
},
|
||||
{
|
||||
"frame": 8372,
|
||||
"x": 272,
|
||||
"y": 47,
|
||||
"shift": 1
|
||||
}
|
||||
],
|
||||
"n_cuda_only": 6,
|
||||
"n_adjacent_to_agreed": 6,
|
||||
"shift_histogram": {
|
||||
"1": 6
|
||||
},
|
||||
"hists": {
|
||||
"cpu": [
|
||||
499.0,
|
||||
919.0,
|
||||
1474.0,
|
||||
2346.0,
|
||||
3498.0,
|
||||
4961.0,
|
||||
6307.0,
|
||||
7820.0,
|
||||
9216.0,
|
||||
10546.0,
|
||||
11475.0,
|
||||
11825.0,
|
||||
12529.0,
|
||||
12832.0,
|
||||
13093.0,
|
||||
13244.0,
|
||||
12611.0,
|
||||
12129.0,
|
||||
11832.0,
|
||||
11677.0,
|
||||
11595.0,
|
||||
11501.0,
|
||||
11484.0,
|
||||
11468.0,
|
||||
11354.0,
|
||||
11579.0,
|
||||
11549.0,
|
||||
11747.0,
|
||||
11641.0,
|
||||
11954.0,
|
||||
12136.0,
|
||||
12360.0,
|
||||
12613.0,
|
||||
13087.0,
|
||||
13153.0,
|
||||
13663.0,
|
||||
13840.0,
|
||||
14463.0,
|
||||
15204.0,
|
||||
16256.0,
|
||||
17384.0,
|
||||
18725.0,
|
||||
20249.0,
|
||||
22251.0,
|
||||
24573.0,
|
||||
27579.0,
|
||||
32342.0,
|
||||
40818.0,
|
||||
58221.0,
|
||||
90431.0,
|
||||
148680.0,
|
||||
247066.0,
|
||||
397450.0,
|
||||
607938.0,
|
||||
877275.0,
|
||||
1178976.0,
|
||||
1480418.0,
|
||||
1724824.0,
|
||||
1875375.0,
|
||||
1898011.0,
|
||||
1802008.0,
|
||||
1612942.0,
|
||||
1371944.0,
|
||||
1118841.0,
|
||||
893351.0,
|
||||
698921.0,
|
||||
540935.0,
|
||||
415181.0,
|
||||
315524.0,
|
||||
238485.0,
|
||||
177935.0,
|
||||
133335.0,
|
||||
101985.0,
|
||||
80613.0,
|
||||
66308.0,
|
||||
57704.0,
|
||||
52551.0,
|
||||
48730.0,
|
||||
46713.0,
|
||||
45263.0,
|
||||
45478.0,
|
||||
46818.0,
|
||||
48748.0,
|
||||
52268.0,
|
||||
55751.0,
|
||||
59752.0,
|
||||
63127.0,
|
||||
64355.0,
|
||||
64278.0,
|
||||
61790.0,
|
||||
57780.0,
|
||||
52398.0,
|
||||
46549.0,
|
||||
41177.0,
|
||||
36929.0,
|
||||
32859.0,
|
||||
29645.0,
|
||||
27949.0,
|
||||
26098.0,
|
||||
25284.0,
|
||||
25948.0,
|
||||
24170.0,
|
||||
23309.0,
|
||||
23092.0,
|
||||
22982.0,
|
||||
22740.0,
|
||||
22928.0,
|
||||
23247.0,
|
||||
24412.0,
|
||||
26263.0,
|
||||
29182.0,
|
||||
33590.0,
|
||||
38978.0,
|
||||
45376.0,
|
||||
52423.0,
|
||||
59269.0,
|
||||
64780.0,
|
||||
67378.0,
|
||||
68667.0,
|
||||
65197.0,
|
||||
61403.0,
|
||||
54807.0,
|
||||
48119.0,
|
||||
41254.0,
|
||||
34742.0,
|
||||
29510.0,
|
||||
24472.0,
|
||||
19725.0,
|
||||
15963.0,
|
||||
12877.0,
|
||||
10251.0,
|
||||
8289.0,
|
||||
6654.0,
|
||||
5526.0,
|
||||
4723.0,
|
||||
4074.0,
|
||||
3730.0,
|
||||
3395.0,
|
||||
3280.0,
|
||||
3208.0,
|
||||
3247.0,
|
||||
3332.0,
|
||||
3451.0,
|
||||
3619.0,
|
||||
3877.0,
|
||||
3903.0,
|
||||
4051.0,
|
||||
3885.0,
|
||||
3814.0,
|
||||
3604.0,
|
||||
3385.0,
|
||||
3074.0,
|
||||
2902.0,
|
||||
2613.0,
|
||||
2402.0,
|
||||
2127.0,
|
||||
1922.0,
|
||||
1855.0,
|
||||
1701.0,
|
||||
1610.0,
|
||||
1467.0,
|
||||
1489.0,
|
||||
1514.0,
|
||||
1444.0,
|
||||
1519.0,
|
||||
1478.0,
|
||||
1442.0,
|
||||
1531.0,
|
||||
1504.0,
|
||||
1641.0,
|
||||
1667.0,
|
||||
1845.0,
|
||||
1837.0,
|
||||
2052.0,
|
||||
2053.0,
|
||||
2149.0,
|
||||
2181.0,
|
||||
2211.0,
|
||||
2096.0,
|
||||
1974.0,
|
||||
1901.0,
|
||||
1720.0,
|
||||
1564.0,
|
||||
1336.0,
|
||||
1111.0,
|
||||
1043.0,
|
||||
860.0,
|
||||
717.0,
|
||||
659.0,
|
||||
529.0,
|
||||
457.0,
|
||||
424.0,
|
||||
298.0,
|
||||
294.0,
|
||||
233.0,
|
||||
236.0,
|
||||
208.0,
|
||||
193.0,
|
||||
164.0,
|
||||
147.0
|
||||
],
|
||||
"frozen": [
|
||||
499.0,
|
||||
919.0,
|
||||
1474.0,
|
||||
2347.0,
|
||||
3501.0,
|
||||
4958.0,
|
||||
6305.0,
|
||||
7819.0,
|
||||
9216.0,
|
||||
10548.0,
|
||||
11475.0,
|
||||
11827.0,
|
||||
12528.0,
|
||||
12831.0,
|
||||
13094.0,
|
||||
13244.0,
|
||||
12612.0,
|
||||
12130.0,
|
||||
11833.0,
|
||||
11677.0,
|
||||
11595.0,
|
||||
11501.0,
|
||||
11484.0,
|
||||
11467.0,
|
||||
11355.0,
|
||||
11578.0,
|
||||
11550.0,
|
||||
11746.0,
|
||||
11641.0,
|
||||
11955.0,
|
||||
12136.0,
|
||||
12360.0,
|
||||
12613.0,
|
||||
13087.0,
|
||||
13153.0,
|
||||
13663.0,
|
||||
13839.0,
|
||||
14464.0,
|
||||
15203.0,
|
||||
16257.0,
|
||||
17384.0,
|
||||
18726.0,
|
||||
20248.0,
|
||||
22251.0,
|
||||
24573.0,
|
||||
27580.0,
|
||||
32341.0,
|
||||
40818.0,
|
||||
58221.0,
|
||||
90432.0,
|
||||
148679.0,
|
||||
247063.0,
|
||||
397447.0,
|
||||
607938.0,
|
||||
877277.0,
|
||||
1178973.0,
|
||||
1480418.0,
|
||||
1724821.0,
|
||||
1875374.0,
|
||||
1898013.0,
|
||||
1802009.0,
|
||||
1612947.0,
|
||||
1371943.0,
|
||||
1118842.0,
|
||||
893348.0,
|
||||
698924.0,
|
||||
540935.0,
|
||||
415183.0,
|
||||
315525.0,
|
||||
238483.0,
|
||||
177935.0,
|
||||
133335.0,
|
||||
101985.0,
|
||||
80614.0,
|
||||
66308.0,
|
||||
57703.0,
|
||||
52552.0,
|
||||
48730.0,
|
||||
46712.0,
|
||||
45264.0,
|
||||
45478.0,
|
||||
46817.0,
|
||||
48749.0,
|
||||
52268.0,
|
||||
55751.0,
|
||||
59751.0,
|
||||
63128.0,
|
||||
64355.0,
|
||||
64278.0,
|
||||
61789.0,
|
||||
57781.0,
|
||||
52397.0,
|
||||
46550.0,
|
||||
41177.0,
|
||||
36929.0,
|
||||
32859.0,
|
||||
29645.0,
|
||||
27949.0,
|
||||
26098.0,
|
||||
25284.0,
|
||||
25947.0,
|
||||
24171.0,
|
||||
23310.0,
|
||||
23091.0,
|
||||
22982.0,
|
||||
22740.0,
|
||||
22928.0,
|
||||
23246.0,
|
||||
24413.0,
|
||||
26263.0,
|
||||
29181.0,
|
||||
33591.0,
|
||||
38978.0,
|
||||
45376.0,
|
||||
52423.0,
|
||||
59269.0,
|
||||
64779.0,
|
||||
67379.0,
|
||||
68667.0,
|
||||
65197.0,
|
||||
61403.0,
|
||||
54807.0,
|
||||
48118.0,
|
||||
41254.0,
|
||||
34742.0,
|
||||
29510.0,
|
||||
24474.0,
|
||||
19724.0,
|
||||
15963.0,
|
||||
12877.0,
|
||||
10251.0,
|
||||
8289.0,
|
||||
6654.0,
|
||||
5526.0,
|
||||
4723.0,
|
||||
4074.0,
|
||||
3730.0,
|
||||
3395.0,
|
||||
3280.0,
|
||||
3208.0,
|
||||
3247.0,
|
||||
3332.0,
|
||||
3451.0,
|
||||
3619.0,
|
||||
3877.0,
|
||||
3903.0,
|
||||
4051.0,
|
||||
3885.0,
|
||||
3814.0,
|
||||
3604.0,
|
||||
3385.0,
|
||||
3074.0,
|
||||
2902.0,
|
||||
2613.0,
|
||||
2402.0,
|
||||
2127.0,
|
||||
1922.0,
|
||||
1855.0,
|
||||
1701.0,
|
||||
1610.0,
|
||||
1467.0,
|
||||
1489.0,
|
||||
1514.0,
|
||||
1444.0,
|
||||
1519.0,
|
||||
1478.0,
|
||||
1442.0,
|
||||
1531.0,
|
||||
1504.0,
|
||||
1641.0,
|
||||
1667.0,
|
||||
1845.0,
|
||||
1837.0,
|
||||
2052.0,
|
||||
2053.0,
|
||||
2149.0,
|
||||
2181.0,
|
||||
2211.0,
|
||||
2096.0,
|
||||
1974.0,
|
||||
1901.0,
|
||||
1720.0,
|
||||
1564.0,
|
||||
1336.0,
|
||||
1111.0,
|
||||
1043.0,
|
||||
860.0,
|
||||
717.0,
|
||||
659.0,
|
||||
529.0,
|
||||
457.0,
|
||||
424.0,
|
||||
298.0,
|
||||
294.0,
|
||||
233.0,
|
||||
236.0,
|
||||
208.0,
|
||||
193.0,
|
||||
164.0,
|
||||
147.0
|
||||
],
|
||||
"cuda": [
|
||||
499.0,
|
||||
919.0,
|
||||
1474.0,
|
||||
2347.0,
|
||||
3501.0,
|
||||
4958.0,
|
||||
6304.0,
|
||||
7820.0,
|
||||
9216.0,
|
||||
10547.0,
|
||||
11476.0,
|
||||
11827.0,
|
||||
12527.0,
|
||||
12831.0,
|
||||
13095.0,
|
||||
13245.0,
|
||||
12611.0,
|
||||
12130.0,
|
||||
11833.0,
|
||||
11678.0,
|
||||
11594.0,
|
||||
11500.0,
|
||||
11484.0,
|
||||
11468.0,
|
||||
11355.0,
|
||||
11577.0,
|
||||
11553.0,
|
||||
11742.0,
|
||||
11642.0,
|
||||
11956.0,
|
||||
12136.0,
|
||||
12361.0,
|
||||
12610.0,
|
||||
13087.0,
|
||||
13155.0,
|
||||
13664.0,
|
||||
13837.0,
|
||||
14465.0,
|
||||
15204.0,
|
||||
16254.0,
|
||||
17386.0,
|
||||
18725.0,
|
||||
20248.0,
|
||||
22248.0,
|
||||
24576.0,
|
||||
27580.0,
|
||||
32341.0,
|
||||
40818.0,
|
||||
58220.0,
|
||||
90432.0,
|
||||
148681.0,
|
||||
247064.0,
|
||||
397436.0,
|
||||
607935.0,
|
||||
877264.0,
|
||||
1178956.0,
|
||||
1480420.0,
|
||||
1724770.0,
|
||||
1875420.0,
|
||||
1897953.0,
|
||||
1802017.0,
|
||||
1612959.0,
|
||||
1371965.0,
|
||||
1118856.0,
|
||||
893348.0,
|
||||
698930.0,
|
||||
540947.0,
|
||||
415204.0,
|
||||
315523.0,
|
||||
238491.0,
|
||||
177935.0,
|
||||
133333.0,
|
||||
101990.0,
|
||||
80619.0,
|
||||
66306.0,
|
||||
57704.0,
|
||||
52554.0,
|
||||
48728.0,
|
||||
46712.0,
|
||||
45263.0,
|
||||
45479.0,
|
||||
46816.0,
|
||||
48746.0,
|
||||
52273.0,
|
||||
55754.0,
|
||||
59745.0,
|
||||
63130.0,
|
||||
64358.0,
|
||||
64273.0,
|
||||
61792.0,
|
||||
57778.0,
|
||||
52400.0,
|
||||
46551.0,
|
||||
41180.0,
|
||||
36927.0,
|
||||
32858.0,
|
||||
29645.0,
|
||||
27949.0,
|
||||
26099.0,
|
||||
25286.0,
|
||||
25945.0,
|
||||
24172.0,
|
||||
23308.0,
|
||||
23091.0,
|
||||
22984.0,
|
||||
22737.0,
|
||||
22928.0,
|
||||
23248.0,
|
||||
24412.0,
|
||||
26263.0,
|
||||
29182.0,
|
||||
33592.0,
|
||||
38976.0,
|
||||
45376.0,
|
||||
52419.0,
|
||||
59270.0,
|
||||
64776.0,
|
||||
67382.0,
|
||||
68665.0,
|
||||
65199.0,
|
||||
61406.0,
|
||||
54805.0,
|
||||
48121.0,
|
||||
41250.0,
|
||||
34746.0,
|
||||
29508.0,
|
||||
24474.0,
|
||||
19725.0,
|
||||
15965.0,
|
||||
12876.0,
|
||||
10251.0,
|
||||
8289.0,
|
||||
6654.0,
|
||||
5527.0,
|
||||
4722.0,
|
||||
4075.0,
|
||||
3730.0,
|
||||
3395.0,
|
||||
3280.0,
|
||||
3208.0,
|
||||
3247.0,
|
||||
3332.0,
|
||||
3451.0,
|
||||
3619.0,
|
||||
3877.0,
|
||||
3903.0,
|
||||
4049.0,
|
||||
3887.0,
|
||||
3813.0,
|
||||
3605.0,
|
||||
3386.0,
|
||||
3073.0,
|
||||
2901.0,
|
||||
2614.0,
|
||||
2401.0,
|
||||
2128.0,
|
||||
1922.0,
|
||||
1855.0,
|
||||
1701.0,
|
||||
1610.0,
|
||||
1467.0,
|
||||
1489.0,
|
||||
1514.0,
|
||||
1444.0,
|
||||
1519.0,
|
||||
1478.0,
|
||||
1442.0,
|
||||
1531.0,
|
||||
1504.0,
|
||||
1641.0,
|
||||
1667.0,
|
||||
1845.0,
|
||||
1837.0,
|
||||
2051.0,
|
||||
2053.0,
|
||||
2150.0,
|
||||
2181.0,
|
||||
2211.0,
|
||||
2096.0,
|
||||
1974.0,
|
||||
1901.0,
|
||||
1720.0,
|
||||
1564.0,
|
||||
1336.0,
|
||||
1111.0,
|
||||
1043.0,
|
||||
860.0,
|
||||
717.0,
|
||||
659.0,
|
||||
529.0,
|
||||
457.0,
|
||||
424.0,
|
||||
298.0,
|
||||
294.0,
|
||||
233.0,
|
||||
236.0,
|
||||
208.0,
|
||||
193.0,
|
||||
164.0,
|
||||
147.0
|
||||
]
|
||||
},
|
||||
"edges": [
|
||||
-2.0,
|
||||
18.01,
|
||||
38.02,
|
||||
58.03,
|
||||
78.04,
|
||||
98.05,
|
||||
118.06,
|
||||
138.07,
|
||||
158.08,
|
||||
178.09,
|
||||
198.1,
|
||||
218.11,
|
||||
238.12,
|
||||
258.13,
|
||||
278.14,
|
||||
298.15,
|
||||
318.16,
|
||||
338.17,
|
||||
358.18,
|
||||
378.19,
|
||||
398.2,
|
||||
418.21,
|
||||
438.22,
|
||||
458.23,
|
||||
478.24,
|
||||
498.25,
|
||||
518.26,
|
||||
538.27,
|
||||
558.28,
|
||||
578.29,
|
||||
598.3,
|
||||
618.31,
|
||||
638.32,
|
||||
658.33,
|
||||
678.34,
|
||||
698.35,
|
||||
718.36,
|
||||
738.37,
|
||||
758.38,
|
||||
778.39,
|
||||
798.4,
|
||||
818.41,
|
||||
838.42,
|
||||
858.43,
|
||||
878.44,
|
||||
898.45,
|
||||
918.46,
|
||||
938.47,
|
||||
958.48,
|
||||
978.49,
|
||||
998.5,
|
||||
1018.51,
|
||||
1038.52,
|
||||
1058.53,
|
||||
1078.54,
|
||||
1098.55,
|
||||
1118.56,
|
||||
1138.57,
|
||||
1158.58,
|
||||
1178.59,
|
||||
1198.6,
|
||||
1218.61,
|
||||
1238.62,
|
||||
1258.63,
|
||||
1278.64,
|
||||
1298.65,
|
||||
1318.66,
|
||||
1338.67,
|
||||
1358.68,
|
||||
1378.69,
|
||||
1398.7,
|
||||
1418.71,
|
||||
1438.72,
|
||||
1458.73,
|
||||
1478.74,
|
||||
1498.75,
|
||||
1518.76,
|
||||
1538.77,
|
||||
1558.78,
|
||||
1578.79,
|
||||
1598.8,
|
||||
1618.81,
|
||||
1638.82,
|
||||
1658.83,
|
||||
1678.84,
|
||||
1698.85,
|
||||
1718.86,
|
||||
1738.87,
|
||||
1758.88,
|
||||
1778.89,
|
||||
1798.9,
|
||||
1818.91,
|
||||
1838.92,
|
||||
1858.93,
|
||||
1878.94,
|
||||
1898.95,
|
||||
1918.96,
|
||||
1938.97,
|
||||
1958.98,
|
||||
1978.99,
|
||||
1999.0,
|
||||
2019.01,
|
||||
2039.02,
|
||||
2059.03,
|
||||
2079.04,
|
||||
2099.05,
|
||||
2119.06,
|
||||
2139.07,
|
||||
2159.08,
|
||||
2179.09,
|
||||
2199.1,
|
||||
2219.11,
|
||||
2239.12,
|
||||
2259.13,
|
||||
2279.14,
|
||||
2299.15,
|
||||
2319.16,
|
||||
2339.17,
|
||||
2359.18,
|
||||
2379.19,
|
||||
2399.2,
|
||||
2419.21,
|
||||
2439.22,
|
||||
2459.23,
|
||||
2479.24,
|
||||
2499.25,
|
||||
2519.26,
|
||||
2539.27,
|
||||
2559.28,
|
||||
2579.29,
|
||||
2599.3,
|
||||
2619.31,
|
||||
2639.32,
|
||||
2659.33,
|
||||
2679.34,
|
||||
2699.35,
|
||||
2719.36,
|
||||
2739.37,
|
||||
2759.38,
|
||||
2779.39,
|
||||
2799.4,
|
||||
2819.41,
|
||||
2839.42,
|
||||
2859.43,
|
||||
2879.44,
|
||||
2899.45,
|
||||
2919.46,
|
||||
2939.47,
|
||||
2959.48,
|
||||
2979.49,
|
||||
2999.5,
|
||||
3019.51,
|
||||
3039.52,
|
||||
3059.53,
|
||||
3079.54,
|
||||
3099.55,
|
||||
3119.56,
|
||||
3139.57,
|
||||
3159.58,
|
||||
3179.59,
|
||||
3199.6,
|
||||
3219.61,
|
||||
3239.62,
|
||||
3259.63,
|
||||
3279.64,
|
||||
3299.65,
|
||||
3319.66,
|
||||
3339.67,
|
||||
3359.68,
|
||||
3379.69,
|
||||
3399.7,
|
||||
3419.71,
|
||||
3439.72,
|
||||
3459.73,
|
||||
3479.74,
|
||||
3499.75,
|
||||
3519.76,
|
||||
3539.77,
|
||||
3559.78,
|
||||
3579.79,
|
||||
3599.8,
|
||||
3619.81,
|
||||
3639.82,
|
||||
3659.83,
|
||||
3679.84,
|
||||
3699.85,
|
||||
3719.86,
|
||||
3739.87,
|
||||
3759.88,
|
||||
3779.89,
|
||||
3799.9,
|
||||
3819.91,
|
||||
3839.92,
|
||||
3859.93,
|
||||
3879.94,
|
||||
3899.95,
|
||||
3919.96,
|
||||
3939.97,
|
||||
3959.98,
|
||||
3979.99,
|
||||
4000.0
|
||||
]
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 217 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 373 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 66 KiB After Width: | Height: | Size: 116 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 113 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 71 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 36 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 48 KiB After Width: | Height: | Size: 51 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 124 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 113 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 55 KiB After Width: | Height: | Size: 86 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 80 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 33 KiB After Width: | Height: | Size: 75 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 62 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 106 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 72 KiB After Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 66 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 96 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 58 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 442 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 100 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 356 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 268 KiB |
+554
-123
@@ -53,6 +53,10 @@ class ClusterFinderCUDA {
|
||||
|
||||
// Per-slot state consumed by collect()
|
||||
bool m_slot_in_flight[NUM_SLOTS] = {false, false};
|
||||
// A BatchView handed out by collect_view() still points into this slot's
|
||||
// pinned buffer. The slot must not be reused until the view is released, so
|
||||
// submit_batch() refuses it rather than overwriting live results.
|
||||
bool m_slot_view_held[NUM_SLOTS] = {false, false};
|
||||
size_t m_slot_n_frames[NUM_SLOTS] = {0, 0};
|
||||
uint64_t m_slot_first_frame[NUM_SLOTS] = {0, 0};
|
||||
int m_slot_streams_used[NUM_SLOTS] = {0, 0};
|
||||
@@ -92,7 +96,18 @@ class ClusterFinderCUDA {
|
||||
float m_total_kernel_ms = 0.0f;
|
||||
size_t m_frames_processed = 0;
|
||||
|
||||
// Per-slot kernel timing event pools (sized lazily to the largest batch)
|
||||
// Opt-in kernel timing. Off by default: it costs two cudaEventRecord per
|
||||
// frame (real stream operations) plus one cudaEventElapsedTime query per
|
||||
// frame on the host, and the number it produces is only meaningful when
|
||||
// kernels cannot queue behind one another — see avg_kernel_time_ms().
|
||||
bool m_time_kernels = false;
|
||||
|
||||
// Frames per internally pipelined chunk in find_clusters_batched().
|
||||
// 0 = auto (batch split so one chunk is marshaled while the next runs).
|
||||
size_t m_batch_chunk = 0;
|
||||
|
||||
// Per-slot kernel timing event pools (sized lazily to the largest batch).
|
||||
// Left empty when m_time_kernels is false — no events are ever created.
|
||||
std::vector<cudaEvent_t> m_kernel_start_pools[NUM_SLOTS];
|
||||
std::vector<cudaEvent_t> m_kernel_stop_pools[NUM_SLOTS];
|
||||
|
||||
@@ -107,6 +122,132 @@ class ClusterFinderCUDA {
|
||||
dim3 block;
|
||||
size_t shmem_bytes;
|
||||
|
||||
/// Copy every frame of a pinned output slot into results, turning the
|
||||
/// packed device layout into owned ClusterVectors. This is exactly the work
|
||||
/// collect_view() exists to avoid.
|
||||
///
|
||||
/// Single-threaded on purpose. Spreading this copy over a thread pool was
|
||||
/// tried and reverted: the work is one 467 kB malloc + first-touch per
|
||||
/// frame at 9x9, so it is allocation-bound rather than bandwidth-bound.
|
||||
/// Extra threads get their own glibc arenas, which destroys heap reuse
|
||||
/// between calls and costs more in page faults than the parallel copy saves
|
||||
/// — measured 2.27 M faults at 8 threads vs 9.7 k at 1, for a 6 % gain at
|
||||
/// best and a 33 % *loss* when results are freed promptly. The fix is to
|
||||
/// stop allocating per frame, not to copy faster.
|
||||
void
|
||||
materialize_slot(const void *slot_base, size_t n_frames,
|
||||
std::vector<ClusterVector<ClusterType>> &results) const {
|
||||
for (size_t frame_idx = 0; frame_idx < n_frames; ++frame_idx) {
|
||||
const void *h_out = static_cast<const char *>(slot_base) +
|
||||
frame_idx * m_output_bytes_per_frame;
|
||||
uint32_t n_found = *reinterpret_cast<const uint32_t *>(h_out);
|
||||
// The device counter increments past the cap (only the write is
|
||||
// guarded), so this clamp is an out-of-bounds guard, not a tuning
|
||||
// choice.
|
||||
n_found = std::min<uint32_t>(
|
||||
n_found, static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
|
||||
if (n_found > 0) {
|
||||
const auto *src = reinterpret_cast<const ClusterType *>(
|
||||
static_cast<const char *>(h_out) + m_clusters_offset);
|
||||
results[frame_idx].resize(n_found);
|
||||
std::memcpy(results[frame_idx].data(), src,
|
||||
n_found * sizeof(ClusterType));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Default number of pipelined chunks per find_clusters_batched() call.
|
||||
/// Two competing costs set this:
|
||||
/// - fill/drain: the first submit and last collect are not overlapped,
|
||||
/// costing min(GPU, host) per batch = min/(C*max) of the total, so this
|
||||
/// term FALLS as C grows;
|
||||
/// - per-chunk tail: every chunk ends with all streams draining,
|
||||
/// ~n_streams * kernel_time of partly-idle GPU, so this term RISES
|
||||
/// linearly with C.
|
||||
/// Measured 9x9 rates (GPU 26.7 us/frame, host 58.8 us/frame, kernel
|
||||
/// 25.4 us, 2000 frames) put the sum at 23 % for C=2, 11 % for C=4,
|
||||
/// 6 % for C=8 and ~4 % for C=16-32 — a flat minimum past 16. C=8 sits
|
||||
/// inside the knee while keeping chunks large enough that each stream still
|
||||
/// gets a healthy number of frames. Override with set_batch_chunk() to
|
||||
/// measure rather than trust this.
|
||||
static constexpr size_t DEFAULT_BATCH_CHUNKS = 8;
|
||||
|
||||
/// Upper bound on one pinned output slot. cudaMallocHost is page-locked, so
|
||||
/// its first touch is charged to whoever triggers it — keep it modest.
|
||||
static constexpr size_t MAX_SLOT_BYTES = 128ull << 20; // 128 MiB
|
||||
|
||||
/// Frames per pipelined chunk in find_clusters_batched(). Rounded up to a
|
||||
/// multiple of n_streams so chunking never changes which stream a frame
|
||||
/// lands on (the device pedestal is per-stream, so a different assignment
|
||||
/// would mean a different pedestal state per frame — a correctness issue,
|
||||
/// not a tidiness one).
|
||||
size_t resolve_batch_chunk(size_t n_frames) const {
|
||||
const size_t ns = static_cast<size_t>(n_streams);
|
||||
size_t chunk = m_batch_chunk;
|
||||
if (chunk == 0) {
|
||||
chunk =
|
||||
(n_frames + DEFAULT_BATCH_CHUNKS - 1) / DEFAULT_BATCH_CHUNKS;
|
||||
// Keep at least a few frames per stream per chunk, or the per-chunk
|
||||
// drain dominates.
|
||||
const size_t min_chunk = ns * 4;
|
||||
if (chunk < min_chunk)
|
||||
chunk = min_chunk;
|
||||
// Cap by BYTES, not frames. Each slot is cudaMallocHost'd at
|
||||
// chunk * m_output_bytes_per_frame, and there are NUM_SLOTS of
|
||||
// them. Without this, find_clusters_batched(whole_array) scales the
|
||||
// pinned allocation with the array: 20 000 frames at 9x9 gives
|
||||
// 2500-frame chunks = 1.23 GB per slot, 2.46 GB pinned, whose
|
||||
// first-touch cost (~600 k page faults) lands inside the caller's
|
||||
// timed region. Bounding it makes one big call behave like a loop
|
||||
// over slices.
|
||||
const size_t max_by_bytes =
|
||||
std::max<size_t>(1, MAX_SLOT_BYTES / m_output_bytes_per_frame);
|
||||
if (chunk > max_by_bytes)
|
||||
chunk = std::max(max_by_bytes, ns);
|
||||
}
|
||||
chunk = ((chunk + ns - 1) / ns) * ns;
|
||||
return std::min(chunk, n_frames);
|
||||
}
|
||||
|
||||
/// Called by BatchView when it is released, making the slot reusable.
|
||||
void release_slot(int slot) {
|
||||
if (slot >= 0 && slot < NUM_SLOTS)
|
||||
m_slot_view_held[slot] = false;
|
||||
}
|
||||
|
||||
/// Wait for a slot's D2H to land and hand back its bookkeeping. Shared by
|
||||
/// collect() and collect_view() so the two cannot drift.
|
||||
void finish_slot(int slot, size_t &n_frames, uint64_t &first_frame) {
|
||||
if (!m_slot_in_flight[slot])
|
||||
throw std::runtime_error(
|
||||
"ClusterFinderCUDA: collect() called on a slot that is not "
|
||||
"in flight");
|
||||
n_frames = m_slot_n_frames[slot];
|
||||
first_frame = m_slot_first_frame[slot];
|
||||
// cudaEventSynchronize (not cudaStreamSynchronize) so a batch already
|
||||
// queued behind this one in the same streams is not waited on.
|
||||
for (int k = 0; k < m_slot_streams_used[slot]; ++k)
|
||||
CUDA_CHECK(cudaEventSynchronize(m_batch_done[slot][k]));
|
||||
accumulate_kernel_times(slot, n_frames);
|
||||
m_frames_processed += n_frames;
|
||||
m_slot_in_flight[slot] = false;
|
||||
}
|
||||
|
||||
/// Accumulate per-frame kernel times for a slot. Serial by construction:
|
||||
/// it mutates m_total_kernel_ms and queries CUDA events.
|
||||
void accumulate_kernel_times(int slot, size_t n_frames) {
|
||||
if (!m_time_kernels)
|
||||
return;
|
||||
for (size_t frame_idx = 0; frame_idx < n_frames; ++frame_idx) {
|
||||
float kernel_ms = 0.0f;
|
||||
CUDA_CHECK(cudaEventElapsedTime(
|
||||
&kernel_ms, m_kernel_start_pools[slot][frame_idx],
|
||||
m_kernel_stop_pools[slot][frame_idx]));
|
||||
m_total_kernel_ms += kernel_ms;
|
||||
}
|
||||
}
|
||||
|
||||
public:
|
||||
/**
|
||||
* @brief Opaque handle returned by submit_batch(). Pass to collect().
|
||||
@@ -115,6 +256,130 @@ class ClusterFinderCUDA {
|
||||
int slot;
|
||||
};
|
||||
|
||||
using value_type = typename ClusterType::value_type;
|
||||
|
||||
/**
|
||||
* @brief Zero-copy view of a batch, still in the pinned D2H buffer.
|
||||
*
|
||||
* The D2H staging buffer already holds correctly laid-out clusters at a
|
||||
* fixed per-frame stride, so there is nothing to copy: this just exposes
|
||||
* offsets into it. Host cost per frame drops to reading one counter.
|
||||
*
|
||||
* @warning The view borrows the finder's slot. It stays valid until
|
||||
* release() (or destruction); until then submit_batch() will refuse that
|
||||
* slot rather than overwrite live results. With NUM_SLOTS == 2 that means
|
||||
* you must finish with a view before submitting two more batches — the
|
||||
* intended pattern is consume-then-release, one chunk at a time.
|
||||
*/
|
||||
class BatchView {
|
||||
friend class ClusterFinderCUDA;
|
||||
|
||||
ClusterFinderCUDA *m_owner = nullptr;
|
||||
const uint8_t *m_base = nullptr;
|
||||
size_t m_n_frames = 0;
|
||||
uint64_t m_first_frame = 0;
|
||||
size_t m_stride = 0; // bytes per frame in the pinned buffer
|
||||
size_t m_cl_offset = 0; // byte offset of the cluster array in a frame
|
||||
size_t m_max_clusters = 0;
|
||||
int m_slot = -1;
|
||||
|
||||
BatchView(ClusterFinderCUDA *owner, const void *base, size_t n_frames,
|
||||
uint64_t first_frame, size_t stride, size_t cl_offset,
|
||||
size_t max_clusters, int slot)
|
||||
: m_owner(owner), m_base(static_cast<const uint8_t *>(base)),
|
||||
m_n_frames(n_frames), m_first_frame(first_frame),
|
||||
m_stride(stride), m_cl_offset(cl_offset),
|
||||
m_max_clusters(max_clusters), m_slot(slot) {}
|
||||
|
||||
public:
|
||||
BatchView() = default;
|
||||
BatchView(const BatchView &) = delete;
|
||||
BatchView &operator=(const BatchView &) = delete;
|
||||
BatchView(BatchView &&o) noexcept { steal(o); }
|
||||
BatchView &operator=(BatchView &&o) noexcept {
|
||||
if (this != &o) {
|
||||
release();
|
||||
steal(o);
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
~BatchView() { release(); }
|
||||
|
||||
/// Give the slot back. Idempotent; the view is unusable afterwards.
|
||||
void release() {
|
||||
if (m_owner)
|
||||
m_owner->release_slot(m_slot);
|
||||
m_owner = nullptr;
|
||||
m_base = nullptr;
|
||||
m_n_frames = 0;
|
||||
m_slot = -1;
|
||||
}
|
||||
|
||||
bool valid() const { return m_base != nullptr; }
|
||||
size_t n_frames() const { return m_n_frames; }
|
||||
uint64_t first_frame() const { return m_first_frame; }
|
||||
|
||||
uint32_t count(size_t i) const {
|
||||
check(i);
|
||||
uint32_t n =
|
||||
*reinterpret_cast<const uint32_t *>(m_base + i * m_stride);
|
||||
// The device counter increments past the cap (only the write is
|
||||
// guarded), so clamp: this is an out-of-bounds guard.
|
||||
return std::min<uint32_t>(n, static_cast<uint32_t>(m_max_clusters));
|
||||
}
|
||||
|
||||
const ClusterType *clusters(size_t i) const {
|
||||
check(i);
|
||||
return reinterpret_cast<const ClusterType *>(m_base + i * m_stride +
|
||||
m_cl_offset);
|
||||
}
|
||||
|
||||
size_t total_clusters() const {
|
||||
size_t n = 0;
|
||||
for (size_t i = 0; i < m_n_frames; ++i)
|
||||
n += count(i);
|
||||
return n;
|
||||
}
|
||||
|
||||
/// Per-cluster sums for the whole batch, concatenated frame by frame.
|
||||
/// The common reduction, done here so callers never have to materialise
|
||||
/// the clusters themselves.
|
||||
std::vector<value_type> sums() const {
|
||||
std::vector<value_type> out;
|
||||
out.reserve(total_clusters());
|
||||
for (size_t i = 0; i < m_n_frames; ++i) {
|
||||
const ClusterType *c = clusters(i);
|
||||
const uint32_t n = count(i);
|
||||
for (uint32_t j = 0; j < n; ++j)
|
||||
out.push_back(c[j].sum());
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private:
|
||||
void steal(BatchView &o) {
|
||||
m_owner = o.m_owner;
|
||||
m_base = o.m_base;
|
||||
m_n_frames = o.m_n_frames;
|
||||
m_first_frame = o.m_first_frame;
|
||||
m_stride = o.m_stride;
|
||||
m_cl_offset = o.m_cl_offset;
|
||||
m_max_clusters = o.m_max_clusters;
|
||||
m_slot = o.m_slot;
|
||||
o.m_owner = nullptr;
|
||||
o.m_base = nullptr;
|
||||
o.m_slot = -1;
|
||||
}
|
||||
void check(size_t i) const {
|
||||
if (!m_base)
|
||||
throw std::runtime_error(
|
||||
"ClusterFinderCUDA::BatchView: view has been released");
|
||||
if (i >= m_n_frames)
|
||||
throw std::out_of_range(
|
||||
"ClusterFinderCUDA::BatchView: frame index out of range");
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* @brief Construct a ClusterFinderCUDA
|
||||
*
|
||||
@@ -125,13 +390,19 @@ class ClusterFinderCUDA {
|
||||
* fixed-size D2H
|
||||
* @param n_streams_ number of CUDA streams for multi-frame
|
||||
* overlap
|
||||
* @param time_kernels enable per-frame CUDA-event kernel
|
||||
* timing. Off by default: it adds two event records per frame to the
|
||||
* streams and one host-side query per frame, and the resulting number is
|
||||
* only meaningful at n_streams == 1 (see avg_kernel_time_ms()).
|
||||
*/
|
||||
ClusterFinderCUDA(Shape<2> shape_, COMPUTE_TYPE nSigma = 5.0,
|
||||
size_t max_clusters_per_frame = 2048, int n_streams_ = 5)
|
||||
size_t max_clusters_per_frame = 2048, int n_streams_ = 5,
|
||||
bool time_kernels = false)
|
||||
: m_shape(shape_), nrows(shape_[0]), ncols(shape_[1]),
|
||||
m_image_size(nrows * ncols), n_streams(n_streams_),
|
||||
m_max_clusters_per_frame(max_clusters_per_frame), m_nSigma(nSigma),
|
||||
m_pedestal(shape_[0], shape_[1]), m_clusters(max_clusters_per_frame) {
|
||||
m_pedestal(shape_[0], shape_[1]), m_clusters(max_clusters_per_frame),
|
||||
m_time_kernels(time_kernels) {
|
||||
if (n_streams_ <= 0) {
|
||||
throw std::invalid_argument(
|
||||
"ClusterFinderCUDA: n_streams must be > 0");
|
||||
@@ -407,22 +678,20 @@ class ClusterFinderCUDA {
|
||||
throw std::runtime_error(
|
||||
"ClusterFinderCUDA: both batch slots are in flight — call "
|
||||
"collect() before submitting a third batch");
|
||||
// Inert unless collect_view() was used: a live BatchView still points
|
||||
// into this slot's pinned buffer, so reusing it would overwrite results
|
||||
// the caller is still reading.
|
||||
if (m_slot_view_held[slot])
|
||||
throw std::runtime_error(
|
||||
"ClusterFinderCUDA: this batch slot is still held by a "
|
||||
"BatchView — release() it before submitting another batch");
|
||||
m_next_slot = 1 - slot;
|
||||
|
||||
const size_t n_frames_batch = static_cast<size_t>(frames.shape(0));
|
||||
const uint32_t n_pd_samples =
|
||||
static_cast<uint32_t>(m_pedestal.n_samples());
|
||||
|
||||
// Grow pinned D2H output buffer for this slot if needed
|
||||
if (n_frames_batch > m_output_slot_capacity[slot]) {
|
||||
if (h_output_slots[slot])
|
||||
CUDA_CHECK(cudaFreeHost(h_output_slots[slot]));
|
||||
CUDA_CHECK(
|
||||
cudaMallocHost(&h_output_slots[slot],
|
||||
n_frames_batch * m_output_bytes_per_frame));
|
||||
m_output_slot_capacity[slot] = n_frames_batch;
|
||||
}
|
||||
|
||||
grow_output_slot(slot, n_frames_batch);
|
||||
ensure_event_pool(slot, n_frames_batch);
|
||||
|
||||
// Launch all frames round-robin across streams
|
||||
@@ -439,16 +708,18 @@ class ClusterFinderCUDA {
|
||||
|
||||
auto *d_clusters = reinterpret_cast<ClusterType *>(
|
||||
sc.d_output + m_clusters_offset);
|
||||
CUDA_CHECK(cudaEventRecord(m_kernel_start_pools[slot][frame_idx],
|
||||
sc.stream));
|
||||
if (m_time_kernels)
|
||||
CUDA_CHECK(cudaEventRecord(
|
||||
m_kernel_start_pools[slot][frame_idx], sc.stream));
|
||||
device::find_clusters_in_single_frame<ClusterType, FRAME_TYPE>
|
||||
<<<grid, block, shmem_bytes, sc.stream>>>(
|
||||
sc.d_frame, sc.d_pd_mean, sc.d_pd_sum, sc.d_pd_sum2,
|
||||
sc.d_pd_off, n_pd_samples, m_nSigma, nrows, ncols,
|
||||
d_clusters, d_cluster_count,
|
||||
static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
CUDA_CHECK(cudaEventRecord(m_kernel_stop_pools[slot][frame_idx],
|
||||
sc.stream));
|
||||
if (m_time_kernels)
|
||||
CUDA_CHECK(cudaEventRecord(m_kernel_stop_pools[slot][frame_idx],
|
||||
sc.stream));
|
||||
CUDA_CHECK(cudaGetLastError());
|
||||
|
||||
void *h_out = static_cast<char *>(h_output_slots[slot]) +
|
||||
@@ -506,140 +777,286 @@ class ClusterFinderCUDA {
|
||||
results.back().set_frame_number(first_frame + i);
|
||||
}
|
||||
|
||||
for (size_t frame_idx = 0; frame_idx < n_frames_batch; ++frame_idx) {
|
||||
const void *h_out =
|
||||
static_cast<const char *>(h_output_slots[slot]) +
|
||||
frame_idx * m_output_bytes_per_frame;
|
||||
uint32_t n_found = *reinterpret_cast<const uint32_t *>(h_out);
|
||||
n_found = std::min<uint32_t>(
|
||||
n_found, static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
// for (size_t frame_idx = 0; frame_idx < n_frames_batch; ++frame_idx) {
|
||||
// const void *h_out =
|
||||
// static_cast<const char *>(h_output_slots[slot]) +
|
||||
// frame_idx * m_output_bytes_per_frame;
|
||||
// uint32_t n_found = *reinterpret_cast<const uint32_t *>(h_out);
|
||||
// n_found = std::min<uint32_t>(
|
||||
// n_found, static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
|
||||
if (n_found > 0) {
|
||||
const auto *src = reinterpret_cast<const ClusterType *>(
|
||||
static_cast<const char *>(h_out) + m_clusters_offset);
|
||||
results[frame_idx].resize(n_found);
|
||||
std::memcpy(results[frame_idx].data(), src,
|
||||
n_found * sizeof(ClusterType));
|
||||
}
|
||||
// if (n_found > 0) {
|
||||
// const auto *src = reinterpret_cast<const ClusterType *>(
|
||||
// static_cast<const char *>(h_out) + m_clusters_offset);
|
||||
// results[frame_idx].resize(n_found);
|
||||
// std::memcpy(results[frame_idx].data(), src,
|
||||
// n_found * sizeof(ClusterType));
|
||||
// }
|
||||
|
||||
float kernel_ms = 0.0f;
|
||||
CUDA_CHECK(cudaEventElapsedTime(
|
||||
&kernel_ms, m_kernel_start_pools[slot][frame_idx],
|
||||
m_kernel_stop_pools[slot][frame_idx]));
|
||||
m_total_kernel_ms += kernel_ms;
|
||||
}
|
||||
// float kernel_ms = 0.0f;
|
||||
// CUDA_CHECK(cudaEventElapsedTime(
|
||||
// &kernel_ms, m_kernel_start_pools[slot][frame_idx],
|
||||
// m_kernel_stop_pools[slot][frame_idx]));
|
||||
// m_total_kernel_ms += kernel_ms;
|
||||
// }
|
||||
|
||||
// materialize_slot() is the original copy loop verbatim, just hoisted
|
||||
// into a helper so find_clusters_batched() shares it;
|
||||
// accumulate_kernel_times is the m_time_kernels branch that used to
|
||||
// live in the same loop.
|
||||
materialize_slot(h_output_slots[slot], n_frames_batch, results);
|
||||
accumulate_kernel_times(slot, n_frames_batch);
|
||||
|
||||
m_frames_processed += n_frames_batch;
|
||||
m_slot_in_flight[slot] = false;
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Collect a batch as a zero-copy view — no allocation, no copy.
|
||||
*
|
||||
* Nothing is copied and nothing is allocated: the returned view points into
|
||||
* the finder's pinned D2H buffer. The slot is held until the view is
|
||||
* released, so at most NUM_SLOTS - 1 further batches can be submitted while
|
||||
* it is alive. Consume, then release.
|
||||
*/
|
||||
BatchView collect_view(BatchToken token) {
|
||||
const int slot = token.slot;
|
||||
size_t n_frames = 0;
|
||||
uint64_t first_frame = 0;
|
||||
finish_slot(slot, n_frames, first_frame);
|
||||
|
||||
m_slot_view_held[slot] = true;
|
||||
return BatchView(this, h_output_slots[slot], n_frames, first_frame,
|
||||
m_output_bytes_per_frame, m_clusters_offset,
|
||||
m_max_clusters_per_frame, slot);
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Synchronous batched cluster finding across multiple frames, using
|
||||
* n_streams CUDA streams to overlap H2D, kernel, and D2H.
|
||||
*
|
||||
* Returns one ClusterVector per input frame (with frame_number set to
|
||||
* first_frame + i). Does not go through submit_batch/collect so it carries
|
||||
* no async-slot overhead.
|
||||
* first_frame + i).
|
||||
*
|
||||
* Internally the batch is split into chunks and pipelined over the two
|
||||
* async slots: chunk i+1 is submitted before chunk i is collected, so the
|
||||
* host marshals one chunk while the GPU runs the next. A single call used
|
||||
* to be strictly `run the whole batch, then copy the whole batch`, leaving
|
||||
* the GPU idle for the entire host phase.
|
||||
*
|
||||
* Chunk size is rounded up to a multiple of n_streams so the frame->stream
|
||||
* assignment is identical to processing the batch in one go. That matters
|
||||
* for correctness, not just tidiness: the device pedestal is per-stream, so
|
||||
* changing which stream a frame lands on would change the pedestal state it
|
||||
* is evaluated against.
|
||||
*
|
||||
* @note Now shares the two batch slots with submit_batch()/collect(). A
|
||||
* batch already in flight is no longer silently overwritten — you get
|
||||
* an exception instead.
|
||||
*/
|
||||
std::vector<ClusterVector<ClusterType>>
|
||||
find_clusters_batched(NDView<FRAME_TYPE, 3> frames,
|
||||
uint64_t first_frame = 0) {
|
||||
if (m_pedestal_dirty) {
|
||||
sync_pedestal_to_device();
|
||||
m_pedestal_dirty = false;
|
||||
}
|
||||
|
||||
const size_t n_frames_batch = static_cast<size_t>(frames.shape(0));
|
||||
const uint32_t n_pd_samples =
|
||||
static_cast<uint32_t>(m_pedestal.n_samples());
|
||||
if (n_frames_batch == 0)
|
||||
return {};
|
||||
|
||||
// Lazy grow D2H output staging buffer (one slot per frame)
|
||||
if (n_frames_batch > m_output_slot_capacity[0]) {
|
||||
if (h_output_slots[0])
|
||||
CUDA_CHECK(cudaFreeHost(h_output_slots[0]));
|
||||
CUDA_CHECK(cudaMallocHost(
|
||||
&h_output_slots[0], n_frames_batch * m_output_bytes_per_frame));
|
||||
m_output_slot_capacity[0] = n_frames_batch;
|
||||
}
|
||||
|
||||
ensure_event_pool(0, n_frames_batch);
|
||||
const size_t chunk = resolve_batch_chunk(n_frames_batch);
|
||||
|
||||
std::vector<ClusterVector<ClusterType>> results;
|
||||
results.reserve(n_frames_batch);
|
||||
for (size_t i = 0; i < n_frames_batch; ++i) {
|
||||
results.emplace_back();
|
||||
results.back().set_frame_number(first_frame + i);
|
||||
|
||||
auto drain = [&results](std::vector<ClusterVector<ClusterType>> part) {
|
||||
for (auto &cv : part)
|
||||
results.push_back(std::move(cv));
|
||||
};
|
||||
|
||||
BatchToken tok = submit_batch(
|
||||
frames.sub_view(
|
||||
0, static_cast<ssize_t>(std::min(chunk, n_frames_batch))),
|
||||
first_frame);
|
||||
for (size_t b = chunk; b < n_frames_batch; b += chunk) {
|
||||
const size_t e = std::min(b + chunk, n_frames_batch);
|
||||
// Submit before collecting: the GPU starts chunk b while the host
|
||||
// is still copying chunk b - chunk out of the pinned slot.
|
||||
BatchToken nxt =
|
||||
submit_batch(frames.sub_view(static_cast<ssize_t>(b),
|
||||
static_cast<ssize_t>(e)),
|
||||
first_frame + b);
|
||||
drain(collect(tok));
|
||||
tok = nxt;
|
||||
}
|
||||
drain(collect(tok));
|
||||
|
||||
// Launch all frames round-robin across streams.
|
||||
// If the caller has called register_input_buffer() on frames.data(),
|
||||
// H2D runs at pinned DMA bandwidth (~22 GB/s); otherwise the CUDA
|
||||
// driver stages it internally (~15 GB/s for pageable memory).
|
||||
for (size_t frame_idx = 0; frame_idx < n_frames_batch; ++frame_idx) {
|
||||
auto &sc = v_sc[frame_idx % n_streams];
|
||||
|
||||
const FRAME_TYPE *h_src = frames.data() + frame_idx * m_image_size;
|
||||
auto *d_cluster_count = reinterpret_cast<uint32_t *>(sc.d_output);
|
||||
|
||||
CUDA_CHECK(cudaMemsetAsync(d_cluster_count, 0, sizeof(uint32_t),
|
||||
sc.stream));
|
||||
CUDA_CHECK(cudaMemcpyAsync(sc.d_frame, h_src, m_image_bytes,
|
||||
cudaMemcpyHostToDevice, sc.stream));
|
||||
|
||||
auto *d_clusters = reinterpret_cast<ClusterType *>(
|
||||
sc.d_output + m_clusters_offset);
|
||||
CUDA_CHECK(
|
||||
cudaEventRecord(m_kernel_start_pools[0][frame_idx], sc.stream));
|
||||
device::find_clusters_in_single_frame<ClusterType, FRAME_TYPE>
|
||||
<<<grid, block, shmem_bytes, sc.stream>>>(
|
||||
sc.d_frame, sc.d_pd_mean, sc.d_pd_sum, sc.d_pd_sum2,
|
||||
sc.d_pd_off, n_pd_samples, m_nSigma, nrows, ncols,
|
||||
d_clusters, d_cluster_count,
|
||||
static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
CUDA_CHECK(
|
||||
cudaEventRecord(m_kernel_stop_pools[0][frame_idx], sc.stream));
|
||||
CUDA_CHECK(cudaGetLastError());
|
||||
|
||||
void *h_out = static_cast<char *>(h_output_slots[0]) +
|
||||
frame_idx * m_output_bytes_per_frame;
|
||||
CUDA_CHECK(cudaMemcpyAsync(h_out, sc.d_output,
|
||||
m_output_bytes_per_frame,
|
||||
cudaMemcpyDeviceToHost, sc.stream));
|
||||
}
|
||||
|
||||
const int streams_used =
|
||||
std::min<int>(n_streams, static_cast<int>(n_frames_batch));
|
||||
for (int k = 0; k < streams_used; ++k)
|
||||
CUDA_CHECK(cudaStreamSynchronize(v_sc[k].stream));
|
||||
|
||||
for (size_t frame_idx = 0; frame_idx < n_frames_batch; ++frame_idx) {
|
||||
const void *h_out = static_cast<const char *>(h_output_slots[0]) +
|
||||
frame_idx * m_output_bytes_per_frame;
|
||||
uint32_t n_found = *reinterpret_cast<const uint32_t *>(h_out);
|
||||
n_found = std::min<uint32_t>(
|
||||
n_found, static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
|
||||
if (n_found > 0) {
|
||||
const auto *src = reinterpret_cast<const ClusterType *>(
|
||||
static_cast<const char *>(h_out) + m_clusters_offset);
|
||||
results[frame_idx].resize(n_found);
|
||||
std::memcpy(results[frame_idx].data(), src,
|
||||
n_found * sizeof(ClusterType));
|
||||
}
|
||||
|
||||
float kernel_ms = 0.0f;
|
||||
CUDA_CHECK(cudaEventElapsedTime(&kernel_ms,
|
||||
m_kernel_start_pools[0][frame_idx],
|
||||
m_kernel_stop_pools[0][frame_idx]));
|
||||
m_total_kernel_ms += kernel_ms;
|
||||
}
|
||||
|
||||
m_frames_processed += n_frames_batch;
|
||||
return results;
|
||||
}
|
||||
|
||||
// Previous implementation: one launch loop over the whole batch, one
|
||||
// cudaStreamSynchronize per stream, then one single-threaded copy loop over
|
||||
// every frame. Kept for reference — it is what the numbers in
|
||||
// docs/ClusterFinderCUDA_benchmark_results.md opt3/opt4 (Act I, sections
|
||||
// 5-6) were measured against.
|
||||
//
|
||||
// std::vector<ClusterVector<ClusterType>>
|
||||
// find_clusters_batched(NDView<FRAME_TYPE, 3> frames,
|
||||
// uint64_t first_frame = 0) {
|
||||
// if (m_pedestal_dirty) {
|
||||
// sync_pedestal_to_device();
|
||||
// m_pedestal_dirty = false;
|
||||
// }
|
||||
//
|
||||
// const size_t n_frames_batch =
|
||||
// static_cast<size_t>(frames.shape(0));
|
||||
// const uint32_t n_pd_samples =
|
||||
// static_cast<uint32_t>(m_pedestal.n_samples());
|
||||
//
|
||||
// // Lazy grow D2H output staging buffer (one slot per frame)
|
||||
// if (n_frames_batch > m_output_slot_capacity[0]) {
|
||||
// if (h_output_slots[0])
|
||||
// CUDA_CHECK(cudaFreeHost(h_output_slots[0]));
|
||||
// CUDA_CHECK(cudaMallocHost(&h_output_slots[0],
|
||||
// n_frames_batch *
|
||||
// m_output_bytes_per_frame));
|
||||
// m_output_slot_capacity[0] = n_frames_batch;
|
||||
// }
|
||||
//
|
||||
// ensure_event_pool(0, n_frames_batch);
|
||||
//
|
||||
// std::vector<ClusterVector<ClusterType>> results;
|
||||
// results.reserve(n_frames_batch);
|
||||
// for (size_t i = 0; i < n_frames_batch; ++i) {
|
||||
// results.emplace_back();
|
||||
// results.back().set_frame_number(first_frame + i);
|
||||
// }
|
||||
//
|
||||
// for (size_t frame_idx = 0; frame_idx < n_frames_batch;
|
||||
// ++frame_idx) {
|
||||
// auto &sc = v_sc[frame_idx % n_streams];
|
||||
//
|
||||
// const FRAME_TYPE *h_src =
|
||||
// frames.data() + frame_idx * m_image_size;
|
||||
// auto *d_cluster_count =
|
||||
// reinterpret_cast<uint32_t *>(sc.d_output);
|
||||
//
|
||||
// CUDA_CHECK(cudaMemsetAsync(d_cluster_count, 0,
|
||||
// sizeof(uint32_t), sc.stream));
|
||||
// CUDA_CHECK(cudaMemcpyAsync(sc.d_frame, h_src, m_image_bytes,
|
||||
// cudaMemcpyHostToDevice, sc.stream));
|
||||
//
|
||||
// auto *d_clusters = reinterpret_cast<ClusterType *>(
|
||||
// sc.d_output + m_clusters_offset);
|
||||
// if (m_time_kernels)
|
||||
// CUDA_CHECK(cudaEventRecord(
|
||||
// m_kernel_start_pools[0][frame_idx], sc.stream));
|
||||
// device::find_clusters_in_single_frame<ClusterType, FRAME_TYPE>
|
||||
// <<<grid, block, shmem_bytes, sc.stream>>>(
|
||||
// sc.d_frame, sc.d_pd_mean, sc.d_pd_sum, sc.d_pd_sum2,
|
||||
// sc.d_pd_off, n_pd_samples, m_nSigma, nrows, ncols,
|
||||
// d_clusters, d_cluster_count,
|
||||
// static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
// if (m_time_kernels)
|
||||
// CUDA_CHECK(cudaEventRecord(
|
||||
// m_kernel_stop_pools[0][frame_idx], sc.stream));
|
||||
// CUDA_CHECK(cudaGetLastError());
|
||||
//
|
||||
// void *h_out = static_cast<char *>(h_output_slots[0]) +
|
||||
// frame_idx * m_output_bytes_per_frame;
|
||||
// CUDA_CHECK(cudaMemcpyAsync(h_out, sc.d_output,
|
||||
// m_output_bytes_per_frame,
|
||||
// cudaMemcpyDeviceToHost, sc.stream));
|
||||
// }
|
||||
//
|
||||
// const int streams_used =
|
||||
// std::min<int>(n_streams, static_cast<int>(n_frames_batch));
|
||||
// for (int k = 0; k < streams_used; ++k)
|
||||
// CUDA_CHECK(cudaStreamSynchronize(v_sc[k].stream));
|
||||
//
|
||||
// for (size_t frame_idx = 0; frame_idx < n_frames_batch;
|
||||
// ++frame_idx) {
|
||||
// const void *h_out =
|
||||
// static_cast<const char *>(h_output_slots[0]) +
|
||||
// frame_idx * m_output_bytes_per_frame;
|
||||
// uint32_t n_found = *reinterpret_cast<const uint32_t *>(h_out);
|
||||
// n_found = std::min<uint32_t>(
|
||||
// n_found, static_cast<uint32_t>(m_max_clusters_per_frame));
|
||||
//
|
||||
// if (n_found > 0) {
|
||||
// const auto *src = reinterpret_cast<const ClusterType *>(
|
||||
// static_cast<const char *>(h_out) + m_clusters_offset);
|
||||
// results[frame_idx].resize(n_found);
|
||||
// std::memcpy(results[frame_idx].data(), src,
|
||||
// n_found * sizeof(ClusterType));
|
||||
// }
|
||||
//
|
||||
// if (m_time_kernels) {
|
||||
// float kernel_ms = 0.0f;
|
||||
// CUDA_CHECK(cudaEventElapsedTime(
|
||||
// &kernel_ms, m_kernel_start_pools[0][frame_idx],
|
||||
// m_kernel_stop_pools[0][frame_idx]));
|
||||
// m_total_kernel_ms += kernel_ms;
|
||||
// }
|
||||
// }
|
||||
//
|
||||
// m_frames_processed += n_frames_batch;
|
||||
// return results;
|
||||
// }
|
||||
|
||||
/// True if per-frame kernel timing was enabled at construction.
|
||||
bool kernel_timing_enabled() const { return m_time_kernels; }
|
||||
|
||||
/**
|
||||
* @brief Frames per internally pipelined chunk in find_clusters_batched().
|
||||
*
|
||||
* 0 (default) = auto: the batch is split into ~8 chunks so the host can
|
||||
* marshal one chunk while the GPU runs the next. Rounded up to a multiple
|
||||
* of n_streams. Set equal to the batch size to disable chunking and get
|
||||
* the old submit-everything-then-copy-everything behaviour.
|
||||
*/
|
||||
void set_batch_chunk(size_t n) { m_batch_chunk = n; }
|
||||
size_t get_batch_chunk() const { return m_batch_chunk; }
|
||||
|
||||
/// The chunk size find_clusters_batched() would use for n_frames. Exposed
|
||||
/// so a caller driving submit/collect_view by hand can match its pipelining
|
||||
/// without duplicating the rounding rules.
|
||||
size_t chunk_size_for(size_t n_frames) const {
|
||||
return resolve_batch_chunk(n_frames);
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Pre-allocate both pinned output slots for batches of n_frames.
|
||||
*
|
||||
* Processes nothing: no frame is transferred, no kernel is launched, and
|
||||
* the pedestal is untouched. Only the two cudaMallocHost calls that
|
||||
* submit_batch() would otherwise make on its first invocation happen here.
|
||||
*
|
||||
* The point is that page-locking is expensive and is charged to whoever
|
||||
* triggers it: measured ~1.0 us per 4 kB page, i.e. ~66 ms for two 128 MiB
|
||||
* slots (and ~40 % more if an undersized slot has to be freed first). Left
|
||||
* to submit_batch() that cost lands inside the first timed region — worth
|
||||
* 2.8 us/frame over 20 000 frames at 3x3, which is 17 % of the 16.2 us
|
||||
* roofline. Call this before starting a timer, or before a
|
||||
* latency-sensitive first batch. Slots only ever grow, so a later smaller
|
||||
* batch is free and a larger one still re-allocates: pass the largest batch
|
||||
* you intend to use, e.g. chunk_size_for(n) when driving
|
||||
* find_clusters_batched() or the submit/collect_view loop over n frames.
|
||||
*/
|
||||
void reserve_output_slots(size_t n_frames) {
|
||||
for (int slot = 0; slot < NUM_SLOTS; ++slot)
|
||||
grow_output_slot(slot, n_frames);
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Average per-frame kernel time in ms, or NaN if timing is disabled.
|
||||
*
|
||||
* @warning Only meaningful at n_streams == 1. The CUDA events bracket the
|
||||
* kernel on its own stream, so under multi-stream contention the measured
|
||||
* interval includes time queued behind kernels from other streams — it
|
||||
* over-reads by up to ~3.5x. Use Nsight Systems for exclusive kernel times.
|
||||
*/
|
||||
float avg_kernel_time_ms() const {
|
||||
if (!m_time_kernels)
|
||||
return std::numeric_limits<float>::quiet_NaN();
|
||||
return m_frames_processed > 0 ? m_total_kernel_ms / m_frames_processed
|
||||
: 0.0f;
|
||||
}
|
||||
@@ -704,7 +1121,21 @@ class ClusterFinderCUDA {
|
||||
CUDA_CHECK(cudaStreamSynchronize(sc.stream));
|
||||
}
|
||||
|
||||
/// Grow one slot's pinned D2H buffer to hold n_frames. Only ever grows, so
|
||||
/// a run whose batches keep the same shape allocates once.
|
||||
void grow_output_slot(int slot, size_t n_frames) {
|
||||
if (n_frames <= m_output_slot_capacity[slot])
|
||||
return;
|
||||
if (h_output_slots[slot])
|
||||
CUDA_CHECK(cudaFreeHost(h_output_slots[slot]));
|
||||
CUDA_CHECK(cudaMallocHost(&h_output_slots[slot],
|
||||
n_frames * m_output_bytes_per_frame));
|
||||
m_output_slot_capacity[slot] = n_frames;
|
||||
}
|
||||
|
||||
void ensure_event_pool(int slot, size_t n_frames) {
|
||||
if (!m_time_kernels)
|
||||
return; // no events are created when timing is disabled
|
||||
const size_t old_size = m_kernel_start_pools[slot].size();
|
||||
if (n_frames <= old_size)
|
||||
return;
|
||||
|
||||
@@ -67,6 +67,14 @@ class ClusterFinderCUDAOpt2 {
|
||||
float m_total_kernel_ms = 0.0f;
|
||||
size_t m_frames_processed = 0;
|
||||
|
||||
/// Per-frame CUDA-event kernel timing. Off by default, matching
|
||||
/// ClusterFinderCUDA. This is a *comparability* requirement, not a
|
||||
/// preference: with events forced on here and off there, opt1/opt2 would
|
||||
/// pay a per-frame tax that opt3+ do not, inflating the opt2 -> opt3 step
|
||||
/// by exactly the tax. That is the same methodological error the CUDA-Graph
|
||||
/// row was faulted for. Kernel times come from nsys either way.
|
||||
bool m_time_kernels = false;
|
||||
|
||||
// Kernel parameters
|
||||
dim3 grid;
|
||||
dim3 block;
|
||||
@@ -82,13 +90,18 @@ class ClusterFinderCUDAOpt2 {
|
||||
* @param capacity device-side cluster buffer size per stream
|
||||
* @param n_streams number of CUDA streams for multi-frame
|
||||
* overlap
|
||||
* @param time_kernels record per-frame CUDA events. Off by
|
||||
* default, matching ClusterFinderCUDA, so the opt1/opt2 rungs of the
|
||||
* benchmark ladder carry the same instrumentation as the rungs above them.
|
||||
*/
|
||||
ClusterFinderCUDAOpt2(Shape<2> shape_, COMPUTE_TYPE nSigma = 5.0,
|
||||
size_t capacity = 1000000, int n_streams_ = 1)
|
||||
size_t capacity = 1000000, int n_streams_ = 1,
|
||||
bool time_kernels = false)
|
||||
: m_shape(shape_), nrows(shape_[0]), ncols(shape_[1]),
|
||||
m_image_size(nrows * ncols), n_streams(n_streams_),
|
||||
m_capacity(capacity), m_nSigma(nSigma),
|
||||
m_pedestal(shape_[0], shape_[1]), m_clusters(capacity) {
|
||||
m_pedestal(shape_[0], shape_[1]), m_clusters(capacity),
|
||||
m_time_kernels(time_kernels) {
|
||||
if (n_streams_ <= 0) {
|
||||
throw std::invalid_argument(
|
||||
"ClusterFinderCUDAOpt2: n_streams must be > 0");
|
||||
@@ -125,8 +138,10 @@ class ClusterFinderCUDAOpt2 {
|
||||
auto &sc = v_sc[k];
|
||||
CUDA_CHECK(
|
||||
cudaStreamCreateWithFlags(&sc.stream, cudaStreamNonBlocking));
|
||||
CUDA_CHECK(cudaEventCreate(&sc.kernel_start));
|
||||
CUDA_CHECK(cudaEventCreate(&sc.kernel_stop));
|
||||
if (m_time_kernels) {
|
||||
CUDA_CHECK(cudaEventCreate(&sc.kernel_start));
|
||||
CUDA_CHECK(cudaEventCreate(&sc.kernel_stop));
|
||||
}
|
||||
CUDA_CHECK(cudaMalloc(&sc.d_frame, m_image_bytes));
|
||||
CUDA_CHECK(cudaMalloc(&sc.d_pd_mean,
|
||||
m_image_size * sizeof(PEDESTAL_TYPE)));
|
||||
@@ -246,14 +261,16 @@ class ClusterFinderCUDAOpt2 {
|
||||
cudaMemcpyHostToDevice, sc.stream));
|
||||
|
||||
// Timed Kernel launch
|
||||
CUDA_CHECK(cudaEventRecord(sc.kernel_start, sc.stream));
|
||||
if (m_time_kernels)
|
||||
CUDA_CHECK(cudaEventRecord(sc.kernel_start, sc.stream));
|
||||
device_opt2::find_clusters_in_single_frame<ClusterType, FRAME_TYPE,
|
||||
PEDESTAL_TYPE>
|
||||
<<<grid, block, shmem_bytes, sc.stream>>>(
|
||||
sc.d_frame, sc.d_pd_mean, sc.d_pd_sum, sc.d_pd_sum2,
|
||||
n_pd_samples, m_nSigma, nrows, ncols, sc.d_clusters,
|
||||
sc.d_cluster_count, static_cast<uint32_t>(m_capacity));
|
||||
CUDA_CHECK(cudaEventRecord(sc.kernel_stop, sc.stream));
|
||||
if (m_time_kernels)
|
||||
CUDA_CHECK(cudaEventRecord(sc.kernel_stop, sc.stream));
|
||||
CUDA_CHECK(cudaGetLastError());
|
||||
|
||||
// Read back cluster count into pinned buffer
|
||||
@@ -326,7 +343,8 @@ class ClusterFinderCUDAOpt2 {
|
||||
cudaMemcpyAsync(sc_k.d_frame, sc_k.h_frame, m_image_bytes,
|
||||
cudaMemcpyHostToDevice, sc_k.stream));
|
||||
|
||||
CUDA_CHECK(cudaEventRecord(sc_k.kernel_start, sc_k.stream));
|
||||
if (m_time_kernels)
|
||||
CUDA_CHECK(cudaEventRecord(sc_k.kernel_start, sc_k.stream));
|
||||
device_opt2::find_clusters_in_single_frame<
|
||||
ClusterType, FRAME_TYPE, PEDESTAL_TYPE>
|
||||
<<<grid, block, shmem_bytes, sc_k.stream>>>(
|
||||
@@ -334,7 +352,8 @@ class ClusterFinderCUDAOpt2 {
|
||||
sc_k.d_pd_sum2, n_pd_samples, m_nSigma, nrows, ncols,
|
||||
sc_k.d_clusters, sc_k.d_cluster_count,
|
||||
static_cast<uint32_t>(m_capacity));
|
||||
CUDA_CHECK(cudaEventRecord(sc_k.kernel_stop, sc_k.stream));
|
||||
if (m_time_kernels)
|
||||
CUDA_CHECK(cudaEventRecord(sc_k.kernel_stop, sc_k.stream));
|
||||
CUDA_CHECK(cudaGetLastError());
|
||||
|
||||
// Queue count D2H immediately after the kernel
|
||||
@@ -370,11 +389,17 @@ class ClusterFinderCUDAOpt2 {
|
||||
return results;
|
||||
}
|
||||
|
||||
/// NaN when timing was not enabled — deliberately not 0.0f, which reads as
|
||||
/// a measurement. Use nsys for kernel times; the event number is inflated
|
||||
/// by queue-wait under multi-stream load anyway.
|
||||
float avg_kernel_time_ms() const {
|
||||
return m_frames_processed > 0 ? m_total_kernel_ms / m_frames_processed
|
||||
: 0.0f;
|
||||
if (!m_time_kernels || m_frames_processed == 0)
|
||||
return std::numeric_limits<float>::quiet_NaN();
|
||||
return m_total_kernel_ms / m_frames_processed;
|
||||
}
|
||||
|
||||
bool kernel_timing_enabled() const { return m_time_kernels; }
|
||||
|
||||
void reset_timers() {
|
||||
m_total_kernel_ms = 0.0f;
|
||||
m_frames_processed = 0;
|
||||
@@ -424,7 +449,13 @@ class ClusterFinderCUDAOpt2 {
|
||||
cv.push_back(sc.h_clusters[i]);
|
||||
}
|
||||
|
||||
/// Called after cudaStreamSynchronize, so cudaEventElapsedTime never blocks
|
||||
/// on an incomplete event. m_frames_processed is only advanced when timing
|
||||
/// is on, which is what makes avg_kernel_time_ms() return NaN rather than a
|
||||
/// plausible-looking zero when it is off.
|
||||
void record_kernel_time(SC &sc) {
|
||||
if (!m_time_kernels)
|
||||
return;
|
||||
float ms = 0.0f;
|
||||
CUDA_CHECK(cudaEventElapsedTime(&ms, sc.kernel_start, sc.kernel_stop));
|
||||
m_total_kernel_ms += ms;
|
||||
|
||||
@@ -400,7 +400,15 @@ class ClusterFinderCUDAGraph {
|
||||
return results;
|
||||
}
|
||||
|
||||
float avg_kernel_time_ms() const { return 0.0f; }
|
||||
/// Always NaN: the graph variant does not instrument individual kernels
|
||||
/// (the whole H2D->kernel->D2H DAG is replayed as one unit). Use Nsight
|
||||
/// Systems, or wall-clock timing around find_clusters_batched().
|
||||
float avg_kernel_time_ms() const {
|
||||
return std::numeric_limits<float>::quiet_NaN();
|
||||
}
|
||||
|
||||
/// False: this variant never instruments kernels.
|
||||
bool kernel_timing_enabled() const { return false; }
|
||||
|
||||
void reset_timers() { m_frames_processed = 0; }
|
||||
|
||||
|
||||
@@ -69,7 +69,8 @@ def _cuda_available():
|
||||
|
||||
|
||||
def ClusterFinderCUDA(image_size, cluster_size=(3,3), n_sigma=5, dtype=np.int32,
|
||||
max_clusters_per_frame=2048, n_streams=4):
|
||||
max_clusters_per_frame=2048, n_streams=4,
|
||||
time_kernels=False):
|
||||
"""
|
||||
Factory function to create a ClusterFinderCUDA object. Provides a cleaner
|
||||
syntax for the templated ClusterFinderCUDA in C++. API mirrors
|
||||
@@ -91,6 +92,14 @@ def ClusterFinderCUDA(image_size, cluster_size=(3,3), n_sigma=5, dtype=np.int32,
|
||||
but as tight as possible to minimize PCIe traffic. Default 2048.
|
||||
n_streams : int, optional
|
||||
Number of CUDA streams for H2D/kernel/D2H pipelining. Default 4.
|
||||
time_kernels : bool, optional
|
||||
Enable per-frame CUDA-event kernel timing, exposed via
|
||||
avg_kernel_time_ms(). Off by default because it adds two event records
|
||||
per frame to the streams plus a host-side query per frame, and the
|
||||
number it yields is only meaningful at n_streams=1 — under multi-stream
|
||||
contention the events measure queue wait, not execution, and over-read
|
||||
by up to ~3.5x. Use Nsight Systems for exclusive kernel times. When
|
||||
disabled, avg_kernel_time_ms() returns NaN.
|
||||
|
||||
Example
|
||||
-------
|
||||
@@ -123,7 +132,71 @@ def ClusterFinderCUDA(image_size, cluster_size=(3,3), n_sigma=5, dtype=np.int32,
|
||||
return cls(image_size,
|
||||
n_sigma=n_sigma,
|
||||
max_clusters_per_frame=max_clusters_per_frame,
|
||||
n_streams=n_streams)
|
||||
n_streams=n_streams,
|
||||
time_kernels=time_kernels)
|
||||
|
||||
def find_cluster_views_batched_iter(cf, frames, first_frame=0, chunk=None):
|
||||
"""
|
||||
Drive a ClusterFinderCUDA over `frames`, yielding zero-copy BatchViews.
|
||||
|
||||
Same pipelining as cf.find_clusters_batched() — chunk i+1 is submitted
|
||||
before chunk i is collected — but nothing is copied out of the pinned D2H
|
||||
buffer, so the host cost per frame collapses to reading one counter. At 9x9
|
||||
that removes ~467 kB of copying per frame.
|
||||
|
||||
Each view is released as soon as the loop body finishes, which is what makes
|
||||
the next submit legal (the finder has only two slots). Consequently:
|
||||
|
||||
**Anything you need after the loop body must be copied out.** Reductions
|
||||
(`v.sums()`) return owned numpy arrays and are safe; `v.frame_data(i)` and
|
||||
`v.frame_xy(i)` are views and are not.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
cf : ClusterFinderCUDA
|
||||
frames : ndarray (n_frames, nrows, ncols), uint16
|
||||
Pin it first with cf.register_input_buffer(frames) for DMA-speed H2D.
|
||||
first_frame : int
|
||||
Frame number of frames[0].
|
||||
chunk : int, optional
|
||||
Frames per chunk; defaults to cf.chunk_size_for(len(frames)).
|
||||
|
||||
Yields
|
||||
------
|
||||
BatchView
|
||||
Valid only until the next iteration.
|
||||
|
||||
Example
|
||||
-------
|
||||
.. code-block:: python
|
||||
|
||||
cf.register_input_buffer(data)
|
||||
for v in find_cluster_views_batched_iter(cf, data):
|
||||
hist.fill(v.sums()) # reduced in C++, nothing materialised
|
||||
cf.unregister_input_buffer()
|
||||
"""
|
||||
n = frames.shape[0]
|
||||
if n == 0:
|
||||
return
|
||||
c = chunk or cf.chunk_size_for(n)
|
||||
bounds = [(s, min(s + c, n)) for s in range(0, n, c)]
|
||||
|
||||
tok = cf.submit_batch(frames[bounds[0][0]:bounds[0][1]],
|
||||
first_frame=first_frame + bounds[0][0])
|
||||
for a, b in bounds[1:]:
|
||||
nxt = cf.submit_batch(frames[a:b], first_frame=first_frame + a)
|
||||
view = cf.collect_view(tok)
|
||||
try:
|
||||
yield view
|
||||
finally:
|
||||
view.release() # frees the slot for the submit after next
|
||||
tok = nxt
|
||||
view = cf.collect_view(tok)
|
||||
try:
|
||||
yield view
|
||||
finally:
|
||||
view.release()
|
||||
|
||||
|
||||
def ClusterFinderCUDAGraph(image_size, cluster_size=(3,3), n_sigma=5, dtype=np.int32,
|
||||
max_clusters_per_frame=2048, n_streams=4):
|
||||
@@ -167,7 +240,8 @@ def ClusterFinderCUDAGraph(image_size, cluster_size=(3,3), n_sigma=5, dtype=np.i
|
||||
|
||||
|
||||
def ClusterFinderCUDAOpt2(image_size, cluster_size=(3, 3), n_sigma=5, dtype=np.int32,
|
||||
max_clusters_per_frame=3000, n_streams=4):
|
||||
max_clusters_per_frame=3000, n_streams=4,
|
||||
time_kernels=False):
|
||||
"""
|
||||
Factory for the OPT2 snapshot finder — the pre-refactor pipeline (per-frame
|
||||
pinned staging, round-robin streams with sync barriers, variable-length D2H),
|
||||
@@ -179,6 +253,11 @@ def ClusterFinderCUDAOpt2(image_size, cluster_size=(3, 3), n_sigma=5, dtype=np.i
|
||||
the opt arc; only the pipeline differs).
|
||||
|
||||
Only the 3x3 cluster size is registered.
|
||||
|
||||
time_kernels defaults to False, matching ClusterFinderCUDA. Leave it off for
|
||||
any throughput comparison: with events on here and off there, opt1/opt2 pay
|
||||
a per-frame tax that opt3+ do not, which inflates the opt2 -> opt3 step by
|
||||
exactly that tax. Kernel times come from nsys.
|
||||
"""
|
||||
if not _cuda_available():
|
||||
raise RuntimeError(
|
||||
@@ -190,7 +269,8 @@ def ClusterFinderCUDAOpt2(image_size, cluster_size=(3, 3), n_sigma=5, dtype=np.i
|
||||
return cls(image_size,
|
||||
n_sigma=n_sigma,
|
||||
max_clusters_per_frame=max_clusters_per_frame,
|
||||
n_streams=n_streams)
|
||||
n_streams=n_streams,
|
||||
time_kernels=time_kernels)
|
||||
|
||||
|
||||
def ClusterCollector(clusterfindermt, dtype=np.int32):
|
||||
|
||||
@@ -32,7 +32,7 @@ from ._aare import corner
|
||||
|
||||
from ._version import __version__
|
||||
from .ClusterFinder import ClusterFinder, ClusterFinderFrozen, ClusterCollector, ClusterFinderMT, ClusterFileSink, ClusterFile
|
||||
from .ClusterFinder import ClusterFinderCUDA, ClusterFinderCUDAGraph, ClusterFinderCUDAOpt2, _cuda_available
|
||||
from .ClusterFinder import ClusterFinderCUDA, ClusterFinderCUDAGraph, ClusterFinderCUDAOpt2, _cuda_available, find_cluster_views_batched_iter
|
||||
from .ClusterVector import ClusterVector
|
||||
from .Cluster import Cluster
|
||||
|
||||
|
||||
@@ -35,10 +35,84 @@ void define_ClusterFinderCUDA(py::module &m, const std::string &typestr) {
|
||||
py::class_<typename CF::BatchToken>(m,
|
||||
(class_name + "_BatchToken").c_str());
|
||||
|
||||
using VT = typename ClusterType::value_type;
|
||||
constexpr size_t NPIX =
|
||||
static_cast<size_t>(ClusterSizeX) * static_cast<size_t>(ClusterSizeY);
|
||||
|
||||
// Zero-copy view into the finder's pinned D2H buffer.
|
||||
py::class_<typename CF::BatchView>(m, (class_name + "_BatchView").c_str())
|
||||
.def_property_readonly("n_frames", &CF::BatchView::n_frames)
|
||||
.def_property_readonly("first_frame", &CF::BatchView::first_frame)
|
||||
.def_property_readonly("valid", &CF::BatchView::valid)
|
||||
.def_property_readonly("total_clusters", &CF::BatchView::total_clusters)
|
||||
.def("count", &CF::BatchView::count, py::arg("frame_index"))
|
||||
.def("release", &CF::BatchView::release,
|
||||
R"(Give the slot back to the finder. The view is unusable
|
||||
afterwards and submit_batch() may reuse the buffer. Called
|
||||
automatically on destruction and on __exit__.)")
|
||||
.def("__enter__", [](py::object self) { return self; })
|
||||
.def("__exit__", [](typename CF::BatchView &v, py::object, py::object,
|
||||
py::object) { v.release(); })
|
||||
.def_property_readonly(
|
||||
"counts",
|
||||
[](const typename CF::BatchView &v) {
|
||||
py::array_t<uint32_t> out(v.n_frames());
|
||||
auto *o = out.mutable_data();
|
||||
for (size_t i = 0; i < v.n_frames(); ++i)
|
||||
o[i] = v.count(i);
|
||||
return out;
|
||||
},
|
||||
R"(Clusters found per frame, as a numpy array.)")
|
||||
.def(
|
||||
"sums",
|
||||
[](const typename CF::BatchView &v) {
|
||||
std::vector<VT> s;
|
||||
{
|
||||
// Reduce without the GIL, but take it back before building
|
||||
// the array — constructing a Python object without it is a
|
||||
// segfault, so this cannot be a call_guard.
|
||||
py::gil_scoped_release nogil;
|
||||
s = v.sums();
|
||||
}
|
||||
return py::array_t<VT>(s.size(), s.data());
|
||||
},
|
||||
R"(Per-cluster sums for the whole batch, reduced in C++ straight out
|
||||
of the pinned buffer — the clusters are never materialised on the
|
||||
host. This is the fast path for spectra/histograms.)")
|
||||
.def(
|
||||
"frame_data",
|
||||
[](py::object self, size_t i) {
|
||||
auto &v = self.cast<typename CF::BatchView &>();
|
||||
const uint32_t n = v.count(i);
|
||||
// Zero-copy (n, NPIX) view; stride skips each cluster's x/y.
|
||||
return py::array_t<VT>(
|
||||
{static_cast<size_t>(n), NPIX},
|
||||
{sizeof(ClusterType), sizeof(VT)},
|
||||
n == 0 ? nullptr : v.clusters(i)->data.data(), self);
|
||||
},
|
||||
py::arg("frame_index"),
|
||||
R"(Zero-copy (n_clusters, ClusterSizeX*ClusterSizeY) view of one
|
||||
frame's pixel data, straight out of the pinned buffer. Valid until
|
||||
this view is released — copy it if you need to keep it.)")
|
||||
.def(
|
||||
"frame_xy",
|
||||
[](py::object self, size_t i) {
|
||||
auto &v = self.cast<typename CF::BatchView &>();
|
||||
const uint32_t n = v.count(i);
|
||||
return py::array_t<CoordType>(
|
||||
{static_cast<size_t>(n), size_t{2}},
|
||||
{sizeof(ClusterType), sizeof(CoordType)},
|
||||
n == 0 ? nullptr : &v.clusters(i)->x, self);
|
||||
},
|
||||
py::arg("frame_index"),
|
||||
R"(Zero-copy (n_clusters, 2) view of one frame's cluster centre
|
||||
coordinates as (x, y).)");
|
||||
|
||||
py::class_<CF>(m, class_name.c_str())
|
||||
.def(py::init<Shape<2>, float, size_t, int>(), py::arg("image_size"),
|
||||
py::arg("n_sigma") = 5.0f,
|
||||
py::arg("max_clusters_per_frame") = 2048, py::arg("n_streams") = 4)
|
||||
.def(py::init<Shape<2>, float, size_t, int, bool>(),
|
||||
py::arg("image_size"), py::arg("n_sigma") = 5.0f,
|
||||
py::arg("max_clusters_per_frame") = 2048, py::arg("n_streams") = 4,
|
||||
py::arg("time_kernels") = false)
|
||||
|
||||
.def_property(
|
||||
"nSigma", &CF::get_nSigma, &CF::set_nSigma,
|
||||
@@ -154,11 +228,77 @@ sqrt(max(sum2/n - mean^2, 0)). Counterpart to `noise` for the device pedestal.)"
|
||||
py::arg("token"), py::call_guard<py::gil_scoped_release>(),
|
||||
R"(Wait for a previously submitted batch and return its results as
|
||||
a list of ClusterVector, one per input frame. Releases the batch
|
||||
slot so it can be reused by the next submit_batch() call.)")
|
||||
slot so it can be reused by the next submit_batch() call.
|
||||
|
||||
One allocation and one copy per frame; see collect_view() for
|
||||
neither.)")
|
||||
|
||||
.def(
|
||||
"collect_view",
|
||||
[](CF &self, typename CF::BatchToken token) {
|
||||
return self.collect_view(token);
|
||||
},
|
||||
py::arg("token"), py::call_guard<py::gil_scoped_release>(),
|
||||
R"(Like collect(), but copies nothing: returns a BatchView onto the
|
||||
finder's pinned D2H buffer.
|
||||
|
||||
The slot stays reserved until the view is released, so with 2 slots
|
||||
you must finish with it before submitting two more batches. Use it
|
||||
as a context manager, or call release():
|
||||
|
||||
tok = cf.submit_batch(frames[a:b], first_frame=a)
|
||||
with cf.collect_view(tok) as v:
|
||||
hist.fill(v.sums()) # reduced in C++, nothing copied
|
||||
|
||||
Anything you want to keep past release() must be copied out.)")
|
||||
|
||||
.def("avg_kernel_time_ms", &CF::avg_kernel_time_ms,
|
||||
R"(Average kernel execution time per frame in milliseconds,
|
||||
excluding PCIe transfers.)")
|
||||
R"(Average per-frame kernel time in ms, or NaN if the finder was
|
||||
constructed with time_kernels=False (the default).
|
||||
|
||||
WARNING: only meaningful with n_streams=1. The CUDA events bracket
|
||||
the kernel on its own stream, so under multi-stream contention the
|
||||
interval includes time spent queued behind other streams' kernels
|
||||
and over-reads by up to ~3.5x. Use Nsight Systems for exclusive
|
||||
kernel times.)")
|
||||
|
||||
.def("kernel_timing_enabled", &CF::kernel_timing_enabled,
|
||||
R"(True if per-frame kernel timing was enabled at construction.)")
|
||||
|
||||
.def("chunk_size_for", &CF::chunk_size_for, py::arg("n_frames"),
|
||||
R"(The chunk size find_clusters_batched() would use for n_frames.
|
||||
Use it to match its pipelining when driving submit_batch/
|
||||
collect_view by hand.)")
|
||||
|
||||
.def("reserve_output_slots", &CF::reserve_output_slots,
|
||||
py::arg("n_frames"), py::call_guard<py::gil_scoped_release>(),
|
||||
R"(Pre-allocate both pinned output slots for batches of n_frames.
|
||||
|
||||
Processes nothing: no transfer, no kernel, and the pedestal is NOT
|
||||
advanced. Only the two cudaMallocHost calls that the first
|
||||
submit_batch() would make happen here.
|
||||
|
||||
Page-locking runs about 1.0 us per 4 kB page (~66 ms for two
|
||||
128 MiB slots), and that cost is charged to whoever triggers it.
|
||||
Call this before starting a timer so it does not land inside the
|
||||
measurement. Slots only grow, so pass the largest batch you intend
|
||||
to use:
|
||||
|
||||
cf.reserve_output_slots(cf.chunk_size_for(len(data)))
|
||||
)")
|
||||
|
||||
.def_property(
|
||||
"batch_chunk", &CF::get_batch_chunk, &CF::set_batch_chunk,
|
||||
R"(Frames per internally pipelined chunk in find_clusters_batched().
|
||||
|
||||
0 (default) = auto: the batch is split into ~8 chunks so the host
|
||||
marshals one chunk while the GPU runs the next. Rounded up to a
|
||||
multiple of n_streams, which keeps the frame->stream assignment —
|
||||
and therefore the per-stream device pedestal each frame sees —
|
||||
identical to processing the batch in one go.
|
||||
|
||||
Set equal to the batch size to disable chunking and recover the old
|
||||
submit-everything-then-copy-everything behaviour.)")
|
||||
|
||||
.def("reset_timers", &CF::reset_timers,
|
||||
R"(Reset the internal kernel timing counters.)")
|
||||
|
||||
@@ -91,8 +91,12 @@ void define_ClusterFinderCUDAGraph(py::module &m, const std::string &typestr) {
|
||||
|
||||
.def(
|
||||
"avg_kernel_time_ms", &CF::avg_kernel_time_ms,
|
||||
R"(Always returns 0.0 — graph version does not instrument individual kernel time.
|
||||
Use wall-clock timing around find_clusters_batched instead.)")
|
||||
R"(Always NaN — the graph version replays the whole H2D->kernel->D2H
|
||||
DAG as one unit and does not instrument individual kernels. Use
|
||||
Nsight Systems, or wall-clock timing around find_clusters_batched.)")
|
||||
|
||||
.def("kernel_timing_enabled", &CF::kernel_timing_enabled,
|
||||
R"(Always False for this variant.)")
|
||||
|
||||
.def("reset_timers", &CF::reset_timers)
|
||||
|
||||
|
||||
@@ -36,9 +36,10 @@ void define_ClusterFinderCUDAOpt2(py::module &m, const std::string &typestr) {
|
||||
py::class_<CF>(m, class_name.c_str())
|
||||
// ctor: (image_size, n_sigma, capacity, n_streams) — capacity is the
|
||||
// per-stream device cluster buffer (upper bound on clusters/frame).
|
||||
.def(py::init<Shape<2>, float, size_t, int>(), py::arg("image_size"),
|
||||
py::arg("n_sigma") = 5.0f,
|
||||
py::arg("max_clusters_per_frame") = 3000, py::arg("n_streams") = 4)
|
||||
.def(py::init<Shape<2>, float, size_t, int, bool>(),
|
||||
py::arg("image_size"), py::arg("n_sigma") = 5.0f,
|
||||
py::arg("max_clusters_per_frame") = 3000, py::arg("n_streams") = 4,
|
||||
py::arg("time_kernels") = false)
|
||||
|
||||
.def_property(
|
||||
"nSigma", &CF::get_nSigma, &CF::set_nSigma,
|
||||
@@ -93,7 +94,12 @@ void define_ClusterFinderCUDAOpt2(py::module &m, const std::string &typestr) {
|
||||
R"(Process a 3D array (n_frames, nrows, ncols) round-robin across
|
||||
n_streams. Returns a list of ClusterVector, one per input frame.)")
|
||||
|
||||
.def("avg_kernel_time_ms", &CF::avg_kernel_time_ms)
|
||||
.def("avg_kernel_time_ms", &CF::avg_kernel_time_ms,
|
||||
R"(Mean per-frame kernel time in ms, or NaN if time_kernels was
|
||||
not enabled at construction. Inflated by queue-wait under
|
||||
multi-stream load — use nsys for the true value.)")
|
||||
.def("kernel_timing_enabled", &CF::kernel_timing_enabled,
|
||||
R"(True if per-frame kernel timing was enabled at construction.)")
|
||||
.def("reset_timers", &CF::reset_timers);
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1,32 +0,0 @@
|
||||
# Minimal probe for nsys: train pedestal, run one batched pass, print summary.
|
||||
# Usage: nsys_kernel_probe.py [n_streams] [n_frames]
|
||||
import sys
|
||||
sys.path.append('/home/ferjao_k/aare/build')
|
||||
|
||||
from pathlib import Path
|
||||
import time
|
||||
from aare import File, ClusterFinderCUDA
|
||||
|
||||
n_streams = int(sys.argv[1]) if len(sys.argv) > 1 else 8
|
||||
N = int(sys.argv[2]) if len(sys.argv) > 2 else 2000
|
||||
|
||||
base = Path('/mnt/sls_det_storage/moench_data/2603_MaxIVBeamtime/2026032408/process/xrf/')
|
||||
f = File(base / 'Cu_factor_10_data_master_0.json')
|
||||
pd = File(base / 'Cu_factor_10_pedestal_master_0.json')
|
||||
|
||||
cf = ClusterFinderCUDA((f.rows, f.cols), (3, 3), n_sigma=5,
|
||||
max_clusters_per_frame=3000, n_streams=n_streams)
|
||||
for _ in range(1000):
|
||||
cf.push_pedestal_frame(pd.read_frame().copy())
|
||||
|
||||
data = f.read_n(N)
|
||||
cf.register_input_buffer(data)
|
||||
|
||||
t0 = time.perf_counter()
|
||||
res = cf.find_clusters_batched(data, first_frame=0)
|
||||
t = time.perf_counter() - t0
|
||||
|
||||
cf.unregister_input_buffer()
|
||||
n = sum(cv.size for cv in res)
|
||||
print(f'n_streams={n_streams} N={N} wall={t:.3f}s ({N/t:.0f} FPS) '
|
||||
f'clusters/frame={n/N:.2f} event kernel_ms={cf.avg_kernel_time_ms():.3f}')
|
||||
@@ -0,0 +1,173 @@
|
||||
# ClusterFinderCUDA measurement campaign
|
||||
|
||||
Everything that produces a number in
|
||||
[`docs/ClusterFinderCUDA_benchmark_results.md`](../../../docs/ClusterFinderCUDA_benchmark_results.md)
|
||||
lives here, and everything it produces lands in `results/<date>_<build>/` with a
|
||||
manifest, so any figure in the report or the deck can be traced back to the run
|
||||
and the build that made it.
|
||||
|
||||
## Scripts
|
||||
|
||||
| script | produces | answers |
|
||||
|---|---|---|
|
||||
| `run_campaign.sh` | both arms | the whole thing: f32 ladder+probes → rebuild → f64 ladder+probes → rebuild back |
|
||||
| `run_ladder.py` | `ladder_<dim>x<dim>.csv` | end-to-end wall / FPS / faults / counts for every step |
|
||||
| `run_probes.py` | `probes.csv` + `.nsys-rep`/`.sqlite` | per-engine GPU times, duty cycles, **the rooflines** |
|
||||
| `ladder.py` | — | the ladder *as a config matrix*; imported, not run |
|
||||
| `common.py` | — | dataset, pedestal, fault bracketing, env capture, CSV format |
|
||||
| `nsys_kernel_probe.py` | — | the workload `run_probes.py` traces; runnable alone under nsys |
|
||||
| `gpu_span.py` | a duty-cycle table | *engine busy* vs *engine idle*; `analyze()` is importable |
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
./run_campaign.sh # both arms, ~1.5 h, unattended
|
||||
./run_campaign.sh f32 # one arm
|
||||
|
||||
python run_ladder.py --dry-run # 2000 frames, 1 rep — proves it executes
|
||||
python run_ladder.py # 3x3 @ 100k, 9x9 @ 20k, 5 reps
|
||||
python run_ladder.py --dims 9 --steps opt7 opt8 # harness labels — see the map below
|
||||
python run_ladder.py --retain # keep every ClusterVector (notebook behaviour)
|
||||
|
||||
python run_probes.py # 4 configs at 20k frames
|
||||
python run_probes.py --only 9x9_s4
|
||||
python gpu_span.py <rep>.sqlite 20000 # re-read duty cycles from an existing profile
|
||||
|
||||
# per-operation durations from a committed profile — no GPU, no rerun.
|
||||
# Pass the .sqlite: the reps are newer than their exports, so the .nsys-rep path
|
||||
# demands --force-export=true and rewrites the export gpu_span.py reads.
|
||||
nsys stats --report cuda_gpu_kern_sum --report cuda_gpu_mem_time_sum \
|
||||
--format table results/2026-08-18_f64/probe_9x9_s1_uncontended.sqlite
|
||||
```
|
||||
|
||||
`nsys stats` gives durations, not duty cycles — it cannot tell "engine busy" from
|
||||
"engine idle waiting". Use it for *how long is one kernel* (`s1`); use `gpu_span.py`
|
||||
for overlap, duty and the roofline.
|
||||
|
||||
**The GPU must be idle.** Both drivers abort above 5 % utilisation: a competing
|
||||
process leaves per-operation averages intact while destroying the duty cycle and
|
||||
the wall clock, so the failure is silent if you don't check. Close any notebook
|
||||
first. `--allow-busy-gpu` exists but the numbers are not quotable.
|
||||
|
||||
## Step labels
|
||||
|
||||
The `--steps` flag and the `step` column of every CSV use the harness's internal labels,
|
||||
which predate the report's numbering. The report carries the same map in its §15.
|
||||
|
||||
| harness label | step in the report | act |
|
||||
|---|---|---|
|
||||
| `cpu` | baseline | — |
|
||||
| `opt1` `opt2` `opt3` `opt4` | opt1 opt2 opt3 opt4 | **I** — feeding the GPU |
|
||||
| `opt5` | **route A** — CUDA Graphs, *rejected* | (fork after opt4) |
|
||||
| `opt7` | **opt5** — chunked host↔GPU overlap | **II** — getting results back |
|
||||
| `opt8` | **opt6** — zero-copy `collect_view()` | **II** |
|
||||
| *(build axis, not a row)* | **opt7** — f32 device pedestal | **III** — the kernel |
|
||||
|
||||
Renaming the labels would rewrite recorded data, so they stay as they are; translate on
|
||||
the way out.
|
||||
|
||||
## Fixed parameters
|
||||
|
||||
| | 3×3 | 9×9 |
|
||||
|---|--:|--:|
|
||||
| `N` | 100 000 | 20 000 |
|
||||
| `max_clusters_per_frame` | 3 000 | 1 500 |
|
||||
| `n_streams` | 4 | 4 |
|
||||
| `BATCH_SIZE` | 2 000 | 2 000 |
|
||||
| pedestal frames / `n_sigma` | 1 000 / 5 | 1 000 / 5 |
|
||||
| reps | 5 | 5 |
|
||||
| nsys probe frames | 20 000 | 20 000 |
|
||||
|
||||
Optimising over these is a separate exercise; what matters here is that every
|
||||
step sees the same ones. Two are deliberate departures from earlier campaigns:
|
||||
`n_streams` was 8 at 9×9 (8 streams buy no kernel concurrency there — instance
|
||||
time +1 % — while inflating the event timer 3.5×), and probes were 2 000 frames
|
||||
(too short for the clocks to ramp, which is how a 26.7 µs roofline was published
|
||||
for a pipeline that sustains 24.25).
|
||||
|
||||
9×9 is held at N=20 000 because its result heap is ~5× larger per frame
|
||||
(1422 × 328 B = 466 kB vs 2330 × 40 B = 93 kB); 100 k would need 46.6 GB to
|
||||
retain against 98 GB free with no swap.
|
||||
|
||||
## Reading the output
|
||||
|
||||
**`cold` = rep 0 in a fresh process. `warm` = best of the remaining reps.**
|
||||
Not the last rep — `collect()` does not converge, it oscillates between allocator
|
||||
states. Measured at 9×9, opt4, one run: 85.8 / 73.7 / 86.6 µs with faults
|
||||
520 k / 127 k / 519 k. Quoting the last rep there reports 86.6 when 73.7 was
|
||||
achieved in the same run; the choice of rep would be doing the work, not the code.
|
||||
|
||||
**The `spread` column is a result, not noise.** Paths that allocate per frame
|
||||
vary 3–28 % run to run; `collect_view()`, which allocates nothing, is
|
||||
reproducible to 0.0–0.2 %. That contrast is the strongest argument for opt6
|
||||
(harness `opt8`) — it is the only path whose throughput is *reproducible*.
|
||||
|
||||
**Each step runs in its own process.** The heap is process-wide, so in a shared
|
||||
process every step inherits what the previous ones grew: `opt3` reports **2**
|
||||
faults after opt1/opt2 have run and **92 251** on its own. `--no-isolate` restores
|
||||
the fast path and makes the fault columns meaningless; throughput is unaffected
|
||||
either way.
|
||||
|
||||
**Reps share a finder within a process** — a new one would reset the heap. The
|
||||
device pedestal advances by `n_frames` each pass, so `n_clusters` drifts ~0.002 %
|
||||
between reps. Compare steps at the same rep index, never across reps.
|
||||
|
||||
**opt7 is not a row.** It is `DEVICE_PED_TYPE` in
|
||||
`include/aare/clusterfinder_kernel.cuh`. Run the matrix once per build and compare
|
||||
directories; `env.json` records which arm you are in. The f64 arm is the report's
|
||||
Acts I–II, the f32 arm is Act III.
|
||||
|
||||
**opt1/opt2 are 3×3 only** — `ClusterFinderCUDAOpt2` is registered for 3×3 in
|
||||
`cuda_bindings.cu`, so the 9×9 ladder starts at opt3. They are also unaffected by
|
||||
the opt7 flip by construction: their binding pins `PEDESTAL_TYPE` to `double`.
|
||||
If they move between arms, something is wrong.
|
||||
|
||||
**The CPU baseline is first-pass only** and forced to 1 rep: `ClusterFinderMT`
|
||||
cannot restart after `stop()`. Both speedup columns therefore divide by a *cold*
|
||||
CPU number, which reads ~9 % generous.
|
||||
|
||||
**At 4 streams the probe's `kernel_us_per_frame` is engine occupancy**, the union
|
||||
of kernel intervals over frames — not per-kernel duration. That is why f64 9×9
|
||||
reads 32.08 µs at `s4` while each kernel is ~39.9 µs. Use `s1` for exclusive
|
||||
kernel times (the opt7 claim), `s4` for "% of roofline".
|
||||
|
||||
**Expect zero-copy to land 2–3 % *under* its roofline.** The roofline is measured
|
||||
under CUPTI, which dilates GPU op durations slightly, so it is a mild
|
||||
over-estimate. "≥100 % of roofline" means *at the floor, within the profiler's
|
||||
own systematic error* — not a measurement error. Take percentages from the **f32**
|
||||
arm: the f64 9×9 `s4` kernel column is an interval union at `overlap = 1.36`, which
|
||||
CUPTI inflates further, and zero-copy reads 6.4 % under it there.
|
||||
|
||||
## Two policies enforced in code
|
||||
|
||||
1. **Never warm up by processing frames.** The kernel pushes a pedestal update
|
||||
per pixel per frame, so a finder that has seen extra frames is no longer
|
||||
comparable with one that has not. Slots are pre-pinned with
|
||||
`reserve_output_slots()`, which allocates without transferring or launching —
|
||||
verified to leave cluster counts bit-identical.
|
||||
2. **`time_kernels=False` everywhere**, including `ClusterFinderCUDAOpt2`, which
|
||||
gained the flag for this reason. With events on for one finder and off for
|
||||
another, the instrumented one pays a per-frame tax the other does not and the
|
||||
step between them absorbs it. Kernel times come from nsys, which is the only
|
||||
source correct under multi-stream load anyway.
|
||||
|
||||
## Results directories
|
||||
|
||||
```
|
||||
results/<date>_<f32|f64>[_tag]/
|
||||
env.json build, git rev, driver, GPU, DEVICE_PED_TYPE, timestamp
|
||||
manifest.csv artifact -> config -> build -> which report section cites it
|
||||
ladder_3x3.csv one row per (step, rep) — every rep kept, nothing averaged
|
||||
ladder_9x9.csv
|
||||
probes.csv per-engine us/frame, duty %, overlap, bottleneck, roofline
|
||||
probe_*.nsys-rep openable in nsys-ui
|
||||
probe_*.sqlite input to gpu_span.py
|
||||
```
|
||||
|
||||
Current campaign: **`2026-08-18_f32/`** and **`2026-08-18_f64/`**.
|
||||
`2026-08-12_f32_legacy/` is retired — see its `SUPERSEDED.md`.
|
||||
|
||||
> **Known wart:** `results_dir()` stamps *today's* date, so a campaign that spans
|
||||
> midnight splits across two directories. That happened once already (the f32
|
||||
> ladder and its probes landed a day apart) and had to be merged by hand, with a
|
||||
> note added to `env.json`. Prefer a campaign tag over a date if this recurs.
|
||||
@@ -0,0 +1,298 @@
|
||||
"""Shared plumbing for the ClusterFinderCUDA measurement campaign.
|
||||
|
||||
Everything that could differ between ladder steps and silently change a number
|
||||
lives here exactly once: dataset loading, pedestal training, fault bracketing,
|
||||
slot pre-pinning and the CSV/manifest format. Steps differ only by the config
|
||||
rows in ladder.py.
|
||||
|
||||
Two policies are enforced here rather than left to each caller, because both
|
||||
have produced wrong numbers in this campaign before:
|
||||
|
||||
1. **Never warm up by processing frames.** The kernel pushes a pedestal update
|
||||
per pixel per frame, so a finder that has seen extra frames is no longer
|
||||
comparable with one that has not. Slots are pre-pinned with
|
||||
reserve_output_slots(), which allocates without transferring or launching.
|
||||
|
||||
2. **Every finder is fresh.** The device pedestal keeps evolving, so two steps
|
||||
sharing a finder see different pedestal states and cannot be compared.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import resource
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass, asdict, field
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
REPO = Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0, str(REPO / "build"))
|
||||
|
||||
DATA_DIR = Path(
|
||||
"/mnt/sls_det_storage/moench_data/2603_MaxIVBeamtime/2026032408/process/xrf/"
|
||||
)
|
||||
DATA_FILE = DATA_DIR / "Cu_factor_10_data_master_0.json"
|
||||
PEDESTAL_FILE = DATA_DIR / "Cu_factor_10_pedestal_master_0.json"
|
||||
|
||||
N_PEDESTAL_FRAMES = 1000
|
||||
N_SIGMA = 5
|
||||
|
||||
# Cost of one first-touch page, measured on this machine (docs §3.1, §12).
|
||||
# Two different values because pinned pages additionally cost driver work.
|
||||
US_PER_HEAP_FAULT = 0.7
|
||||
US_PER_PINNED_FAULT = 1.0
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# process-level instrumentation
|
||||
# --------------------------------------------------------------------------
|
||||
def faults():
|
||||
"""(minor, major) fault counters. Bracket every timed region with these."""
|
||||
r = resource.getrusage(resource.RUSAGE_SELF)
|
||||
return r.ru_minflt, r.ru_majflt
|
||||
|
||||
|
||||
def _sh(cmd, default="unknown"):
|
||||
try:
|
||||
return subprocess.run(
|
||||
cmd, shell=True, capture_output=True, text=True, timeout=30
|
||||
).stdout.strip() or default
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
|
||||
def capture_env() -> dict:
|
||||
"""Everything needed to know whether a later run is comparable to this one."""
|
||||
import aare
|
||||
|
||||
return {
|
||||
"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"host": _sh("hostname"),
|
||||
"git_rev": _sh(f"git -C {REPO} rev-parse --short HEAD"),
|
||||
"git_branch": _sh(f"git -C {REPO} rev-parse --abbrev-ref HEAD"),
|
||||
"git_dirty": bool(_sh(f"git -C {REPO} status --porcelain")),
|
||||
"aare_version": getattr(aare, "__version__", "unknown"),
|
||||
# The opt6 axis. Read from the header rather than trusted from memory:
|
||||
# a stale build is the single easiest way to mislabel a whole campaign.
|
||||
"device_ped_type": device_ped_type(),
|
||||
"gpu": _sh("nvidia-smi --query-gpu=name --format=csv,noheader"),
|
||||
"driver": _sh("nvidia-smi --query-gpu=driver_version --format=csv,noheader"),
|
||||
"gpu_busy_pct": _sh(
|
||||
"nvidia-smi --query-gpu=utilization.gpu --format=csv,noheader"
|
||||
),
|
||||
"nvcc": _sh("nvcc --version | tail -1"),
|
||||
"python": sys.version.split()[0],
|
||||
}
|
||||
|
||||
|
||||
def assert_build_fresh() -> None:
|
||||
"""Abort if the installed extension predates the headers it was compiled from.
|
||||
|
||||
device_ped_type() below parses the SOURCE header, so on an un-rebuilt tree it
|
||||
reports the type the source *claims* while the loaded module is still the
|
||||
previous build — and env.json is stamped with the wrong build identity. That
|
||||
is not hypothetical: on 2026-08-20 it produced a full-f64 9x9 probe labelled
|
||||
"float" (header edited 10:54, binary built 09:57). See
|
||||
results/2026-08-20_INVALID_stale_build/INVALID.md.
|
||||
|
||||
mtime only. Cheap, and it catches the case that actually occurred.
|
||||
"""
|
||||
so = list((REPO / "build" / "aare").glob("_aare_cuda*.so"))
|
||||
if not so:
|
||||
raise RuntimeError("assert_build_fresh: no _aare_cuda*.so under build/aare")
|
||||
built = max(f.stat().st_mtime for f in so)
|
||||
|
||||
headers = [REPO / "include/aare/clusterfinder_kernel.cuh",
|
||||
REPO / "include/aare/ClusterFinderCUDA.hpp"]
|
||||
stale = [h for h in headers if h.exists() and h.stat().st_mtime > built]
|
||||
if stale:
|
||||
names = ", ".join(h.name for h in stale)
|
||||
raise RuntimeError(
|
||||
f"assert_build_fresh: {names} newer than the installed extension.\n"
|
||||
f" header(s) edited after the build -> env.json would record the "
|
||||
f"SOURCE type, not the compiled one.\n"
|
||||
f" Rebuild and reinstall before measuring.")
|
||||
|
||||
|
||||
def device_ped_type() -> str:
|
||||
"""Parse DEVICE_PED_TYPE out of the kernel header — the opt6 build axis.
|
||||
|
||||
Reads SOURCE, not the binary; there is no binding that exposes the compiled
|
||||
type. Only meaningful once assert_build_fresh() has passed.
|
||||
"""
|
||||
hdr = REPO / "include/aare/clusterfinder_kernel.cuh"
|
||||
try:
|
||||
for line in hdr.read_text().splitlines()[:40]:
|
||||
if "using DEVICE_PED_TYPE" in line and not line.strip().startswith("//"):
|
||||
return line.split("=")[1].split(";")[0].strip()
|
||||
except Exception:
|
||||
pass
|
||||
return "unknown"
|
||||
|
||||
|
||||
def assert_idle_gpu(max_pct: int = 5):
|
||||
"""A competing process leaves per-op averages intact while destroying the
|
||||
duty cycle and the wall clock (docs §14). Fail loudly rather than record it."""
|
||||
pct = _sh("nvidia-smi --query-gpu=utilization.gpu --format=csv,noheader")
|
||||
try:
|
||||
val = int(pct.split()[0])
|
||||
except Exception:
|
||||
return
|
||||
if val > max_pct:
|
||||
raise SystemExit(
|
||||
f"GPU is {val}% busy — another process is running. "
|
||||
f"Numbers taken now are not quotable (docs §14). Aborting."
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# dataset + finders
|
||||
# --------------------------------------------------------------------------
|
||||
_data_cache: dict[int, np.ndarray] = {}
|
||||
|
||||
|
||||
def load_frames(n_frames: int) -> np.ndarray:
|
||||
"""Data frames, held in RAM so file I/O is outside every timing loop."""
|
||||
from aare import File
|
||||
|
||||
if n_frames not in _data_cache:
|
||||
f = File(DATA_FILE)
|
||||
_data_cache[n_frames] = f.read_n(n_frames)
|
||||
return _data_cache[n_frames]
|
||||
|
||||
|
||||
def image_size() -> tuple[int, int]:
|
||||
from aare import File
|
||||
|
||||
f = File(DATA_FILE)
|
||||
return (f.rows, f.cols)
|
||||
|
||||
|
||||
def train_pedestal(finder) -> None:
|
||||
"""The same 1000 pedestal frames for every finder in every step."""
|
||||
from aare import File
|
||||
|
||||
pd = File(PEDESTAL_FILE)
|
||||
for _ in range(N_PEDESTAL_FRAMES):
|
||||
finder.push_pedestal_frame(pd.read_frame().copy())
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# results
|
||||
# --------------------------------------------------------------------------
|
||||
@dataclass
|
||||
class Row:
|
||||
"""One measurement. Written verbatim to CSV; every report number cites one."""
|
||||
|
||||
step: str # opt1 … opt8, cpu
|
||||
label: str # human-readable description
|
||||
cluster_dim: int
|
||||
cap: int
|
||||
n_frames: int
|
||||
n_streams: int
|
||||
pinned: bool
|
||||
batch_chunk: str # "auto" | "off" | explicit int
|
||||
collection: str # collect | collect_view | per-frame | n/a
|
||||
device_ped_type: str
|
||||
rep: int
|
||||
wall_s: float
|
||||
us_per_frame: float
|
||||
fps: float
|
||||
minor_faults: int
|
||||
major_faults: int
|
||||
n_clusters: int
|
||||
clusters_per_frame: float
|
||||
notes: str = ""
|
||||
|
||||
|
||||
def write_rows(path: Path, rows: list[Row]) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", newline="") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=list(Row.__dataclass_fields__))
|
||||
w.writeheader()
|
||||
for r in rows:
|
||||
w.writerow(asdict(r))
|
||||
|
||||
|
||||
def write_env(path: Path, env: dict) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(env, indent=2) + "\n")
|
||||
|
||||
|
||||
def append_manifest(path: Path, entry: dict) -> None:
|
||||
"""artifact -> config -> build -> which report section cites it."""
|
||||
cols = ["artifact", "kind", "config", "build", "cites", "produced_by", "timestamp"]
|
||||
new = not path.exists()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("a", newline="") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=cols)
|
||||
if new:
|
||||
w.writeheader()
|
||||
w.writerow({c: entry.get(c, "") for c in cols})
|
||||
|
||||
|
||||
def results_dir(tag: str = "") -> Path:
|
||||
"""perf/results/<date>_<f32|f64>[_tag]/ — one directory per campaign."""
|
||||
ped = device_ped_type()
|
||||
short = {"float": "f32", "double": "f64"}.get(ped, ped)
|
||||
name = f"{time.strftime('%Y-%m-%d')}_{short}" + (f"_{tag}" if tag else "")
|
||||
d = Path(__file__).resolve().parent / "results" / name
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def print_table(rows: list[Row], floor_us: float | None = None) -> None:
|
||||
"""Cold (rep 0) vs warm (BEST of the rest), with the spread.
|
||||
|
||||
"Best of the rest", not "last", because collect() does not converge: it
|
||||
oscillates between allocator states. Measured at 9x9, opt4, one run:
|
||||
85.8 / 73.7 / 86.6 us per frame with faults 520k / 127k / 519k. Quoting the
|
||||
last rep there reports 86.6 when 73.7 was achieved in the same run — the
|
||||
choice of rep would be doing the work, not the code.
|
||||
|
||||
The spread column is therefore part of the result, not noise to hide: the
|
||||
paths that allocate per frame vary by 3-17 %, and collect_view(), which
|
||||
allocates nothing, is reproducible to a few tenths of a percent. That
|
||||
contrast is itself a finding.
|
||||
"""
|
||||
by_step: dict[str, list[Row]] = {}
|
||||
for r in rows:
|
||||
by_step.setdefault(r.step, []).append(r)
|
||||
for v in by_step.values():
|
||||
v.sort(key=lambda r: r.rep)
|
||||
|
||||
def warm_of(v: list[Row]) -> Row:
|
||||
return min(v[1:] or v, key=lambda r: r.us_per_frame)
|
||||
|
||||
cpu = by_step.get("cpu")
|
||||
cpu_us = warm_of(cpu).us_per_frame if cpu else None
|
||||
|
||||
hdr = (f"{'step':<6} {'label':<38} {'cold FPS':>9} {'warm FPS':>9} "
|
||||
f"{'warm us/f':>10} {'spread':>7} {'faults c->w':>18}")
|
||||
if cpu_us:
|
||||
hdr += f" {'vs CPU':>7}"
|
||||
if floor_us:
|
||||
hdr += f" {'%floor':>7}"
|
||||
print(hdr)
|
||||
print("-" * len(hdr))
|
||||
for step, v in by_step.items():
|
||||
cold, warm = v[0], warm_of(v)
|
||||
us = [r.us_per_frame for r in v[1:]] or [v[0].us_per_frame]
|
||||
spread = 100 * (max(us) - min(us)) / min(us)
|
||||
line = (f"{step:<6} {warm.label[:38]:<38} {cold.fps:9,.0f} {warm.fps:9,.0f} "
|
||||
f"{warm.us_per_frame:10.2f} {spread:6.1f}% "
|
||||
f"{cold.minor_faults:>8,} ->{warm.minor_faults:>8,}")
|
||||
if cpu_us:
|
||||
line += f" {cpu_us / warm.us_per_frame:6.2f}x"
|
||||
if floor_us:
|
||||
line += f" {100 * floor_us / warm.us_per_frame:6.0f}%"
|
||||
print(line)
|
||||
print(" cold = rep 0 (fresh process). warm = best of reps 1..n; spread = "
|
||||
"(max-min)/min over those reps.")
|
||||
if cpu and len(cpu) == 1:
|
||||
print(" cpu is first-pass only (stop() is terminal): cold == warm.")
|
||||
@@ -0,0 +1,143 @@
|
||||
"""How many threads should the CPU baseline actually use?
|
||||
|
||||
The campaign's CPU reference was ClusterFinderMT with n_threads=48. This machine
|
||||
is a Ryzen 9 7950X: 16 physical cores, 32 logical. 48 threads oversubscribes it
|
||||
by 1.5x, so the baseline was slower than the CPU can go and every GPU speedup
|
||||
quoted against it was correspondingly flattered.
|
||||
|
||||
This sweeps the thread count at both cluster sizes and reports the best. The
|
||||
result is the number the deck and the report should divide by.
|
||||
|
||||
Two timings are recorded per point, because the campaign and the notebook do not
|
||||
measure the same thing:
|
||||
|
||||
loop_s -- the find_clusters() loop alone. This is what
|
||||
ClusterFinderCUDA_perf.ipynb prints as `CPU clustering`.
|
||||
wall_s -- loop + stop() + draining the ClusterCollector, which is what
|
||||
ladder.py's `cpu` step records, because _drive() does the drain
|
||||
inside the timed region.
|
||||
|
||||
wall_s is the one to compare against the GPU rows in ladder_*.csv: those include
|
||||
their own result collection. loop_s is here so the notebook's number can be
|
||||
reconciled with this one rather than looking like a contradiction.
|
||||
|
||||
Matches the ladder's CPU step in every other respect: same 1000 pedestal frames,
|
||||
same caps, same frame counts, a fresh finder per point, clusters retained.
|
||||
|
||||
python python/tests/perf/cpu_threads.py [--tag 2026-08-19_cpu]
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import gc
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
import common
|
||||
from common import Row, faults
|
||||
|
||||
# 8 = half the physical cores
|
||||
# 16 = one per physical core
|
||||
# 24 = 1.5x cores (the count that beat 48 in the notebook)
|
||||
# 32 = one per logical thread
|
||||
# 48 = the campaign's original, 1.5x oversubscribed
|
||||
THREADS = [8, 16, 24, 32, 48]
|
||||
|
||||
# (cluster_dim, cap, n_frames) -- exactly the ladder's `cpu` row for each size.
|
||||
# 9x9 ran at 20 000 frames there, so it stays at 20 000 here.
|
||||
CONFIGS = [(3, 3000, 100_000), (9, 1500, 20_000)]
|
||||
|
||||
|
||||
def measure(dim: int, cap: int, n_frames: int, n_threads: int, data) -> Row:
|
||||
"""One point. The finder is built, trained, driven and destroyed here.
|
||||
|
||||
ClusterFinderMT.stop() is terminal, so a point cannot be repeated on the
|
||||
same finder -- which is also why the ladder runs the CPU step with reps=1.
|
||||
"""
|
||||
from aare import ClusterFinderMT, ClusterCollector
|
||||
|
||||
cf = ClusterFinderMT(common.image_size(), (dim, dim), n_sigma=common.N_SIGMA,
|
||||
capacity=cap, n_threads=n_threads)
|
||||
sink = ClusterCollector(cf)
|
||||
common.train_pedestal(cf)
|
||||
|
||||
gc.collect()
|
||||
mf0, Mf0 = faults()
|
||||
t0 = time.perf_counter()
|
||||
for i in range(n_frames):
|
||||
cf.find_clusters(data[i])
|
||||
loop_s = time.perf_counter() - t0
|
||||
|
||||
# The drain is inside ladder.py's timed region, so it is inside ours too.
|
||||
cf.stop()
|
||||
sink.stop()
|
||||
n_clusters = 0
|
||||
for cv in sink.steal_clusters():
|
||||
n_clusters += cv.size
|
||||
wall_s = time.perf_counter() - t0
|
||||
mf1, Mf1 = faults()
|
||||
|
||||
del cf, sink
|
||||
gc.collect()
|
||||
|
||||
return Row(
|
||||
step="cpu", label=f"ClusterFinderMT, {n_threads} threads",
|
||||
cluster_dim=dim, cap=cap, n_frames=n_frames, n_streams=0, pinned=False,
|
||||
batch_chunk="n/a", collection="n/a",
|
||||
device_ped_type=common.device_ped_type(), rep=0,
|
||||
wall_s=wall_s, us_per_frame=wall_s * 1e6 / n_frames,
|
||||
fps=n_frames / wall_s, minor_faults=mf1 - mf0, major_faults=Mf1 - Mf0,
|
||||
n_clusters=n_clusters, clusters_per_frame=n_clusters / n_frames,
|
||||
notes=f"retain threads={n_threads} loop_s={loop_s:.3f} "
|
||||
f"loop_fps={n_frames / loop_s:.1f}",
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--tag", default=time.strftime("%Y-%m-%d") + "_cpu")
|
||||
ap.add_argument("--threads", type=int, nargs="*", default=THREADS)
|
||||
args = ap.parse_args()
|
||||
|
||||
out = common.results_dir(args.tag)
|
||||
rows: list[Row] = []
|
||||
|
||||
for dim, cap, n_frames in CONFIGS:
|
||||
data = common.load_frames(n_frames)
|
||||
print(f"\n=== {dim}x{dim}, cap {cap}, {n_frames:,} frames "
|
||||
f"{'=' * 30}", flush=True)
|
||||
print(f"{'threads':>8} {'wall_s':>9} {'FPS':>10} {'us/fr':>9} "
|
||||
f"{'loop FPS':>10} {'faults':>12}", flush=True)
|
||||
for n_threads in args.threads:
|
||||
r = measure(dim, cap, n_frames, n_threads, data)
|
||||
rows.append(r)
|
||||
loop_fps = float(r.notes.split("loop_fps=")[1])
|
||||
print(f"{n_threads:>8} {r.wall_s:>9.3f} {r.fps:>10,.1f} "
|
||||
f"{r.us_per_frame:>9.1f} {loop_fps:>10,.1f} "
|
||||
f"{r.minor_faults:>12,}", flush=True)
|
||||
|
||||
common.write_rows(out / "cpu_threads.csv", rows)
|
||||
common.write_env(out / "env.json", common.capture_env())
|
||||
common.append_manifest(out / "manifest.csv", dict(
|
||||
artifact="cpu_threads.csv", kind="ladder",
|
||||
config="ClusterFinderMT thread sweep, 3x3 + 9x9",
|
||||
build=common.device_ped_type(),
|
||||
cites="CPU baseline for every speedup in deck + report",
|
||||
produced_by="cpu_threads.py",
|
||||
timestamp=time.strftime("%Y-%m-%d %H:%M:%S")))
|
||||
|
||||
print(f"\nwrote {out / 'cpu_threads.csv'}")
|
||||
print("\nbest per size (wall_s convention, comparable to ladder GPU rows):")
|
||||
for dim, _, _ in CONFIGS:
|
||||
best = max((r for r in rows if r.cluster_dim == dim), key=lambda r: r.fps)
|
||||
old = {3: 5228.6, 9: 1304.0}[dim]
|
||||
print(f" {dim}x{dim}: {best.label:<32} {best.fps:>9,.1f} FPS "
|
||||
f"({best.us_per_frame:6.1f} us/fr) "
|
||||
f"vs 48-thread {old:,.1f} = {best.fps / old:.2f}x")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+78
@@ -0,0 +1,78 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Engine duty cycle over the processing window, from an nsys SQLite export.
|
||||
|
||||
nsys stats --force-export=true --report cuda_gpu_sum <rep>.nsys-rep # makes the .sqlite
|
||||
python gpu_span.py <rep>.sqlite <n_frames>
|
||||
|
||||
Reports, per engine (kernel / H2D / D2H):
|
||||
sum = total of individual op durations (what `nsys stats` gives you)
|
||||
busy = length of the UNION of those intervals (no double-counting overlaps)
|
||||
duty = busy / processing-window
|
||||
|
||||
Processing window = first kernel start -> last D2H end. Scoping matters: the trace
|
||||
also contains the pedestal-upload H2Ds that precede any kernel, and including them
|
||||
deflates every duty cycle.
|
||||
"""
|
||||
import sqlite3, sys
|
||||
|
||||
Q = {'kernel': "SELECT start,end FROM CUPTI_ACTIVITY_KIND_KERNEL ORDER BY start",
|
||||
'H2D': "SELECT start,end FROM CUPTI_ACTIVITY_KIND_MEMCPY WHERE copyKind=1 ORDER BY start",
|
||||
'D2H': "SELECT start,end FROM CUPTI_ACTIVITY_KIND_MEMCPY WHERE copyKind=2 ORDER BY start"}
|
||||
|
||||
|
||||
def union(rows):
|
||||
"""Total wall time covered by at least one interval."""
|
||||
busy, cs, ce = 0, *rows[0]
|
||||
for s, e in rows[1:]:
|
||||
if s > ce:
|
||||
busy += ce - cs
|
||||
cs, ce = s, e
|
||||
else:
|
||||
ce = max(ce, e)
|
||||
return busy + ce - cs
|
||||
|
||||
|
||||
def analyze(sqlite_path, n_frames):
|
||||
"""Per-engine sum / busy / overlap / duty / per-frame, plus the window.
|
||||
|
||||
Returns a plain dict so a driver can write it to CSV. The roofline is
|
||||
max(per_frame_us) over the three engines: PCIe is full-duplex, so the floor
|
||||
is the tallest bar, never the sum.
|
||||
"""
|
||||
db = sqlite3.connect(str(sqlite_path))
|
||||
ops = {k: db.execute(q).fetchall() for k, q in Q.items()}
|
||||
lo = ops['kernel'][0][0] # first kernel start
|
||||
hi = max(ops['kernel'][-1][1], ops['D2H'][-1][1])
|
||||
span = hi - lo
|
||||
|
||||
out = {'n_frames': n_frames, 'window_ms': span / 1e6,
|
||||
'window_us_per_frame': span / 1e3 / n_frames,
|
||||
'window_fps': n_frames / (span / 1e9)}
|
||||
for k, rows in ops.items():
|
||||
rows = [r for r in rows if r[1] > lo] # drop the pedestal-phase copies
|
||||
tot, busy = sum(e - s for s, e in rows), union(rows)
|
||||
out[f'{k}_n'] = len(rows)
|
||||
out[f'{k}_sum_ms'] = tot / 1e6
|
||||
out[f'{k}_busy_ms'] = busy / 1e6
|
||||
out[f'{k}_overlap'] = tot / busy
|
||||
out[f'{k}_duty_pct'] = 100 * busy / span
|
||||
out[f'{k}_us_per_frame'] = busy / 1e3 / n_frames
|
||||
engines = ('kernel', 'H2D', 'D2H')
|
||||
tallest = max(engines, key=lambda k: out[f'{k}_us_per_frame'])
|
||||
out['bottleneck'] = tallest
|
||||
out['roofline_us_per_frame'] = out[f'{tallest}_us_per_frame']
|
||||
out['roofline_fps'] = 1e6 / out['roofline_us_per_frame']
|
||||
return out
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
N = int(sys.argv[2]) if len(sys.argv) > 2 else 2000
|
||||
r = analyze(sys.argv[1], N)
|
||||
print(f"processing window: {r['window_ms']:.1f} ms / {N} frames = "
|
||||
f"{r['window_us_per_frame']:.1f} us/frame -> {r['window_fps']:,.0f} FPS")
|
||||
for k in ('kernel', 'H2D', 'D2H'):
|
||||
print(f" {k:6s} sum={r[k+'_sum_ms']:7.2f} ms busy={r[k+'_busy_ms']:7.2f} ms "
|
||||
f"overlap={r[k+'_overlap']:4.2f}x duty={r[k+'_duty_pct']:5.1f}% "
|
||||
f"per-frame={r[k+'_us_per_frame']:5.1f} us")
|
||||
print(f" roofline = {r['bottleneck']} at {r['roofline_us_per_frame']:.1f} us/frame "
|
||||
f"= {r['roofline_fps']:,.0f} FPS")
|
||||
Executable
+167
@@ -0,0 +1,167 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Register pressure, spills and occupancy for every ClusterFinder kernel.
|
||||
|
||||
Reads the *compiled* extension — no rebuild and no source parsing:
|
||||
|
||||
cuobjdump -res-usage <lib.so>
|
||||
|
||||
then applies the sm_89 occupancy arithmetic. Reproduces the figures on slides 7
|
||||
and 8 of docs/cf_cuda_fused.pptx, and cross-checks against
|
||||
cudaOccupancyMaxActiveBlocksPerMultiprocessor (same blocks/SM).
|
||||
|
||||
python kernel_resources.py # shipping build, 16x16 blocks
|
||||
python kernel_resources.py --blocks 64 256 1024 # the block-size sweep
|
||||
python kernel_resources.py --lib path/to/other.so
|
||||
|
||||
Two things to know when reading the output:
|
||||
|
||||
1. **Register count depends on the cluster payload type.** At 9x9 it is 96
|
||||
(float) / 120 (double) / 128 (int). The Python bindings register the *int*
|
||||
variants, so those are the rows to quote.
|
||||
|
||||
2. **Register count also depends on DEVICE_PED_TYPE**, at 3x3. The f64 pedestal
|
||||
costs 9 extra registers there — 47 vs 38 — which is enough to lose a block per
|
||||
SM and drop occupancy from 100 % to 83 %. At 9x9 nothing moves: the limiter is
|
||||
the clusterData[CSX][CSY] staging array, not the pedestal accumulators. Always
|
||||
record which build a number came from; `common.device_ped_type()` reports it.
|
||||
|
||||
SHARED reads 0 in the ELF because the finder passes shared memory dynamically at
|
||||
launch, so it is recomputed here with the launcher's own formula
|
||||
(ClusterFinderCUDA.hpp): (BLOCK_X + 2*col_radius) * (BLOCK_Y + 2*row_radius) *
|
||||
sizeof(COMPUTE_TYPE).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import math
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[3]
|
||||
DEFAULT_LIB = REPO / "build/aare/_aare_cuda.cpython-311-x86_64-linux-gnu.so"
|
||||
|
||||
# --- RTX 4090 / sm_89, from cudaDeviceProp ---------------------------------
|
||||
REGS_PER_SM = 65536
|
||||
MAX_THREADS_PER_SM = 1536
|
||||
MAX_BLOCKS_PER_SM = 24
|
||||
SMEM_PER_SM = 102400 # opt-in maximum; 48 KB is the default carveout
|
||||
WARP = 32
|
||||
REG_ALLOC_GRANULARITY = 256 # registers are allocated per warp, rounded up
|
||||
|
||||
KERNEL_RE = re.compile(r"\s*Function (\S+):")
|
||||
USAGE_RE = re.compile(r"REG:(\d+) STACK:(\d+) SHARED:(\d+) LOCAL:(\d+)")
|
||||
TEMPLATE_RE = re.compile(
|
||||
r"Cluster<(\w+), \(unsigned char\)(\d+), \(unsigned char\)(\d+)")
|
||||
|
||||
|
||||
def demangle(name: str) -> str:
|
||||
try:
|
||||
import cxxfilt
|
||||
return cxxfilt.demangle(name)
|
||||
except Exception:
|
||||
out = subprocess.run(["c++filt", name], capture_output=True, text=True)
|
||||
return out.stdout.strip() or name
|
||||
|
||||
|
||||
def read_res_usage(lib: Path) -> list[tuple[str, int, int, int, int]]:
|
||||
"""[(mangled_name, regs, stack, shared, local), ...] straight from the ELF."""
|
||||
proc = subprocess.run(["cuobjdump", "-res-usage", str(lib)],
|
||||
capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
sys.exit(f"cuobjdump failed:\n{proc.stderr}")
|
||||
rows, cur = [], None
|
||||
for line in proc.stdout.splitlines():
|
||||
m = KERNEL_RE.match(line)
|
||||
if m:
|
||||
cur = m.group(1)
|
||||
continue
|
||||
m = USAGE_RE.search(line)
|
||||
if m and cur:
|
||||
rows.append((cur, *map(int, m.groups())))
|
||||
cur = None
|
||||
return rows
|
||||
|
||||
|
||||
def occupancy(regs: int, smem_per_block: int, block: int):
|
||||
"""Blocks/SM, warps/SM, occupancy %, and which resource binds."""
|
||||
warps_per_block = math.ceil(block / WARP)
|
||||
per_warp = math.ceil(regs * WARP / REG_ALLOC_GRANULARITY) * REG_ALLOC_GRANULARITY
|
||||
limits = {
|
||||
"registers": (REGS_PER_SM // per_warp) // warps_per_block,
|
||||
"threads/SM": MAX_THREADS_PER_SM // block,
|
||||
"shared mem": SMEM_PER_SM // smem_per_block if smem_per_block else 1 << 20,
|
||||
"blocks/SM": MAX_BLOCKS_PER_SM,
|
||||
}
|
||||
blocks = min(limits.values())
|
||||
binder = min(limits, key=lambda k: (limits[k], k))
|
||||
warps = blocks * warps_per_block
|
||||
return blocks, warps, 100.0 * warps / (MAX_THREADS_PER_SM // WARP), binder
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--lib", type=Path, default=DEFAULT_LIB)
|
||||
ap.add_argument("--blocks", type=int, nargs="+", default=[256],
|
||||
help="threads per block (default 256 = the 16x16 the finder launches)")
|
||||
ap.add_argument("--all-types", action="store_true",
|
||||
help="also show the float/double cluster payloads (bindings use int)")
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.lib.exists():
|
||||
sys.exit(f"no such library: {args.lib}\nBuild it first: cmake --build build -j16")
|
||||
|
||||
try:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import common
|
||||
build = common.device_ped_type()
|
||||
except Exception:
|
||||
build = "unknown"
|
||||
|
||||
kernels = []
|
||||
for name, regs, stack, shared, local in read_res_usage(args.lib):
|
||||
d = demangle(name)
|
||||
if "find_clusters_in_single_frame" not in d:
|
||||
continue
|
||||
m = TEMPLATE_RE.search(d)
|
||||
if not m:
|
||||
continue
|
||||
ctype, sx, sy = m.group(1), int(m.group(2)), int(m.group(3))
|
||||
if not args.all_types and ctype != "int":
|
||||
continue
|
||||
pipeline = "opt2" if "device_opt2" in d else "current"
|
||||
kernels.append((pipeline, sx, sy, ctype, regs, stack + local))
|
||||
|
||||
kernels.sort(key=lambda k: (k[0] != "current", k[1]))
|
||||
|
||||
print(f"library {args.lib}")
|
||||
print(f"build DEVICE_PED_TYPE = {build}")
|
||||
print(f"payload {'all cluster types' if args.all_types else 'int (what the bindings register)'}")
|
||||
print()
|
||||
|
||||
for block in args.blocks:
|
||||
side = int(math.isqrt(block))
|
||||
print(f"--- {side}x{side} blocks · {block} threads ---")
|
||||
print(f"{'kernel':28} {'regs':>5} {'spill':>6} {'smem/blk':>9} "
|
||||
f"{'blk/SM':>7} {'warps':>6} {'occupancy':>10} limited by")
|
||||
print("-" * 92)
|
||||
for pipeline, sx, sy, ctype, regs, spill in kernels:
|
||||
# the launcher's own formula; COMPUTE_TYPE is float in every build
|
||||
smem = (side + sy - 1) * (side + sx - 1) * 4
|
||||
blocks, warps, occ, binder = occupancy(regs, smem, block)
|
||||
label = f"[{pipeline}] {sx}x{sy} Cluster<{ctype}>"
|
||||
note = " <- will not launch" if blocks == 0 else ""
|
||||
print(f"{label:28} {regs:5} {spill:6} {smem:8} B {blocks:7} {warps:6} "
|
||||
f"{occ:9.1f}% {binder}{note}")
|
||||
print()
|
||||
|
||||
print(f"sm_89: {REGS_PER_SM:,} regs/SM · {MAX_THREADS_PER_SM:,} threads/SM · "
|
||||
f"{MAX_BLOCKS_PER_SM} blocks/SM · {REG_ALLOC_GRANULARITY}-reg granularity")
|
||||
print("Spills must be 0. A non-zero STACK/LOCAL means ptxas gave up and went "
|
||||
"to local memory,\nwhich costs far more than the occupancy it buys back.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,319 @@
|
||||
"""The opt1 → opt8 ladder as a configuration matrix.
|
||||
|
||||
The point of this file is that the ladder is *data*, not code. Every step runs
|
||||
through the same measure() function, so nothing can differ between two steps
|
||||
except the fields in their STEP entry. Previous campaigns drifted precisely
|
||||
because each step lived in its own notebook cell with its own warm-up policy.
|
||||
|
||||
Three classes cover eight steps:
|
||||
|
||||
ClusterFinderCUDAOpt2 opt1, opt2 frozen pre-refactor pipeline (88e0e8d)
|
||||
ClusterFinderCUDAGraph opt5 graph-based
|
||||
ClusterFinderCUDA opt3, opt4, opt7, opt8
|
||||
|
||||
opt3/opt4 are reachable on the *current* ClusterFinderCUDA because batch_chunk
|
||||
disables the internal chunking that opt7 added: set it to the batch size and
|
||||
find_clusters_batched degenerates to submit-everything-then-collect-everything.
|
||||
This is why the old pipeline does not need to be kept alive in a second class.
|
||||
|
||||
opt6 is NOT a row here — it is the DEVICE_PED_TYPE build axis, so the whole
|
||||
matrix is run once per build and the two result sets are compared.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import gc
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Callable
|
||||
|
||||
import numpy as np
|
||||
|
||||
import common
|
||||
from common import Row, faults, load_frames, image_size, train_pedestal
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Step:
|
||||
step: str
|
||||
label: str
|
||||
cls: str # "opt2cls" | "graph" | "cuda"
|
||||
n_streams: int = 4
|
||||
pinned: bool = True
|
||||
batch_chunk: str = "auto" # "auto" | "off" | int
|
||||
collection: str = "collect"
|
||||
per_frame: bool = False # opt1: one find_clusters() call per frame
|
||||
act: str = ""
|
||||
status: str = "adopted"
|
||||
# ClusterFinderCUDAOpt2 is registered for 3x3 only (cuda_bindings.cu), so
|
||||
# the 9x9 ladder necessarily starts at opt3. Not a limitation of the
|
||||
# measurement — the deck's arc is the 3x3 one.
|
||||
cluster_dims: tuple[int, ...] = (3, 5, 7, 9)
|
||||
|
||||
|
||||
# ---- the ladder ----------------------------------------------------------
|
||||
STEPS: list[Step] = [
|
||||
# ClusterFinderMT cannot restart after stop(), so this is ALWAYS a
|
||||
# first-pass number carrying its own ~1.8 s of allocator faults (docs §3.4).
|
||||
# measure() forces reps=1 for it. Every speedup quoted against it must say
|
||||
# whether it uses the raw or the fault-corrected baseline.
|
||||
Step("cpu", "ClusterFinderMT", "cpu", # thread count appended per size
|
||||
n_streams=0, pinned=False, batch_chunk="n/a", collection="n/a",
|
||||
act="baseline", status="reference"),
|
||||
Step("opt1", "1 stream, one launch per frame", "opt2cls",
|
||||
n_streams=1, pinned=False, collection="per-frame", per_frame=True,
|
||||
act="I - getting frames to the GPU", cluster_dims=(3,)),
|
||||
Step("opt2", "4 streams + host-side batching", "opt2cls",
|
||||
n_streams=4, pinned=False, act="I - getting frames to the GPU",
|
||||
cluster_dims=(3,)),
|
||||
Step("opt3", "pipeline rework, no pinning", "cuda",
|
||||
pinned=False, batch_chunk="off", act="I - getting frames to the GPU"),
|
||||
Step("opt4", "+ pinned input (DMA H2D)", "cuda",
|
||||
pinned=True, batch_chunk="off", act="I - getting frames to the GPU"),
|
||||
Step("opt5", "CUDA Graphs", "graph",
|
||||
pinned=True, act="I - getting frames to the GPU", status="rejected"),
|
||||
Step("opt7", "host<->GPU overlap, chunked internally", "cuda",
|
||||
pinned=True, batch_chunk="auto", act="III - getting results back"),
|
||||
Step("opt8", "zero-copy collection (collect_view)", "cuda",
|
||||
pinned=True, batch_chunk="auto", collection="collect_view",
|
||||
act="III - getting results back"),
|
||||
]
|
||||
|
||||
STEPS_BY_NAME = {s.step: s for s in STEPS}
|
||||
|
||||
# The CPU baseline's ClusterCollector must outlive the finder's worker threads
|
||||
# but cannot be attached to it — pybind11 classes have no __dict__, and the
|
||||
# AttributeError from trying leaves ClusterFinderMT half-built with 48 threads
|
||||
# running, which aborts the process at destruction. Only one CPU step runs at a
|
||||
# time, so a module-level holder is sufficient and explicit.
|
||||
_cpu_sink = None
|
||||
|
||||
# Frames whose returned cluster count landed exactly ON the cap. The kernel bumps
|
||||
# its counter for every detection but guards only the write
|
||||
# (clusterfinder_kernel.cuh: `if (write_idx >= max_clusters) return;`) and the host
|
||||
# then clamps n_found to the cap, so a truncated frame is indistinguishable from a
|
||||
# frame that happened to contain exactly `cap` clusters -- and silently short.
|
||||
# With the cap set well above the observed maximum, landing on it is effectively
|
||||
# impossible by chance, so this counter is a sound truncation detector. It exists
|
||||
# because a cap that was believed non-truncating silently discarded 0.0095 % of
|
||||
# 9x9 clusters through an entire campaign.
|
||||
_at_cap = 0
|
||||
|
||||
# Best ClusterFinderMT thread count per cluster size, swept 2026-08-19
|
||||
# (results/2026-08-19_cpu_threads/). This is a 16-core / 32-thread 7950X: the
|
||||
# campaign's original n_threads=48 oversubscribed it by 1.5x and understated the
|
||||
# CPU by 24 % at 3x3 and 15 % at 9x9, inflating every speedup. The optima differ
|
||||
# by size because ClusterCollector's drain is inside the timed region and 9x9
|
||||
# clusters are 9x larger.
|
||||
CPU_THREADS = {3: 24, 9: 32}
|
||||
|
||||
|
||||
# ---- finder construction -------------------------------------------------
|
||||
def build_finder(step: Step, cluster_dim: int, cap: int, batch_size: int):
|
||||
"""A FRESH finder, trained on the same 1000 pedestal frames.
|
||||
|
||||
Never reuse one across steps: the device pedestal keeps evolving, so two
|
||||
steps sharing a finder are not measuring the same thing.
|
||||
"""
|
||||
from aare import ClusterFinderCUDA, ClusterFinderCUDAGraph, ClusterFinderCUDAOpt2
|
||||
|
||||
dim = (cluster_dim, cluster_dim)
|
||||
common_kw = dict(image_size=image_size(), cluster_size=dim, n_sigma=common.N_SIGMA,
|
||||
max_clusters_per_frame=cap, n_streams=step.n_streams)
|
||||
|
||||
if step.cls == "cpu":
|
||||
from aare import ClusterFinderMT, ClusterCollector
|
||||
|
||||
global _cpu_sink
|
||||
cf = ClusterFinderMT(image_size(), dim, n_sigma=common.N_SIGMA,
|
||||
capacity=cap,
|
||||
n_threads=CPU_THREADS[cluster_dim])
|
||||
_cpu_sink = ClusterCollector(cf)
|
||||
train_pedestal(cf)
|
||||
return cf
|
||||
|
||||
if step.cls == "opt2cls":
|
||||
# time_kernels kept OFF for comparability with the ClusterFinderCUDA
|
||||
# steps, which default to off. Without this, opt1/opt2 pay an event tax
|
||||
# that opt3+ do not, inflating the opt2 -> opt3 step (docs §11).
|
||||
cf = ClusterFinderCUDAOpt2(**common_kw, time_kernels=False)
|
||||
elif step.cls == "graph":
|
||||
cf = ClusterFinderCUDAGraph(**common_kw)
|
||||
elif step.cls == "cuda":
|
||||
cf = ClusterFinderCUDA(**common_kw, time_kernels=False)
|
||||
else:
|
||||
raise ValueError(step.cls)
|
||||
|
||||
train_pedestal(cf)
|
||||
|
||||
if step.cls == "cuda":
|
||||
cf.batch_chunk = batch_size if step.batch_chunk == "off" else (
|
||||
0 if step.batch_chunk == "auto" else int(step.batch_chunk))
|
||||
return cf
|
||||
|
||||
|
||||
# ---- the measured region -------------------------------------------------
|
||||
def steps_for(cluster_dim: int) -> list[Step]:
|
||||
"""The steps that can actually run at this cluster size."""
|
||||
return [s for s in STEPS if cluster_dim in s.cluster_dims]
|
||||
|
||||
|
||||
def measure(step: Step, cluster_dim: int, cap: int, n_frames: int,
|
||||
batch_size: int, reps: int, retain: bool = False) -> list[Row]:
|
||||
"""Run one ladder step `reps` times, returning one Row per rep.
|
||||
|
||||
Policy applied identically to every step:
|
||||
- fresh finder, same pedestal
|
||||
- input pinned only if the step says so (that IS opt4)
|
||||
- output slots pre-pinned OUTSIDE the timer, without processing frames
|
||||
- faults bracketed around the timed region only
|
||||
|
||||
Reps deliberately SHARE one finder, because the whole point of repeating is
|
||||
to reach the heap plateau, and a new finder would reset it. Two consequences
|
||||
to keep in mind when reading the CSV:
|
||||
|
||||
* quote the last rep, not the mean — earlier reps carry first-touch faults;
|
||||
* n_clusters drifts by ~0.002 % between reps, because the device pedestal
|
||||
advances by n_frames each pass. This is expected and is why steps are
|
||||
compared against each other at the same rep index, never across reps.
|
||||
"""
|
||||
data = load_frames(n_frames)
|
||||
rows: list[Row] = []
|
||||
if step.cls == "cpu":
|
||||
reps = 1 # stop() is terminal; a second pass is impossible
|
||||
cf = build_finder(step, cluster_dim, cap, batch_size)
|
||||
|
||||
if step.pinned:
|
||||
cf.register_input_buffer(data)
|
||||
|
||||
# Pre-pin the output slots. Allocation only: no transfer, no launch, and
|
||||
# crucially no pedestal advance, so this cannot perturb the result. Sized to
|
||||
# the largest batch this step will submit (docs §12.1).
|
||||
if hasattr(cf, "reserve_output_slots"):
|
||||
slot_frames = batch_size if step.batch_chunk == "off" else (
|
||||
cf.chunk_size_for(min(n_frames, batch_size))
|
||||
if hasattr(cf, "chunk_size_for") else batch_size)
|
||||
cf.reserve_output_slots(slot_frames)
|
||||
|
||||
try:
|
||||
rows = _reps(cf, step, cluster_dim, cap, n_frames, batch_size, reps,
|
||||
retain, data)
|
||||
finally:
|
||||
# A finder that escapes here still owns worker threads (CPU) or CUDA
|
||||
# streams, and destroying it implicitly aborts the process. Release it
|
||||
# explicitly whatever happened.
|
||||
if step.pinned:
|
||||
try:
|
||||
cf.unregister_input_buffer()
|
||||
except Exception:
|
||||
pass
|
||||
del cf
|
||||
gc.collect()
|
||||
return rows
|
||||
|
||||
|
||||
def _reps(cf, step: Step, cluster_dim: int, cap: int, n_frames: int,
|
||||
batch_size: int, reps: int, retain: bool, data) -> list[Row]:
|
||||
global _at_cap
|
||||
rows: list[Row] = []
|
||||
for rep in range(reps):
|
||||
gc.collect()
|
||||
_at_cap = 0
|
||||
mf0, Mf0 = faults()
|
||||
t0 = time.perf_counter()
|
||||
n_clusters = _drive(cf, step, data, n_frames, batch_size, cap, retain)
|
||||
wall = time.perf_counter() - t0
|
||||
mf1, Mf1 = faults()
|
||||
|
||||
if _at_cap:
|
||||
print(f" !! {step.step} {cluster_dim}x{cluster_dim} rep {rep}: "
|
||||
f"{_at_cap} frame(s) returned exactly cap={cap} clusters — "
|
||||
f"the cap is truncating and this row undercounts. Raise it.",
|
||||
file=sys.stderr, flush=True)
|
||||
|
||||
# The CPU baseline's thread count varies by cluster size, so it goes in
|
||||
# the label rather than being hardcoded: a row must say what it ran.
|
||||
label = (f"{step.label}, {CPU_THREADS[cluster_dim]} threads"
|
||||
if step.cls == "cpu" else step.label)
|
||||
rows.append(Row(
|
||||
step=step.step, label=label, cluster_dim=cluster_dim, cap=cap,
|
||||
n_frames=n_frames, n_streams=step.n_streams, pinned=step.pinned,
|
||||
batch_chunk=step.batch_chunk, collection=step.collection,
|
||||
device_ped_type=common.device_ped_type(), rep=rep,
|
||||
wall_s=wall, us_per_frame=wall * 1e6 / n_frames,
|
||||
fps=n_frames / wall, minor_faults=mf1 - mf0, major_faults=Mf1 - Mf0,
|
||||
n_clusters=n_clusters, clusters_per_frame=n_clusters / n_frames,
|
||||
notes=" ".join(filter(None, [
|
||||
step.status if step.status != "adopted" else "",
|
||||
"retain" if retain else "",
|
||||
f"TRUNCATED at_cap={_at_cap}" if _at_cap else "",
|
||||
])),
|
||||
))
|
||||
|
||||
return rows
|
||||
|
||||
|
||||
def _drive(cf, step: Step, data, n_frames: int, batch_size: int, cap: int,
|
||||
retain: bool = False) -> int:
|
||||
"""The timed work. One branch per collection strategy, nothing else.
|
||||
|
||||
`retain=False` (default) counts each batch and lets it die, so peak result
|
||||
memory is one batch. This is what makes N=100 000 possible at 9x9, where
|
||||
retaining every ClusterVector would need 46.6 GB, and it is also the only
|
||||
mode in which opt8 is comparable — a BatchView cannot be retained.
|
||||
|
||||
`retain=True` keeps everything, which is what the notebook does. It measures
|
||||
the finder PLUS a growing result heap; at 9x9 that heap is re-faulted every
|
||||
pass because it sits above glibc's mmap threshold (docs §12.2).
|
||||
"""
|
||||
global _at_cap
|
||||
total = 0
|
||||
kept: list = [] if retain else None
|
||||
|
||||
# The CPU finder's `cap` is a ClusterVector capacity, which grows on demand,
|
||||
# so it cannot truncate and is not checked.
|
||||
if step.cls == "cpu":
|
||||
for i in range(n_frames):
|
||||
cf.find_clusters(data[i])
|
||||
cf.stop() # terminal — this is why the CPU step cannot be repped
|
||||
_cpu_sink.stop()
|
||||
for cv in _cpu_sink.steal_clusters():
|
||||
total += cv.size
|
||||
return total
|
||||
|
||||
if step.per_frame:
|
||||
# opt1: no batching at all — one find_clusters() call per frame, and the
|
||||
# result stolen out of the finder's internal vector each time. This is
|
||||
# the fully synchronous path: H2D, kernel and D2H cannot overlap because
|
||||
# there is only one frame in flight.
|
||||
for i in range(n_frames):
|
||||
cf.find_clusters(data[i], i)
|
||||
cv = cf.steal_clusters()
|
||||
total += cv.size
|
||||
_at_cap += cv.size == cap
|
||||
if retain:
|
||||
kept.append(cv)
|
||||
return total
|
||||
|
||||
if step.collection == "collect_view":
|
||||
from aare import find_cluster_views_batched_iter
|
||||
|
||||
for start in range(0, n_frames, batch_size):
|
||||
stop = min(start + batch_size, n_frames)
|
||||
for v in find_cluster_views_batched_iter(
|
||||
cf, data[start:stop], first_frame=start):
|
||||
# No retain branch: a view is borrowed and released at the end of
|
||||
# this iteration. That constraint IS opt8.
|
||||
total += v.total_clusters
|
||||
_at_cap += int(np.count_nonzero(np.asarray(v.counts) == cap))
|
||||
return total
|
||||
|
||||
for start in range(0, n_frames, batch_size):
|
||||
stop = min(start + batch_size, n_frames)
|
||||
batch = cf.find_clusters_batched(data[start:stop], first_frame=start)
|
||||
for cv in batch:
|
||||
total += cv.size
|
||||
_at_cap += cv.size == cap
|
||||
if retain:
|
||||
kept.extend(batch)
|
||||
return total
|
||||
@@ -0,0 +1,66 @@
|
||||
# Minimal probe for nsys: train pedestal, run one batched pass, print summary.
|
||||
# Usage: nsys_kernel_probe.py [n_streams] [n_frames] [cluster_dim] [cap] [batch]
|
||||
#
|
||||
# nsys profile --trace=cuda --sample=none --cpuctxsw=none -o rep \
|
||||
# python nsys_kernel_probe.py 4 2000 9 1700
|
||||
# nsys stats --report cuda_gpu_sum rep.nsys-rep # per-op totals
|
||||
# python gpu_span.py rep.sqlite 2000 # engine duty cycles
|
||||
#
|
||||
# The wall time printed here is NOT a throughput number: a fresh process pays the
|
||||
# full first-touch page-fault tax inside the timed call, and nsys inflates it
|
||||
# further. Take per-operation GPU times from the reports, wall times from the
|
||||
# notebook. See docs/ClusterFinderCUDA_benchmark_results.md sections 3.3 and 14.
|
||||
import sys
|
||||
sys.path.append('/home/ferjao_k/aare/build')
|
||||
|
||||
from pathlib import Path
|
||||
import time
|
||||
from aare import File, ClusterFinderCUDA
|
||||
|
||||
n_streams = int(sys.argv[1]) if len(sys.argv) > 1 else 8
|
||||
N = int(sys.argv[2]) if len(sys.argv) > 2 else 2000
|
||||
cdim = int(sys.argv[3]) if len(sys.argv) > 3 else 9
|
||||
cap = int(sys.argv[4]) if len(sys.argv) > 4 else 1700
|
||||
# Loop in BATCH_SIZE slices exactly as run_ladder.py does, so the duty cycles
|
||||
# describe the configuration the throughput numbers were taken on. One giant
|
||||
# call would chunk differently and give a different overlap picture.
|
||||
batch = int(sys.argv[5]) if len(sys.argv) > 5 else 2000
|
||||
|
||||
base = Path('/mnt/sls_det_storage/moench_data/2603_MaxIVBeamtime/2026032408/process/xrf/')
|
||||
f = File(base / 'Cu_factor_10_data_master_0.json')
|
||||
pd = File(base / 'Cu_factor_10_pedestal_master_0.json')
|
||||
|
||||
cf = ClusterFinderCUDA((f.rows, f.cols), (cdim, cdim), n_sigma=5,
|
||||
max_clusters_per_frame=cap, n_streams=n_streams)
|
||||
for _ in range(1000):
|
||||
cf.push_pedestal_frame(pd.read_frame().copy())
|
||||
|
||||
data = f.read_n(N)
|
||||
cf.register_input_buffer(data)
|
||||
|
||||
cf.reserve_output_slots(cf.chunk_size_for(min(N, batch)))
|
||||
|
||||
t0 = time.perf_counter()
|
||||
n = 0
|
||||
for start in range(0, N, batch):
|
||||
stop = min(start + batch, N)
|
||||
# Counted and discarded per batch, matching run_ladder.py's default
|
||||
# consumer: peak result memory is one batch, not the whole run.
|
||||
for cv in cf.find_clusters_batched(data[start:stop], first_frame=start):
|
||||
n += cv.size
|
||||
t = time.perf_counter() - t0
|
||||
|
||||
cf.unregister_input_buffer()
|
||||
|
||||
# Transfer sizes per frame, so the nsys memcpy rows can be checked against the
|
||||
# payload they carry: the D2H is cap-sized regardless of how many clusters were
|
||||
# found, which is what makes `cap` a throughput knob and not just a safety bound.
|
||||
h2d = f.rows * f.cols * 2
|
||||
slot = 2 + 2 + cdim * cdim * 4 # x, y (uint16) + data (int32)
|
||||
print(f'n_streams={n_streams} N={N} cluster={cdim}x{cdim} cap={cap} batch={batch}')
|
||||
print(f' H2D/frame={h2d:,} B D2H/frame={cap * slot:,} B '
|
||||
f'({slot} B/slot, {100 * n / N / cap:.0f}% filled)')
|
||||
print(f' wall={t:.3f}s ({N/t:.0f} FPS, profiler-inflated) clusters/frame={n/N:.2f}')
|
||||
if cf.kernel_timing_enabled():
|
||||
print(f' event kernel_ms={cf.avg_kernel_time_ms():.3f} '
|
||||
f'(only meaningful at n_streams=1)')
|
||||
@@ -0,0 +1,5 @@
|
||||
# Binary profiler artifacts: large (0.5-2 MB each, ~40 MB per campaign) and
|
||||
# regenerable from the scripts. They stay on disk for nsys-ui; the CSV/JSON
|
||||
# summaries next to them are what the report cites and what gets tracked.
|
||||
*.nsys-rep
|
||||
*.sqlite
|
||||
@@ -0,0 +1,32 @@
|
||||
# Superseded — do not cite
|
||||
|
||||
These profiles predate the `perf/` harness. They are kept only because they are
|
||||
the provenance for numbers still quoted in older revisions of
|
||||
`docs/ClusterFinderCUDA_benchmark_results.md`. **Every one of them has been replaced** by
|
||||
`../2026-08-18_f32/` and `../2026-08-18_f64/`.
|
||||
|
||||
Three reasons they were retired rather than reused:
|
||||
|
||||
1. **2 000 frames.** Too short for the GPU clocks to ramp (210 MHz idle ->
|
||||
3.1 GHz boost), which under-reports the GPU by ~7-10 %. This is how a
|
||||
26.7 us/frame roofline was published for a pipeline that sustains 24.25.
|
||||
2. **No build record.** Nothing in these files says which `DEVICE_PED_TYPE` they
|
||||
were taken on. `probe_s1.nsys-rep` was described in the report as the f64
|
||||
reference; `nsys stats` shows a 25.2 us kernel, i.e. it is **f32**. The
|
||||
entire f64 arm was therefore unsupported until the 2026-08-18 campaign.
|
||||
3. **One giant call, not the ladder's batching.** They drove
|
||||
`find_clusters_batched(all_frames)` rather than looping in BATCH_SIZE
|
||||
slices, so their overlap and duty cycles describe a configuration the
|
||||
throughput numbers were never taken on.
|
||||
|
||||
| file | what it actually is |
|
||||
|---|---|
|
||||
| `probe_s1.nsys-rep` | 9x9, 1 stream, **f32** (25.2 us kernel) — mislabelled as f64 in the report |
|
||||
| `probe3x3_s4.*` | 3x3, 4 streams, events ON |
|
||||
| `probe3x3_s4_no_timing.*` | 3x3, 4 streams, events OFF |
|
||||
| `probe3x3_s1_no_timing.*` | 3x3, 1 stream |
|
||||
| `probe9x9_s1_no_timing_cap_1500.*` | 9x9, 1 stream, cap 1500 |
|
||||
| `probe9x9_s1_no_timing.*` | 9x9, 1 stream, cap 3000 |
|
||||
| `probe9x9_s4_no_timing.*` | 9x9, 4 streams, cap 3000 |
|
||||
|
||||
Safe to delete once the report cites only the 2026-08-18 campaign.
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"timestamp": "2026-08-18 11:10:35",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "float",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15",
|
||||
"note":
|
||||
"ladder_*.csv were taken 2026-08-13 and probes.csv 2026-08-18, same f32 build (git 7177f00, DEVICE_PED_TYPE=float), no rebuild between them. Merged here so the arm is one directory."
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 48 threads",3,3000,100000,0,False,n/a,n/a,float,0,19.125628537032753,191.25628537032753,5228.58633411033,2664805,0,233085343,2330.85343,reference
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,float,0,6.339732066029683,63.39732066029683,15773.537266004043,797,0,233094770,2330.9477,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,float,1,6.308290184941143,63.08290184941143,15852.155983362236,0,0,233094984,2330.94984,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,float,2,6.3270947767887264,63.270947767887264,15805.04220781632,0,0,233094984,2330.94984,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,float,3,6.326483318815008,63.26483318815008,15806.569773542162,0,0,233094984,2330.94984,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,float,4,6.329101589974016,63.29101589974016,15800.030790848872,0,0,233094984,2330.94984,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,float,0,4.077369901817292,40.77369901817292,24525.613914849815,196111,0,233093553,2330.93553,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,float,1,4.201830665813759,42.01830665813759,23799.150406893714,308497,0,233094462,2330.94462,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,float,2,3.986788455862552,39.86788455862552,25082.84578103223,60192,0,233094462,2330.94462,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,float,3,3.9811147190630436,39.811147190630436,25118.5929210136,60024,0,233094462,2330.94462,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,float,4,4.107359660090879,41.073596600908786,24346.54091085547,150292,0,233094462,2330.94462,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,float,0,3.4708277301397175,34.708277301397175,28811.571122250578,92259,0,233093554,2330.93554,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,float,1,3.4449734720401466,34.449734720401466,29027.79972374621,51811,0,233094465,2330.94465,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,float,2,3.448159068124369,34.48159068124369,29000.982270343793,51808,0,233094465,2330.94465,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,float,3,3.440851571969688,34.40851571969688,29062.573002169866,51803,0,233094465,2330.94465,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,float,4,3.413981437915936,34.13981437915936,29291.31332976578,3461,0,233094465,2330.94465,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,float,0,2.558677193010226,25.58677193010226,39082.69486794942,92309,0,233093554,2330.93554,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,float,1,2.524539733072743,25.245397330727428,39611.180877824816,51427,0,233094465,2330.94465,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,float,2,2.490160754881799,24.901607548817992,40158.04995880546,951,0,233094465,2330.94465,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,float,3,2.5080041729379445,25.080041729379445,39872.34195183068,26340,0,233094465,2330.94465,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,float,4,2.5344345020130277,25.344345020130277,39456.5335661951,51751,0,233094465,2330.94465,
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,float,0,2.5488168040756136,25.488168040756136,39233.89073710508,151070,0,233093554,2330.93554,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,float,1,2.419733207905665,24.19733207905665,41326.87011662427,953,0,233094465,2330.94465,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,float,2,2.4206148399971426,24.206148399971426,41311.818116474096,46,0,233094465,2330.94465,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,float,3,2.4250832318793982,24.250832318793982,41235.698092927596,1094,0,233094465,2330.94465,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,float,4,2.4093486091587692,24.093486091587692,41504.994179698755,1026,0,233094465,2330.94465,rejected
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,float,0,1.994967577047646,19.94967577047646,50126.127938374855,95899,0,233093554,2330.93554,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,float,1,1.9801588000264019,19.80158800026402,50501.000222137074,30450,0,233094465,2330.94465,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,float,2,1.9894863478839397,19.894863478839397,50264.230315710454,67370,0,233094465,2330.94465,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,float,3,1.980688118841499,19.80688118841499,50487.504341920234,57917,0,233094465,2330.94465,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,float,4,2.0119610340334475,20.119610340334475,49702.7518468022,95398,0,233094465,2330.94465,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,float,0,1.6354960668832064,16.354960668832064,61143.52826942089,1438,0,233093554,2330.93554,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,float,1,1.6309949120040983,16.309949120040983,61312.26974652189,0,0,233094465,2330.94465,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,float,2,1.631136188050732,16.31136188050732,61306.9593652408,0,0,233094465,2330.94465,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,float,3,1.6322539390530437,16.322539390530437,61264.97697902034,0,0,233094465,2330.94465,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,float,4,1.6319843179080635,16.319843179080635,61275.098604001054,0,0,233094465,2330.94465,
|
||||
|
@@ -0,0 +1,27 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 48 threads",9,1500,20000,0,False,n/a,n/a,float,0,15.337030207971111,766.8515103985555,1304.0334229507748,2967117,0,28438072,1421.9036,reference
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,float,0,1.8587155409622937,92.93577704811469,10760.11878054541,520359,0,28439289,1421.96445,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,float,1,1.6278529588598758,81.39264794299379,12286.12196890787,126986,0,28447972,1422.3986,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,float,2,1.8940563639625907,94.70281819812953,10559.347852857782,519312,0,28452043,1422.60215,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,float,3,1.7711672731675208,88.55836365837604,11291.988228888395,356562,0,28453565,1422.67825,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,float,4,1.7310039640869945,86.55019820434973,11553.988560938307,355909,0,28454173,1422.70865,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,float,0,1.7293723039329052,86.46861519664526,11564.889731676854,520357,0,28439288,1421.9644,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,float,1,1.4968323530629277,74.84161765314639,13361.549781493257,126986,0,28447972,1422.3986,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,float,2,1.7610446740873158,88.05223370436579,11356.895310089314,519296,0,28452045,1422.60225,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,float,3,1.6390001631807536,81.95000815903768,12202.561323231756,355738,0,28453565,1422.67825,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,float,4,1.731389197986573,86.56945989932865,11551.417799797951,482775,0,28454174,1422.7087,
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,float,0,2.0218078319448978,101.09039159724489,9892.136969694497,760653,0,28439289,1421.96445,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,float,1,1.9103619859088212,95.51809929544106,10469.220047050585,683396,0,28447973,1422.39865,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,float,2,1.7221080309245735,86.10540154622868,11613.673266050739,484224,0,28452044,1422.6022,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,float,3,1.630410891957581,81.52054459787905,12266.846411941382,420603,0,28453565,1422.67825,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,float,4,1.4879669798538089,74.39834899269044,13441.158487243432,191861,0,28454174,1422.7087,rejected
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,float,0,1.5269549458753318,76.34774729376659,13097.963403586173,465095,0,28439289,1421.96445,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,float,1,1.2762275428976864,63.81137714488432,15671.186624439883,130398,0,28447972,1422.3986,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,float,2,1.2400420890189707,62.002104450948536,16128.484812819956,53083,0,28452043,1422.60215,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,float,3,1.2918187589384615,64.59093794692308,15482.047974310877,130484,0,28453565,1422.67825,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,float,4,1.3914556668605655,69.57278334302828,14373.436737028398,244033,0,28454174,1422.7087,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,float,0,0.47659120289608836,23.829560144804418,41964.68562253471,1445,0,28439289,1421.96445,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,float,1,0.4731572079472244,23.65786039736122,42269.24934055065,0,0,28447971,1422.39855,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,float,2,0.4732611121144146,23.66305560572073,42259.96915454326,0,0,28452042,1422.6021,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,float,3,0.473247610963881,23.66238054819405,42261.174777544584,0,0,28453565,1422.67825,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,float,4,0.4731025758665055,23.655128793325275,42274.13043222018,0,0,28454174,1422.7087,
|
||||
|
@@ -0,0 +1,7 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
probe_3x3_s4.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,3x3 cap=3000 N=20000 streams=4 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:10:35
|
||||
probe_3x3_s1_uncontended.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,3x3 cap=3000 N=20000 streams=1 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:10:35
|
||||
probe_9x9_s4.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=4 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:10:35
|
||||
probe_9x9_s1_uncontended.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=1 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:10:35
|
||||
ladder_3x3.csv,end-to-end wall/FPS,3x3 cap=3000 N=100000 batch=2000 streams=4 consumer=streaming,float,"docs/benchmark_opt1_opt6_results.md §4, §6, §10, §10.3",perf/run_ladder.py,2026-08-13 17:57:20
|
||||
ladder_9x9.csv,end-to-end wall/FPS,9x9 cap=1500 N=20000 batch=2000 streams=4 consumer=streaming,float,"docs/benchmark_opt1_opt6_results.md §4, §6, §10, §10.3",perf/run_ladder.py,2026-08-13 17:57:20
|
||||
|
@@ -0,0 +1,5 @@
|
||||
n_frames,window_ms,window_us_per_frame,window_fps,kernel_n,kernel_sum_ms,kernel_busy_ms,kernel_overlap,kernel_duty_pct,kernel_us_per_frame,H2D_n,H2D_sum_ms,H2D_busy_ms,H2D_overlap,H2D_duty_pct,H2D_us_per_frame,D2H_n,D2H_sum_ms,D2H_busy_ms,D2H_overlap,D2H_duty_pct,D2H_us_per_frame,bottleneck,roofline_us_per_frame,roofline_fps,label,cluster_dim,cap,n_streams,batch,device_ped_type
|
||||
20000,457.218964,22.8609482,43742.71754834736,20000,110.724674,110.663005,1.0005572684385355,24.203502853831758,5.53315025,19999,332.555825,332.555825,1.0,72.73447761016317,16.62779125,20000,151.466742,151.466742,1.0,33.12783456637201,7.5733371,H2D,16.62779125,60140.27870358307,3x3_s4,3,3000,4,2000,float
|
||||
20000,860.686761,43.03433805,23237.257625251194,20000,86.459044,86.459044,1.0,10.045355397304641,4.3229522,19999,263.014103,263.014103,1.0,30.558632352426763,13.15070515,20000,105.444691,105.444691,1.0,12.251227249910029,5.27223455,H2D,13.15070515,76041.54975674441,3x3_s1_uncontended,3,3000,1,2000,float
|
||||
20000,1395.476764,69.7738382,14332.019361377197,20000,502.711249,485.060864,1.0363879799628608,34.75950847147176,24.2530432,19999,394.701491,394.701491,1.0,28.28434705488224,19.73507455,20000,456.813167,456.813167,1.0,32.735275769880175,22.84065835,kernel,24.2530432,41231.939091255976,9x9_s4,9,1500,4,2000,float
|
||||
20000,1791.736919,89.58684595,11162.353015063367,20000,473.307274,473.307274,1.0,26.41611438492662,23.6653637,19999,264.749131,264.749131,1.0,14.776116303266283,13.23745655,20000,388.783565,388.783565,1.0,21.698696994924173,19.43917825,kernel,23.6653637,42255.847519469986,9x9_s1_uncontended,9,1500,1,2000,float
|
||||
|
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-18 11:44:48",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "double",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 48 threads",3,3000,100000,0,False,n/a,n/a,double,0,20.115794302197173,201.15794302197173,4971.218063662412,2680160,1,233085343,2330.85343,reference
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,double,0,6.398641797946766,63.98641797946766,15628.316626832993,795,0,233094770,2330.9477,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,double,1,6.403091112151742,64.03091112151742,15617.45698264713,0,0,233094984,2330.94984,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,double,2,6.361006764927879,63.61006764927879,15720.781897507351,0,0,233094984,2330.94984,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,double,3,6.333786997012794,63.33786997012794,15788.342747737339,0,0,233094984,2330.94984,
|
||||
opt1,"1 stream, one launch per frame",3,3000,100000,1,False,auto,per-frame,double,4,6.3262022321578115,63.262022321578115,15807.272093148196,0,0,233094984,2330.94984,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,double,0,4.187054253881797,41.87054253881797,23883.139299494505,196127,0,233093553,2330.93553,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,double,1,4.259959110058844,42.59959110058844,23474.403724644828,308518,0,233094462,2330.94462,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,double,2,4.044346384936944,40.44346384936944,24725.874216028387,60192,0,233094462,2330.94462,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,double,3,4.052164989057928,40.52164989057928,24678.165935994773,60024,0,233094462,2330.94462,
|
||||
opt2,4 streams + host-side batching,3,3000,100000,4,False,auto,collect,double,4,4.14804248791188,41.4804248791188,24107.756921829383,150310,0,233094462,2330.94462,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,double,0,3.460056332172826,34.60056332172826,28901.26356330233,92408,0,233093484,2330.93484,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,double,1,3.4260347769595683,34.26034776959568,29188.26179830694,1274,0,233094390,2330.9439,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,double,2,3.498826839029789,34.98826839029789,28581.00860679622,91720,0,233094390,2330.9439,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,double,3,3.483933220151812,34.83933220151812,28703.190813640944,91721,0,233094390,2330.9439,
|
||||
opt3,"pipeline rework, no pinning",3,3000,100000,4,False,off,collect,double,4,3.438609245000407,34.38609245000407,29081.524789533963,60308,0,233094390,2330.9439,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,double,0,2.694697377970442,26.94697377970442,37109.918470814235,142473,0,233093484,2330.93484,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,double,1,2.6183871990069747,26.183871990069747,38191.44855196553,46349,0,233094390,2330.9439,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,double,2,2.5983168808743358,25.983168808743358,38486.45280184221,1010,0,233094390,2330.9439,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,double,3,2.680369977839291,26.80369977839291,37308.28237399239,91787,0,233094390,2330.9439,
|
||||
opt4,+ pinned input (DMA H2D),3,3000,100000,4,True,off,collect,double,4,2.6633800519630313,26.633800519630313,37546.2750523702,91845,0,233094390,2330.9439,
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,double,0,2.608076024800539,26.08076024800539,38342.440576534886,151236,0,233093484,2330.93484,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,double,1,2.5156038589775562,25.156038589775562,39751.886865304805,26430,0,233094390,2330.9439,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,double,2,2.524822043022141,25.24822043022141,39606.751801130034,51870,0,233094390,2330.9439,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,double,3,2.525553680025041,25.25553680025041,39595.27797445528,51852,0,233094390,2330.9439,rejected
|
||||
opt5,CUDA Graphs,3,3000,100000,4,True,auto,collect,double,4,2.5333995749242604,25.333995749242604,39472.6520797611,51964,0,233094390,2330.9439,rejected
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,double,0,2.012347515905276,20.12347515905276,49693.20617319615,95888,0,233093484,2330.93484,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,double,1,2.007254082011059,20.07254082011059,49819.30334390474,40011,0,233094390,2330.9439,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,double,2,1.9837507978081703,19.837507978081703,50409.557546486765,30795,0,233094390,2330.9439,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,double,3,1.9911192271392792,19.911192271392792,50223.00957018732,33910,0,233094390,2330.9439,
|
||||
opt7,"host<->GPU overlap, chunked internally",3,3000,100000,4,True,auto,collect,double,4,1.9873879398219287,19.873879398219287,50317.30242307904,39200,0,233094390,2330.9439,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,double,0,1.715774823911488,17.15774823911488,58282.706219005995,2062,0,233093484,2330.93484,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,double,1,1.712581568164751,17.12581568164751,58391.37934151815,0,0,233094390,2330.9439,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,double,2,1.7117952490225434,17.117952490225434,58418.201626100585,0,0,233094390,2330.9439,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,double,3,1.7099325810559094,17.099325810559094,58481.83788523901,0,0,233094390,2330.9439,
|
||||
opt8,zero-copy collection (collect_view),3,3000,100000,4,True,auto,collect_view,double,4,1.7095515080727637,17.095515080727637,58494.873964185754,0,0,233094390,2330.9439,
|
||||
|
@@ -0,0 +1,27 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 48 threads",9,1500,20000,0,False,n/a,n/a,double,0,15.480465562082827,774.0232781041414,1291.9508085717475,2968664,0,28438072,1421.9036,reference
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,double,0,1.9789065518416464,98.94532759208232,10106.591431205901,520482,0,28439276,1421.9638,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,double,1,2.038018618011847,101.90093090059236,9813.453038770884,683444,0,28447962,1422.3981,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,double,2,1.6433628669474274,82.16814334737137,12170.166676061215,191845,0,28452026,1422.6013,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,double,3,1.6958885919302702,84.79442959651351,11793.227512212865,191546,0,28453556,1422.6778,
|
||||
opt3,"pipeline rework, no pinning",9,1500,20000,4,False,off,collect,double,4,1.901584326988086,95.0792163494043,10517.545667658052,519380,0,28454157,1422.70785,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,double,0,1.9237553810235113,96.18776905117556,10396.332193420161,520481,0,28439276,1421.9638,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,double,1,2.005056690890342,100.2528345445171,9974.780309637546,683443,0,28447962,1422.3981,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,double,2,1.6088325418531895,80.44162709265947,12431.374602208323,191845,0,28452027,1422.60135,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,double,3,1.6616083378903568,83.08041689451784,12036.530838183435,191546,0,28453556,1422.6778,
|
||||
opt4,+ pinned input (DMA H2D),9,1500,20000,4,True,off,collect,double,4,1.8773560170084238,93.86780085042119,10653.280368137153,519380,0,28454157,1422.70785,
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,double,0,2.122308956924826,106.1154478462413,9423.698625378047,760811,0,28439276,1421.9638,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,double,1,1.9905071242246777,99.52535621123388,10047.690740011894,519681,0,28447962,1422.3981,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,double,2,1.896492162020877,94.82460810104385,10545.785740916674,519307,0,28452027,1422.60135,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,double,3,1.8956730319187045,94.78365159593523,10550.342629370536,519388,0,28453556,1422.6778,rejected
|
||||
opt5,CUDA Graphs,9,1500,20000,4,True,auto,collect,double,4,1.9486719449050725,97.43359724525362,10263.400185080553,582642,0,28454157,1422.70785,rejected
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,double,0,1.535099176922813,76.75495884614065,13028.474186333062,464904,0,28439276,1421.9638,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,double,1,1.2591699638869613,62.95849819434807,15883.4792550654,48608,0,28447962,1422.3981,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,double,2,1.309328624047339,65.46643120236695,15275.00402318929,129226,0,28452027,1422.60135,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,double,3,1.3087106021121144,65.43553010560572,15282.217449543243,130365,0,28453556,1422.6778,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1500,20000,4,True,auto,collect,double,4,1.4137493839953095,70.68746919976547,14146.77893155238,244576,0,28454157,1422.70785,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,double,0,0.6129775650333613,30.648878251668066,32627.62153279052,2066,0,28439276,1421.9638,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,double,1,0.6005836641415954,30.029183207079768,33300.93905998206,0,0,28447962,1422.3981,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,double,2,0.6006644780281931,30.033223901409656,33296.45872460144,0,0,28452027,1422.60135,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,double,3,0.6017205759417266,30.08602879708633,33238.01910662416,0,0,28453556,1422.6778,
|
||||
opt8,zero-copy collection (collect_view),9,1500,20000,4,True,auto,collect_view,double,4,0.601737436838448,30.086871841922402,33237.08776552242,0,0,28454157,1422.70785,
|
||||
|
@@ -0,0 +1,7 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
ladder_3x3.csv,end-to-end wall/FPS,3x3 cap=3000 N=100000 batch=2000 streams=4 consumer=streaming,double,"docs/benchmark_opt1_opt6_results.md §4, §6, §10, §10.3",perf/run_ladder.py,2026-08-18 11:36:53
|
||||
ladder_9x9.csv,end-to-end wall/FPS,9x9 cap=1500 N=20000 batch=2000 streams=4 consumer=streaming,double,"docs/benchmark_opt1_opt6_results.md §4, §6, §10, §10.3",perf/run_ladder.py,2026-08-18 11:36:53
|
||||
probe_3x3_s4.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,3x3 cap=3000 N=20000 streams=4 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:44:48
|
||||
probe_3x3_s1_uncontended.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,3x3 cap=3000 N=20000 streams=1 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:44:48
|
||||
probe_9x9_s4.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=4 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:44:48
|
||||
probe_9x9_s1_uncontended.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=1 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-18 11:44:48
|
||||
|
@@ -0,0 +1,5 @@
|
||||
n_frames,window_ms,window_us_per_frame,window_fps,kernel_n,kernel_sum_ms,kernel_busy_ms,kernel_overlap,kernel_duty_pct,kernel_us_per_frame,H2D_n,H2D_sum_ms,H2D_busy_ms,H2D_overlap,H2D_duty_pct,H2D_us_per_frame,D2H_n,D2H_sum_ms,D2H_busy_ms,D2H_overlap,D2H_duty_pct,D2H_us_per_frame,bottleneck,roofline_us_per_frame,roofline_fps,label,cluster_dim,cap,n_streams,batch,device_ped_type
|
||||
20000,465.941344,23.2970672,42923.85781503004,20000,320.923507,303.431619,1.0576468861671269,65.12227835270184,15.171580950000001,19999,323.314466,323.314466,1.0,69.38952084063182,16.1657233,20000,153.775667,153.775667,1.0,33.003224328597035,7.68878335,H2D,16.1657233,61859.279751497415,3x3_s4,3,3000,4,2000,double
|
||||
20000,1081.033554,54.0516777,18500.81334293163,20000,294.470605,294.470605,1.0,27.239728490425748,14.72353025,19999,262.833965,262.833965,1.0,24.313210633238125,13.141698250000001,20000,106.121461,106.121461,1.0,9.816666708200993,5.30607305,kernel,14.72353025,67918.49393592274,3x3_s1_uncontended,3,3000,1,2000,double
|
||||
20000,1380.430047,69.02150235,14488.238678565942,20000,873.141638,641.583455,1.3609166994494895,46.47707114129486,32.07917275,19999,401.770313,401.770313,1.0,29.104720943530722,20.08851565,20000,450.35086,450.35086,1.0,32.62395374388718,22.517543,kernel,32.07917275,31172.87368328412,9x9_s4,9,1500,4,2000,double
|
||||
20000,2090.896126,104.54480629999999,9565.276701842242,20000,798.56177,798.56177,1.0,38.192321467814516,39.9280885,19999,264.869784,264.869784,1.0,12.667763869585935,13.243489199999999,20000,388.851,388.851,1.0,18.597337053940286,19.44255,kernel,39.9280885,25045.025634022022,9x9_s1_uncontended,9,1500,1,2000,double
|
||||
|
@@ -0,0 +1,11 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 8 threads",3,3000,100000,0,False,n/a,n/a,float,0,26.28347669984214,262.8347669984214,3804.6717008560972,2476464,0,233091568,2330.91568,retain threads=8 loop_s=25.833 loop_fps=3871.0
|
||||
cpu,"ClusterFinderMT, 16 threads",3,3000,100000,0,False,n/a,n/a,float,0,15.164910248946398,151.64910248946398,6594.170249503958,2597376,0,233089481,2330.89481,retain threads=16 loop_s=14.680 loop_fps=6811.8
|
||||
cpu,"ClusterFinderMT, 24 threads",3,3000,100000,0,False,n/a,n/a,float,0,14.78826567903161,147.8826567903161,6762.118166553549,2476158,0,233087992,2330.87992,retain threads=24 loop_s=14.494 loop_fps=6899.3
|
||||
cpu,"ClusterFinderMT, 32 threads",3,3000,100000,0,False,n/a,n/a,float,0,16.830063453875482,168.30063453875482,5941.748245576155,2393978,0,233086801,2330.86801,retain threads=32 loop_s=16.783 loop_fps=5958.3
|
||||
cpu,"ClusterFinderMT, 48 threads",3,3000,100000,0,False,n/a,n/a,float,0,19.52668195287697,195.2668195287697,5121.197766283404,2444826,0,233085343,2330.85343,retain threads=48 loop_s=19.479 loop_fps=5133.8
|
||||
cpu,"ClusterFinderMT, 8 threads",9,1500,20000,0,False,n/a,n/a,float,0,27.1391425980255,1356.957129901275,736.9429571240431,2373525,1,28440010,1422.0005,retain threads=8 loop_s=24.983 loop_fps=800.5
|
||||
cpu,"ClusterFinderMT, 16 threads",9,1500,20000,0,False,n/a,n/a,float,0,16.167955602984875,808.3977801492438,1237.0147773233411,2521827,0,28438925,1421.94625,retain threads=16 loop_s=14.145 loop_fps=1413.9
|
||||
cpu,"ClusterFinderMT, 24 threads",9,1500,20000,0,False,n/a,n/a,float,0,14.831497456878424,741.5748728439212,1348.4815041871968,2662966,0,28438530,1421.9265,retain threads=24 loop_s=12.445 loop_fps=1607.0
|
||||
cpu,"ClusterFinderMT, 32 threads",9,1500,20000,0,False,n/a,n/a,float,0,13.304477212950587,665.2238606475294,1503.253354482203,2784433,0,28438219,1421.91095,retain threads=32 loop_s=9.328 loop_fps=2144.0
|
||||
cpu,"ClusterFinderMT, 48 threads",9,1500,20000,0,False,n/a,n/a,float,0,14.94528856384568,747.264428192284,1338.21437535721,3021388,0,28438072,1421.9036,retain threads=48 loop_s=8.166 loop_fps=2449.2
|
||||
|
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-19 17:43:09",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "float",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
cpu_threads.csv,ladder,"ClusterFinderMT thread sweep, 3x3 + 9x9",float,CPU baseline for every speedup in deck + report,cpu_threads.py,2026-08-19 17:43:09
|
||||
|
@@ -0,0 +1,32 @@
|
||||
# INVALID — do not cite these numbers
|
||||
|
||||
Probes run 2026-08-20 for 9x9 at cap 1700, discarded.
|
||||
|
||||
`env.json` in this directory says `"device_ped_type": "float"`. **It is wrong.**
|
||||
`common.device_ped_type()` parses `include/aare/clusterfinder_kernel.cuh` — the
|
||||
*source* — and the tree had not been rebuilt after that header was edited:
|
||||
|
||||
header edited : 2026-08-20 10:54:29
|
||||
binary built : 2026-08-20 09:57:53 <- 57 minutes EARLIER
|
||||
|
||||
So the header read `float/float` while the loaded module was still the
|
||||
`COMPUTE_TYPE = double; DEVICE_PED_TYPE = double` build used for the 9x9
|
||||
validation study. The measurement is a full-f64 kernel labelled f32.
|
||||
|
||||
The giveaway is the kernel time, which the cap cannot affect:
|
||||
|
||||
9x9 s4 kernel 80.73 us (the genuine f32 build reads 24.25)
|
||||
9x9 s1 kernel 87.92 us (the genuine f32 build reads 23.67)
|
||||
|
||||
Re-run after `make install`, and only once the guard in common.py confirms the
|
||||
binary is newer than the header.
|
||||
|
||||
One number here is still worth reading, because the cluster payload type is
|
||||
`int32` regardless of COMPUTE_TYPE, so the D2H byte count is build-independent:
|
||||
|
||||
cap 1500 -> 492 004 B/frame -> D2H 22.84 us (2026-08-18_f32)
|
||||
cap 1700 -> 557 600 B/frame -> D2H 22.70 us (here)
|
||||
|
||||
13.3 % more bytes, no more time. D2H is not bandwidth-bound at this size, which
|
||||
contradicts the linear extrapolation used to predict "cap 1700 makes D2H
|
||||
overtake the kernel". To be confirmed on a clean build.
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:12:32",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "float",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
probe_9x9_s4_cap1700.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1700 N=20000 streams=4 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:12:32
|
||||
probe_9x9_s1_uncontended_cap1700.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1700 N=20000 streams=1 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:12:32
|
||||
|
@@ -0,0 +1,3 @@
|
||||
n_frames,window_ms,window_us_per_frame,window_fps,kernel_n,kernel_sum_ms,kernel_busy_ms,kernel_overlap,kernel_duty_pct,kernel_us_per_frame,H2D_n,H2D_sum_ms,H2D_busy_ms,H2D_overlap,H2D_duty_pct,H2D_us_per_frame,D2H_n,D2H_sum_ms,D2H_busy_ms,D2H_overlap,D2H_duty_pct,D2H_us_per_frame,bottleneck,roofline_us_per_frame,roofline_fps,label,cluster_dim,cap,n_streams,batch,device_ped_type
|
||||
20000,1819.694496,90.9847248,10990.855906836792,20000,1961.014812,1614.614064,1.214540896009438,88.7299526128808,80.7307032,19999,295.218095,295.218095,1.0,16.223497716179278,14.760904749999998,20000,453.962236,453.962236,1.0,24.94716761510719,22.6981118,kernel,80.7307032,12386.861012750353,9x9_s4,9,1700,4,2000,float
|
||||
20000,2957.254181,147.86270905,6763.030424810142,20000,1758.386908,1758.386908,1.0,59.460120786959166,87.9193454,19999,263.332636,263.332636,1.0,8.904633145567272,13.1666318,20000,438.000176,438.000176,1.0,14.811042581800987,21.9000088,kernel,87.9193454,11374.061026619063,9x9_s1_uncontended,9,1700,1,2000,float
|
||||
|
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:26:21",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "float",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:25:16",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "float",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 32 threads",9,1700,20000,0,False,n/a,n/a,float,0,13.512256105896086,675.6128052948043,1480.1377240972352,2777514,0,28438219,1421.91095,reference
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,float,0,2.034773570019752,101.73867850098759,9829.103490766227,684506,0,28442005,1422.10025,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,float,1,1.9132814179174602,95.66407089587301,10453.245305528184,520617,0,28452049,1422.60245,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,float,2,1.9234019841533154,96.17009920766577,10398.242366794704,520181,0,28456746,1422.8373,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,float,3,1.9693005678709596,98.46502839354798,10155.89002831716,585114,0,28458528,1422.9264,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,float,4,1.9842164178844541,99.2108208942227,10079.545668372073,584154,0,28459216,1422.9608,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,float,0,1.9024269040673971,95.12134520336986,10512.887489784713,684506,0,28442005,1422.10025,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,float,1,1.7872322199400514,89.36161099700257,11190.487602484502,520617,0,28452049,1422.60245,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,float,2,1.632499601924792,81.6249800962396,12251.15153254499,355879,0,28456746,1422.8373,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,float,3,1.5366189870983362,76.83094935491681,13015.588228391516,256027,0,28458528,1422.9264,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,float,4,1.5039158621802926,75.19579310901463,13298.616300918009,127010,0,28459216,1422.9608,
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,float,0,2.0413262380752712,102.06631190376356,9797.552016407544,792580,0,28442005,1422.10025,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,float,1,1.603434408083558,80.1717204041779,12473.226157036392,255982,0,28452049,1422.60245,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,float,2,1.7725593589711934,88.62796794855967,11283.120025728305,519534,0,28456746,1422.8373,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,float,3,1.648476961068809,82.42384805344045,12132.410990465265,420630,0,28458528,1422.9264,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,float,4,1.5012192442081869,75.06096221040934,13322.504409107101,127494,0,28459216,1422.9608,rejected
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,float,0,1.4985969089902937,74.92984544951469,13345.816930501582,460008,0,28442005,1422.10025,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,float,1,1.5564289251342416,77.82144625671208,12849.928240876792,413394,0,28452048,1422.6024,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,float,2,1.488227722933516,74.4113861466758,13438.80354585591,413549,0,28456746,1422.8373,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,float,3,1.237005049129948,61.8502524564974,16168.082752828755,10128,0,28458528,1422.9264,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,float,4,1.4880174759309739,74.4008737965487,13440.702359686373,341096,0,28459216,1422.9608,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,float,0,0.5066126601304859,25.330633006524295,39477.89223200362,1443,0,28442005,1422.10025,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,float,1,0.5032505870331079,25.162529351655394,39741.63272795991,0,0,28452049,1422.60245,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,float,2,0.5028966551180929,25.144832755904645,39769.60235558436,0,0,28456746,1422.8373,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,float,3,0.5031049428507686,25.15524714253843,39753.13755947816,0,0,28458528,1422.9264,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,float,4,0.502825811970979,25.14129059854895,39775.2054963207,0,0,28459216,1422.9608,
|
||||
|
@@ -0,0 +1,2 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
ladder_9x9.csv,end-to-end wall/FPS,9x9 cap=1700 N=20000 batch=2000 streams=4 consumer=streaming,float,"docs/ClusterFinderCUDA_benchmark_results.md §5, §6, §7, §8, §9, §11",perf/run_ladder.py,2026-08-20 11:25:16
|
||||
|
@@ -0,0 +1,90 @@
|
||||
# 9×9 cap A/B — what `max_clusters_per_frame` actually buys and costs
|
||||
|
||||
**Build**: `[f32]`, `COMPUTE_TYPE = float`, `DEVICE_PED_TYPE = float`, git `7177f00`.
|
||||
**Config**: 9×9, 20 000 frames, batch 2000, RTX 4090, idle GPU.
|
||||
Both caps measured **on one build in one session, back to back**, which is the
|
||||
whole point of this directory.
|
||||
|
||||
## Why this exists
|
||||
|
||||
The cap is not a safety bound. The D2H slot is
|
||||
|
||||
output_bytes_per_frame = 4 + cap × sizeof(ClusterType) (328 B at 9×9)
|
||||
|
||||
and `ClusterFinderCUDA` copies that slot **whole**, every frame, regardless of how
|
||||
many clusters were actually found. So at 9×9 the cap sets the height of the D2H
|
||||
bar directly. At 3×3 the cluster is 40 B and the same headroom is nearly free —
|
||||
which is why this only ever mattered at 9×9.
|
||||
|
||||
The campaign ran 9×9 at **cap 1500**. Measured against the true per-frame
|
||||
distribution over the same 20 000-frame block:
|
||||
|
||||
mean 1422.1 std 25.0 min 1327 MAX 1633
|
||||
|
||||
so 1500 is **below the maximum**. It truncated 64 frames and discarded 2 715
|
||||
clusters — 0.0095 %. The loss was silent: the kernel bumps its counter for every
|
||||
detection and guards only the write (`if (write_idx >= max_clusters) return;`),
|
||||
and the host then clamps the count, so a truncated frame is indistinguishable
|
||||
from a short one. `ladder.py` now detects this and marks such rows `TRUNCATED`.
|
||||
|
||||
## The measurement
|
||||
|
||||
| cap | streams | kernel | H2D | D2H | binds | roofline | FPS |
|
||||
|--:|--:|--:|--:|--:|---|--:|--:|
|
||||
| 1500 | 4 | 24.11 | 19.64 | 22.77 | **kernel** | 24.11 µs | 41 480 |
|
||||
| 1500 | 1 | 23.97 | 13.22 | 19.50 | kernel | 23.97 µs | 41 715 |
|
||||
| 1700 | 4 | 23.94 | 20.54 | **25.24** | **D2H** | 25.24 µs | 39 614 |
|
||||
| 1700 | 1 | 23.70 | 13.22 | 21.95 | kernel | 23.70 µs | 42 197 |
|
||||
|
||||
Sustained, unprofiled, same session (`ladder_9x9.csv` in
|
||||
`../2026-08-20_f32_cap1700/`): opt8 reaches **39 775 FPS / 25.14 µs** at cap 1700,
|
||||
against 42 274 FPS / 23.66 µs at cap 1500. So covering every cluster costs
|
||||
**5.9 % of throughput**.
|
||||
|
||||
The kernel is unchanged across the two caps (24.11 vs 23.94 at s4, 0.7 % apart),
|
||||
as it must be — the cap affects only the write guard. D2H scales with the slot:
|
||||
at 1 stream, 492 000 B in 19.50 µs = 25.2 GB/s and 557 600 B in 21.95 µs =
|
||||
25.4 GB/s. Linear, and bandwidth-bound.
|
||||
|
||||
## What it means
|
||||
|
||||
**The cap chooses which engine binds.** Same code, same data, one parameter:
|
||||
|
||||
- **cap 1500** — 0.0095 % of clusters discarded, **kernel-bound** at 24.11 µs.
|
||||
Act III's premise holds: the kernel is the tallest bar, and opt7's −40 % kernel
|
||||
translates into end-to-end gain.
|
||||
- **cap 1700** — lossless, **D2H-bound** at 25.24 µs. opt7 still shortens the
|
||||
kernel by 40 %, but the frame no longer follows it: the result path is now the
|
||||
constraint.
|
||||
|
||||
Both are legitimate operating points and the choice belongs to the experiment,
|
||||
not to the library. If 1 in 10 000 clusters is inside your statistical error —
|
||||
and at 14 M clusters per 10 000 frames it usually is — cap 1500 is the faster,
|
||||
kernel-bound configuration and the kernel-optimization argument is the right one.
|
||||
If you need every cluster, take the 5.9 % and read D2H as the next target.
|
||||
|
||||
What is **not** legitimate is the state this campaign was in before today: a cap
|
||||
believed to be non-truncating, silently discarding clusters, with a
|
||||
kernel-bound conclusion resting on it and no way to notice.
|
||||
|
||||
## Artifacts
|
||||
|
||||
| file | what |
|
||||
|---|---|
|
||||
| `probes.csv` | the four rows above; `cap` is a column, so the A/B is machine-readable |
|
||||
| `probe_9x9_{s4,s1_uncontended}_cap{1500,1700}.nsys-rep` / `.sqlite` | nsys traces. **The cap is in the filename** — two probes of one label at two caps are different measurements and must not overwrite each other |
|
||||
| `env.json` | build identity. Trustworthy here: `common.assert_build_fresh()` ran first |
|
||||
|
||||
## Caveat on reading these
|
||||
|
||||
`roofline_fps` here is the **profiled** engine-occupancy estimate. Peak, as the
|
||||
report defines it, is the *lower* of that estimate and the best rate the
|
||||
unprofiled pipeline sustained. At cap 1700 the sustained 25.14 µs beats the
|
||||
25.24 µs estimate, so peak is 25.14 µs / 39 775 FPS and opt8 sits **on** the D2H
|
||||
floor.
|
||||
|
||||
An earlier attempt at this A/B on 2026-08-20 was discarded — see
|
||||
`../2026-08-20_INVALID_stale_build/INVALID.md`. The tree had not been rebuilt
|
||||
after the kernel header was edited, so an f64 kernel was measured and labelled
|
||||
`float`. `common.assert_build_fresh()` was added in response and now guards both
|
||||
`run_probes.py` and `run_ladder.py`.
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:22:24",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "float",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
probe_9x9_s4_cap1500.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=4 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:21:53
|
||||
probe_9x9_s1_uncontended_cap1500.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=1 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:21:53
|
||||
probe_9x9_s4_cap1700.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1700 N=20000 streams=4 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:22:24
|
||||
probe_9x9_s1_uncontended_cap1700.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1700 N=20000 streams=1 batch=2000,float,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:22:24
|
||||
|
@@ -0,0 +1,5 @@
|
||||
n_frames,window_ms,window_us_per_frame,window_fps,kernel_n,kernel_sum_ms,kernel_busy_ms,kernel_overlap,kernel_duty_pct,kernel_us_per_frame,H2D_n,H2D_sum_ms,H2D_busy_ms,H2D_overlap,H2D_duty_pct,H2D_us_per_frame,D2H_n,D2H_sum_ms,D2H_busy_ms,D2H_overlap,D2H_duty_pct,D2H_us_per_frame,bottleneck,roofline_us_per_frame,roofline_fps,label,cluster_dim,cap,n_streams,batch,device_ped_type
|
||||
20000,1427.94668,71.397334,14006.125214703396,20000,497.912253,482.159812,1.0326705805999443,33.765953501849246,24.107990599999997,19999,392.881663,392.881663,1.0,27.51374883269451,19.64408315,20000,455.391991,455.391991,1.0,31.89138623859541,22.76959955,kernel,24.107990599999997,41480.02281036231,9x9_s4,9,1500,4,2000,float
|
||||
20000,1812.389592,90.61947959999999,11035.154962421568,20000,479.445687,479.445687,1.0,26.45378726054834,23.97228435,19999,264.357072,264.357072,1.0,14.586106274660178,13.2178536,20000,390.014943,390.014943,1.0,21.519376668325073,19.500747150000002,kernel,23.97228435,41714.83974575831,9x9_s1_uncontended,9,1500,1,2000,float
|
||||
20000,1398.131903,69.90659515,14304.801969746628,20000,486.97025,478.701589,1.0172731012179699,34.2386571662402,23.93507945,19999,410.704818,410.704818,1.0,29.37525544755415,20.5352409,20000,504.877265,504.877265,1.0,36.11084647426145,25.24386325,D2H,25.24386325,39613.58806679481,9x9_s4,9,1700,4,2000,float
|
||||
20000,1796.174109,89.80870544999999,11134.778026131764,20000,473.972052,473.972052,1.0,26.38786794805091,23.6986026,19999,264.40628,264.40628,1.0,14.720526182576213,13.220314000000002,20000,439.06546,439.06546,1.0,24.444482180207174,21.953273,kernel,23.6986026,42196.580822870965,9x9_s1_uncontended,9,1700,1,2000,float
|
||||
|
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:36:32",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "double",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:35:26",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "double",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
step,label,cluster_dim,cap,n_frames,n_streams,pinned,batch_chunk,collection,device_ped_type,rep,wall_s,us_per_frame,fps,minor_faults,major_faults,n_clusters,clusters_per_frame,notes
|
||||
cpu,"ClusterFinderMT, 32 threads",9,1700,20000,0,False,n/a,n/a,double,0,13.765519716078416,688.2759858039208,1452.9055504268088,2774547,0,28438219,1421.91095,reference
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,double,0,1.8636698939371854,93.18349469685927,10731.514237077705,520379,0,28441991,1422.09955,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,double,1,1.7463798618409783,87.31899309204891,11452.262154991087,291716,0,28452038,1422.6019,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,double,2,1.6718548401258886,83.59274200629443,11962.761072303398,256041,0,28456731,1422.83655,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,double,3,1.648889538133517,82.44447690667585,12129.375278006357,127503,0,28458520,1422.926,
|
||||
opt3,"pipeline rework, no pinning",9,1700,20000,4,False,off,collect,double,4,1.8827759760897607,94.13879880448803,10622.61270272683,519376,0,28459199,1422.95995,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,double,0,1.8503628321923316,92.51814160961658,10808.690950792481,520376,0,28441991,1422.09955,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,double,1,1.6989846539217979,84.9492326960899,11771.736698052939,291717,0,28452038,1422.6019,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,double,2,1.6217983360402286,81.08991680201143,12331.989468451337,256040,0,28456731,1422.83655,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,double,3,1.5965895410627127,79.82947705313563,12526.701124878793,127503,0,28458520,1422.926,
|
||||
opt4,+ pinned input (DMA H2D),9,1700,20000,4,True,off,collect,double,4,1.8154389860574156,90.77194930287078,11016.619205382358,519489,0,28459199,1422.95995,
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,double,0,2.212348804110661,110.61744020553306,9040.1658015404,792737,0,28441991,1422.09955,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,double,1,2.0249657810200006,101.24828905100003,9876.710109108979,683768,0,28452038,1422.6019,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,double,2,1.8624301289673895,93.12150644836947,10738.6578905319,519533,0,28456731,1422.83655,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,double,3,1.8063955381512642,90.31977690756321,11071.772254524489,420440,0,28458520,1422.926,rejected
|
||||
opt5,CUDA Graphs,9,1700,20000,4,True,auto,collect,double,4,1.902979239821434,95.1489619910717,10509.83614612459,548306,0,28459199,1422.95995,rejected
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,double,0,1.5712034509051591,78.56017254525796,12729.096278701618,460283,0,28441991,1422.09955,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,double,1,1.3277159039862454,66.38579519931227,15063.463456266012,151601,0,28452038,1422.6019,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,double,2,1.3467270429246128,67.33635214623064,14850.819329035752,95884,0,28456731,1422.83655,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,double,3,1.6251181659754366,81.25590829877183,12306.797387865947,506241,0,28458521,1422.92605,
|
||||
opt7,"host<->GPU overlap, chunked internally",9,1700,20000,4,True,auto,collect,double,4,1.4793297110591084,73.96648555295542,13519.636528952858,334303,0,28459199,1422.95995,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,double,0,0.611520430073142,30.576021503657103,32705.366847037083,2072,0,28441991,1422.09955,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,double,1,0.6003484949469566,30.01742474734783,33313.98374167172,0,0,28452038,1422.6019,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,double,2,0.600202470086515,30.010123504325747,33322.08878966649,0,0,28456731,1422.83655,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,double,3,0.6002302598208189,30.011512991040945,33320.54602840319,0,0,28458520,1422.926,
|
||||
opt8,zero-copy collection (collect_view),9,1700,20000,4,True,auto,collect_view,double,4,0.6001883989665657,30.009419948328286,33322.87000954533,0,0,28459199,1422.95995,
|
||||
|
@@ -0,0 +1,2 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
ladder_9x9.csv,end-to-end wall/FPS,9x9 cap=1700 N=20000 batch=2000 streams=4 consumer=streaming,double,"docs/ClusterFinderCUDA_benchmark_results.md §5, §6, §7, §8, §9, §11",perf/run_ladder.py,2026-08-20 11:35:26
|
||||
|
@@ -0,0 +1,68 @@
|
||||
# 9×9 cap A/B — the `[f64]` arm
|
||||
|
||||
**Build**: `COMPUTE_TYPE = float`, `DEVICE_PED_TYPE = double`, git `7177f00` — the
|
||||
mixed configuration the campaign's "f64 arm" has always been, not double/double.
|
||||
**Config**: 9×9, 20 000 frames, batch 2000, idle GPU, `assert_build_fresh()` passed.
|
||||
|
||||
Companion to `../2026-08-20_f32_capAB/README.md`, which explains the mechanism
|
||||
(the D2H slot is `4 + cap × 328 B` at 9×9 and is copied whole every frame) and
|
||||
why cap 1500 was truncating. Read that one first.
|
||||
|
||||
## The measurement
|
||||
|
||||
| cap | streams | kernel | H2D | D2H | binds | roofline |
|
||||
|--:|--:|--:|--:|--:|---|--:|
|
||||
| 1500 | 4 | 31.86 | 19.90 | 22.44 | kernel | 31.86 µs → 31 392 |
|
||||
| 1500 | 1 | 39.78 | 13.18 | 19.48 | kernel | 39.78 µs → 25 139 |
|
||||
| 1700 | 4 | 32.66 | 20.77 | 25.25 | **kernel** | 32.66 µs → 30 621 |
|
||||
| 1700 | 1 | 39.86 | 13.20 | 21.97 | kernel | 39.86 µs → 25 086 |
|
||||
|
||||
Sustained (`../2026-08-20_f64_cap1700/ladder_9x9.csv`): opt8 reaches
|
||||
**33 323 FPS / 30.01 µs** at cap 1700, against 33 301 / 30.03 at cap 1500.
|
||||
|
||||
## Why the cap is free here and not on `[f32]`
|
||||
|
||||
D2H grows identically on both arms — 22.4 → 25.2 µs — because the cluster payload
|
||||
is `int32` regardless of `COMPUTE_TYPE`, so the slot is the same 328 B either way.
|
||||
What differs is what it has to climb over:
|
||||
|
||||
arm kernel @ s4 D2H @ cap 1700 binds opt8 cost of the cap
|
||||
f64 32.66 25.25 kernel +0.1 % (nothing)
|
||||
f32 23.94 25.24 D2H -5.9 %
|
||||
|
||||
On the f64 arm the kernel is tall enough to hide any cap worth setting: 25.25 µs
|
||||
of D2H disappears entirely underneath a 32.66 µs kernel, and opt8 lands on the
|
||||
same 30.0 µs it did at cap 1500.
|
||||
|
||||
**opt7 is what makes the cap expensive.** Dropping the kernel 40 % — 32.66 → 23.94
|
||||
µs — moves it *below* the enlarged D2H bar. The result path was never the
|
||||
constraint until the kernel stopped being one. That is the same rule the whole
|
||||
ladder is ordered by, appearing once more and at the last possible moment: you
|
||||
cannot see the result path until the kernel gets out of its way.
|
||||
|
||||
So the honest statement about Act III is not that the cap invalidates it. It is:
|
||||
|
||||
- the kernel optimization is worth its full −40 % at cap 1500 (kernel-bound), and
|
||||
- at a lossless cap it is worth −40 % of a bar that is no longer the tallest,
|
||||
which is what success looks like when you optimize in bottleneck order.
|
||||
|
||||
## Non-obvious: opt5 loses 5.2 %, opt3/opt4 lose nothing
|
||||
|
||||
opt3 12 170 -> 12 129 -0.3 % run noise
|
||||
opt4 12 431 -> 12 527 +0.8 % run noise
|
||||
opt5 15 883 -> 15 063 -5.2 % real
|
||||
opt8 33 301 -> 33 323 +0.1 % run noise
|
||||
|
||||
opt3 and opt4 sit at ~30 % of the floor: they are host-bound with the GPU idle
|
||||
most of the frame, so 65 kB more per frame vanishes into slack. opt8 sits *on*
|
||||
the floor, but on this arm the floor is the kernel, which did not move. opt5 is
|
||||
the one caught in between — overlapped enough that transfer time is on the
|
||||
critical path, not fast enough to be floor-bound — so the extra bytes bill in
|
||||
full. The cap's cost is not uniform across the ladder; it depends on what each
|
||||
step is limited by.
|
||||
|
||||
## Artifacts
|
||||
|
||||
`probe_9x9_{s4,s1_uncontended}_cap{1500,1700}.nsys-rep` / `.sqlite`, and
|
||||
`probes.csv` carrying `cap` as a column. Filenames include the cap because two
|
||||
probes of one label at two caps are different measurements.
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"timestamp": "2026-08-20 11:35:07",
|
||||
"host": "pc-moench-04.psi.ch",
|
||||
"git_rev": "7177f00",
|
||||
"git_branch": "bench/opt2-pipeline",
|
||||
"git_dirty": true,
|
||||
"aare_version": "2026.7.2",
|
||||
"device_ped_type": "double",
|
||||
"gpu": "NVIDIA GeForce RTX 4090",
|
||||
"driver": "595.71.05",
|
||||
"gpu_busy_pct": "0 %",
|
||||
"nvcc": "Build cuda_12.4.r12.4/compiler.34097967_0",
|
||||
"python": "3.11.15"
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
artifact,kind,config,build,cites,produced_by,timestamp
|
||||
probe_9x9_s4_cap1500.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=4 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:34:47
|
||||
probe_9x9_s1_uncontended_cap1500.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1500 N=20000 streams=1 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:34:47
|
||||
probe_9x9_s4_cap1700.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1700 N=20000 streams=4 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:35:07
|
||||
probe_9x9_s1_uncontended_cap1700.nsys-rep / .sqlite,nsys per-engine GPU times + duty cycles,9x9 cap=1700 N=20000 streams=1 batch=2000,double,"docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",perf/run_probes.py,2026-08-20 11:35:07
|
||||
|
@@ -0,0 +1,5 @@
|
||||
n_frames,window_ms,window_us_per_frame,window_fps,kernel_n,kernel_sum_ms,kernel_busy_ms,kernel_overlap,kernel_duty_pct,kernel_us_per_frame,H2D_n,H2D_sum_ms,H2D_busy_ms,H2D_overlap,H2D_duty_pct,H2D_us_per_frame,D2H_n,D2H_sum_ms,D2H_busy_ms,D2H_overlap,D2H_duty_pct,D2H_us_per_frame,bottleneck,roofline_us_per_frame,roofline_fps,label,cluster_dim,cap,n_streams,batch,device_ped_type
|
||||
20000,1371.403475,68.57017375000001,14583.600205621471,20000,867.936005,637.105409,1.3623114679913195,46.456452868474756,31.85527045,19999,397.97064,397.97064,1.0,29.01922353667654,19.898532,20000,448.886648,448.886648,1.0,32.731917060367664,22.4443324,kernel,31.85527045,31391.979596268033,9x9_s4,9,1500,4,2000,double
|
||||
20000,2095.919101,104.79595505,9542.353037604194,20000,795.58624,795.58624,1.0,37.958823869700495,39.779312,19999,263.508265,263.508265,1.0,12.572444464782803,13.17541325,20000,389.599353,389.599353,1.0,18.588472847740892,19.47996765,kernel,39.779312,25138.695209208247,9x9_s1_uncontended,9,1500,1,2000,double
|
||||
20000,1390.7386,69.53693,14380.847702077155,20000,863.510171,653.140187,1.3220900936539677,46.963547786765965,32.65700935,19999,415.436516,415.436516,1.0,29.871646332387698,20.7718258,20000,505.007753,505.007753,1.0,36.31219792130599,25.25038765,kernel,32.65700935,30621.29753776734,9x9_s4,9,1700,4,2000,double
|
||||
20000,2095.135366,104.75676829999999,9545.922580736962,20000,797.245324,797.245324,1.0,38.05221070379278,39.8622662,19999,263.965791,263.965791,1.0,12.598985024244968,13.198289550000002,20000,439.36096,439.36096,1.0,20.970528545791346,21.968048,kernel,39.8622662,25086.3810648076,9x9_s1_uncontended,9,1700,1,2000,double
|
||||
|
Executable
+50
@@ -0,0 +1,50 @@
|
||||
#!/bin/bash
|
||||
# Both arms of the campaign, end to end: f32 ladder+probes, rebuild, f64
|
||||
# ladder+probes, rebuild back. ~1.5 h, mostly unattended.
|
||||
#
|
||||
# ./run_campaign.sh # both arms
|
||||
# ./run_campaign.sh f32 # one arm
|
||||
#
|
||||
# The GPU must be idle: run_ladder.py and run_probes.py both abort above 5 %
|
||||
# utilisation, because a competing process leaves per-operation averages intact
|
||||
# while destroying the duty cycle and the wall clock. Close any notebook first.
|
||||
#
|
||||
# The arm is selected by ONE line in the kernel header. Nothing else differs --
|
||||
# same commands, same parameters -- and env.json records which arm produced each
|
||||
# result set, so a stale build cannot silently mislabel a campaign.
|
||||
set -euo pipefail
|
||||
|
||||
REPO=/home/ferjao_k/aare
|
||||
HDR=$REPO/include/aare/clusterfinder_kernel.cuh
|
||||
PERF=$REPO/python/tests/perf
|
||||
PY=${PY:-/home/ferjao_k/.conda/envs/py/bin/python3.11}
|
||||
|
||||
set_ped() { # set_ped float|double
|
||||
sed -i "s/^using DEVICE_PED_TYPE = .*;/using DEVICE_PED_TYPE = $1;/" "$HDR"
|
||||
echo "=== rebuilding with DEVICE_PED_TYPE = $1 ==="
|
||||
cmake --build "$REPO/build" -j 16 2>&1 | grep -E "Built target _aare_cuda|error" || {
|
||||
echo "BUILD FAILED"; exit 1; }
|
||||
got=$(cd "$PERF" && $PY -c "import sys;sys.path.insert(0,'.');import common;print(common.device_ped_type())")
|
||||
[ "$got" = "$1" ] || { echo "header reports '$got', expected '$1' — aborting"; exit 1; }
|
||||
echo "verified: $got"
|
||||
}
|
||||
|
||||
run_arm() {
|
||||
cd "$PERF"
|
||||
echo "=== ladder ==="; $PY run_ladder.py
|
||||
echo "=== probes ==="; $PY run_probes.py
|
||||
}
|
||||
|
||||
arm=${1:-both}
|
||||
if [ "$arm" = "f32" ] || [ "$arm" = "both" ]; then
|
||||
set_ped float
|
||||
run_arm
|
||||
fi
|
||||
if [ "$arm" = "f64" ] || [ "$arm" = "both" ]; then
|
||||
set_ped double
|
||||
run_arm
|
||||
# f32 is the shipping default: never leave the tree on the f64 build, or the
|
||||
# next person's notebook silently runs a 40 % slower kernel at 9x9.
|
||||
set_ped float
|
||||
fi
|
||||
echo "=== CAMPAIGN COMPLETE ==="
|
||||
@@ -0,0 +1,229 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the opt1 → opt8 ladder and write one CSV row per (step, rep).
|
||||
|
||||
python run_ladder.py --dry-run # 2000 frames, 1 rep, both sizes
|
||||
python run_ladder.py # the real campaign
|
||||
python run_ladder.py --dims 9 --reps 3
|
||||
|
||||
Output lands in perf/results/<date>_<f32|f64>/ together with env.json and a
|
||||
manifest row, so every number in the report can be traced back to the run that
|
||||
produced it and the build it was taken on.
|
||||
|
||||
The build axis (opt6) is NOT a command-line option: it is compiled in. Check
|
||||
env.json's device_ped_type to know which arm a result set belongs to.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
import common
|
||||
import ladder
|
||||
from common import Row
|
||||
|
||||
|
||||
# Per-cluster-size configuration. FIXED for the campaign. Optimising over these
|
||||
# is a separate exercise; what matters here is that every step sees the same ones.
|
||||
#
|
||||
# cap MEASURED against the per-frame maximum over each campaign block,
|
||||
# not guessed. With a probing cap high enough never to bind:
|
||||
#
|
||||
# frames mean max cap truncates?
|
||||
# 3x3 100 000 2 330.9 2 545 3000 no
|
||||
# 9x9 20 000 1 422.1 1 633 1700 no
|
||||
#
|
||||
# The 9x9 cap was 1500 for the whole earlier campaign, which is
|
||||
# BELOW the maximum: it silently dropped 2 715 clusters (0.0095 %)
|
||||
# across 64 frames, because the kernel guards only the write and the
|
||||
# host clamps the count. ladder.py now detects this — any frame that
|
||||
# returns exactly `cap` clusters is flagged and the row is marked
|
||||
# TRUNCATED — so the constant can never again be quietly wrong.
|
||||
#
|
||||
# What raising it costs, MEASURED (results/2026-08-20_{f32,f64}_capAB):
|
||||
# D2H copies the whole fixed-size slot (4 + cap * sizeof(ClusterType))
|
||||
# regardless of occupancy, so at 9x9 the cap is a throughput knob.
|
||||
# 1500 -> 1700 grows the slot 480.5 -> 544.5 KiB and D2H 22.8 -> 25.2
|
||||
# us/frame on BOTH arms (the payload is int32 either way). The cost
|
||||
# is arm-dependent, because it depends what the extra bytes hide under:
|
||||
#
|
||||
# arm kernel@s4 D2H@1700 binds opt8 cost of the cap
|
||||
# f64 32.66 25.25 kernel +0.1 % (free)
|
||||
# f32 23.94 25.24 D2H -5.9 %
|
||||
#
|
||||
# i.e. opt7's -40 % kernel is exactly what makes the cap expensive:
|
||||
# it drops the kernel below the enlarged D2H bar. At 3x3 the cluster
|
||||
# is 40 B and the cap is free at any sane value.
|
||||
# n_streams 4 everywhere. Campaign C used 8 at 9x9; §8 then showed 8 streams
|
||||
# buy no kernel concurrency there (instance time +1%) while
|
||||
# inflating the event timer 3.5x. Fixed at 4 and re-measured.
|
||||
# batch 2000 everywhere. The §10 opt7 measurement used 3000; nothing else did.
|
||||
# n_frames 100k at 3x3, 20k at 9x9 — the historical Campaign A and C values.
|
||||
# 9x9 is held at 20k because the result heap is ~5x larger per frame
|
||||
# (1422 x 328 B = 466 kB vs 2330 x 40 B = 93 kB), so 100k would need
|
||||
# 46.6 GB to retain against 98 GB free with no swap. At 20k it is
|
||||
# 9.3 GB, which keeps --retain feasible at both sizes.
|
||||
CONFIGS = {
|
||||
3: dict(cap=3000, n_frames=100_000, batch_size=2000, n_streams=4),
|
||||
9: dict(cap=1700, n_frames=20_000, batch_size=2000, n_streams=4),
|
||||
}
|
||||
|
||||
# Sustained per-frame roofline = tallest engine bar, from the nsys probes.
|
||||
# Used only to annotate the printed table; never written to the CSV, because a
|
||||
# roofline is a derived quantity and belongs in the report, not the raw data.
|
||||
ROOFLINE_US = {3: 16.2, 9: 23.9}
|
||||
|
||||
|
||||
def _run_isolated(step_name: str, dim: int, args, outdir: Path) -> list[Row]:
|
||||
"""Run one step in a FRESH process and read its rows back.
|
||||
|
||||
Required for the fault columns to mean anything: the heap is process-wide,
|
||||
so in a shared process every step inherits what the previous ones grew.
|
||||
The dataset is re-read per step, which is the cost of the isolation.
|
||||
"""
|
||||
tmp = outdir / f".{step_name}_{dim}.csv"
|
||||
cmd = [sys.executable, str(Path(__file__).resolve()),
|
||||
"--dims", str(dim), "--steps", step_name, "--reps", str(args.reps),
|
||||
"--_worker-out", str(tmp), "--allow-busy-gpu"]
|
||||
if args.frames:
|
||||
cmd += ["--frames", str(args.frames)]
|
||||
if args.retain:
|
||||
cmd += ["--retain"]
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if not tmp.exists():
|
||||
raise RuntimeError(
|
||||
f"worker for {step_name} produced nothing "
|
||||
f"(exit {proc.returncode}): {proc.stderr.strip()[-400:]}")
|
||||
rows = []
|
||||
for d in csv.DictReader(tmp.open()):
|
||||
for k, f in Row.__dataclass_fields__.items():
|
||||
if f.type in ("int", int):
|
||||
d[k] = int(d[k])
|
||||
elif f.type in ("float", float):
|
||||
d[k] = float(d[k])
|
||||
elif f.type in ("bool", bool):
|
||||
d[k] = d[k] == "True"
|
||||
rows.append(Row(**d))
|
||||
tmp.unlink()
|
||||
return rows
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--dims", type=int, nargs="+", default=[3, 9],
|
||||
help="cluster sizes to run (default: 3 9)")
|
||||
ap.add_argument("--reps", type=int, default=5,
|
||||
help="repetitions per step. 5 because collect() is bistable and 3 cannot separate a plateau from an oscillation (default: 5)")
|
||||
ap.add_argument("--frames", type=int, default=None,
|
||||
help="override n_frames for every size")
|
||||
ap.add_argument("--steps", nargs="+", default=None,
|
||||
help="subset of step names, e.g. --steps opt7 opt8")
|
||||
ap.add_argument("--tag", default="", help="suffix for the results directory")
|
||||
ap.add_argument("--dry-run", action="store_true",
|
||||
help="2000 frames, 1 rep — proves the matrix executes")
|
||||
ap.add_argument("--allow-busy-gpu", action="store_true",
|
||||
help="skip the idle-GPU check (results will not be quotable)")
|
||||
ap.add_argument("--no-isolate", action="store_true",
|
||||
help="run every step in ONE process. Faster (the dataset is "
|
||||
"loaded once) but fault counts become meaningless: the "
|
||||
"heap is process-wide, so each step inherits whatever "
|
||||
"the previous ones grew. Measured: opt3 reports 2 faults "
|
||||
"after opt1/opt2 have run, and 92,251 on its own. "
|
||||
"Throughput is unaffected either way.")
|
||||
ap.add_argument("--_worker-out", default=None,
|
||||
help=argparse.SUPPRESS) # internal: one step, write CSV, exit
|
||||
ap.add_argument("--retain", action="store_true",
|
||||
help="keep every ClusterVector instead of discarding each "
|
||||
"batch after counting. Measures the finder PLUS a "
|
||||
"growing result heap, which is what the notebook did. "
|
||||
"Needs N<=20000 at 9x9 or it will OOM (46.6 GB at 100k).")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.dry_run:
|
||||
args.frames, args.reps = 2000, 1
|
||||
args.tag = args.tag or "dryrun"
|
||||
|
||||
if not args.allow_busy_gpu and not getattr(args, "_worker_out", None):
|
||||
common.assert_build_fresh()
|
||||
common.assert_idle_gpu()
|
||||
|
||||
env = common.capture_env()
|
||||
outdir = common.results_dir(args.tag)
|
||||
common.write_env(outdir / "env.json", env)
|
||||
|
||||
print(f"build: DEVICE_PED_TYPE={env['device_ped_type']} "
|
||||
f"git={env['git_rev']}{'+dirty' if env['git_dirty'] else ''} "
|
||||
f"gpu={env['gpu']}")
|
||||
print(f"out: {outdir}\n")
|
||||
|
||||
for dim in args.dims:
|
||||
cfg = dict(CONFIGS[dim])
|
||||
if args.frames:
|
||||
cfg["n_frames"] = args.frames
|
||||
steps = ladder.steps_for(dim)
|
||||
if args.steps:
|
||||
steps = [s for s in steps if s.step in args.steps]
|
||||
if not steps:
|
||||
print(f"{dim}x{dim}: no runnable steps, skipping")
|
||||
continue
|
||||
|
||||
print(f"=== {dim}x{dim} cap={cfg['cap']} N={cfg['n_frames']:,} "
|
||||
f"batch={cfg['batch_size']} reps={args.reps} ===")
|
||||
skipped = [s.step for s in ladder.STEPS if s not in steps]
|
||||
if skipped and not args.steps:
|
||||
print(f" not available at this size: {', '.join(skipped)}")
|
||||
|
||||
rows: list[Row] = []
|
||||
for step in steps:
|
||||
try:
|
||||
if args.no_isolate or getattr(args, "_worker_out", None):
|
||||
got = ladder.measure(step, dim, cfg["cap"], cfg["n_frames"],
|
||||
cfg["batch_size"], args.reps,
|
||||
retain=args.retain)
|
||||
else:
|
||||
got = _run_isolated(step.step, dim, args, outdir)
|
||||
except Exception as exc: # one broken step must not lose the rest
|
||||
print(f" {step.step:<6} FAILED: {type(exc).__name__}: {exc}")
|
||||
continue
|
||||
if not got:
|
||||
print(f" {step.step:<6} produced no rows")
|
||||
continue
|
||||
rows.extend(got)
|
||||
last = got[-1]
|
||||
print(f" {step.step:<6} {last.fps:9,.0f} FPS "
|
||||
f"{last.us_per_frame:6.1f} us/f faults {last.minor_faults:>9,}")
|
||||
|
||||
if not rows:
|
||||
continue
|
||||
if getattr(args, "_worker_out", None):
|
||||
common.write_rows(Path(getattr(args, "_worker_out", None)), rows)
|
||||
continue
|
||||
csv_path = outdir / f"ladder_{dim}x{dim}.csv"
|
||||
common.write_rows(csv_path, rows)
|
||||
common.append_manifest(outdir / "manifest.csv", {
|
||||
"artifact": csv_path.name,
|
||||
"kind": "end-to-end wall/FPS",
|
||||
"config": f"{dim}x{dim} cap={cfg['cap']} N={cfg['n_frames']} "
|
||||
f"batch={cfg['batch_size']} streams={cfg['n_streams']} "
|
||||
f"consumer={'retain' if args.retain else 'streaming'}",
|
||||
"build": env["device_ped_type"],
|
||||
"cites": "docs/ClusterFinderCUDA_benchmark_results.md §5, §6, §7, §8, §9, §11",
|
||||
"produced_by": "perf/run_ladder.py",
|
||||
"timestamp": env["timestamp"],
|
||||
})
|
||||
|
||||
print(f"\n -> {csv_path.name} (all reps; cold = rep 0, warm = last)")
|
||||
common.print_table(rows, ROOFLINE_US.get(dim))
|
||||
print()
|
||||
|
||||
print(f"manifest: {outdir / 'manifest.csv'}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,170 @@
|
||||
#!/usr/bin/env python3
|
||||
"""nsys probe sweep: per-engine GPU times, duty cycles and the roofline.
|
||||
|
||||
python run_probes.py # the campaign sweep
|
||||
python run_probes.py --frames 2000 # quick check
|
||||
|
||||
Produces, per config, an .nsys-rep + .sqlite in the results directory and one
|
||||
row in probes.csv. This is the ONLY source of the rooflines that run_ladder.py's
|
||||
"% of roofline" column divides by.
|
||||
|
||||
Why this cannot be merged with run_ladder.py: nsys inflates wall clock ~4x by
|
||||
tracing every CUDA API call, so a profiled run cannot produce a throughput
|
||||
number, and an unprofiled run cannot produce a per-engine breakdown. Two tools,
|
||||
two questions.
|
||||
|
||||
Why 20 000 frames and not 2 000: over a short run the GPU clocks never fully
|
||||
ramp (210 MHz idle -> 3.1 GHz boost), which under-reports the GPU by ~10 %. Every
|
||||
retained probe from the previous campaign used 2 000 frames, which is how a
|
||||
26.7 us/frame roofline was published for a pipeline that sustains 23.9.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
import common
|
||||
import gpu_span
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
NSYS = "/opt/nvidia/nsight-systems/2024.5.1/bin/nsys"
|
||||
|
||||
# (cluster_dim, cap, n_streams, label)
|
||||
# s4 = the configuration the ladder runs, so its roofline is the one to quote.
|
||||
# Caps MUST track run_ladder.py's CONFIGS. At 9x9 the D2H slot is
|
||||
# 4 + cap * sizeof(ClusterType) and is copied whole regardless of
|
||||
# occupancy, so a probe at a different cap measures a different D2H bar
|
||||
# and its "roofline" would not be the ladder's.
|
||||
# s1 = the uncontended control: with one stream H2D and D2H never coexist, so
|
||||
# it separates "this engine is slow" from "these engines are fighting"
|
||||
# (docs §8.2 — H2D loses 23 % of its bandwidth against a busy D2H).
|
||||
CONFIGS = [
|
||||
(3, 3000, 4, "3x3_s4"),
|
||||
(3, 3000, 1, "3x3_s1_uncontended"),
|
||||
(9, 1700, 4, "9x9_s4"),
|
||||
(9, 1700, 1, "9x9_s1_uncontended"),
|
||||
]
|
||||
|
||||
|
||||
def run_one(cdim, cap, streams, label, n_frames, batch, outdir) -> dict | None:
|
||||
# The cap is in the filename because at 9x9 it SETS the D2H bar: the slot is
|
||||
# 4 + cap * sizeof(ClusterType) and is copied whole regardless of occupancy.
|
||||
# Two probes of the same label at different caps are different measurements,
|
||||
# and the earlier campaign's 9x9 probes were taken at cap=1500. Without the
|
||||
# suffix they overwrite each other and the difference disappears.
|
||||
rep = outdir / f"probe_{label}_cap{cap}"
|
||||
print(f"\n--- {label}: {cdim}x{cdim} cap={cap} streams={streams} "
|
||||
f"N={n_frames} ---")
|
||||
|
||||
prof = [NSYS, "profile", "--trace=cuda", "--sample=none", "--cpuctxsw=none",
|
||||
"--force-overwrite=true", "-o", str(rep),
|
||||
sys.executable, str(HERE / "nsys_kernel_probe.py"),
|
||||
str(streams), str(n_frames), str(cdim), str(cap), str(batch)]
|
||||
p = subprocess.run(prof, capture_output=True, text=True)
|
||||
for line in p.stdout.splitlines():
|
||||
if line.strip().startswith(("n_streams", "H2D/frame", "wall")):
|
||||
print(" ", line.strip())
|
||||
if not (rep.with_suffix(".nsys-rep")).exists():
|
||||
print(f" FAILED: {p.stderr.strip()[-400:]}")
|
||||
return None
|
||||
|
||||
# --force-export makes the .sqlite gpu_span.py reads
|
||||
subprocess.run([NSYS, "stats", "--force-export=true", "--report",
|
||||
"cuda_gpu_sum", str(rep.with_suffix(".nsys-rep"))],
|
||||
capture_output=True, text=True)
|
||||
sq = rep.with_suffix(".sqlite")
|
||||
if not sq.exists():
|
||||
print(" FAILED: no sqlite export")
|
||||
return None
|
||||
|
||||
r = gpu_span.analyze(sq, n_frames)
|
||||
r.update(label=label, cluster_dim=cdim, cap=cap, n_streams=streams,
|
||||
batch=batch, device_ped_type=common.device_ped_type())
|
||||
print(f" kernel {r['kernel_us_per_frame']:5.1f} us (duty {r['kernel_duty_pct']:4.1f}%) "
|
||||
f"H2D {r['H2D_us_per_frame']:5.1f} ({r['H2D_duty_pct']:4.1f}%) "
|
||||
f"D2H {r['D2H_us_per_frame']:5.1f} ({r['D2H_duty_pct']:4.1f}%)")
|
||||
print(f" -> roofline: {r['bottleneck']}-bound at "
|
||||
f"{r['roofline_us_per_frame']:.1f} us/frame = {r['roofline_fps']:,.0f} FPS")
|
||||
return r
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--frames", type=int, default=20_000)
|
||||
ap.add_argument("--batch", type=int, default=2000)
|
||||
ap.add_argument("--tag", default="")
|
||||
ap.add_argument("--only", nargs="+", default=None, help="subset of labels")
|
||||
ap.add_argument("--cap", type=int, default=None,
|
||||
help="override the cap for every selected config. At 9x9 the "
|
||||
"cap sets the D2H bar (the slot is copied whole), so this "
|
||||
"is how you A/B two caps ON ONE BUILD IN ONE SESSION "
|
||||
"rather than against a probe taken days earlier. Artifact "
|
||||
"filenames carry the cap, so runs do not overwrite.")
|
||||
args = ap.parse_args()
|
||||
|
||||
common.assert_build_fresh()
|
||||
common.assert_idle_gpu()
|
||||
env = common.capture_env()
|
||||
outdir = common.results_dir(args.tag)
|
||||
common.write_env(outdir / "env.json", env)
|
||||
print(f"build: DEVICE_PED_TYPE={env['device_ped_type']} git={env['git_rev']}")
|
||||
print(f"out: {outdir}")
|
||||
|
||||
rows = []
|
||||
for cdim, cap, streams, label in CONFIGS:
|
||||
if args.only and label not in args.only:
|
||||
continue
|
||||
if args.cap:
|
||||
cap = args.cap
|
||||
r = run_one(cdim, cap, streams, label, args.frames, args.batch, outdir)
|
||||
if r:
|
||||
rows.append(r)
|
||||
common.append_manifest(outdir / "manifest.csv", {
|
||||
"artifact": f"probe_{label}_cap{cap}.nsys-rep / .sqlite",
|
||||
"kind": "nsys per-engine GPU times + duty cycles",
|
||||
"config": f"{cdim}x{cdim} cap={cap} N={args.frames} "
|
||||
f"streams={streams} batch={args.batch}",
|
||||
"build": env["device_ped_type"],
|
||||
"cites": "docs §7 rooflines, §8 kernel/memcpy, §8.1 duty cycles, §9",
|
||||
"produced_by": "perf/run_probes.py",
|
||||
"timestamp": env["timestamp"],
|
||||
})
|
||||
|
||||
if rows:
|
||||
out = outdir / "probes.csv"
|
||||
# MERGE, do not clobber. A results directory may legitimately hold
|
||||
# several probe runs -- a cap A/B is exactly that -- and the artifact
|
||||
# filenames already carry the cap. Writing "w" here silently discarded
|
||||
# the first half of the first such A/B. Rows are keyed by
|
||||
# (label, cap, n_streams): re-running one config replaces its own row
|
||||
# and leaves every other row alone.
|
||||
def _key(r):
|
||||
return (str(r["label"]), str(r["cap"]), str(r["n_streams"]))
|
||||
|
||||
prior = list(csv.DictReader(out.open())) if out.exists() else []
|
||||
fresh = {_key(r) for r in rows}
|
||||
merged = [r for r in prior if _key(r) not in fresh] + rows
|
||||
with out.open("w", newline="") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=list(rows[0]))
|
||||
w.writeheader()
|
||||
w.writerows(merged)
|
||||
print(f"\n=== rooflines ({env['device_ped_type']} build) ===")
|
||||
print(f"{'config':<22} {'kernel':>8} {'H2D':>8} {'D2H':>8} "
|
||||
f"{'bottleneck':<10} {'roofline':>10} {'FPS':>10}")
|
||||
for r in rows:
|
||||
print(f"{r['label']:<22} {r['kernel_us_per_frame']:8.2f} "
|
||||
f"{r['H2D_us_per_frame']:8.2f} {r['D2H_us_per_frame']:8.2f} "
|
||||
f"{r['bottleneck']:<10} {r['roofline_us_per_frame']:9.2f}u "
|
||||
f"{r['roofline_fps']:10,.0f}")
|
||||
print(f"\n-> {out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Tiered CPU/CUDA agreement study for the deck's validation slides.
|
||||
|
||||
Runs the three finders that differ in exactly one thing each over the same
|
||||
frames, from the same pedestal, and scores every pair:
|
||||
|
||||
ClusterFinder serial CPU, pedestal pushed DURING the raster scan
|
||||
ClusterFinderFrozen same logic, pedestal frozen per frame + deferred push
|
||||
ClusterFinderCUDA frozen per frame, float32 device pedestal
|
||||
|
||||
so that serial vs frozen = update timing alone (a CPU-only effect)
|
||||
and frozen vs cuda = everything CUDA changes.
|
||||
|
||||
The question the deck needs answered is not "how many disagree" but "in which
|
||||
direction, and is a CUDA-only centre an invented photon or a second copy of one
|
||||
both finders already found". So every CUDA-only centre is also scored at tol=1:
|
||||
if it has a counterpart in the agreed set's 8-neighbourhood it is a duplicate,
|
||||
not an invention.
|
||||
|
||||
Writes tiers.json + spectra_valid.png next to itself.
|
||||
"""
|
||||
import sys, json, time
|
||||
sys.path.append('/home/ferjao_k/aare/build')
|
||||
sys.path.append('/home/ferjao_k/aare/python/tests')
|
||||
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
import boost_histogram as bh
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
|
||||
from aare import File, ClusterFinder, ClusterFinderFrozen, ClusterFinderCUDA
|
||||
from helper import centers, only_sets, shift_dist
|
||||
|
||||
OUT = Path(__file__).resolve().parent
|
||||
BASE = Path('/mnt/sls_det_storage/moench_data/2603_MaxIVBeamtime/2026032408/'
|
||||
'process/xrf/')
|
||||
|
||||
N_PED, N, N_SIGMA, N_STREAMS = 1000, 10000, 5, 4
|
||||
CLUSTER = (3, 3)
|
||||
IMG = (400, 400)
|
||||
CAP = 50_000
|
||||
NBINS, ERANGE = 200, (-2, 4000)
|
||||
|
||||
f = File(BASE / 'Cu_factor_10_data_master_0.json')
|
||||
pd = File(BASE / 'Cu_factor_10_pedestal_master_0.json')
|
||||
|
||||
cf_cpu = ClusterFinder(IMG, CLUSTER, n_sigma=N_SIGMA, capacity=CAP)
|
||||
cf_frz = ClusterFinderFrozen(IMG, CLUSTER, n_sigma=N_SIGMA, capacity=CAP)
|
||||
cf_cud = ClusterFinderCUDA(IMG, CLUSTER, n_sigma=N_SIGMA,
|
||||
max_clusters_per_frame=3000, n_streams=N_STREAMS)
|
||||
finders = {'cpu': cf_cpu, 'frozen': cf_frz, 'cuda': cf_cud}
|
||||
|
||||
t0 = time.perf_counter()
|
||||
pd.seek(0)
|
||||
for _ in range(N_PED):
|
||||
img = pd.read_frame().copy()
|
||||
for cf in finders.values():
|
||||
cf.push_pedestal_frame(img)
|
||||
print(f'pedestal train: {time.perf_counter()-t0:.1f}s', flush=True)
|
||||
|
||||
f.seek(0)
|
||||
data = f.read_n(N)
|
||||
print('data:', data.shape, data.dtype, flush=True)
|
||||
|
||||
names = list(finders)
|
||||
totals = {n: 0 for n in names}
|
||||
hists = {n: bh.Histogram(bh.axis.Regular(NBINS, *ERANGE)) for n in names}
|
||||
pairs = {(a, b): dict(a_only=0, b_only=0) for i, a in enumerate(names)
|
||||
for b in names[i + 1:]}
|
||||
|
||||
# every CUDA-only centre, scored against the agreed set
|
||||
extras = [] # one record per frozen-vs-cuda cuda-only centre
|
||||
n_dup_tol1 = 0
|
||||
|
||||
t0 = time.perf_counter()
|
||||
for fid in range(N):
|
||||
cs = {}
|
||||
for n, cf in finders.items():
|
||||
cf.find_clusters(data[fid])
|
||||
cv = cf.steal_clusters(realloc_same_capacity=True)
|
||||
cs[n] = centers(cv)
|
||||
totals[n] += len(cs[n])
|
||||
if cv.size:
|
||||
hists[n].fill(np.asarray(cv.sum()).ravel())
|
||||
|
||||
for (a, b), acc in pairs.items():
|
||||
a_only, b_only = only_sets(cs[a], cs[b], tol=0)
|
||||
acc['a_only'] += len(a_only)
|
||||
acc['b_only'] += len(b_only)
|
||||
|
||||
# the tier that matters: frozen vs cuda, one record per extra
|
||||
_, cu_only = only_sets(cs['frozen'], cs['cuda'], tol=0)
|
||||
for p in cu_only:
|
||||
d = shift_dist(p, cs['frozen'], R=4)
|
||||
extras.append(dict(frame=int(fid), x=int(p[0]), y=int(p[1]),
|
||||
shift=int(d)))
|
||||
if cu_only:
|
||||
_, cu_only_1 = only_sets(cs['frozen'], cs['cuda'], tol=1)
|
||||
n_dup_tol1 += len(cu_only) - len(cu_only_1)
|
||||
|
||||
if fid % 1000 == 0:
|
||||
print(f' {fid}/{N} {time.perf_counter()-t0:.0f}s', flush=True)
|
||||
|
||||
print(f'scan: {time.perf_counter()-t0:.0f}s', flush=True)
|
||||
|
||||
res = dict(n_frames=N, totals=totals,
|
||||
pairs={f'{a} vs {b}': v for (a, b), v in pairs.items()},
|
||||
extras=extras,
|
||||
n_cuda_only=len(extras),
|
||||
n_adjacent_to_agreed=n_dup_tol1,
|
||||
shift_histogram={str(k): int(v) for k, v in
|
||||
zip(*np.unique([e['shift'] for e in extras],
|
||||
return_counts=True))} if extras else {},
|
||||
hists={n: h.values().tolist() for n, h in hists.items()},
|
||||
edges=hists[names[0]].axes[0].edges.tolist())
|
||||
(OUT / 'tiers.json').write_text(json.dumps(res))
|
||||
|
||||
print('\n=== totals ===')
|
||||
for n in names:
|
||||
print(f' {n:8s} {totals[n]:>12,}')
|
||||
print('\n=== pairwise (tol=0) ===')
|
||||
for (a, b), v in pairs.items():
|
||||
tot = v['a_only'] + v['b_only']
|
||||
print(f' {a:>6s} vs {b:<6s} {a}-only {v["a_only"]:>4} '
|
||||
f'{b}-only {v["b_only"]:>4} total {tot:>4} '
|
||||
f'({tot/max(totals[a],1):.2e})')
|
||||
print('\n=== the frozen-vs-cuda extras ===')
|
||||
print(f' cuda-only centres (tol=0): {len(extras)}')
|
||||
print(f' of which adjacent to an agreed centre (tol=1): {n_dup_tol1}')
|
||||
print(f' chebyshev shift to nearest frozen centre: {res["shift_histogram"]}')
|
||||
Reference in New Issue
Block a user