Every cuFFT plan has the driver compile cuFFT's PTX kernels: cufftMakePlanMany ->
cuModuleLoadData -> the PTX JIT, under cuFFT's process-wide lock. A second plan of the same
shape pays again, even while the first is alive. The CUDA JIT cache makes that ~1 ms. Without
one (CUDA_CACHE_DISABLE, no writable $HOME) a plan costs 20-250 ms. That holds for every size
(64^3 and 1D 256 included), every transform type and batch, on sm_75 and sm_120, with cuFFT 11.2,
12.0 and 12.4. No CUFFT_* setting changes it, so a choice of grid sizes cannot avoid it; only
the number of plans made can be cut.
Before this change, a plan was lent to one user at a time, so each concurrent user of a shape
on a card made its own: 9-24 rigid-body engines, the translation slices and the score engines.
Now a plan is made once per card and shape and shared. Each user keeps its own stream and work
area, and CudaFFTPlan::R2C/C2R set both and launch under the plan's mutex. Shared plans give the
same bits as private ones: 8 threads on one plan with their own work areas, 9 shapes x 160
transforms, 0 mismatches on RTX 5080 and T4. A shared work area, the negative control, gives
2-13 mismatches per shape where the plan has a work area.
The rigid-body engine also dropped its batch-3 plan: the Jacobian transforms its three grids one
at a time on the batch-1 plan. That gives the same bits (10 zone-grid shapes, sm_75 and sm_120)
and one plan less per zone and card.
The cytochrome C and thaumatin MR test sets, AF models (MR step, s):
4x T4 cytc cache on 2.22 -> 2.22 cache off 8.41 -> 4.00 plans 86 -> 24
4x T4 thau cache on 1.58 -> 1.61 cache off 11.13 -> 3.08 plans 131 -> 23
1x T4 cytc cache on 2.87 -> 2.81 cache off 8.82 -> 3.84 plans 80 -> 12
1x T4 thau cache on 2.41 -> 2.39 cache off 10.42 -> 3.17 plans 104 -> 11
RTX5080 cytc cache on 1.41 -> 1.12 cache off 6.5-10.7 -> 2.02
RTX5080 thau cache on 0.90 -> 0.85 cache off 8.03 -> 1.60
Outputs are bit-identical before and after, cache on or off: model, merged hkl, maps, and every
MR and validation log line. The mtz/cif differ only in the command line they record.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
leonarski_fandClaude Opus 5.5
97ef0936b2