A service started by systemd without User= runs as root with no HOME, so
the driver keeps no JIT cache and every cuFFT plan is compiled again
(50-130 ms each, under cuFFT's process-wide lock) - seconds of MR in the
broker. Before anything touches CUDA, the broker and rugnux now point
CUDA_CACHE_PATH at /var/tmp/jfjoch-cuda-cache-<uid> when the default
cache is missing or unusable and the site has set no CUDA_CACHE_PATH.
The directory is used only if it is a real directory of this user's,
mode 0700; otherwise the start-up warning stays as before.
Measured without HOME: a cuFFT plan 51 ms -> 0.5 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
cuFFT ships its transforms as PTX for every card (cuFFT 11 and 12 alike), so each plan has the driver
compile kernels, and a second plan of the same shape compiles them again. From the driver's JIT cache
that costs ~0.5 ms a plan; without it 45-130 ms, under cuFFT's process-wide lock - which, over the
~100 plans of molecular replacement, matches the stage pattern of the slow MR in the production broker.
The cache is unusable in more ways than CUDA_CACHE_DISABLE: with no HOME at all (a systemd system
service without User= gets none, and the driver does not fall back to the passwd entry), on a read-only
or unwritable directory, on a filesystem whose fcntl locks fail (NFS without locking) - each measured
as slow as no cache - or on a slow network filesystem (each plan opens and locks the cache ~10 times).
- common/CudaJitCache: the path the driver uses (CUDA_CACHE_PATH, else $HOME/.nv/ComputeCache), its
filesystem type, whether it is writable, whether its index opens read-write and takes the driver's
fcntl read lock, CUDA_CACHE_DISABLE and a CUDA_CACHE_MAXSIZE below 64 MiB. One line: info when fine,
a warning with the cause and the remedy (CUDA_CACHE_PATH on a local disk) when not, or when a plan
takes over 10 ms.
- WarmUpCuFFT returns the fastest of three 64^3 R2C plan makes after a first one that pays the
initialisation (and a cold-cache compile): ~2 ms in all with a cache, ~0.3 s without.
- jfjoch_broker runs it after its version line (0.15 s at start, cuFFT init and card 0's context
included); rugnux on its existing background warm-up thread.
- CUDAMemHelpers.h: the modules are PTX on every platform, not only on Windows.
No result changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXc1zD1yxHqaR1DGdCTHL4
leonarski_fandClaude Opus 5.5