4.1 KiB
CUDA and Proprietary Nvidia GPU Drivers on RHEL 8
Managing Nvidia software comes with its own set of challenges. For the most common cases are covered by our Puppet configuration. Those are discussed in the first chapter, more details you find more below.
Hiera Configuration
Changes in Hiera are forwared by Puppet to the node, but not applied.
They are applied on reboot.
Alternatively you might execute /opt/pli/libexec/ensure-nvidia-software in a safe moment (no process using CUDA and the desktop will be restarted).
I need CUDA
Set in Hiera nvidia::cuda::enable true and it will automatically install the suitable Nvidia drivers and newest possible CUDA version.
The nvidia_persistenced service is automatically started. If you do not want it, to set nvidia::cuda::nvidia_persistenced::enable: false.
I need a specific CUDA version
Then you can additionally set nvidia::cuda::version to the desired version.
The version must be fully specified (all three numbers, with X.Y.0 for the GA version).
Note that newer CUDA versions do not support older drivers, for details see Table 3 in the CUDA Release Notes.
I just need the Nvidia drivers
Nothing needs to be done, they are installed by default when Nvidia GPUs or accelerators are found.
I do not want the Nvidia drivers
Set in Hiera nvidia::driver::enable: false. Note this will be ignored if CUDA is enabled (see above).
Note they do not get automatically removed when already installed. That you would need to do by hand.
I need the Nvidia drivers from a specific driver branch
The driver branch can be selected in Hiera with nvidia::driver::branch. It will then use the latest driver version of that branch. Note that only production branches are available in the PSI package repository.
I need a Nvidia driver of a given version
This is not recommended, still it is possible to do so by setting the exact driver version (X.Y.Z, excluding the package iteration number) in Hiera with nvidia::driver::version.
If the driver version is too old, it will install an older kernel version and you will need a second reboot to activate it.
Versioning Mess
I did not find much information about Nvidia driver version structure and policy. Still I concluded that they use following pattern.
Their drivers are oranized in driver branches. As you see for example in their Unix Driver Archive noted as e.g. 470.xx series.
There are Production and New Feature branches (and, on the above linked page, a Beta Version which is not linked to any of the above branches (yet?).
Such a branch can be considered a major release and with new braches adding support for new hardware or removing support for old hardware. The drivers within a branch are maintained quite a long time. Individual drivers in that branch get increasing version numbers which just start with the same first "branch" number.
In the RPM repo there are more branches available than listed in the Unix Driver Archive. It is not possible to find out retrospectively to what type of branch it belongs. Also it is not possible to find out from the package meta information if a driver is considered beta or not. That you only find out by googling "Nvidia $DRIVER_VERSION" and looking at the respective driver page. In my experience the first few driver versions of a branch are usually "beta".
To figure out what driver branch to use for given hardware, go to their Download page and search its Linux driver. It will then point out a driver version and its first number points out the driver branch to use. Note that this is not always the full story. For example the Tesla K40c gives driver 460.106.00, where as the 470 branch driver still works, though the 470 driver still works, even though the hardware is not listed as supported there.