NAME

Rex::GPU - GPU detection and driver management for Rex

VERSION

version 0.002

SYNOPSIS

use Rex::GPU;

# Detect GPUs only — returns a hashref
my $gpus = gpu_detect();
if (@{ $gpus->{nvidia} }) {
  say "NVIDIA GPU: ", $gpus->{nvidia}[0]{name};
}

# Full GPU setup for an RKE2 cluster (detect + drivers + toolkit + containerd)
gpu_setup(
  containerd_config => 'rke2',   # 'rke2', 'k3s', 'containerd', or 'none'
  reboot            => 1,        # reboot after driver install (first deploy)
);

# For a K3s cluster
gpu_setup(containerd_config => 'k3s');

# Just drivers + toolkit, no containerd config
gpu_setup(containerd_config => 'none');

DESCRIPTION

Rex::GPU provides GPU detection and driver management for Rex. It automates the complete software stack needed to make NVIDIA GPUs available to workloads running in a Kubernetes cluster.

The full pipeline, as executed by "gpu_setup":

1. GPU detection — PCI class code scan via lspci -nn to identify NVIDIA and AMD hardware, filtering out virtual GPUs (virtio, QEMU, VMware). Only CUDA-capable NVIDIA GPUs trigger driver installation, decided by the GPU generation read from the PCI device ID, not by name or PCI class: Maxwell and newer count (GeForce MX, GT and GTX 9xx included), Kepler and older are skipped with a warning at any PCI class. See "gpu_setup".
2. NVIDIA driver installation — Distribution-appropriate packages via DKMS for kernel-version independence, chosen so one driver fits every detected GPU. Nouveau is blacklisted and the initramfs is regenerated. An NVIDIA vGPU guest without a working driver dies before any driver package is installed (see "gpu_setup").
3. NVIDIA Container Toolkit — Installs nvidia-container-toolkit from the official NVIDIA repository for all supported distributions, unless it is already installed (then it is left as it is, not upgraded; see "install_container_toolkit" in Rex::GPU::NVIDIA).
4. CDI spec generation — Writes /etc/cdi/nvidia.yaml so the Kubernetes device plugin can enumerate GPU resources without privileged container access.
5. Containerd runtime configuration — Injects the NVIDIA runtime into the containerd config for the target Kubernetes distribution.

Tested on Hetzner dedicated servers (bare metal) running:

Debian 11 (bullseye), 12 (bookworm), 13 (trixie)
Ubuntu 22.04 (jammy), 24.04 (noble)
RHEL / Rocky Linux / AlmaLinux 8, 9, 10 — CentOS Stream 9, 10

The verified target set is the RKE2 Linux family above. openSUSE Leap / SLES is unverified and unsupported — SUSE is not a deploy target for the GPU-on-Rancher pipeline.

GPUs tested include the NVIDIA RTX 4000 SFF Ada Generation (PCI class 0302, datacenter compute profile).

This module requires Rex::LibSSH (or SFTP) on the connection backend. Hetzner servers do not enable the SFTP subsystem by default; use set connection => "LibSSH" in your Rexfile.

Rex::LibSSH version 0.004 and later verify the server's host key against known_hosts by default. A freshly provisioned host has no entry yet, so the first connection fails with host key is not in known_hosts and strict_hostkeycheck is on. There are two ways to handle this:

Recommended (keeps host-key verification) — scan the key into known_hosts before deploying:
```
ssh-keyscan <host> >> ~/.ssh/known_hosts
```
The bundled eg/Rexfile and eg/hetzner-gpu.pl do this in a before 'ALL' hook.
Disable the check Rexfile-wide — for first-contact provisioning; a deliberate security tradeoff:
```
use Rex -feature => ['1.4', 'disable_strict_host_key_checking'];
```

FUNCTIONS

gpu_detect

Detect GPU hardware on the remote host by scanning PCI devices. Installs pciutils only if lspci is not on the host's PATH (dies if it still is not afterwards), then parses lspci -nn output. See "detect" in Rex::GPU::Detect.

Returns a hashref with detected GPUs grouped by vendor, plus the HGX NVSwitch chips under nvswitch:

my $gpus = gpu_detect();
# {
#   nvidia => [
#     {
#       name      => "AD104GL [RTX 4000 SFF Ada Generation]",
#       vendor    => "nvidia",
#       pci_class => "0302",   # 0300 = VGA, 0302 = 3D/compute
#       compute   => 1,        # 1 if CUDA-capable
#       device_id => "27b0",
#       subsystem_vendor_id => "10de",
#       subsystem_id        => "16fa",
#       vgpu      => 0,        # 1 for an NVIDIA vGPU guest device
#     }
#   ],
#   amd => [
#     {
#       name      => "Radeon RX 7900 XTX",
#       vendor    => "amd",
#       pci_class => "0300",
#       compute   => 0,        # always 0 (AMD not yet supported)
#     }
#   ],
#   nvswitch => [],            # HGX NVSwitch chips, see Rex::GPU::Detect
# }

Virtual display devices (virtio, QEMU, VMware, VirtualBox) are skipped. If they are the only display devices all three arrays (nvidia, amd, nvswitch) are empty; a real card passed through next to one (vfio-pci, cloud GPU VM) is still detected. An NVIDIA vGPU guest device is told apart from a physical or passed-through card by its PCI subsystem ID, read with another read-only lspci only on hosts with an NVIDIA GPU: vgpu => 1 and vgpu_type (see "NVIDIA vGPU guests" in Rex::GPU::Detect). See Rex::GPU::Detect for details on the classification logic.

gpu_setup

Detect GPUs and run the full installation pipeline: NVIDIA driver, Container Toolkit, CDI spec generation, and containerd runtime configuration. This is the single call needed to make a node GPU-ready for Kubernetes.

Every CUDA-capable NVIDIA GPU detected (name and PCI device ID) is passed to "install_driver" in Rex::GPU::NVIDIA as gpus, and one driver is chosen that can drive them all -- the intersection of what each GPU needs (see Rex::GPU::NVIDIA::Requirement). Blackwell-architecture silicon (B200/GB200/B300, GeForce RTX 50xx, RTX PRO Blackwell, the GB10 / NVIDIA DGX Spark) has no proprietary kernel module at all, on x86_64 and aarch64 alike: Ubuntu gets the -open driver package variant instead of the default -server one, Debian 12 and 13 NVIDIA's CUDA-repository open-module driver instead of Debian's non-free one (which cannot drive them); on any other Debian release it dies before any driver package is installed. A pre-Turing GPU (Maxwell/Pascal/Volta, e.g. the V100, a GeForce GT 1030 or GTX 980) gets the proprietary 580-branch driver on Ubuntu, RHEL and openSUSE, Debian's non-free driver on Debian 12 and 13. A host whose GPUs are all Turing to Hopper gets the same driver as before.

Which GPUs count as CUDA-capable is decided by generation, not by marketing name (see "NVIDIA compute classification" in Rex::GPU::Detect): every Maxwell or newer GPU does, GeForce MX, GT and GTX 9xx included. A Kepler-or-older GPU at any PCI class (GeForce GT 710, GTX 780, Quadro K4000, and the class 0302 Tesla K80/K40/K20) is detected with compute => 0 and skipped with a warning: no driver is installed for it, and it does not stop the installation for a newer GPU on the same host. GPUs that cannot share one driver (a V100 next to a B200) make gpu_setup die before any driver package is installed, unless a working driver is already installed.

On an NVIDIA vGPU guest (vgpu => 1 in the "gpu_detect" result: Azure NVadsA10 v5, AWS G6f, ...) the GPU needs NVIDIA's licensed vGPU guest driver, which Rex::GPU does not install. If that driver already works (nvidia-smi -L lists the GPU and libcuda.so.1 is in the linker cache) gpu_setup goes on as on any host with a working driver: container toolkit, CDI specs, containerd. If not, it dies in "install_driver" in Rex::GPU::NVIDIA before any driver package is installed, naming the vGPU type -- also when a GPU that is not a vGPU sits next to it.

On an HGX baseboard with NVSwitches (HGX-2, HGX A100, HGX H100/H200: nvswitch in the "gpu_detect" result is not empty) the NVSwitches are passed as nvswitches, and NVIDIA Fabric Manager is installed with the driver at exactly its version and nvidia-fabricmanager.service enabled -- without it CUDA does not initialise on those hosts. A distro source that has no Fabric Manager is not used (Debian's non-free; Debian 12/13 takes NVIDIA's CUDA repository instead, Debian 11 and openSUSE die before any driver package is installed). If the driver is already installed, Fabric Manager is added only when the host's own package sources offer it at exactly the loaded driver's version (asked after an apt-get update on Debian/Ubuntu); otherwise it warns and installs nothing. No package source is added and the driver is not touched either way. See the nvswitches option of "install_driver" in Rex::GPU::NVIDIA. Hosts without NVSwitch are unchanged. HGX B200/B300 have no NVSwitch on the host PCI bus (nvswitch is empty there); they are recognised by the GPU device IDs, and gpu_setup installs Fabric Manager with their driver the same way, then the NVLink Subnet Manager nvlsm, infiniband-diags and libibumad (unversioned, from NVIDIA's CUDA repository -- on Ubuntu added for this, pinned to nvlsm alone), loads ib_umad, warns on a kernel older than 5.17 (except on the RHEL family), and after the start checks that every GPU reports Fabric State: Completed -- a loud warning if not, never a die. With the driver already installed, the missing ones of these packages are installed from the host's own package sources (after an apt-get update on Debian/Ubuntu; no source is added, one not offered only warns) and ib_umad is loaded. Where no nvlsm source is known it dies before any driver package is installed. GB200/GB300 NVL72 compute trays need no Fabric Manager (it runs on the NVLink switch trays); an info line notes that multi-node NVLink needs nvidia-imex, which is not set up either. See the nvswitches option of "install_driver" in Rex::GPU::NVIDIA.

Each die named above comes from "plan" in Rex::GPU::NVIDIA::Setup, known without a refreshed package index: by then no package source has been added and nothing has been installed, except pciutils when "gpu_detect" found no lspci on the host.

After the last step "verify_nvidia" in Rex::GPU::NVIDIA checks the result -- kernel module, nvidia-smi -L, container toolkit -- and logs a warning for anything missing (e.g. the module before the first reboot); it never dies. It runs on every call with a CUDA-capable GPU, also when the driver was already installed.

AMD GPUs are detected and logged but not yet supported (a warning is emitted).

gpu_setup(
  containerd_config => 'rke2',  # containerd integration target
  reboot            => 1,       # reboot after driver install
);

Options:

containerd_config

Which containerd configuration variant to write. Controls where the NVIDIA runtime snippet is placed:

rke2 (default) — registers the NVIDIA runtime additively under /var/lib/rancher/rke2/agent/etc/containerd/: a no-op if RKE2 already wired it natively, otherwise a config-v3.toml.d/ drop-in (modern, config v3) or a base-extending config.toml.tmpl (legacy, config v2). The RKE2-generated base config is never replaced.
k3s — same logic as rke2, under /var/lib/rancher/k3s/...; K3s and RKE2 share the same containerd config mechanism
containerd — runs nvidia-ctk runtime configure --runtime=containerd for a standalone containerd installation
none — skip containerd configuration entirely (driver and toolkit are still installed)

Any other value makes gpu_setup die before detection, naming the valid values -- even on a host without a GPU.

reboot

If true, the host is rebooted after driver installation and the function waits (up to 5 minutes, polling every 5 seconds) for it to come back before continuing with toolkit installation and containerd configuration. Default: 0.

Rebooting is required on the first deployment if the nouveau open-source driver was previously loaded, because nouveau must be unloaded before the NVIDIA driver can bind to the GPU.

setup

Experimental. The Rex::GPU::NVIDIA::Setup class (a name) or object that installs the driver, instead of the one Rex::GPU picks for the OS -- typically a subclass of one of the built-in ones, kept in your project's lib/ (see "WRITING YOUR OWN SETUP" in Rex::GPU::NVIDIA::Setup). Without it, set gpu_nvidia_setup => 'My::GPU::Setup' in the Rexfile does the same for every call; this option wins over it. A class that cannot be loaded or is not a Setup makes gpu_setup die before anything is done on the host, even on a host without a GPU. Passed to "install_driver" in Rex::GPU::NVIDIA.

requirement

Experimental. An extra constraint on the driver, e.g. { kernel_module => 'open', min_branch => 580 } (keys kernel_module, min_branch, max_branch), intersected with what the detected GPUs need: it can narrow the choice but never override a GPU's hard limit. If no driver meets both, gpu_setup dies before the driver install changes the host. Passed to "install_driver" in Rex::GPU::NVIDIA.

Neither option changes anything for a caller that does not pass it; in particular Rex::Rancher's gpu => 1 passes neither, and picks up a custom setup through set gpu_nvidia_setup.

Returns the result of "detect" in Rex::GPU::Detect — a hashref with nvidia, amd and nvswitch array keys.

Dies if the connection backend is neither LibSSH nor SFTP-capable.

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/rex-gpu/issues.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <getty@cpan.org>

COPYRIGHT AND LICENSE

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

To install Rex::GPU, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Rex::GPU

CPAN shell

perl -MCPAN -e shell
install Rex::GPU

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)