NAME
Rex::Rancher::Distribution - What RKE2 and K3s differ in, and the host steps they share
VERSION
version 0.003
SYNOPSIS
use Rex::Rancher::Distribution;
my $dist = Rex::Rancher::Distribution->new_for($opts{distribution});
$dist->kubeconfig; # /etc/rancher/rke2/rke2.yaml
$dist->service; # rke2-server
my $agent = Rex::Rancher::Distribution->new_for('k3s', role => 'agent');
$agent->service; # k3s-agent.service
$agent->wait_for_service(hint => 'It joins the cluster via ...');DESCRIPTION
Everything Rex::Rancher::Server, Rex::Rancher::Agent and Rex::Rancher::Cilium need to know about RKE2 versus K3s lives in one object: paths, service names, install script and release artifacts, and the host steps both distributions share (artifact download and checksum, version check, service start and wait, secret files, the NVIDIA runtime PATH). Rex::Rancher::Distribution::RKE2 and Rex::Rancher::Distribution::K3s implement the distribution-specific methods; a change to one wants the other in the same edit. Use "new_for" to get one: it loads the class.
What is the same for both and needs no host lives apart: the option checks in Rex::Rancher::Options, the checksum parsing in Rex::Rancher::Checksum.
This is an internal building block of the Rex tasks; the functions those modules export are the interface for a Rexfile. The host steps use the Rex DSL (run, file) of the current connection, like those functions.
role
server (default) or agent: which side of the distribution this object speaks for. "service" and "env_file", and the installer command lines ("script_install_cmd", "artifact_install_cmds") follow it; everything else is the same for both.
is_agent
True for role => 'agent'.
distribution_classes
Rex::Rancher::Distribution->distribution_classes
# { rke2 => 'Rex::Rancher::Distribution::RKE2', k3s => '...::K3s' }The distribution names distribution options accept, each with the class that implements it. A subclass may override it to add or replace one: its own "new_for" then loads and builds those classes, and "unknown_distribution" lists their names. The functions Rex::Rancher and its modules export ask this class, not a subclass.
new_for
my $dist = Rex::Rancher::Distribution->new_for('k3s');
my $agent = Rex::Rancher::Distribution->new_for('rke2', role => 'agent');The object for a distribution option value; undef means "default_distribution". Anything not in "distribution_classes" dies with "unknown_distribution" (Unknown distribution: NAME (expected 'rke2' or 'k3s')). Further arguments go to new. The class is loaded here, when it is first asked for.
default_distribution
rke2: what an absent distribution option means.
unknown_distribution
die Rex::Rancher::Distribution->unknown_distribution($name) . "\n";The message for a distribution option that is not in "distribution_classes", without a newline: Unknown distribution: NAME (expected 'rke2' or 'k3s'), the "default_distribution" named first.
name
rke2 or k3s, as in the distribution option.
label
RKE2 or K3s, for log lines.
config_dir
/etc/rancher/rke2 / /etc/rancher/k3s, without a trailing slash.
install_url
The official install script: https://get.rke2.io / https://get.k3s.io.
channel_url
The release channel the install script resolves an unpinned install through: https://update.rke2.io/v1-release/channels/stable / https://update.k3s.io/v1-release/channels/stable. It redirects to the GitHub release tag of the version it would install.
kubeconfig
The kubeconfig the server writes (rke2.yaml / k3s.yaml).
token_file
The node join token the server writes (server/node-token).
server_token
The token the control plane is sealed with (server/token).
default_disable
A new arrayref of the packaged components switched off when disable is not given.
default_cluster_cidr
The pod network written out (and handed to Cilium's cluster-pool) when cilium is on and cluster_cidr is not given: 10.42.0.0/16 on K3s, undef on RKE2, which keeps its own default unwritten.
binary
The distribution binary (rke2 / k3s), asked for --version.
uninstall_scripts
The uninstall scripts the distribution's installer puts on PATH, for either "role": rke2-uninstall.sh (RKE2, server and agent) / k3s-uninstall.sh and k3s-agent-uninstall.sh (K3s names it after the service it set up). "uninstall_node" in Rex::Rancher::Uninstall runs every one that is there.
release_url
The GitHub release download base the artifacts come from.
artifact_dir
Where install_method => 'artifact' downloads to on the host.
containerd_dir
The directory holding the generated containerd config.toml.
server_service
The server's systemd unit: rke2-server / k3s.
agent_service
The agent's systemd unit: rke2-agent.service / k3s-agent.service. Unlike "server_service" with the .service suffix: both are handed to systemctl and journalctl and named in log lines and errors exactly like this, so the asymmetry is kept.
env_file
The EnvironmentFile of the "role"'s unit that gets the PATH for the NVIDIA runtime lookup, or undef where none is needed (K3s).
default_start_verb
How the service is started when its containerd config is not stale: start (RKE2: a running service is restarted only for "restart_reasons") or restart (K3s, on every run, as its install script did, unless hold_running: see "start_verb"). Either way "start_verb" checks the version skew first, and an unpinned new minor version leaves a running service of either distribution alone.
installer_unit_files
The files the distribution's install script rewrites on every run and the "service" reads only when it starts, for "installer_unit_digest": on K3s the unit and its env file, /etc/systemd/system/k3s.service and k3s.service.env (agent: k3s-agent.service, .env); none on RKE2, whose unit comes with the version installed.
asset_name
$dist->asset_name('arm64') # rke2.linux-arm64.tar.gz / k3s-arm64The release artifact for a GOARCH.
cilium_config
The config.yaml keys that leave pod networking, network policy and kube-proxy to Cilium, as a new hashref with real booleans.
default_ipam_mode
Cilium's IPAM mode on a fresh install when ipam_mode is not given: kubernetes on RKE2 (pods take the node podCIDRs cut from the cluster's cluster-cidr), cluster-pool on K3s (Cilium's own pool).
cilium_helm_defaults
$dist->cilium_helm_defaults(cluster_cidr => $cidr, k8s_service_host => $host,
ipam_mode => $mode)The distribution's part of Cilium's default Helm values, as a new hashref with real booleans, for Rex::Rancher::Cilium to merge over the values both share: cni.exclusive, k8sServiceHost and the IPAM mode and pool. The mode is ipam_mode, or "default_ipam_mode" without one. RKE2: not exclusive (its CNI config stays), 127.0.0.1, cluster_cidr as the pool (used only in cluster-pool mode). K3s: exclusive, k8s_service_host, the pool on cluster_cidr or "default_cluster_cidr".
needs_k8s_service_host
True where Cilium's kube-proxy replacement needs the control plane's address from the caller (K3s: its agents serve the API on 127.0.0.1:6444, not 6443); false where every node serves it on 127.0.0.1:6443 (RKE2), which then refuses a k8s_service_host.
gateway_api_crd_chart
The packaged Helm chart that brings its own Gateway API CRDs and has to be disabled for gateway_api: rke2-gateway-api-crd on RKE2 (v1.37+), undef on K3s.
live_verified
True for a distribution that has been deployed live through Rex::Rancher: RKE2 and K3s both. For one that has not, "install_server" in Rex::Rancher::Server warns.
script_install_cmd
$dist->script_install_cmd($server, $version)The curl | sh install line for the "role". $server (join URL) and $version may be undef. The token is never on it.
artifact_install_cmds
$dist->artifact_install_cmds($spec, $server, $version)The command lines that install from a verified "fetch_artifacts" spec, in order, for the "role". Each has to succeed.
run_server_install_script
$dist->run_server_install_script($server, $version)Run "script_install_cmd" for a server, with the exit-status handling the distribution's install script needs.
service
The unit of the "role": "server_service" or "agent_service".
config_file
config.yaml in "config_dir".
registries_file
registries.yaml in "config_dir".
cni_bin_dir
Where the kubelet of either distribution looks for CNI binaries once its own CNI is off, /opt/cni/bin: Cilium's cni.binPath.
cni_conf_dir
Where it looks for CNI configuration, /etc/cni/net.d: Cilium's cni.confPath.
restart_services_cmd
The line "update_registries" in Rex::Rancher::Server runs: restart the server unit, or the agent unit if that fails.
resolve_install_method
The same as "resolve_install_method" in Rex::Rancher::Options, which is not distribution-specific; kept here for callers of this class.
check_cluster_cidr
The same as "check_cluster_cidr" in Rex::Rancher::Options; kept here for callers of this class.
goarch
Rex::Rancher::Distribution->goarch("aarch64\n") # arm64The GOARCH release artifacts are named by, from uname -m output. Dies for anything but amd64 and arm64.
artifact_spec
my $spec = $dist->artifact_spec($arch, $version);Where the install script, the artifact and its checksum file come from and go to: a hashref with dir, script, script_url, asset, asset_url, sums and sums_url. Dies without a version or for one with characters a URL or shell line must not carry.
expected_sha256
The same as "expected_sha256" in Rex::Rancher::Checksum, which is not distribution-specific; kept here for callers of this class, and as what "fetch_artifacts" calls, so a subclass can override it.
sha256_of
The same as "sha256_of" in Rex::Rancher::Checksum, kept as "expected_sha256" is.
verify_sha256
The same as "verify_sha256" in Rex::Rancher::Checksum, kept as "expected_sha256" is.
download_cmd
$dist->download_cmd($url, $dest, $progress)The curl line that downloads $url to $dest on the host; with $progress it shows a progress bar.
fetch_artifacts
my $spec = $dist->fetch_artifacts($version);Download install script, artifact and checksum file on the host (curl, no SFTP, no upload) into an emptied "artifact_dir", for the host's own architecture, and verify the artifact. Dies on any failure. Returns the "artifact_spec".
install_server_package
$dist->install_server_package($server, $version, $method);Put the distribution onto a server host with $method (script or artifact). Installing only: the version check and the service start are the caller's.
parse_version_output
The version in --version output of rke2 or k3s, or nothing. Its first word is the name the binary was called by: rke2 version ... / k3s version ... for "binary", exe version ... through /proc/PID/exe ("running_version"). The go version ... line never counts.
same_version
1 when both versions are given and equal, ignoring a leading v.
verify_installed_version
$dist->verify_installed_version($version);Without $version returns 1 at once. Otherwise asks "binary" for its version and dies unless it is $version: a failed pinned install or upgrade leaves the old binary in place.
parse_release
my @v = $dist->parse_release('v1.30.4+rke2r1'); # (1, 30, 4, 1)Pure: major, minor, patch and the distribution revision (rke2rN / k3sN; 0 without one) of a version, or nothing for anything not shaped vMAJOR.MINOR.PATCH.
compare_versions
$dist->compare_versions($x, $y) # -1, 0 or 1Pure: <=> over "parse_release", undef when either does not parse.
version_skew
$dist->version_skew($running, $new)Pure: what moving from $running to $new is. same; patch (same minor, newer, or only a pre-release tag differs); minor (exactly the next minor); jump (more than one minor, or another major); downgrade (older, patch or revision included). undef when either does not parse.
parse_channel_redirect
Pure: the version at the end of the release tag URL "channel_url" redirects to (.../releases/tag/v1.36.4+rke2r1, %2B decoded), or nothing.
channel_version
The version an unpinned install would get: "channel_url" resolved on the host with curl, as the install script resolves it there. Nothing when that fails.
main_pid
The main PID of the "service", or nothing when it is not running. Reads the host (systemctl show -p MainPID).
running_version
$dist->running_version($pid)The version the process $pid runs (/proc/PID/exe --version: the binary it was started from, even when the installer has replaced it on disk since; called that way it answers exe version ...), or nothing.
installed_version
The version the installed "binary" reports, or nothing.
held_version
my $version = $dist->held_version(version => $pinned);What hold_running holds a run to, read from the host and meant to be used as version from there on: for "check_version_skew", the installer, "verify_installed_version" and "start_verb" (pinned). Writes nothing, so it can be asked before anything else is.
The "running_version" of the "service" when it runs ("main_pid"); when it does not run, the "installed_version", the binary it would start on; with neither, version as given, or undef without one (unpinned, the stable channel, as without hold_running). A running service whose version cannot be read falls back to the installed binary, and without one to version, each with a warning. A version that is not shaped like a release ("parse_release") counts as unreadable: it goes onto an installer line.
A held version other than version wins; a warning names both. Every other outcome is logged as info.
check_version_skew
$dist->check_version_skew(version => $pinned, server_version => $cp);Before anything is installed: die when what would be installed breaks the version skew policy, with nothing on the host changed.
The version to install is version, or without one "channel_version". Against a running "service" (its "running_version"), a "version_skew" of jump or downgrade dies. With server_version (the control plane's version, for an agent), a version of a newer minor than it dies too.
When the "service" is not running (stopped, crashed, never started), the reference is the "installed_version" instead, the binary it would start on: jump or downgrade dies the same way, and minor without version warns that the service starts on the new minor (nothing holds a stopped service back once the installer replaced its binary). No binary there: a fresh install, nothing to check.
Nothing to compare against and no server_version: nothing asked beyond "main_pid" and "installed_version". A channel that cannot be resolved, or a running version that cannot be read, is a warning, not a die: for a running service "start_verb" checks the installed binary against the running one again before it restarts anything; a stopped one is not checked again.
check_agent_version
$dist->check_agent_version($agent_version, $control_plane_version);
$dist->check_agent_version($agent_version, $control_plane_version, 1);Die when the agent version is of a newer minor (or major) than the control plane's: a kubelet must never be newer than the API server. A newer patch of the same minor is fine. The message says nothing was installed, or with a true third argument that the binary is installed but the "service" was not (re)started. Two versions that do not parse only warn.
builtin_cluster_cidr
10.42.0.0/16: the cluster-cidr RKE2 and K3s run with when none is configured. Unlike "default_cluster_cidr" it says nothing about what is written.
is_established
True when a server of the distribution is set up on the host: its "server_service" is active, or "server_token" is there (a server that bootstrapped and is stopped starts on its configuration again). Reads the host.
established_cluster_cidr
my ( $cidr, $from ) = $dist->established_cluster_cidr;The cluster-cidr of the server set up on the host ("is_established") and the file it comes from (in scalar context the cluster-cidr alone); nothing when none is set up. Read over the exec channel (cat, no SFTP) from "config_file" and its config.yaml.d drop-ins, merged as RKE2 and K3s merge them: config.yaml first, then the drop-ins named *.yaml or *.yml (any case) sorted by name; the last value wins, and cluster-cidr+ appends to an earlier one. A list comes back comma-separated. Set in none of them: "builtin_cluster_cidr", and no file.
A file that is not there is skipped. One that cannot be read or parsed, is not a YAML mapping, or holds anything but a string or a list of strings as cluster-cidr dies with its name (never its content: config.yaml holds the join token), since the value is then unknown.
check_established_cluster_cidr
$dist->check_established_cluster_cidr(cluster_cidr => $cidr, cilium => $cilium);Before anything is written: die when a server is set up on the host and the "established_cluster_cidr" is not the cluster-cidr this run gives it -- cluster_cidr, else what "cilium_config" writes when cilium is true, else "builtin_cluster_cidr" -- compared as strings, trimmed. The message names both and says to pass the established one. Returns that, or nothing when no server is set up.
start_verb
$dist->start_verb(pinned => defined $version);
$dist->start_verb(pinned => 1, hold => 1, unit_digest => $before);start or restart for the "service". Reads the host.
First the installed "binary" against the running one ("version_skew"), for both distributions: jump or downgrade dies, and the service is not touched. minor without pinned is start, which leaves a running service on its old binary, with a warning that says so and how to restart it; nothing else below is asked. patch, or minor with pinned, restarts as any other change does.
Then restart when a running service's containerd config.toml is still the output of Rex::GPU 0.001's bare config.toml.tmpl and the template is gone ("remove_bare_containerd_template", which "install_server" in Rex::Rancher::Server and "install_agent" in Rex::Rancher::Agent run first) or replaced by another one, with a warning, so it renders that config anew; the bare template still in place only warns with what to do. Otherwise "default_start_verb", and where that is start (RKE2), restart when "restart_reasons" has any, logging them: a running service reads its configuration only when it starts.
With hold (hold_running, whose held version is pinned) it is start for K3s too, turned into restart the same way, and also when unit_digest is given ("installer_unit_digest" taken before the installer ran) and the "installer_unit_files" now have other content -- the check K3s' own install script makes before it restarts K3s. For RKE2 hold changes nothing.
installer_unit_digest
my $before = $dist->installer_unit_digest;sha256sum of the "installer_unit_files" on the host, errors included (a file that is not there yet), as K3s' install script takes it to decide whether it restarts K3s; undef without such files (RKE2), with nothing asked. Compared with a second one after the installer ran by "start_verb"'s unit_digest.
remove_bare_containerd_template
$dist->remove_bare_containerd_template;Before the "service" is (re)started: remove config.toml.tmpl in "containerd_dir" when it is Rex::GPU 0.001's bare template ("is_bare_template_output": only imports and version = 2), which the distribution would render instead of its own containerd config. Decided by content, never by path: any other template is somebody's configuration and stays, with a log line. No template: nothing. Reads with cat and removes with rm -f (no SFTP); dies when the removal fails. Restarts nothing itself: "start_verb" then restarts a running service whose config.toml is still that template's output. 1 when it removed the template, else 0.
restart_watch
The paths the "role"'s service reads only when it starts and that Rex::Rancher or Rex::GPU write: "config_file", config.yaml.d, "registries_file", "env_file" (if any), and in "containerd_dir" config.toml.tmpl, config-v3.toml.tmpl and the config-v3.toml.d drop-ins (where Rex::GPU puts 99-nvidia.toml). Paths that do not exist are fine.
Not the distribution's own output (the kubeconfig, config.toml), which it rewrites on every start.
restart_reasons
my @why = $dist->restart_reasons;Why the running "service" would have to be restarted to run what is on the host now, as log-ready strings; empty when it is not running or nothing changed. Reads the host:
a "restart_watch" path modified (mtime) after the service's main process started. Rex's
fileleaves a file with unchanged content untouched, so a re-run with the same options changes nothing; a change left behind by an interrupted earlier run, or made by hand, counts too. This covers the NVIDIA runtime's registration: the "env_file"PATHline and Rex::GPU'sconfig-v3.toml.ddrop-in. An upgradednvidia-container-runtimebinary alone is no reason: containerd runs it anew for every container it creates.the running binary (
/proc/PID/exe --version) reports another version than the installed "binary". Whether it may be restarted onto that version is "start_verb"'s decision, which asks first.
What it cannot determine (no ps, a binary that does not answer --version) counts as unchanged, with a warning that names it.
parse_main_pid
The PID in systemctl show -p MainPID output (MainPID=1234), or nothing for 0 (not running) and anything else.
is_bare_template_output
$dist->is_bare_template_output($config_toml)Pure: is this config.toml the verbatim output of Rex::GPU 0.001's template -- ignoring blank lines and comments, exactly an imports = and a version = 2 line and nothing else? The distributions' own configs always have [plugins...] sections.
wait_for_service
$dist->wait_for_service;
$dist->wait_for_service(hint => 'It joins the cluster via ...');Poll systemctl is-active for the "service" until it is active (attempts, default 60, every interval seconds, default 10). failed dies at once, any other state is polled until the timeout, which dies too. Either die message carries the reason, then hint if given, then the last 50 lines of the unit's journal.
runtime_path_line
The PATH= line "ensure_nvidia_runtime_path" writes: systemd's own default directories, not the SSH session's PATH, since it becomes the environment of a service running as root.
env_with_runtime_path
$dist->env_with_runtime_path($current)Pure: the env file with exactly one PATH line ("runtime_path_line", last), every other line kept in order. undef when the file already is exactly that.
ensure_nvidia_runtime_path
When the "role"'s unit has an "env_file" and nvidia-container-runtime is on the host's PATH, make that file carry "runtime_path_line", other lines kept. Restarts nothing itself: the file is in "restart_watch", so the "start_verb" of an RKE2 service running since before the write is restart.
env_files
my @files = $dist->env_files;The "env_file" of either "role", server first, leaving out a role without one: /etc/default/rke2-server and /etc/default/rke2-agent on RKE2, none on K3s. Pure; also a class method. For "uninstall_cmd" in Rex::Rancher::Uninstall, which does not know the role a host had.
write_secret_file
$dist->write_secret_file($path, $content);Write $content to $path on the host as 0600 root:root, with no moment in which it is readable by others. Dies if the mode cannot be set.
write_registries
$dist->write_registries($registries);Write the registry mirror hashref to "registries_file" through "write_secret_file".
SEE ALSO
Rex::Rancher, Rex::Rancher::Server, Rex::Rancher::Agent, Rex::Rancher::Cilium, Rex::Rancher::Distribution::RKE2, Rex::Rancher::Distribution::K3s, Rex::Rancher::Options, Rex::Rancher::Checksum
SUPPORT
Issues
Please report bugs and feature requests on GitHub at https://github.com/Getty/rex-rancher/issues.
CONTRIBUTING
Contributions are welcome! Please fork the repository and submit a pull request.
AUTHOR
Torsten Raudssus <getty@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2026 by Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.