NAME
App::FuguVM::Mirror - the OpenBSD mirror of one guest
SYNOPSIS
use App::FuguVM::Mirror;
my $mirror = App::FuguVM::Mirror->new(
cache => App::FuguVM::Proxy::Cache->new($cache_dir),
version => '7.8',
arch => 'arm64',
);
my $path = $mirror->ensure('source', 'ports.tar.gz')
or die $mirror->error, "\n";DESCRIPTION
The module is the one home of the mirror facts: the host, the URLs of one version and one architecture, the downloader, the release key, and the verification. Fugu::Curl downloads each file, Fugu::Signify proves a signed SHA256 manifest, and this module decides which manifest signs which file. The proof runs under the signify engine of that module. A release of OpenBSD signs its SHA256 in the embedded form, and the perl engine of Fugu::Signify does not read that form, so the command makes the proof.
A file must verify before it enters the cache, because the cache is what a later run reads. A verification failure therefore leaves no file in the cache.
THE TWO SCOPES
Each method that takes a $scope takes release or source. The release scope is the architecture directory, /pub/OpenBSD/<version>/<arch>/: the file sets, the kernels and the miniroot. The source scope is the version directory, /pub/OpenBSD/<version>/: ports.tar.gz, src.tar.gz, sys.tar.gz and xenocara.tar.gz. Each scope holds one SHA256 manifest with one SHA256.sig signature, and the base key of the release signs both manifests.
THE RELEASE KEY
The version names the key file: 7.8 gives openbsd-78-base.pub. key_path resolves the file in this order:
- 1. The
keys_dirargument, which carries thesignify_dirdirective. - 2. /etc/signify/. An OpenBSD host holds the authentic key from its own installation, which is a better anchor than a copy in a repository.
The share tree holds one key file for each release that the tool supports. Today that is the default release, 7.8. A guest of an older release needs /etc/signify/ or the signify_dir directive. A key in a repository is a trust anchor, so a human adds each file, and the commit that adds one states the comparison against /etc/signify/ on a real OpenBSD host of that release.
The verifier takes one key. The release directory of a numbered release carries one signature, under the base key of that release. A second key would accept a file that the version does not own. A snapshot release carries the key of the next release, so this module does not serve -current.
METHODS
new
App::FuguVM::Mirror->new(%args)cache, version and arch are necessary, and the method dies without one of them. keys_dir names the directory of the public keys; the default is the resolution order above. verify is 1 or 0, and the default is 1. log is the Fugu::Log object that the download line of fetch takes, and the default is Fugu::Log->default. A caller that holds its own logger passes it, so the --quiet mode of the tool reaches that line.
version, url, source_url
version returns the OpenBSD version of the mirror, so a consumer holds no copy of it. url($file) returns the URL of a file of the release scope, and source_url($file) returns the URL of a file of the source scope. Both scopes live on cdn.openbsd.org, and this module is the one home of that host. The distfile tree is not a scope of this module: the guest fetches a distfile itself, from a mirror host that carries the tree, and cdn.openbsd.org does not.
key_path
Return the resolved public key file, or undef. On a miss, error names the key file and each directory that the method tried.
fetch
my $tmp = $mirror->fetch($url);Download the URL to a temporary file, through Fugu::Curl. The method returns the File::Temp object, or undef with the reason in error. This method is the one home of the download call.
The mirror builds one downloader in new, and that object resolves its command one time. A host without curl, wget and ftp therefore fails here with the reason of the downloader. The bound of one fetch is 3600 seconds, because a set file of OpenBSD is hundreds of megabytes.
Each temporary file lands in the private download directory of the mirror, at mode 0700. Fugu::Curl writes a sibling file next to its destination, with a name that another process can guess, and it opens that file without O_EXCL. A download in the shared temporary directory therefore takes a file that another user made first.
The first fetch makes that directory, and the directory then lives as long as the mirror. new makes no directory: it dies for a programming error alone, and a url call needs no directory. A host whose temporary directory is absent or unwritable therefore fails here, with the reason in error.
A caller must read $tmp->filename, and must never read the object as a filehandle. Fugu::Curl renames its own file onto that path, so the filehandle of the object holds the old inode.
The mirror logs the URL before each fetch that reaches the network, because the downloader writes no progress. A download of a hundred megabytes is a wait that an operator wants to see. The line goes to the logger of the mirror, so --quiet drops it.
manifest
my $path = $mirror->manifest($scope);Make sure that the SHA256 and the SHA256.sig of the scope are cached, and prove the signature under the release key. The method returns the cached manifest path, or undef. The module keeps the proven path for the length of the object. With verify set to 0 the method returns undef with a reason, because a method must not report a proof that it did not make.
The manifest pair enters the cache before the proof. That store is safe: the signature proves the manifest on every read, and a tampered manifest fails that proof.
manifest_names
my $names = $mirror->manifest_names($scope);Return the file names that the manifest of the scope holds, as a sorted array reference, or undef. With verification on, the names come from the proven manifest. With verification off, the manifest is a routing table only, and the method reads it without a proof.
verify_file
$mirror->verify_file($scope, $name, $path);Verify one local file against the proven manifest of the scope, under its manifest name. The method returns 1, or undef. The manifest name can differ from the local path, so a temporary download verifies under the name that the mirror serves.
The manifest proof runs the signature check, through Fugu::Signify. The digest check runs in this module, with core Digest::SHA, because a real OpenBSD release manifest repeats the install image lines: Fugu::Signify refuses a manifest with a duplicate name, and a repeated identical fact changes nothing. The parser here accepts a duplicate line whose digest is identical, and it refuses a duplicate name with two digests, like a line that does not parse.
ensure
my $path = $mirror->ensure($scope, $file);Return the cached path of a file. On a cache miss the method works in this order: prove the manifest of the scope, fetch the file, verify the temporary file, and store it. The order matters: the file verifies before it enters the cache. The manifest proof runs before the fetch, so a bad key and a bad signature fail before any download.
With verify set to 0 the method skips the proof and the file check, and it logs one warning. That warning goes to Fugu::Log->default, and never to the logger of the mirror, because --quiet must not drop it.
verify_cache
my $report = $mirror->verify_cache;Verify every cached file of the two scopes. The method returns { ok, failed, unknown }, each one a list of { scope, name } entries sorted by name, or undef. A cached file that the proven manifest of its scope names is ok or failed. A cached file that no proven manifest names is unknown: index.txt, a package, and every file of a scope whose manifest is absent or unproven, are all in that class. The method never skips a cached file. The SHA256 and SHA256.sig pairs stay out of the three lists while their proof holds, because that proof covers them.
A cached manifest pair that fails its own proof is a failure: the method removes the pair and reports its SHA256 under failed, so a poisoned pair cannot pin every later run. A pair that the method cannot fetch leaves the files of its scope unknown.
The manifest proof makes one signify(1) run for each scope, and each digest check runs in this module, so a tree of a hundred files costs one signify(1) run for each scope.
cached_path
my $path = $mirror->cached_path($scope, $file);Return the cache path of a file of the scope, cached or not. The mirror verify verb of the tool removes a failed file through this answer.
error
Return the reason of the most recent failure, or undef. Every public method clears the reason before it starts. When signify(1) is absent, the reason names the command and the packages: signify-openbsd on Linux and signify-osx on Darwin.
SEE ALSO
Fugu::Curl, Fugu::Signify, App::FuguVM::Proxy, App::FuguVM::Miniroot, fuguvm(1)
AUTHOR
Dick Olsson <hi@senzilla.io>