NAME
Data::Resolver - resolve keys to data
VERSION
This document describes Data::Resolver version 0.006.
SYNOPSIS
# generate is a single entry point useful to instantiate from
# metadata/configurations
use Data::Resolver 'generate';
my $spec = { -factory => 'resolver_from_tar',
path => '/to/archive.tar' };
my $dr_tar = generate($spec);
my $dr_dir = generate({ -factory => 'resolver_from_dir',
path => '.' });
# functions can be imported and used directly though
use Data::Resolver qw< resolver_from_tar resolver_from_dir >;
my $dr_tar = resolver_from_tar(path => 'to/somewhere.tar');
# getting stuff is easy, this is how to get the default
# representation
my ($thing, $meta) = $dr_something->($key);
my $type = $meta->{type};
if ($type eq 'file') { say 'got a file' }
elsif ($type eq 'filehandle') { say 'got a filehandle' }
elsif ($type eq 'data') ) { say 'got data' }
elsif ($type eq 'error') { ... }
# you can specify the type and just get the value
my $data = $dr_something->($key, 'data');
my $path = $dr_something->($key, 'file');
my $fh = $dr_something->($key, 'filehandle');
# it's possible to get a list of available keys
my $list_aref = $dr_something->(undef, 'list');
DESCRIPTION
While coding, two problems often arise:
* Using several modules, there can be a variety of ways on how they
get access to data. Many times they support reading from a file, but
often times they expect to receive data (e.g. JSON::PP). Other times
modules an be OK with both, and even accept filehandles.
* Deciding on where to store data and what to use as a source can be
limiting, especially when multiple things might be needed. What is
best at that point? A directory? An archive? A few URLs?
This module aims at providing a way forward to cope for both problems,
by providing a unified interface that can get three types of data types
(i.e. data, file, or filehandle) while at the same time providing a
very basic interface that can be backed by several different fetching
approaches, like reading from a directory, taking items from an
archive, or download stuff on the fly from a URL.
The Resolver Interface
A valid resolver is a function that supports at least the following
interface:
* The resolver has the following signature:
my $resolver = sub ($key, [$type]) { ... }
where $key is a mandatory parameter providing the key that we want to
resolve to a data representation, and $type is an optional parameter
that specifies what representation is needed.
* When called in list context, two items are provided back, i.e. the
value and the metadata:
my ($value, $meta) = $resolver->(@args);
The $meta is a hash reference that contains at least one key type,
indicating the type of the $value. Allowed types are at least the
following:
data
The $value is directly data. It might be provided either as a plain
scalar, or as a reference to a plain scalar (ref() will help
disambiguate the two cases).
error
The $value should be ignored because an error occurred during
retrieval. When the resolver is set for throwing an exception, this
is never returned.
file
The $value represents a file path in the filesystem.
filehandle
The $value is a filehandle, suitable for reading. Characteristics
of thi filehandle may vary, although it SHOULD support seeking.
* When called in scalar context, only the $value is provided back:
my $value = $resolver->(@args);
In this case it is usually better to also provide a type as the
second argument, unless the default return type for the resolver is
already known in advance.
* The following invocation provides back a list of all supported
keys:
my $list = $resolver->(undef, 'list');
Examples:
# get list of supported keys, as an array ref
my $list = $resolver->(undef, 'list');
# get value associated to key 'foo', as raw data
my $data = $resolver->(foo => 'data');
# get value and metadata, decide later how to use them
my ($value, $meta) = $resolver->('foo');
if ($meta->{type} eq 'file') { ... }
Stock Factories
The module comes with a few stock factory functions to generate
resolvers in a few cases:
* A directory in the filesystem, via "resolver_from_dir".
* A TAR archive, via "resolver_from_tar".
* A list of resolvers, via "resolver_from_alternatives". This allows
e.g. looking for a resolution of the key from multiple sources
(possibly of different kinds).
INTERFACE
The interface provided by this module can be broadly divided into three
areas:
* factories to generate resolvers;
* transformers to ease turning a data representation into another
(e.g. turning data into a file or a filehandle)
* utilities for coding new resolvers/resolver factories.
Factories
These functions generate resolvers.
generate
my $resolver = generate($specification_hash);
Generate a new resolver based on a hash containing a specification. The
following meta-keys are supported:
-factory
The name of the factory function (e.g. "resolver_from_tar").
-package
The package where the factory function above is located. By default
this is Data::Resolver.
--recursed-args
A sub-hash where values are array references, holding
sub-specifications that are generated recursively via generate
itself. The key and the resulting array are then inserted as new
keys/value pairs in the hash.
All the rest of the hash is passed to the factory function as key/value
pairs.
Example:
my $spec = { -factory => 'resolver_from_tar',
path => '/to/archive.tar' };
my $dr_tar = generate($spec);
my $spec = {
-factory => 'resolver_from_alternatives',
-recursed => {
alternatives => [
{ -factory => resolver_from_tar => archive => $tar },
{ -factory => resolver_from_dir => root => $dir },
],
}
};
my $dr_multi = generate($spec);
resolver_from_alternatives
my $dr = resolver_from_alternatives(%args);
Generate a resolver that wraps other resolvers and tries them in
sequence, until the first supporing the input key.
It cares about two keys in %args:
alternatives
Accepts a reference to an array of sub-resolvers (i.e. CODE
references) or sub-resolver specifications (which will be
instantiated via "generate").
throw
If set to a true value, raise an exception in case of errors.
The list type is supported for the undef key only. It replicates the
call over all alternatives, aggregating the result in the order they
appear while filtering out duplicates (it does not try to normalize
keys in any way, so this might give practical duplicates out).
The search for a key is performed in the same order as the
sub-resolvers appear in alternatives; when a result is found, it is
returned. Exceptions from sub-resolvers are trapped.
If throw is set, errors will raise exceptions thrown as hashes, via
"resolved_error"/"resolved". This happens in two cases:
* the call for type list does not provide an undef key. In this case,
the error code is set to 400.
* the call for any other type does not provide a result back. In this
case, the error code is set to 404.
resolver_from_dir
my $dr = resolver_from_dir(%args);
Generate a resolver that serves files from a directory in the local
filesystem. It supports the following keys in %args:
path
root
The path to the directory containing the files. root takes precedence
over path.
throw
If set to a true value, raise an exception in case of errors.
Errors are handled via "resolved_error"/"resolved", including raising
exceptions.
The call to type list with an undef key will generate a lit of all
files in the subtree starting from path/root. As an extension, it's
also possible to pass the name of a sub-directory and get only that
subtree back; this is prone to errors if the sub-directory does not
exist (error code 404) or is a file instead of a directory (error code
400).
The call for other types will resolve the key and provide back the
requested type, defaulting to type file for effort minimization. The
code tries to restrict looking for the file only inside the sub-tree
but you should check by yourself if this is really critical (patches
welcome!).
If the key cannot be found, error code 404 is set; if the key refers to
a directory, error code 400 is set; if the type cannot be handled via
"transform", error code 400 is set.
resolver_from_tar
my $dr = resolver_from_tar(%args);
Generate a resolver that serves file from a TAR file in the local
filesystem. It supports the following keys in %args:
archive
path
The path to the TAR file. archive takes precedence over path.
throw
If set to a true value, raise an exception in case of errors.
Errors are handled via "resolved_error"/"resolved", including raising
exceptions.
The call for type list only supports key undef and will lead to error
code 400 otherwise.
The call for any other type will look for the key and return the
requested type, defaulting to type data. Two keys are actually
searched, to cater for the equivalence of path/to/file and
./path/to/file; this means that it's possible to ask for somefile and
get back the contents of ./somefile, or vice-versa.
If a file for a key cannot be found, error code 404 is returned. This
also applies if the key is present, but represents a directory item.
Type transformation is performed via "transform"; unsupported types
will lead to error code 400 after the search and extraction of data
from the archive (i.e. there is no attempt to pre-validate the type and
this is by design).
resolver_from_passthrough
my $dr = resolver_from_passthrough(%args);
Generate a minimal, fake-like resolver that always returns what is
provided. Arguments %args are added to the return value via "resolved",
so key throw might lead to exceptions.
In the generated resolver, the type defaults to undef. No attempt at
validation is done, by design.
Transformers
data_to_fh
my $data = 'Some thing';
my $fh1 = data_to_fh($data); # first way
my $fh2 = data_to_fh(\$data); # second way
Gets a scalar or a scalar reference and provides a filehandle back,
suitable for reading/seeking.
data_to_file
my $data = 'Some thing';
my $path1 = data_to_file($data); # plain scalar in
my $path2 = data_to_file(\$data); # scalar reference in
my $persistent_path = data_to_file($data, 1);
Gets a scalar or a scalar reference and saves it to a temporary file in
the filesystem. A second parameter, when true, makes it possible to
persist the file after the process exists (the file would be removed
otherwise).
fh_to_data
open my $fh, '<:raw', $some_path or die '...';
my $data = fh_to_data($fh);
Slurps data from a filehandle. The filehandle is not changed otherwise,
so it's up to the caller to set the right Perl IO layers if needed.
fh_to_file
open my $fh, '<:raw', $some_path or die '...';
my $path = fh_to_file($fh);
my $persistent_path = fh_to_file($fh, 1);
Slurps data from a filehandle and saves it into a temporary file (or
persistent, if the second parameter is present and true).
file_to_data
my $data = file_to_data($path);
Slurps data from a file in the filesystem. Data are read in raw mode.
file_to_fh
my $fh = file_to_fh($path);
Opens a file in the filesystem in read raw mode.
transform
my $ref_to_that = transform($this, $this_type, $that_type);
Treat input $this as having $this_type and return it as a reference to
a $that_type.
Input and output types can be:
fh
filehandle
the input is a file handle
data
the input is raw data
file
path
the input is a path in the filesystem.
NOTE the return value is a reference to the target data form, to avoid
transferring too much data around.
Utilities For New Resolvers
This module comes with two non-trivial resolvers, one for wrapping a
directory and another one for tar archives. There can be other possible
resolvers, e.g. using different archive formats (like ZIP), leveraging
any file format that supports carrying metadata (like PDF, or many
image formats), or wrapping remote resources (plain HTTP or some fancy
API).
These functions help complying with the output API of a resolver, i.e.:
* throw an exception when errors occur and the resolver was created
with throw parameter set;
* return just the content in scalar context;
* return the content and additional metadata in list context.
A typical way of using these function is like this:
sub resolver_for_whatever (%args) {
my $OK = resolved_factory($args{throw});
my $KO = resolved_error_factory($args{throw});
return sub ($key, $type = 'xxx') {
return $KO->(400 => 'Wrong inputs!') if $some_error;
return $OK->($data, type => 'data');
};
}
resolved
return resolved($throw, $value, $meta_as_href);
return resolved($throw, $value, %meta);
Throw an exception if $throw is true and metadata have type set to
error.
Otherwise, return $value if called in scalar context.
Otherwise, return a list with $value and a hash reference with the
metadata.
resolved_error
return resolved_error($throw, $code, $message, $meta);
return resolved_error($throw, $code, $message, %meta);
If an error has to be returned, this is a shorthand to integrate the
optional metadata with a code and a message. If $throw is set, an
exception is thrown.
resolved_error_factory
my $error_return = resolved_error_factory($throw);
Wrap "resolved_error" with the specific value for $throw. This can be
useful because whether a resolver should throw exceptions or not is
usually set at resolver creation time, so it makes sense to wrap this
characteristic.
resolved_factory
my $return = resolved_factory($throw);
Wrap "resolved" with the specific value for $throw. This can be useful
because whether a resolver should throw exceptions or not is usually
set at resolver creation time, so it makes sense to wrap this
characteristic.
BUGS AND LIMITATIONS
Minimum perl version 5.24 because reasons (it's been around since 2016
and signatures just make sense).
Report bugs through Codeberg (patches welcome) at
AUTHOR
Flavio Poletti <flavio@polettix.it>
COPYRIGHT AND LICENSE
Copyright 2023 by Flavio Poletti <flavio@polettix.it>
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License.