NAME

Data::Tubes::Plugin::Source

DESCRIPTION

This module contains factory functions to generate tubes that ease handling data sources.

FUNCTIONS

iterate_array

my $tube = iterate_array(%args); # OR
my $tube = iterate_array(\%args);

iterate over an array, that can be provided to the factory as an array field, or provided as input record. In case both are provided, the array from the factory (the global array) is iterated for each call to the tube, before the array provided to the tube call itself.

The input record for the generated tube MUST be either false or a reference to an array. The returned tube ALWAYS returns an iterator (i.e. it returns the string iterator, followed by a sub reference that acts as an iterator).

The following arguments are supported:

array

the global array to iterate over. Defaults to an empty array reference;

name

name of the tube, useful when debugging;

open_file

my $tube = open_file(%args); # OR
my $tube = open_file(\%args);

open a file from the input record.

Depending on what is provided as parameter input in %args, the input channel will be either the record itself (if input is undefined or zero-length) or what input points inside the input record, that in this case MUST be a hash.

The input channel might be either a filehandle or a string; in either case, normalize_filename from Data::Tubes::Util is used to resolve a handle or a filename.

The tube returns a data structure that copies the input one (if input is defined and not empty) and sets an output field (set in the factory, defaults to source) to a hash reference with the following fields:

fh

a filehandle that can be used to read from the source;

input

what was passed as input;

type

the type of the particular source, might be handle, scalar (if the input is a reference to a scalar) or file;

name

a name that will provide a hint about what the channel is. If the input channel is a handle, this will be set to handle:%s (with %s expanded to a string); if it is a reference to a scalar (which is a valid filename for CORE::open) it sets it to scalar:%s and if it is a plain filename it sets it to file:%s.

The arguments accepted by the factory function are the following:

binmode

whenever a filehandle is opened, the generated filehandle is set to this via CORE::binmode. Defaults to :encoding(UTF-8);

input

if defined and not empty, the records input to the tube MUST be hash references with a field named as the provided input; otherwise, the whole input record is the channel to open;

name

name for the tube, useful for debugging;

output

name of the output field in the returned record, defaults to source.

iterate_files

my $tube = iterate_files(@channels_list); # OR
my $tube = iterate_files(@channels_list, \%args);

generate a tube that iterates over a list of files, either provided to the factory function directly, or provided as a reference to an array when invoking the tube. The two following cases are equivalent:

# pass the file list to the factory
my $tube1 = iterate_files('foo.txt', 'bar.txt');
my (undef, $iterator) = $tube1->();

# leave factory without list, pass to the tube
my $tube2 = iterate_files();
my (undef, $iterator)= $tube2->(['foo.txt', 'bar.txt']);

All elements in @channels_list MUST be valid channels for the tube generated by "open_file". Optionally, the last item in the argument list can be a reference to a hash, with further options for the factory.

The factory is built as a sequence (see Data::Tubes::Plugin::Plumbing) of "iterate_array", "open_file" and a logger (see Data::Tubes::Plugin::Plumbing for the latter).

The following arguments can be (optionally) supplied via %args:

iterate_array

optional hash with arguments for "iterate_array", where the array field is overridden with @channels_list;

logger

optional hash with arguments for logger in Data::Tubes::Plugin::Plubming. Defaults to setting a target pointing to a sub reference that gives out message reading from %s, where %s is expanded with the content of $record->{source}{name};

name

open_file

optional hash with arguments for the "open_file" sub-tube, defaults to the empty hash which means taking all defaults from "open_file".

BUGS AND LIMITATIONS

Report bugs either through RT or GitHub (patches welcome).

AUTHOR

Flavio Poletti <polettix@cpan.org>

COPYRIGHT AND LICENSE

Copyright (C) 2016 by Flavio Poletti <polettix@cpan.org>

This module is free software. You can redistribute it and/or modify it under the terms of the Artistic License 2.0.

This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.