return( $self->error( "Unable to decode $size bytes of data into the stream with $dec and input '", $self->_normalise( $src), "' and output '", $tempfile, "': $err") ) if( !defined( $rv) );
# otherwise, it is either a scalar reference, a glob or a file, and if it is none
# of those, we return an error
else
{
$data= $_[1];
return( $self->error( "Unsupported data type '", overload::StrVal( $data), "'. You can only provide a string, a scalar reference, a code reference, a glob or a file path.") ) if( ref( $data) && $typene 'scalar'&& $typene 'glob'&& $typene 'code');
}
# If we are dealing with a file, open it and use its file glob instead,
# because some encoder like IO::Compress::Zip actually creates and archive of the file with the file path included, rather than just the file content as advertised.
HTTP::Promise::Stream - Data Stream Encoding and Decoding
=head1 SYNOPSIS
use HTTP::Promise::Stream;
my $this = HTTP::Promise::Stream->new ||
die( HTTP::Promise::Stream->error, "\n" );
=head1 VERSION
v0.1.0
=head1 DESCRIPTION
L<HTTP::Promise::Stream> serves to set a stream of data tha that optionally may need to be encoding or decoding, and read or write data from or to it that may also need to be compressed or decompressed.
Once those versatile parameters are set, one can use the class method to access or write the data and the necessary encoding or decoding is done automatically.
=head1 CONSTRUCTOR
=head2 new
Provided with a stream source, and some optional parameters and this will return a new L<HTTP::Promise::Stream> object.
Currently supported stream sources are: scalar reference, glob and file path.
If an error occurred, this sets an L<error|Module::Generic/error> and returns C<undef>
Supported parameters are:
=over 4
=item * C<decoding>
A string representing the encoding to use for decoding data. Currently supported encodings are: gzip, bzip2, deflate/inflate and zip
=item * C<encoding>
A string representing the encoding to use for encoding data. Currently supported encodings are: gzip, bzip2, deflate/inflate and zip
=back
=head1 METHODS
=head2 as_string
Returns the source stream as a string, or C<undef> and an L<error|Module::Generic/error> occurred.
=head2 compress_params
Sets or gets an hash of parameters-value pairs to be used for the compression algorithm used.
=head2 decodable
Provided with a C<target> and this returns an L<array object|Module::Generic::Array> of decoders that are installed.
The C<target> can be a string or an array reference of decoder names. If the target string C<all> is specified, then, this will check all supported encodings. See L</supported>. If the target string C<browser> is specified, then ths will check only the supported encodings that are also supported by web browsers. If no target is specified, it defaults to C<all>.
If the target is an array reference, it will return the list of supported decoders in the order provided.
my $all = $stream->decodable;
# Same as above
my $all = $stream->decodable( 'all' );
my $all = $stream->decodable( 'browser' );
my $all = $stream->decodable( [qw( gzip br deflate )] );
# $all could contain gzip and br for example
Note that for most encoding, encoding and decoding is done by different classes.
=head2 decode
$stream->decode( $data );
$stream->decode( $data, { encoding => bzip2 } );
$stream->decode( $data, { decoding => bzip2 } );
my $decoded = $stream->decode;
my $decoded = $stream->decode( { encoding => bzip2 } );
my $decoded = $stream->decode( { decoding => bzip2 } );
This behaves in two different ways depending on the parameters provided:
=over 4
=item 1. with C<data> provided
This will decode the C<data> provided using the encoding specified and write the decoded data to the source stream.
=item 2. without C<data> provided
This will decode the source stream directly and return the data thus decoded.
=back
This method will take the required encoding in the following order: from the C<decoding> parameter, from the C<encoding> parameter, or from L</decoding> as set during object instantiation.
If the encoding specified is not supported this will return an error.
It returns true upon success, or sets an L<error|Module::Generic/error> and returns C<undef>
=head2 decoding
This is a string. Sets or gets the encoding used for decoding. Supported encodings are: gzip, bzip2, inflate/deflate and zip
=head2 encodable
Provided with a C<target> and this returns an L<array object|Module::Generic::Array> of encoders that are installed.
The C<target> can be a string or an array reference of decoder names. If the target string C<all> is specified, then, this will check all supported encodings. See L</supported>. If the target string C<browser> is specified, then ths will check only the supported encodings that are also supported by web browsers. If no target is specified, it defaults to C<all>.
If the target is an array reference, it will return the list of supported encoders in the order provided.
my $all = $stream->encodable;
# Same as above
my $all = $stream->encodable( 'all' );
my $all = $stream->encodable( 'browser' );
my $all = $stream->encodable( [qw( gzip br deflate )] );
# $all could contain gzip and br for example
Note that for most encoding, encoding and decoding is done by different classes.
=head2 encode
$stream->encode( $data );
$stream->encode( $data, { encoding => bzip2 } );
$stream->encode( $data, { decoding => bzip2 } );
my $encoded = $stream->encode;
my $encoded = $stream->encode( { encoding => bzip2 } );
my $encoded = $stream->encode( { decoding => bzip2 } );
This is the alter ego of L</decode>
This behaves in two different ways depending on the parameters provided:
=over 4
=item 1. with C<data> provided
This will encode the C<data> provided using the encoding specified and write the encoded data to the source stream.
=item 2. without C<data> provided
This will encode the source stream directly and return the data thus encoded.
=back
This method will take the required encoding in the following order: from the C<encoding> parameter, or from L</encoding> as set during object instantiation.
If the encoding specified is not supported this will return an error.
It returns true upon success, or sets an L<error|Module::Generic/error> and returns C<undef>
=head2 encoding
This is a string. Sets or gets the encoding used for encoding. Supported encodings are: gzip, bzip2, inflate/deflate and zip
=head2 encoding2suffix
Provided with a string of comma-separated encodings, or an array reference of encodings and this will return an L<array object|Module::Generic::Array> of associated file extensions.
For example:
my $a = HTTP::Promise::Stream->encoding2suffix( [qw( base64 gzip )] );
# $a contains: b64 and gz
my $a = HTTP::Promise::Stream->encoding2suffix( 'gzip' );
# $a contains: gz
=head2 load
This attempts the load the specified encoding related class and returns true upon success or false otherwise.
It sets an L<error|Module::Generic/error> and returns C<undef> upon error.
For example:
if( HTTP::Promise::Stream->load( 'bzip2' ) )
{
my $s = HTTP::Promise::Stream->new( \$data, encoding => 'bzip2' );
say "Encoder/decoder bzip2 related modules are not installed on this system.";
}
See also L</supported>, which will tell you if L<HTTP::Promise::Stream> even supports the specified encoding.
=head2 read
$stream->read( $buffer );
$stream->read( $buffer, $len );
$stream->read( $buffer, $len, $offset );
$stream->read( *buffer );
$stream->read( *buffer, $len );
$stream->read( sub{} );
$stream->read( sub{}, $len );
$stream->read( \$buffer );
$stream->read( \$buffer, $len );
$stream->read( \$buffer, $len, $offset );
$stream->read( '/some/where/file.txt' );
$stream->read( '/some/where/file.txt', $len );
Provided with some parameters, as detailed below, and this will either encode or decode the stream if any encoding was provided at all and into the read buffer specified.
Possible read buffers are:
=over 4
=item * scalar
=item * scalar reference
=item * file handle (glob)
=item * subroutine reference or anonymous subroutine
=item * file path
=back
It takes as optional parameters the C<length> of data, possibly encoded or decoded if any encoding was provided, and an optional C<offset>. However, note that the C<offset> argument is not used and ignored if the data buffer is not a string or a scalar reference.
Also you can specify an hash reference of options as the last parameter. Recognised options are:
=over 4
=item * autoflush
Boolean value. If true, this will set the auto flush.
=item * binmode
The encoding to be used when opening the file specified, if one is specified. See L</binmode>
=item * mode
The mode in which to open the file specified, if one is specified.
Possible modes can be >, +>, >>, +<, w, w+, r+, a, a+, < and r or an integer representing a bitwise value such as O_APPEND, O_ASYNC, O_CREAT, O_DEFER, O_EXCL, O_NDELAY, O_NONBLOCK, O_SYNC, O_TRUNC, O_RDONLY, O_WRONLY, O_RDWR. For example: C<O_WRONLY|O_APPEND> For that see L<Fcntl>
=item * other parameters starting with an uppercase letter
Those parameters will be passed directly to the encoder/decoder.
my $s = HTTP::Promise::Stream->new( \$data, decoding => 'inflate' );
# Transparent and its value are passed directly to IO::Uncompress::Inflate
$s->read( \$output, { Transparent => 0 } );
=back
A typical recommended parameter used for the C<IO::Compress> and C<IO::Uncompress> families is C<Transparent> set to C<0>, otherwise, the default is C<1> and it would be lenient and any encoding/decoding issues with the data would be ignored.
For example, when using C<inflate> to uncompress data compressed with C<deflate>, some encoder do not format the data correctly, or declare it as C<deflate> when they really meant C<rawdeflate>, i.e. without the zlib headers and trailers. By default with C<Transparent> set to C<1>, L<IO::Uncompress::Inflate> will simply pass through the data. However, you are better of catching the error and resort to using C<rawinflate> instead.
For example:
use v5.17;
use HTTP::Promise::Stream;
my $data = '80jNyclXCM8vyklRBAA=';
my $buff = '';
my $s = HTTP::Promise::Stream->new( \$data, decoding => 'base64' ) ||
die( HTTP::Promise::Stream->error );
my $len = $s->read( \$buff );
die( $s->error ) if( !defined( $len ) );
say "Now inflating data.";
$data = $buff;
$buff = '';
my $s = HTTP::Promise::Stream->new( \$data, decoding => 'deflate' ) ||
say "Found deflate encoding error (", $s->error->message, "), trying with rawinflate instead.";
my $s = HTTP::Promise::Stream->new( \$data, decoding => 'rawdeflate' ) ||
die( HTTP::Promise::Stream->error );
$len = $s->read( \$buff, { Transparent => 0 } );
die( $s->error ) if( !defined( $len ) );
}
else
{
die( $s->error );
}
}
say $buff; # Hello world
=head2 source
Set or get the source stream.
=head2 suffix2encoding
Provided with a filename, and this will return an L<array object|Module::Generic::Array> containing the encoding naes associated with the extensions found.
For example:
my $a = HTTP::Promise::Stream->suffix2encoding( 'file.html.gz' );
# $a contains: gzip
my $a = HTTP::Promise::Stream->suffix2encoding( 'file.html' );
# $a contains nothing
=head2 supported
Provided with an encoding name and this returns true if it is supported, or false otherwise.
Currently supported encodings are:
=over 4
=item Base64
Supported natively. See L<HTTP::Promise::Stream::Base64>
=item Brotli
Requires L<IO::Compress::Brotli> for encoding and L<IO::Uncompress::Brotli> for decoding.
Note that some web server announce data encoded with C<deflate> whereas they really mean C<rawdeflate>, so you might want to use the C<Transparent> parameter set to C<0> when using L</read>
=item Gzip
Requires L<IO::Compress::Gzip> for encoding and L<IO::Uncompress::Gunzip> for decoding.
Requires L<IO::Compress::Lzop> for encoding and L<IO::Uncompress::UnLzop> for decoding.
"lzop is a file compressor which is very similar to L<gzip|http://www.gzip.org/>. lzop uses the L<LZO data compression library|http://www.oberhumer.com/opensource/lzo/> for compression services, and its main advantages over gzip are much higher compression and decompression speed (at the cost of some compression ratio)."
Requires L<Compress::LZW> for encoding and for decoding.
A.k.a C<compress>, this was used commonly until some corporation purchased the patent and started asking everyone for royalties. The patent expired in 2003. Gzip took over since then.
=item QuptedPrint
Requires the XS module L<MIME::QuotedPrint> for encoding and decoding.
Supported natively. See L<HTTP::Promise::Stream::UU>
=item Xz
Requires L<IO::Compress::Xz> for encoding and L<IO::Uncompress::UnXz> for decoding.
Reportedly, "xz achieves higher compression rates than alternatives like gzip and bzip2. Decompression speed is higher than bzip2, but lower than gzip. Compression can be much slower than gzip, and is slower than bzip2 for high levels of compression, and is most useful when a compressed file will be used many times."
See also L</load>, which will tell you if the specified encoding related modules are installed on the system or not.
=head2 write
$stream->write( $data );
$stream->write( \$data );
$stream->write( *$data );
$stream->write( '/some/where/file.txt' );
$stream->write( sub{} );
Provided with some data, and this will read the data provided, and write it, possibly encoded or decoded, depending on whether a decoding or encoding was provided, to the stream source.
It returns the amount of bytes written to the source stream, but before any possible encoding or decoding.
The data that can be provided are:
=over 4
=item * string
Note that the difference between a file and a string is slim. To distinguish the two, this method will treat as a string any value that is not a reference and that either contains a CRLF sequence, or that does not contain a CRLF sequence and is not an existing file.
=item * scalar reference
=item * file handle (glob)
=item * file path
Note that the difference between a file and a string is slim. So to distinguish the two, this method will treat as a file a value that has no CRLR sequence
=item * code reference (anonymous subroutine or subroutine reference)
It will be called once and expect data in return. If the code executed dies, the exception will be trapped using try-catch block from L<Nice::Try>
=back
The behaviour is different depending on the source type and the data type being provided. Below is an in-depth explanation:
=over 4
=item 1. Source stream is a code reference
=over 8
=item 1.1 Data is to be encoded
Data is encoded with L</encode>, then the source code reference is executed, passing it the encoded data
=item 1.2 Data is to be decoded
Data is decoded with L</decode>, then the source code reference is executed, passing it the decoded data
=item 1.3 Data is scalar reference
The source code reference is executed, passing it the content of the scalar reference
=item 1.4 Data is a glob
The file handle is read by chunks of 10Kb (10240 bytes) and each time the source code reference is called passing it the data chunk read.
=item 1.5 Data is a file path
The file is opened in read mode, and all its content is provided in one pass to the source code reference.
=back
=item 2. Data is the be encoded
The appropriate encoder is called to encode the data and write to the source stream.
=item 3. Data is to be decoded
The appropriate decoder is called to decode the data and write to the source stream.
=item 4. Source stream is a scalar reference
=over 8
=item 4.1 Data is a scalar reference
The provided data is simply appended to the source stream.
=item 4.2 Data is a glob
The file handle is read by chunks of 10Kb (10240 bytes) and appended to the source stream.
=item 4.3 Data is a file path
The file is opened in read mode and its content appended to the source stream.
=back
=item 5. Source stream is a glob
=over 8
=item 5.1 Data is a scalar reference
The file handle of the source stream is called with L</print> and the data is printed to it.
=item 5.2 Data is a glob
The data file handle is read by chunks of 10Kb (10240 bytes) and each one printed to the source stream file handle.
=item 5.3 Data is a file path
The given file path is read in read mode and each chunk of 10Kb (10240 bytes) read is printed to the source stream file handle.
=back
=item 6. Source stream is a file path
The source file is opened in write clobbering mode.
=over 8
=item 6.1 Data is a scalar reference
The data is printed to the source stream
=item 6.2 Data is a glob
Data from the glob is read by chunks of 10Kb (10240 bytes) and each one printed to the source stream
=item 6.3 Data is a file path.
The file is opened in read mode and its content is read by chunks o 10Kb (10240 bytes) and each chunk printed to the source stream.