NAME
HTML::Encapsulate - rewrites an HTML page as a self-contained set of
files
VERSION
This document describes HTML::Encapsulate version 0.1
SYNOPSIS
use HTML::Encapsulate qw(download);
# This will download the page at the URL given in the first
# argument into a folder named in the second, here called
# C<bar.html>. The folder will contain all the images and other
# components required to view the page. The page itself will be
# saved as C<index.html>
# It also has an OO interface, allows various defaults to be
# adjusted via the %options hash.
my $he = HTML::Encapsulate->new(%options);
# HTTP::Requests can also be passed. This enables the result of
# form posts to be captured.
my $download_dir = 'some/directory/path';
$he->download($request, $download_dir);
DESCRIPTION
The main motivation for this module is for archiving and printing web
pages: these typically come in various separate pieces and aren't simple
to download as one chunk.
However, it is possible to preserve the content of a web page, but to
rewrite the links to the embedded contend like images, stylesheets, etc.
so that the downloaded version can be viewed entirely offline.
Once web pages have been downloaded in an "encapsulated" form, they can
then be archived, and/or converted into other formats.
The "wget" command line utility has an option for downloading web pages
with their images and stylesheets, rewriting the links to point to the
downloaded copies, like this:
This command isn't always convenient, nor available, so it's a fairly
non-portable option. This module aims to perform the same function in a
portable, pure-perl fashion.
See the documentation for the "->download" method for more details.
EXPORTABLE FUNCTIONS
"download($url_or_request, $download_dir)"
"download($url_or_request, $download_dir, $user_agent)"
Essentially constructs a default instance and delegates to its
"->download" method. See the appropriate documentation for that method.
Note that, once created, this instance will be re-used by future calls
to "download".
Optionally, a LWP::UserAgent instance $user_agent may be supplied for
use, e.g. if the download needs to be performed as part of an ongoing
session, or needs to have specific properties or behaviour.
If no $user_agent is supplied, a new LWP::UserAgent instance will be
created by the default instance used. See the "->new" method for
details.
CLASS METHODS
"$obj = $class->new(%options)"
Constructs a new instance with tweaked properties.
Only one option is currently available:
"ua"
Supplies a "LWP::UserAgent" instance to use instead of the default.
If not supplied, a default new instance will be constructed like
this:
$ua = LWP::UserAgent->new(
requests_redirectable => [qw(GET POST HEAD)]
);
This means that redirects will be followed for "GET", "HEAD", and
(unlike a default instance), "POST".
One reason for using an externally supplied user agent might be to
download within the context of a session it has created.
OBJECT METHODS
"$obj->download($url_or_request, $download_dir)"
This downloads the page obtained by the HTTP::Request $request (which
could be a post, or any other request returning HTML) in the directory
$download_dir, plus all images and other dependencies needed to render
it.
The main HTML document will be saved in $download_dir as 'index.html'.
Other dependencies will be saved with filenames composed of an index
number (1 for the first item saved, 2 for the second, etc.), plus an
extension (taken from the source URL).
By design, this function will dowload but not attempt to process
non-html content (i.e. if the 'content-type' header does not end in
html). Note also that I've been lazy, so it will still save the content
with as "index.html" as for a HTML page.
The content of the HTML is re-written so that links to dependencies
refer to the downloaded files. External dependencies (anything not
downloaded) are left as-is.
The following dependencies *are* handled:
* "<img href="...">" linked images
* "<style href="...">" stylesheet links
* CSS "@import url(...)" linked stylesheets
* "<script src="...">" linked scripts.
* "<input src="...">" linked images.
* CSS "url()" links.
TODO
The following constructs are not handled, but ought to be:
* Frames and "iframe" tags.
* "<embed>", "object"
Unsupported
These are not handled, and may or may not get implemented:
* excessivly funky javascript which constructs content dynamically
DEPENDENCIES
Dependencies are intentionally kept fairly minimal, but do exist. The
main non-core ones are HTML::Tidy, HTML::Entities and
HTML::TreeBuilder::XPath. See the "META.yaml" included the distribution
for full details.
BUGS / CAVEATS
The internals are a bit of an ugly hack. If I could find something off
the shelf which does the job equivalently, I'd have used that. Since I
couldn't find anything suitable I whipped this up in a jiffy, and then
warped it to support as much as I could.
See the description of "->download" for details of what is and isn't
implemented.
Please report any bugs or feature requests to
"bug-HTML-Encapsulate@rt.cpan.org", or through the web interface at
AUTHOR
Nick Woolley "<npw@cpan.org>"
LICENCE AND COPYRIGHT
Copyright (c) 2009, Nick Woolley "<npw@cpan.org>". All rights reserved.
This module is free software; you can redistribute it and/or modify it
under the same terms as Perl itself. See perlartistic.
DISCLAIMER OF WARRANTY
BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH
YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
NECESSARY SERVICING, REPAIR, OR CORRECTION.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE
TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.