NAME
HTML::Manipulator - Perl extension for manipulating HTML files
SYNOPSIS
use HTML::Manipulator;
my $html = <<HTML;
<h1 id=title>Old news</h1>
<a href='http://www.google.com' id=link>Google</a>....
HTML
# replace a tag content
my $new = HTML::Manipulator::replace($html, title => 'New news');
# replace a tag attribute and content
my $new = HTML::Manipulator::replace($html, link => {
_content => 'Slashdot',
href=>'http://www.slashdot.org/' }
);
# extract a tag content
my $content = HTML::Manipulator::extract_content($html, 'link');
# extract a tag content and attributes
my $tag = HTML::Manipulator::extract($html, 'link');
# returns a hash ref like
# { href => 'http://www.google.com', id => 'link', _content => 'Google' }DESCRIPTION
This module manipulates of the contents of HTML files. It can query and replace the content or attributes of any HTML tag.
The advertised usage pattern is to update static HTML files.
ANOTHER TEMPLATE ENGINE ? NO !
HTML::Manipulator is NOT yet another templating module. There are, for example, no template files. It works on normal HTML files without any special markup (you only have to give element IDs to tags you are interested in, or wrap them in comments).
While you could probably use this module for producing your web application's output, DON'T. It does not offer a lot of features for this area (no loops, no conditionals, no includes) and is not optimized for performance. Have a look at HTML::Template instead.
ABOUT THE INPUT HTML FILES
HTML::Manipulator is meant to work on real-life HTML files (in all their non-standards-compliant ugliness). It uses the HTML::Parser module to find elements (tags) inside those files, which you can then replace or modify. All you have to do is give those elements a DOM ID, for example
<h3 id=headline77>Headline</h3>No other markup is necessary.
As an alternative to element ID, HTML::Manipulator can also identify sections enclosed in HTML comments.
<h3><!-- headline77 -->Headline<!-- --></h3>Malformed HTML (is fine)
HTML::Manipulator tries to cope with malformed input data. All you have to ensure is that you properly close the element you are working on (any other tags can be unbalanced) and that the IDs are unique. It will also preserve the content outside the element you asked it to operate on. It does not rewrite your HTML any more than it has to.
Case insensitivity issues
HTML is case insensitive in its tag and attribute names. This means that
<h3 id=headline77>Headline</h3>and
<H3 iD=headline77>Headline</h3>are treated as identical.
However, HTML::Manipulator respects case when comparing the IDs of elements (not sure about the HTML standard here), so that you could NOT address above h3 element as HeadLine77.
When HTML::Manipulator has to rewrite tags (this happens when you ask it to change element attributes) it will output the tag and attribute names as lower-case. It will also rearrange their order. When changing only the content of an element, it preserves the original opening and closing tags.
When matching HTML comments (see below), case and whitespace are ignored.
FUNCTIONS TO CHANGE CONTENT
You can change the content or attributes of any HTML element with an attached ID.
Replace the content of one element
my $new = HTML::Manipulator::replace($html, title => 'New news');The function takes as input the HTML data and returns the modified data (as a long scalar).
Replace the content of many elements
my $new = HTML::Manipulator::replace($html,
title => 'New news', headline77=>'All clear?');You can just pass many IDs and new contents to the function as well. The caveat here is that if those elements are nested, only the outermost will be applied: The complete content of the outermost element will be replaced with the new content, eliminating any nested tags. Even if the new content contains nested elements, these will not be evaluated. No recursion today.
Replace attribute values
If you want to replace attribute values (such as a link href), you use the same function described above, but pass a hashref instead of the string with the new content:
my $new = HTML::Manipulator::replace($html, link => {
href=>'http://www.slashdot.org/' }
);The hashref can contain as many key/value pairs as you want. Any attributes that you specify here will appear in the output HTML. Any attributes that you do not specify will retain their old value.
Replace attribute values and content
You can also change content and attributes at the same time, by adding the special "attribute" _content to the attribute hashref.
my $new = HTML::Manipulator::replace($html, link => {
_content => 'Slashdot',
href=>'http://www.slashdot.org/' }
);Insert adjacent HTML
There is a family of functions to insert some text into the source HTML document before or after a given DOM element.
my $new = HTML::Manipulator::insert_before_begin($html,
headline77 => '<b>',
);
my $new = HTML::Manipulator::insert_after_begin($html,
headline77 => '[**] ',
);
my $new = HTML::Manipulator::insert_before_end($html,
headline77 => '[**]',
);
my $new = HTML::Manipulator::insert_after_end($html,
headline77 => '</b>',
);Replace the document title
You can set the document title (the stuff between the <title> tags) like
my $new = HTML::Manipulator::replace_title($html, 'new title');FUNCTIONS TO EXTRACT CONTENT
In addition to replacing parts of the HTML document, you can also query it for the current content.
Extract the content of an element
my $content = HTML::Manipulator::extract_content($html, 'link');gives you a scalar containing the content of the tag with the ID 'link'.
Extract the content of all elements
my $content = HTML::Manipulator::extract_all_content($html);gives you a hashref with all element IDs as keys and their contents as values.
Extract the content and attributes
my $content = HTML::Manipulator::extract($html, 'link');gives you a hashref with information about the tag with the ID 'link'. There is a key for every attribute in the tag, and the special key '_content' which contains the content. The structure of the hashref is identical to what you would use when calling the replace function.
There is also a function to get information about all elements (that have an ID):
my $content = HTML::Manipulator::extract_all($html);This returns a hashref of hashrefs, so that you could get the href of the "link" element like $content->{link}{href}.
Extract some elements
You can selectively use the extract_all* functions by passing in the IDs you are interested in. This is optional. The default returns data for all elements with IDs.
my $content = HTML::Manipulator::extract_all_content
($html, 'one', 'two', 'three');
You can also mix in regular expressions. Any elements with IDs that match will be returned. This way you can also achieve case-insensitivity with IDs.
my $content = HTML::Manipulator::extract_all_content
($html, qr/^...$/i, 'two', qr/^some.*/);Find out all element IDs
You can query for a list of all element IDs and their tag type.
$data = HTML::Manipulator::extract_all_ids($html);This returns a hashref where the element IDs of the document are they keys. The associated value is the type of the element (the tag type, such as div, span, a), which is returned as lowercase.
You can filter this in the same way as with the extract_all_content function above:
$data = HTML::Manipulator::extract_all_ids($html,
qr/^...$/i, 'two', qr/^some.*/);Find out the document title
my $title = HTML::Manipulator::extract_title($html);USING HTML COMMENTS INSTEAD OF ELEMENT ID
Instead of (or in addition to) using a DOM element ID to identify the section of the document you want to work on, you can also enclose that section in HTML comments. This approach is used by many software packages, for example Dreamweaver.
extracting content
<!-- start description --> blah blah blah <!-- end -->If you have HTML like above, you can get the section enclosed by the two comments using the same extraction functions as with element ID. Instead of an ID, you just use the opening comment tag:
my $content = HTML::Manipulator::extract_content($html, '<!-- start description -->');There are some caveats:
The section starts with the comment tag you used in lieu of an element ID. It ends with the first subsequent comment tag, no matter how that tag looks like. This means that the enclosed section cannot contain HTML comments itself.
When matching the opening comment tag, all whitespace and case is ignored. '<!-- start description -->' and '<!--STARTDESCRIPTION-->' are the same thing
When using extract_all_content or extract_all without any parameters, the functions return the content of all elements with an ID. They do not return any sections marked up by comments (because it is impossible to figure out if a given HTML comment is supposed to mark a section in that manner without more information). If you want to get comment-enclosed sections, you have to explicitly name them. You also cannot use regular expressions directly in the extract_* functions like you could to match an ID.
You can query the document for all comments that have a certain form by using another function (extract_all_comments) and use the results of this function to specify what you need in extract_all_content.
my @array = HTML::Manipulator::extract_all_comments($html, qr/START/i);This will give you something like: ( '<!-- start description -->', '<!-- start footer -->').
replacing content
You can use the replace() function with the opening comment tag instead of an element ID:
my $new = HTML::Manipulator::replace($html,
'<!-- title -->' => 'New news');USING FILEHANDLES
You can also call all of the above functions with a file handle instead of the string holding the HTML. HTML::Manipulator (or HTML::Parser deeper down the line) will read from the file.
use FileHandle;
my $new = HTML::Manipulator::replace(new FileHandle('myfile.html'), title => 'New news');
open IN, 'myfile.html';
my $new = HTML::Manipulator::replace(*IN, title => 'New news');
close IN;HTML::Manipulator will only read from the file handles you give it. It does not change them. Nor does it open them, you have to have done that yourself. Or you can use HTML::Manipulator::Document, which does open files.
EXPORT
The module exports none of its functions. You have to prefix the full module name to use them.
If you want an object-oriented interface instead, consider HTML::Manipulator::Document.
SEE ALSO
Processing HTML documents
HTML::Manipulator uses HTML::Parser for parsing the input file. If you find yourself in the unfortunate situation of having to process HTML, have a look at that module.
For specific purposes there are also some other modules to work with, for example
HTML::HeadParser extracts info from the <head> section
HTML::LinkExtor and HTML::LinkExtractor extract links
HTML::FillInForm populates HTML forms with data
Producing HTML output (templating engines)
HTML::Manipulator is not a templating engine. You should not use it to produce output for your CGI script. If you want to do that, take a look at
HTML::Template (personal favorite)
Template Toolkit (also very popular)
Petal (innovative, uses attributes rather than tags for markup)
Managing complete (static) web sites
A great tool to produce and manage a large number of related static HTML pages is HTML::WebMake.
BUGS
This is a young module. It works for me, but it has not been extensively tested in the wild. Handle with care. Report bugs to get them fixed.
AUTHOR
Thilo Planz, <thilo@cpan.org>
COPYRIGHT AND LICENSE
Copyright 2004/05 by Thilo Planz
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.