NAME

URI::Find - Find URIs in arbitrary text

SYNOPSIS

require URI::Find;

my $finder = URI::Find->new(\&callback);

$how_many_found = $finder->find(\$text);

DESCRIPTION

This module does one thing: Finds URIs and URLs in plain text. It finds them quickly and it finds them all (or what URI::URL considers a URI to be.) It employs a series of heuristics to:

Find schemeless URIs (ie. www.foo.com)

Avoid picking up trailing characters from the text

Avoid picking up URL-like things such as perl module names.

Public Methods

new

my $finder = URI::Find->new(\&callback);

Creates a new URI::Find object.

&callback is a function which is called on each URI found. It is passed two arguments, the first is a URI::URL object representing the URI found. The second is the original text of the URI found. The return value of the callback will replace the original URI in the text.

find

my $how_many_found = $finder->find(\$text);

$text is a string to search and possibly modify with your callback.

Protected Methods

I got a bunch of mail from people asking if I'd add certain features to URI::Find. Most wanted the search to be less restrictive, do more heuristics, etc... Since many of the requests were contradictory, I'm letting people create their own custom subclasses to do what they want.

The following are methods internal to URI::Find which a subclass can override to change the way URI::Find acts. They are only to be called inside a URI::Find subclass. Users of this module are NOT to use these methods.

uri_re

my $uri_re = $self->uri_re;

Returns the regex for finding absolute, schemed uris (http://www.foo.com and such). This, combined with schemeless_uri_re() is what finds candidate uris.

Usually this method does not have to be overridden.

schemeless_uri_re

my $schemeless_re = $self->schemeless_uri_re;

Returns the regex for finding schemeless uris (www.foo.com and such) and other things which might be uris. The default implementation only looks for things starting with www and ftp. It does this to limit the number of false positives.

Many people will want to override this method.

uric_set

my $uric_set = $self->uric_set;

Returns a set matching the 'uric' set defined in RFC 2396 suitable for putting into a character set ([]) in a regex.

You almost never have to override this.

cruft_set

my $cruft_set = $self->cruft_set;

Returns a set of characters which are considered garbage. Used by decruft().

decruft

my $uri = $self->decruft($uri);

Sometimes garbage characters like periods and parenthesis get accidentally matched along with the URI. In order for the URI to be properly identified, it must sometimes be "decrufted", the garbage characters stripped.

This method takes a candidate URI and strips off any cruft it finds.

recruft

my $uri = $self->recruft($uri);

This method puts back the cruft taken off with decruft(). This is necessary... for reasons I'm not going to go into at the moment.

schemeless_to_schemed

my $schemed_uri = $self->schemeless_to_schemed($schemeless_uri);

This takes a schemeless URI and returns an absolute, schemed URI. If you overrode schemeless_uri_re(), you probably want to override this.

is_schemed

$obj->is_schemed($uri);

Returns whether or not the given uri is schemed or schemeless. True for schemed, false for schemeless.

Old Functions

The old find_uri() function is still around and it works, but its deprecated.

EXAMPLES

Simply print the original URI text found and the normalized representation.

my $finder = URI::Find->new(
                    sub {
                        my($uri, $orig_uri) = @_;
                        print "The text '$orig_uri' represents '$uri'\n";
                        return $orig_uri;
                    });
$finder->find(\$text);

Check each URI in document to see if it exists.

use LWP::Simple;

my $finder = URI::Find->new(sub {
                                my($uri, $orig_uri) = @_;
                                if( head $uri ) {
                                    print "$orig_uri is okay\n";
                                }
                                else {
                                    print "$orig_uri cannot be found\n";
                                }
                                return $orig_uri;
                            });
$finder->find(\$text);

Turn plain text into HTML, with each URI found wrapped in an HTML anchor.

use CGI qw(escapeHTML);

$text = "<pre>\n" . escapeHTML($text) . "</pre>\n";
my $finder = URI::Find->new(
                            sub {
                                my($uri, $orig_uri) = @_;
                                return qq|<a href="$uri">$orig_uri</a>|;
                            });
$finder->find(\$text);

CAVEATS, BUGS, ETC...

RFC 2396 Appendix E suggests using the form '<http://www.foo.com>' or '<URL:http://www.foo.com>' when putting URLs in plain text. URI::Find accomidates this suggestion and considers the entire thing (brackets and all) to be part of the URL found. This means that when find_uris() sees '<URL:http://www.foo.com>' it will hand that entire string to your callback, not just the URL.

NOTE: The prototype on find_uris() is already getting annoying to me. I might remove it in a future version.

SEE ALSO

L<URI::Find::Schemeless>, L<URI::URL>, L<URI>,
RFC 2396 (especially Appendix E)

AUTHOR

Michael G Schwern <schwern@pobox.com> with insight from Uri Gutman, Greg Bacon, Jeff Pinyan, Roderick Schertler and others.

Currently maintained by Roderick Schertler <roderick@argon.org>.