NAME
Xango - Event Based High Performance Web Crawler Framework
SYNOPSIS
use Xango;DESCRIPTION
Xango is a frameworlk for writing web crawlers. As such, it doesn't do a whole lot by itself - you need to create custom handlers to do the grunt work. See the documentation for Xango::Broker for more details on how to write your own crawler.
Please note that Xango is still in beta. Some behavior may change as we keep on developing it.
COMPONENTS
The main component that comes with Xango is the Broker component. The Broker handles the basic flow of a web crawler - it accepts data to fetch, applies some policies to it, fetches it, and then sends it for final processing.
The only concrete implementation provided by the Broker of the above flow is the part where the requested URI is fetched. You must provide the rest of the logic. See Xango::Broker for details
The Handler is the component in which the Broker delegates the above processing to.
DEBUGGING
To turn debugging on, you need to pre-declare some constants (which, in Perl, are subroutines). For example, to turn debugging on, you need to say something like this:
sub Xango::DEBUG { 1 }
use Xango;Xango will recognize that the DEBUG flag is already set, and turn debugging on.
BUGS
Plenty, I'm sure. Please report bugs to RT http://rt.cpan.org/NoAuth/Bugs.html?Dist=Xango
TODO
- Documentation.
-
Documentation for this distribution is half-baked at best. It needs a lot of work, including a tutorial.
SEE ALSO
AUTHOR
Copyright 2005 Daisuke Maki <dmaki@cpan.org>. All rights reserved. Development funded by Brazil, Ltd. <http://b.razil.jp>