Revision history for Perl module WWW-3172-Crawler-WebInterface

0.001     2011-10-03
 - First stab at an interface to the crawler
    - Required stats: meta description and keywords
    - Extra stats: load time, size
    - Extra features: Obeys robots.txt, crawls from most linked-to to least
    - It's pretty!