Changes for version 0.001 - 2011-10-03
- First stab at an interface to the crawler
- Required stats: meta description and keywords
- Extra stats: load time, size
- Extra features: Obeys robots.txt, crawls from most linked-to to least
- It's pretty!
Documentation
run your WWW::3172::Crawler::WebInterface application
Modules
Provides a web frontend to WWW::3172::Crawler