Changes for version 0.11 - 2015-02-19
- Updated dependency to Mojolicious v5.79.
- Fixed a bug where connection count didn't detected for redirected urls.
- Fixed a bug on checkbot example.
- Fixed small bug on form submittion.
- Fixed small bug on base tag detection.
- Fixed small bug on charset detection.
- Removed refer event in favor of callback for scraper.
- Removed peeping server feature at all.
- Deprecated resolved_uri attribute of Job class in favor of url.
- Deprecated original_uri in favor of original_url.
- Improved to crawl sitemap.xml too.
- Improved html document detection by accepting more html like mime types.
- Improved to use less memory.
- Improved internal codes.
- Added clock_speed attribute.
Modules
A web crawling framework for Perl
Single crawler job
Crawler queue base class
Crawler queue with memory
Scraper utitlities
Mojo::UserAgent sub class for userinfo storage