NAME

WWW::Crawler::Mojo::ScraperUtil - Scraper utitlities

SYNOPSIS

DESCRIPTION

This class inherits Mojo::UserAgent and override start method for storing user info

ATTRIBUTES

WWW::Crawler::Mojo::UserAgent inherits all attributes from Mojo::UserAgent.

METHODS

WWW::Crawler::Mojo::UserAgent inherits all methods from Mojo::UserAgent.

decoded_body

Returns decoded response body for given Mojo::Message::Request using guess_encoding and encoder.

encoder

Generates Encode instance for given name. Defaults to Encode::utf8.

resolve_href

Resolves URLs with a base URL.

WWW::Crawler::Mojo::resolve_href($base, $uri);

guess_encoding

Guesses encoding of HTML or CSS with given Mojo::Message::Response instance.

$encode = WWW::Crawler::Mojo::guess_encoding($res) || 'utf-8'

AUTHOR

Sugama Keita, <sugama@jamadam.com>

COPYRIGHT AND LICENSE

Copyright (C) Sugama Keita.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.