NAME
Hailo::Role::Tokenizer - A role representing a Hailo tokenizer
METHODS
new
This is the constructor. It takes no arguments.
make_tokens
Takes a line of input and returns a list of tokens.
make_output
Takes a list of tokens and returns a line of output.
find_key_tokens
Takes a list of tokens and returns those which are deemed interesting enough to base a reply on.
AUTHOR
Hinrik Örn Sigurðsson, hinrik.sig@gmail.com
LICENSE AND COPYRIGHT
Copyright 2010 Hinrik Örn Sigurðsson
This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself.