NAME
Hailo::Tokenizer::Words - A word tokenizer for Hailo
DESCRIPTION
This tokenizer does its best to handle various languages. It knows about most apostrophes, quotes, and sentence terminators.
AUTHOR
Hinrik Örn Sigurðsson, hinrik.sig@gmail.com
LICENSE AND COPYRIGHT
Copyright 2010 Hinrik Örn Sigurðsson
This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself.