There is an ongoing outage on the primary CPAN mirror. It is possible to work around the issue by using MetaCPAN as a mirror.

 / 
Plucene-1.25
River stage two • 6 direct dependents • 19 total dependents
/ Plucene::Analysis::LetterTokenizer

NAME

Plucene::Analysis::LetterTokenizer - Letter tokenizer

SYNOPSIS

# isa Plucene::Analysis::CharTokenizer

DESCRIPTION

This is the letter tokenizer class, which divides text at non-letters.

Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces