######################################################################
Text::Language::Guess 0.02
######################################################################
NAME
Text::Language::Guess - Trained module to guess a document's language
SYNOPSIS
use Text::Language::Guess;
my $guesser = Text::Language::Guess->new();
my $lang = $guesser->language_guess("bill.txt");
# prints 'en'
print "Best fit: $lang\n";
DESCRIPTION
Text::Language::Guess guesses a document's language. Its implementation
is simple: Using "Text::ExtractWords" and "Lingua::StopWords" from CPAN,
it determines how many of the known stopwords the document contains for
each language supported by "Lingua::StopWords".
Each word in the document recognized as stopword of a particular
language scores one point for this language.
The "language_guess()" function takes a document as a parameter and
returns the abbreviation of the language that it is most likely written
in.
Supported Languages:
* English (en)
* French (fr)
* Spanish (es)
* Portugese (pt)
* Italian (it)
* German (de)
* Dutch (nl)
* Swedish (sv)
* Norwegian (no)
* Danish (da)
Methods
"new()"
Initializes the guesser with all stopwords available for all
supported languges. If "new" has been called before, subsequent
calls will return the same precomputed stoplist map, avoiding
collecting all stopwords again (as long as the number of languages
stays the same, see next paragraph).
You can limit the number of searched languages by specifying the
"language" parameter and passing it an array ref of wanted
languages:
# Only guess between English and German
$guesser = Text::Language::Guess->new(languages => ['en', 'de']);
"language_guess($textfile)"
Reads in a text file, extracts all words, scores them using the
stopword maps and returns a single two-letter string indicating the
language the document is most likely written in.
"language_guess_string($string)"
Just like "language_guess", but takes a string instead of a file
name.
"scores($textfile)"
Like "language_guess($textfile)", just returning a ref to a hash
mapping language strings (e.g. 'en') to a score number. The entry
with the highest score is the most likely one.
"scores_string($string)"
Like "scores", but takes a string instead of a file name.
EXAMPLES
use Text::Language::Guess;
# Guess language in a string instead of a file
my $guesser = Text::Language::Guess->new();
my $lang = $guesser->language_guess_string("Make love not war");
# 'en'
# Limit number of languages to choose from
my $guesser = Text::Language::Guess->new(languages => ['da', 'nl']);
my $lang = $guesser->language_guess_string(
"Which is closer to English, danish or dutch?");
# 'nl'
# Show different scores
my $guesser = Text::Language::Guess->new();
my $scores = $guesser->scores_string(
"This text is English, but other languages are scoring as well");
use Data::Dumper;
print Dumper($scores);
# $VAR1 = {
# 'pt' => 1,
# 'en' => 6,
# 'fr' => 1,
# 'nl' => 1
# };
LEGALESE
Copyright 2005 by Mike Schilli, all rights reserved. This program is
free software, you can redistribute it and/or modify it under the same
terms as Perl itself.
AUTHOR
2005, Mike Schilli <cpan@perlmeister.com>