NAME
tscore.pm
SYNOPSIS
Statistical library package to calculate the T-score. This package should be used with statistic.pl and rank.pl.
DESCRIPTION
Assume that the frequency count data associated with a bigram <word1><word2> is stored in a 2x2 contingency table:
word2 ~word2
word1 n11 n12 | n1p
~word1 n21 n22 | n2p
--------------
np1 np2 nppwhere n11 is the number of times <word1><word2> occur together, and n12 is the number of times <word1> occurs with some word other than word2, and n1p is the number of times in total that word1 occurs as the first word in a bigram.
The T-score is defined as a ratio of difference between the observed and the expected mean to the variance of the sample. Note that this is a variant of the standard t-test that was proposed for use in the identification of collocations in large samples of text.
Thus, the T-score is defined as follows:
m11 = n1p * np1 / npp
T-score = (n11 - m11)/sqrt(n11)AUTHORS
Ted Pedersen <tpederse@d.umn.edu>
Satanjeev Banerjee <banerjee@cs.cmu.edu>
Bridget Thomson McInnes <bthomson@d.umn.edu>
BUGS
This measure currently only defined for bigram data stored in 2x2 contingency table.
SEE ALSO
Mailing List: http://groups.yahoo.com/ngram
COPYRIGHT
Copyright 2000-2004 by Ted Pedersen and Satanjeev Banerjee
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.