NAME

Mail::SpamAssassin::Plugin::NeuralNetwork - check messages using Fast Artificial Neural Network library

SYNOPSIS

loadplugin Mail::SpamAssassin::Plugin::NeuralNetwork

DESCRIPTION

This plugin checks emails using Neural Network algorithm.

CAVEATS

Training, forgetting and prediction only require the generic use_learner 1 switch (on by default) and use_neuralnetwork 1.

sa-learn --dump --plugin NeuralNetwork can be used to display this plugin's stats and vocabulary data.

use_neuralnetwork (0|1) (default: 1)

Whether to use Neural Network, if it is available.

neuralnetwork_data_dir dirname (default: undef)

Where NeuralNetwork plugin will store its data.

neuralnetwork_min_text_len n (default: 256)

Minimum number of characters of visible text required to run prediction or learning on a message.

neuralnetwork_min_word_len n (default: 4)

Minimum token length considered when building the vocabulary and feature vectors.

neuralnetwork_max_word_len n (default: 24)

Maximum token length considered when building the vocabulary and feature vectors.

neuralnetwork_vocab_cap n (default: 10000)

Maximum number of vocabulary terms to retain; least-frequent terms are pruned when exceeded. Because the trained network is (by default) fully connected, its on-disk model size grows roughly as vocab_cap^1.5, so raising this value has a superlinear effect on .model file size. Values above 50000 are rejected.

neuralnetwork_cache_ttl n (default: 300)

Time-to-live in seconds for the in-memory vocabulary and model caches Set to 0 to disable caching.

neuralnetwork_min_spam_count n (default: 100)

Minimum number of spam messages in the vocabulary required to enable prediction.

neuralnetwork_min_ham_count n (default: 100)

Minimum number of ham messages in the vocabulary required to enable prediction.

neuralnetwork_spam_threshold f (default: 0.6)

Prediction values above this threshold are considered spam.

neuralnetwork_ham_threshold f (default: 0.4)

Prediction values below this threshold are considered ham.

neuralnetwork_learning_rate f (default: 0.1)

Learning rate used by the underlying FANN network during incremental training.

neuralnetwork_momentum f (default: 0.1)

Momentum used for training updates.

neuralnetwork_train_epochs n (default: 50)

Number of training epochs to perform when learning a single message.

neuralnetwork_train_algorithm FANN_TRAIN_QUICKPROP|FANN_TRAIN_RPROP|FANN_TRAIN_BATCH|FANN_TRAIN_INCREMENTAL (default: FANN_TRAIN_RPROP)

Algorithm used by Fann neural network used when training, might increase speed depending on the data volume.

neuralnetwork_lock_timeout n (default: 10)

Maximum number of seconds to wait for the exclusive training lock before giving up and skipping the learn operation. Set to 0 to wait indefinitely.

neuralnetwork_rprop_delta_max n (default: 0.5)

Delta value to apply to RPROP training replay loop.

neuralnetwork_retrain_interval n (default: 100)

Number of successful learn_message calls between forced full retrains of the neural network from the persisted vocabulary. After each retrain the incrementally-trained network is replaced with the freshly rebuilt one. Set to 0 to disable periodic retraining and keep online learning only.

neuralnetwork_stopwords words (default: "the and for with that this from there their have be not but you your")

Space-separated list of stopwords to ignore when tokenizing text.

neuralnetwork_autolearn 0|1 (default 0)

When SpamAssassin declares a message a clear spam or ham during the message scan, and launches the auto-learn process, message is autolearned as spam/ham in the same way as during the manual learning. Value 0 at this option disables the auto-learn process for this plugin.

neuralnetwork_autolearn_vocab_only 0|1 (default 0)

When set to 1 and auto-learn is enabled, autolearned messages update the vocabulary and training buffer but skip FANN model training and saving. This avoids slow I/O operations and temporary files during spamd processing. The model is rebuilt from the accumulated vocabulary on the next manual sa-learn run.

neuralnetwork_dsn (default: none)

The DBI dsn of the database to use.

For SQLite, the database will be created automatically if it does not already exist, the supplied path and file must be read/writable by the user running spamassassin or spamd.

For MySQL/MariaDB or PostgreSQL, see sql-directory for database table creation clauses.

You will need to have the proper DBI module for your database. For example DBD::SQLite, DBD::mysql, DBD::MariaDB or DBD::Pg.

Minimum required SQLite version is 3.24.0 (available from DBD::SQLite 1.59_01).

Examples:

neuralnetwork_dsn dbi:SQLite:dbname=/var/lib/spamassassin/NeuralNetwork.db

neuralnetwork_username (default: none)

The username that should be used to connect to the database. Not used for SQLite.

neuralnetwork_password (default: none)

The password that should be used to connect to the database. Not used for SQLite.

neuralnetwork_min_vocab_hits n (default: 10)

Minimum number of tokens in the email that must exist in the vocabulary for prediction to run.

EVAL RULES

check_neuralnetwork_spam()

Body eval rule. Returns true when the neural network prediction score exceeds neuralnetwork_spam_threshold (default 0.6).

check_neuralnetwork_ham()

Body eval rule. Returns true when the neural network prediction score is below neuralnetwork_ham_threshold (default 0.4).

check_neuralnetwork(low, high)

Body eval rule accepting two optional floating-point arguments. Returns true when the raw prediction score falls within the inclusive range [low, high]. Defaults: low = 0.0, high = 1.0.

Use this rule to define finer-grained confidence tiers.

body      NN_CONFIDENT_SPAM  eval:check_neuralnetwork(0.75, 1.0)
describe  NN_CONFIDENT_SPAM  Email classified as spam with high confidence by Neural Network
score     NN_CONFIDENT_SPAM  2.0

body      NN_PROBABLE_SPAM   eval:check_neuralnetwork(0.55, 0.75)
describe  NN_PROBABLE_SPAM   Email classified as probable spam by Neural Network
score     NN_PROBABLE_SPAM   1.0

body      NN_PROBABLE_HAM    eval:check_neuralnetwork(0.0, 0.4)
describe  NN_PROBABLE_HAM    Email classified as ham by Neural Network
score     NN_PROBABLE_HAM    -1.0