NAME
AI::NNFlex - A base class for implementing neural networks
SYNOPSIS
use AI::NNFlex::backprop;
my $network = AI::NNFlex::backprop->new(config parameter=>value);
$network->add_layer(nodes=>x,activationfunction=>'function');
$network->init();
use AI::NNFlex::Dataset;
my $dataset = AI::NNFlex::Dataset->new([
[INPUTARRAY],[TARGETOUTPUT],
[INPUTARRAY],[TARGETOUTPUT]]);
my $sqrError = 10;
while ($sqrError >0.01)
{
$sqrError = $dataset->learn($network);
}
$network->lesion({'nodes'=>PROBABILITY,'connections'=>PROBABILITY});
$network->dump_state(filename=>'badgers.wts');
$network->load_state(filename=>'badgers.wts');
my $outputsRef = $dataset->run($network);
my $outputsRef = $network->output(layer=>2,round=>1);DESCRIPTION
AI::NNFlex is intended to be a highly flexible, modular NN framework. It's written entirely in native perl, so there are essentially no prereq's. The following modular divisions are made:
* NNFlex.pm
the core module. Contains methods to construct and
lesion a network
* feedforward.pm
the network type module. Feedforward is the only type
currently defined, but others may be created and
imported at runtime
* <learning>.pm
the learning algorithm. Currently the options are
backprop - standard vanilla backprop
momentum - backprop with momentum
* <activation>.pm
node activation function. Currently the options are
tanh, linear & sigmoid.
* Dataset.pm
methods for constructing a set of input/output data
and applying to a network.Syntax (as of v0.2) is to call the network type constructor, which typically will inherit most methods from AI::NNFlex. The base class should no longer be called directly.
The code should be simple enough to use for teaching purposes, but a simpler implementation of a simple backprop network is included in the example file bp.pl. This is derived from Phil Brierleys freely available java code at www.philbrierley.com.
AI::NNFlex leans towards teaching NN and cognitive modelling applications. Future modules are likely to include more biologically plausible nets like DeVries & Principes Gamma model.
CONSTRUCTOR
AI::NNFlex::<networktype>
new ( parameter => value );
randomweights=>MAXIMUM VALUE FOR INITIAL WEIGHT
fixedweights=>WEIGHT TO USE FOR ALL CONNECTIONS
debug=>[LIST OF CODES FOR MODULES TO DEBUG]
learningrate=>the learning rate of the network
momentum=>the momentum value (momentum learning only)
round=>0 or 1 - 1 sets the network to round output values to
nearest of 1, -1 or 0Currently, network type subclasses of backprop and momentum are implemented. Hopfield is intended to follow fairly soon.
The following parameters are optional: randomweights fixedweights debug round
(Note, if randomweights is not specified the network will default to a random value from 0 to 1.
AI::NNFlex::Dataset
new ( [[INPUT VALUES],[OUTPUT VALUES],[INPUT VALUES],[OUTPUT VALUES],..])INPUT VALUES
These should be comma separated values. They can be applied to the network with ::run or ::learn
OUTPUT VALUES
These are the intended or target output values. Comma separated. These will be used by ::learn
METHODS
This is a short list of the main methods implemented in AI::NNFlex. Subclasses may implement other methods.
AI::NNFlex
add_layer
Syntax:
$network->add_layer( nodes=>NUMBER OF NODES IN LAYER,
persistentactivation=>RETAIN ACTIVATION BETWEEN PASSES,
decay=>RATE OF ACTIVATION DECAY PER PASS,
randomactivation=>MAXIMUM STARTING ACTIVATION,
threshold=>NYI,
activationfunction=>"ACTIVATION FUNCTION",
randomweights=>MAX VALUE OF STARTING WEIGHTS);init
Syntax:
$network->init();Initialises connections between nodes, sets initial weights and loads external components. The base AI::NNFlex init method implementes connections backwards and forwards from each node in each layer to each node in the preceeding and following layers.
lesion
$network->lesion ({'nodes'=>PROBABILITY,'connections'=>PROBABILITY})
Damages the network.PROBABILITY
A value between 0 and 1, denoting the probability of a given node or connection being damaged.
Note: this method may be called on a per network, per node or per layer basis using the appropriate object.
AN::NNFlex::Dataset
learn
$dataset->learn($network)'Teaches' the network the dataset using the networks defined learning algorithm. Returns sqrError;
run
$dataset->run($network)Runs the dataset through the network and returns a reference to an array of output patterns.
EXAMPLES
See the code in ./examples. For any given version of NNFlex, xor.pl will contain the latest functionality.
PREREQs
None. NNFlex should run OK on any version of Perl 5 >.
ACKNOWLEDGEMENTS
Phil Brierley, for his excellent free java code, that solved my backprop problem
Dr Martin Le Voi, for help with concepts of NN in the early stages
Dr David Plaut, for help with the project that this code was originally intended for.
Graciliano M.Passos for suggestions & improved code (see SEE ALSO).
SEE ALSO
AI::NNEasy - Developed by Graciliano M.Passos
Shares some common code with NNFlex. Much faster, and more suitable for
backprop projects with large datasets.TODO
Lots of things:
clean up the perldocs some more
write gamma modules
write BPTT modules
write a perceptron learning module
speed it up
write a tk guiCHANGES
v0.11 introduces the lesion method, png support in the draw module and datasets.
v0.12 fixes a bug in reinforce.pm & adds a reflector in feedforward->run to make $network->run($dataset) work.
v0.13 introduces the momentum learning algorithm and fixes a bug that allowed training to proceed even if the node activation function module can't be loaded
v0.14 fixes momentum and backprop so they are no longer nailed to tanh hidden units only.
v0.15 fixes a bug in feedforward, and reduces the debug overhead
v0.16 changes some underlying addressing of weights, to simplify and speed
v0.17 is a bugfix release, plus some cleaning of UI
v0.20 changes AI::NNFlex to be a base class, and ships three different network types (i.e. training algorithms). Backprop & momentum are both networks of the feedforward class, and inherit their 'run' method from feedforward.pm. 0.20 also fixes a whole raft of bugs and 'not nices'.
COPYRIGHT
Copyright (c) 2004-2005 Charles Colbourn. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
CONTACT
charlesc@nnflex.g0n.net