NAME
AI::MXNet::InitDesc - A container for the initialization pattern serialization.
new
Parameters
---------
name : str
name of variable
attrs : hash ref of str to str
attributes of this variable taken from AI::MXNet::Symbol->attr_dict
NAME
AI::MXNet::Initializer - Base class for all Initializers
DESCRIPTION
The base class AI::MXNet::Initializer defines the default behaviors to initialize various parameters,
such as set bias to 1, except for the weight. Other classes then define how to initialize the weights.
Currently following classes are available:
mx->init->Uniform Initializes weights with random values uniformly sampled from a given range.
mx->init->Normal Initializes weights with random values sampled from a normal distribution with a mean of zero and standard deviation of sigma.
mx->init->Load Initializes variables by loading data from file or dict.
mx->init->Mixed Initialize parameters using multiple initializers.
mx->init->Zero Initializes weights to zero.
mx->init->One Initializes weights to one.
mx->init->Constant Initializes the weights to a given value.
mx->init->Orthogonal Initialize weight as orthogonal matrix.
mx->init->Xavier Returns an initializer performing Xavier initialization for weights.
mx->init->MSRAPrelu Initialize the weight according to a MSRA paper.
mx->init->Bilinear Initialize weight for upsampling layers.
mx->init->FusedRNN Initialize parameters for fused rnn layers.
register
Register an initializer class to the AI::MXNet::Initializer factory.
set_verbosity
Switch on/off verbose mode
Parameters
----------
$verbose : bool
switch on/off verbose mode
$print_func : CodeRef
A function that computes statistics of initialized arrays.
Takes an AI::MXNet::NDArray and returns a scalar. Defaults to mean
absolute value |x|/size(x)
init
Parameters
----------
$desc : AI::MXNet::InitDesc|str
a name of corresponding ndarray
or the object that describes the initializer.
$arr : AI::MXNet::NDArray
an ndarray to be initialized.
NAME
AI::MXNet::Load - Initialize by loading a pretrained param from a hash ref.
new
Parameters
----------
param: HashRef[AI::MXNet::NDArray]
default_init: Initializer
default initializer when a name is not found in the param hash ref.
verbose: bool
log the names when initializing.
NAME
AI::MXNet::Mixed - A container with multiple initializer patterns.
new
patterns: array ref of str
array ref of regular expression patterns to match parameter names.
initializers: array ref of AI::MXNet::Initializer objects.
array ref of Initializers corresponding to the patterns.
NAME
AI::MXNet::Uniform - Initialize the weight with uniform random values.
DESCRIPTION
Initialize the weight with uniform random values contained within of [-scale, scale]
Parameters
----------
scale : float, optional
The scale of the uniform distribution.
NAME
AI::MXNet::Normal - Initialize the weight with gaussian random values.
DESCRIPTION
Initialize the weight with gaussian random values contained within of [0, sigma]
Parameters
----------
sigma : float, optional
Standard deviation for the gaussian distribution.
NAME
AI::MXNet::Orthogonal - Intialize the weight as an Orthogonal matrix.
DESCRIPTION
Intialize weight as Orthogonal matrix
Parameters
----------
scale : float, optional
scaling factor of weight
rand_type: string optional
use "uniform" or "normal" random number to initialize weight
Reference
---------
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
arXiv preprint arXiv:1312.6120 (2013).
NAME
AI::MXNet::Xavier - Initialize the weight with Xavier or similar initialization scheme.
DESCRIPTION
Parameters
----------
rnd_type: str, optional
Use gaussian or uniform.
factor_type: str, optional
Use avg, in, or out.
magnitude: float, optional
The scale of the random number range.
NAME
AI::MXNet::MSRAPrelu - Custom initialization scheme.
DESCRIPTION
Initialize the weight with initialization scheme from
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.
Parameters
----------
factor_type: str, optional
Use avg, in, or out.
slope: float, optional
initial slope of any PReLU (or similar) nonlinearities.
NAME
AI::MXNet::LSTMBias - Custom initializer for LSTM cells.
DESCRIPTION
Initializes all biases of an LSTMCell to 0.0 except for
the forget gate's bias that is set to a custom value.
Parameters
----------
forget_bias: float,a bias for the forget gate.
Jozefowicz et al. 2015 recommends setting this to 1.0.
NAME
AI::MXNet::FusedRNN - Custom initializer for fused RNN cells.
DESCRIPTION
Initializes parameters for fused rnn layer.
Parameters
----------
init : Initializer
initializer applied to unpacked weights.
All parameters below must be exactly the same as ones passed to the
FusedRNNCell constructor.
num_hidden : int
num_layers : int
mode : str
bidirectional : bool
forget_bias : float