Class MultilayerPerceptron

java.lang.Object
smile.base.mlp.MultilayerPerceptron
All Implemented Interfaces:
Serializable, AutoCloseable
Direct Known Subclasses:
MLP, MLP
public abstract class MultilayerPerceptron extends Object implements AutoCloseable, Serializable
Fully connected multilayer perceptron neural network. An MLP consists of at least three layers of nodes: an input layer, a hidden layer and an output layer. The nodes are interconnected through weighted acyclic arcs from each preceding layer to the following, without lateral or feedback connections. Each node calculates a transformed weighted linear combination of its inputs (output activations from the preceding layer), with one of the weights acting as a trainable bias connected to a constant input. The transformation, called activation function, is a bounded non-decreasing (non-linear) function.
See Also:

  • Field Details

    • p

      protected int p
      The dimensionality of input data.

    • output

      protected OutputLayer output
      The output layer.

    • net

      protected Layer[] net
      The input and hidden layers.

    • target

      protected transient ThreadLocal<Vector> target
      The buffer to store desired target value of training instance.

    • learningRate

      protected TimeFunction learningRate
      The learning rate.

    • momentum

      protected TimeFunction momentum
      The momentum factor.

    • rho

      protected double rho
      The discounting factor for the history/coming gradient in RMSProp.

    • epsilon

      protected double epsilon
      A small constant for numerical stability in RMSProp.

    • lambda

      protected double lambda
      The L2 regularization factor, which is also the weight decay factor.

    • clipValue

      protected double clipValue
      The gradient clipping value.

    • clipNorm

      protected double clipNorm
      The gradient clipping norm.

    • t

      protected int t
      The training iterations.

  • Constructor Details

    • MultilayerPerceptron

      public MultilayerPerceptron(Layer... net)
      Constructor.
      Parameters:
      net - the input layer, hidden layers, and output layer in order.

  • Method Details

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • setLearningRate

      public void setLearningRate(TimeFunction rate)
      Sets the learning rate.
      Parameters:
      rate - the learning rate.

    • setMomentum

      public void setMomentum(TimeFunction momentum)
      Sets the momentum factor. momentum = 0.0 means no momentum.
      Parameters:
      momentum - the momentum factor.

    • setRMSProp

      public void setRMSProp(double rho, double epsilon)
      Sets RMSProp parameters.
      Parameters:
      rho - The discounting factor for the history/coming gradient.
      epsilon - A small constant for numerical stability.

    • setWeightDecay

      public void setWeightDecay(double lambda)
      Sets the weight decay factor. After each weight update, every weight is simply "decayed" or shrunk according to w = w * (1 - 2 * eta * lambda).
      Parameters:
      lambda - the weight decay factor.

    • setClipValue

      public void setClipValue(double clipValue)
      Sets the gradient clipping value. If clip value is set, the gradient of each weight is clipped to be no higher than this value.
      Parameters:
      clipValue - the gradient clipping value.

    • setClipNorm

      public void setClipNorm(double clipNorm)
      Sets the gradient clipping norm. If clip norm is set, the gradient of each weight is individually clipped so that its norm is no higher than this value.
      Parameters:
      clipNorm - the gradient clipping norm.

    • getLearningRate

      public double getLearningRate()
      Returns the learning rate.
      Returns:
      the learning rate.

    • getMomentum

      public double getMomentum()
      Returns the momentum factor.
      Returns:
      the momentum factor.

    • getWeightDecay

      public double getWeightDecay()
      Returns the weight decay factor.
      Returns:
      the weight decay factor.

    • getClipValue

      public double getClipValue()
      Returns the gradient clipping value.
      Returns:
      the gradient clipping value.

    • getClipNorm

      public double getClipNorm()
      Returns the gradient clipping norm.
      Returns:
      the gradient clipping norm.

    • vector

      protected Vector vector(double[] x)
      Wraps an array into vector.
      Parameters:
      x - the input array.
      Returns:
      the vector.

    • propagate

      protected void propagate(Vector x, boolean training)
      Propagates the signals through the neural network.
      Parameters:
      x - the input signal.
      training - true if this is in training pass.

    • backpropagate

      protected void backpropagate(boolean update)
      Propagates the errors back through the network.
      Parameters:
      update - the flag if update the weights directly. It should be false for (mini-)batch.

    • update

      protected void update(int m)
      Updates the weights for mini-batch training.
      Parameters:
      m - the mini-batch size.

    • setParameters

      public void setParameters(Properties params)
      Sets MLP hyperparameters such as learning rate, weight decay, momentum, RMSProp, etc.
      Parameters:
      params - the MLP hyperparameters.