Package smile.deep

Class Optimizer

java.lang.Object
smile.deep.Optimizer
public class Optimizer extends Object
Optimizer functions.

  • Method Summary

    Modifier and Type
    Method
    Description
    static Optimizer
    Adam(Model model, double rate)
    Returns an Adam optimizer.
    static Optimizer
    Adam(Model model, double rate, double beta1, double beta2, double eps, double decay, boolean amsgrad)
    Returns an Adam optimizer.
    static Optimizer
    AdamW(Model model, double rate)
    Returns an AdamW optimizer.
    static Optimizer
    AdamW(Model model, double rate, double beta1, double beta2, double eps, double decay, boolean amsgrad)
    Returns an AdamW optimizer.
    void
    reset()
    Resets gradients.
    static Optimizer
    RMSprop(Model model, double rate)
    Returns an RMSprop optimizer.
    static Optimizer
    RMSprop(Model model, double rate, double alpha, double eps, double decay, double momentum, boolean centered)
    Returns an RMSprop optimizer.
    void
    setLearningRate(double rate)
    Sets the learning rate.
    static Optimizer
    SGD(Model model, double rate)
    Returns a stochastic gradient descent optimizer without momentum.
    static Optimizer
    SGD(Model model, double rate, double momentum, double decay, double dampening, boolean nesterov)
    Returns a stochastic gradient descent optimizer with momentum.
    void
    step()
    Updates the parameters based on the calculated gradients.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • reset

      public void reset()
      Resets gradients.

    • step

      public void step()
      Updates the parameters based on the calculated gradients.

    • setLearningRate

      public void setLearningRate(double rate)
      Sets the learning rate.
      Parameters:
      rate - the learning rate.

    • SGD

      public static Optimizer SGD(Model model, double rate)
      Returns a stochastic gradient descent optimizer without momentum.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      Returns:
      the optimizer.

    • SGD

      public static Optimizer SGD(Model model, double rate, double momentum, double decay, double dampening, boolean nesterov)
      Returns a stochastic gradient descent optimizer with momentum.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      momentum - the momentum factor.
      decay - the weight decay (L2 penalty).
      dampening - dampening for momentum.
      nesterov - enables Nesterov momentum.
      Returns:
      the optimizer.

    • Adam

      public static Optimizer Adam(Model model, double rate)
      Returns an Adam optimizer.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      Returns:
      the optimizer.

    • Adam

      public static Optimizer Adam(Model model, double rate, double beta1, double beta2, double eps, double decay, boolean amsgrad)
      Returns an Adam optimizer.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      beta1 - coefficients used for computing running averages of gradient and its square.
      beta2 - coefficients used for computing running averages of gradient and its square.
      eps - term added to the denominator to improve numerical stability.
      decay - the weight decay (L2 penalty).
      amsgrad - whether to use the AMSGrad variant of this algorithm from the paper On the Convergence of Adam and Beyond.
      Returns:
      the optimizer.

    • AdamW

      public static Optimizer AdamW(Model model, double rate)
      Returns an AdamW optimizer.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      Returns:
      the optimizer.

    • AdamW

      public static Optimizer AdamW(Model model, double rate, double beta1, double beta2, double eps, double decay, boolean amsgrad)
      Returns an AdamW optimizer.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      beta1 - coefficients used for computing running averages of gradient and its square.
      beta2 - coefficients used for computing running averages of gradient and its square.
      eps - term added to the denominator to improve numerical stability.
      decay - the weight decay (L2 penalty).
      amsgrad - whether to use the AMSGrad variant of this algorithm from the paper On the Convergence of Adam and Beyond.
      Returns:
      the optimizer.

    • RMSprop

      public static Optimizer RMSprop(Model model, double rate)
      Returns an RMSprop optimizer.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      Returns:
      the optimizer.

    • RMSprop

      public static Optimizer RMSprop(Model model, double rate, double alpha, double eps, double decay, double momentum, boolean centered)
      Returns an RMSprop optimizer.
      Parameters:
      model - the model to be optimized.
      rate - the learning rate.
      alpha - smoothing constant.
      eps - term added to the denominator to improve numerical stability.
      decay - the weight decay (L2 penalty).
      momentum - the momentum factor.
      centered - if true, compute the centered RMSProp, the gradient is normalized by an estimation of its variance.
      Returns:
      the optimizer.