Package smile.deep.layer

Interface Layer

All Superinterfaces:
Function<Tensor,Tensor>
All Known Implementing Classes:
ActivationFunction, AdaptiveAvgPool2dLayer, AvgPool2dLayer, BatchNorm1dLayer, BatchNorm2dLayer, Conv2dLayer, Conv2dNormActivation, DropoutLayer, EfficientNet, EmbeddingLayer, FusedMBConv, GELU, GLU, GroupNormLayer, HardShrink, LayerBlock, LeakyReLU, LinearLayer, LogSigmoid, LogSoftmax, MaxPool2dLayer, MBConv, PositionalEncoding, ReLU, RMSNormLayer, SequentialBlock, Sigmoid, SiLU, Softmax, SoftShrink, SqueezeExcitation, StochasticDepth, Tanh, TanhShrink, Transformer
public interface Layer extends Function<Tensor,Tensor>
A layer in the neural network.

  • Method Summary

    Modifier and Type
    Method
    Description
    static AdaptiveAvgPool2dLayer
    adaptiveAvgPool2d(int size)
    Returns an adaptive average pooling layer.
    default Tensor
    apply(Tensor input)
     
    org.bytedeco.pytorch.Module
    asTorch()
    Returns the PyTorch Module object.
    static AvgPool2dLayer
    avgPool2d(int size)
    Returns an average pooling layer that reduces a tensor by combining cells, and assigning the average value of the input cells to the output cell.
    static BatchNorm1dLayer
    batchNorm1d(int in)
    Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another.
    static BatchNorm1dLayer
    batchNorm1d(int in, double eps, double momentum, boolean affine)
    Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another.
    static BatchNorm2dLayer
    batchNorm2d(int in)
    Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another.
    static BatchNorm2dLayer
    batchNorm2d(int in, double eps, double momentum, boolean affine)
    Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another.
    static Conv2dLayer
    conv2d(int in, int out, int kernel)
    Returns a convolutional layer.
    static Conv2dLayer
    conv2d(int in, int out, int kernel, int stride, int padding, int dilation, int groups, boolean bias, String paddingMode)
    Returns a convolutional layer.
    static Conv2dLayer
    conv2d(int in, int out, int size, int stride, String padding, int dilation, int groups, boolean bias, String paddingMode)
    Returns a convolutional layer.
    static DropoutLayer
    dropout(double p)
    Returns a dropout layer that randomly zeroes some of the elements of the input tensor with probability p during training.
    static EmbeddingLayer
    embedding(int numTokens, int dim)
    Returns an embedding layer that is a simple lookup table that stores embeddings of a fixed dictionary and size.
    static EmbeddingLayer
    embedding(int numTokens, int dim, double alpha)
    Returns an embedding layer that is a simple lookup table that stores embeddings of a fixed dictionary and size.
    Tensor
    forward(Tensor input)
    Forward propagation (or forward pass) through the layer.
    static SequentialBlock
    gelu(int in, int out)
    Returns a fully connected layer with GELU activation function.
    static SequentialBlock
    gelu(int in, int out, double dropout)
    Returns a fully connected layer with GELU activation function.
    static SequentialBlock
    hardShrink(int in, int out)
    Returns a fully connected layer with hard shrink activation function.
    static SequentialBlock
    leaky(int in, int out, double negativeSlope)
    Returns a fully connected layer with leaky ReLU activation function.
    static SequentialBlock
    leaky(int in, int out, double negativeSlope, double dropout)
    Returns a fully connected layer with leaky ReLU activation function.
    static LinearLayer
    linear(int in, int out)
    Returns a linear fully connected layer.
    static SequentialBlock
    logSigmoid(int in, int out)
    Returns a fully connected layer with log sigmoid activation function.
    static SequentialBlock
    logSoftmax(int in, int out)
    Returns a fully connected layer with log softmax activation function.
    static MaxPool2dLayer
    maxPool2d(int size)
    Returns a max pooling layer that reduces a tensor by combining cells, and assigning the maximum value of the input cells to the output cell.
    static SequentialBlock
    relu(int in, int out)
    Returns a fully connected layer with ReLU activation function.
    static SequentialBlock
    relu(int in, int out, double dropout)
    Returns a fully connected layer with ReLU activation function.
    static SequentialBlock
    sigmoid(int in, int out)
    Returns a fully connected layer with sigmoid activation function.
    static SequentialBlock
    silu(int in, int out)
    Returns a fully connected layer with SiLU activation function.
    static SequentialBlock
    silu(int in, int out, double dropout)
    Returns a fully connected layer with SiLU activation function.
    static SequentialBlock
    softmax(int in, int out)
    Returns a fully connected layer with softmax activation function.
    static SequentialBlock
    softShrink(int in, int out)
    Returns a fully connected layer with soft shrink activation function.
    static SequentialBlock
    tanh(int in, int out)
    Returns a fully connected layer with tanh activation function.
    static SequentialBlock
    tanhShrink(int in, int out)
    Returns a fully connected layer with tanh shrink activation function.
    default Layer
    to(Device device)
    Moves the layer block to a device.
    default Layer
    to(Device device, ScalarType dtype)
    Moves the layer block to a device.

    Methods inherited from interface java.util.function.Function

    andThen, compose

  • Method Details

    • forward

      Tensor forward(Tensor input)
      Forward propagation (or forward pass) through the layer.
      Parameters:
      input - the input tensor.
      Returns:
      the output tensor.

    • apply

      default Tensor apply(Tensor input)
      Specified by:
      apply in interface Function<Tensor,Tensor>

    • asTorch

      org.bytedeco.pytorch.Module asTorch()
      Returns the PyTorch Module object.
      Returns:
      the PyTorch Module object.

    • to

      default Layer to(Device device)
      Moves the layer block to a device.
      Parameters:
      device - the compute device.

    • to

      default Layer to(Device device, ScalarType dtype)
      Moves the layer block to a device.
      Parameters:
      device - the compute device.
      dtype - the data type.

    • linear

      static LinearLayer linear(int in, int out)
      Returns a linear fully connected layer.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • relu

      static SequentialBlock relu(int in, int out)
      Returns a fully connected layer with ReLU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • relu

      static SequentialBlock relu(int in, int out, double dropout)
      Returns a fully connected layer with ReLU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      dropout - the optional dropout probability.
      Returns:
      a fully connected layer.

    • leaky

      static SequentialBlock leaky(int in, int out, double negativeSlope)
      Returns a fully connected layer with leaky ReLU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      negativeSlope - Controls the angle of the negative slope in leaky ReLU, which is used for negative input values.
      Returns:
      a fully connected layer.

    • leaky

      static SequentialBlock leaky(int in, int out, double negativeSlope, double dropout)
      Returns a fully connected layer with leaky ReLU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      negativeSlope - Controls the angle of the negative slope in leaky ReLU, which is used for negative input values.
      dropout - the optional dropout probability.
      Returns:
      a fully connected layer.

    • gelu

      static SequentialBlock gelu(int in, int out)
      Returns a fully connected layer with GELU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • gelu

      static SequentialBlock gelu(int in, int out, double dropout)
      Returns a fully connected layer with GELU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      dropout - the optional dropout probability.
      Returns:
      a fully connected layer.

    • silu

      static SequentialBlock silu(int in, int out)
      Returns a fully connected layer with SiLU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • silu

      static SequentialBlock silu(int in, int out, double dropout)
      Returns a fully connected layer with SiLU activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      dropout - the optional dropout probability.
      Returns:
      a fully connected layer.

    • tanh

      static SequentialBlock tanh(int in, int out)
      Returns a fully connected layer with tanh activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • sigmoid

      static SequentialBlock sigmoid(int in, int out)
      Returns a fully connected layer with sigmoid activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • logSigmoid

      static SequentialBlock logSigmoid(int in, int out)
      Returns a fully connected layer with log sigmoid activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • softmax

      static SequentialBlock softmax(int in, int out)
      Returns a fully connected layer with softmax activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • logSoftmax

      static SequentialBlock logSoftmax(int in, int out)
      Returns a fully connected layer with log softmax activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • tanhShrink

      static SequentialBlock tanhShrink(int in, int out)
      Returns a fully connected layer with tanh shrink activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • softShrink

      static SequentialBlock softShrink(int in, int out)
      Returns a fully connected layer with soft shrink activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • hardShrink

      static SequentialBlock hardShrink(int in, int out)
      Returns a fully connected layer with hard shrink activation function.
      Parameters:
      in - the number of input features.
      out - the number of output features.
      Returns:
      a fully connected layer.

    • conv2d

      static Conv2dLayer conv2d(int in, int out, int kernel)
      Returns a convolutional layer.
      Parameters:
      in - the number of input channels.
      out - the number of output features.
      kernel - the window/kernel size.
      Returns:
      a convolutional layer.

    • conv2d

      static Conv2dLayer conv2d(int in, int out, int kernel, int stride, int padding, int dilation, int groups, boolean bias, String paddingMode)
      Returns a convolutional layer.
      Parameters:
      in - the number of input channels.
      out - the number of output channels/features.
      kernel - the window/kernel size.
      stride - controls the stride for the cross-correlation.
      padding - controls the amount of padding applied on both sides.
      dilation - controls the spacing between the kernel points.
      groups - controls the connections between inputs and outputs. The in channels and out channels must both be divisible by groups.
      bias - If true, adds a learnable bias to the output.
      paddingMode - "zeros", "reflect", "replicate" or "circular".
      Returns:
      a convolutional layer.

    • conv2d

      static Conv2dLayer conv2d(int in, int out, int size, int stride, String padding, int dilation, int groups, boolean bias, String paddingMode)
      Returns a convolutional layer.
      Parameters:
      in - the number of input channels.
      out - the number of output channels/features.
      size - the window/kernel size.
      stride - controls the stride for the cross-correlation.
      padding - "valid" or "same". With "valid" padding, there's no "made-up" padding inputs. It drops the right-most columns (or bottom-most rows). "same" tries to pad evenly left and right, but if the amount of columns to be added is odd, it will add the extra column to the right. If stride is 1, the layer's outputs will have the same spatial dimensions as its inputs.
      dilation - controls the spacing between the kernel points.
      groups - controls the connections between inputs and outputs. The in channels and out channels must both be divisible by groups.
      bias - If true, adds a learnable bias to the output.
      paddingMode - "zeros", "reflect", "replicate" or "circular".
      Returns:
      a convolutional layer.

    • maxPool2d

      static MaxPool2dLayer maxPool2d(int size)
      Returns a max pooling layer that reduces a tensor by combining cells, and assigning the maximum value of the input cells to the output cell.
      Parameters:
      size - the window/kernel size.
      Returns:
      a max pooling layer.

    • avgPool2d

      static AvgPool2dLayer avgPool2d(int size)
      Returns an average pooling layer that reduces a tensor by combining cells, and assigning the average value of the input cells to the output cell.
      Parameters:
      size - the window/kernel size.
      Returns:
      a max pooling layer.

    • adaptiveAvgPool2d

      static AdaptiveAvgPool2dLayer adaptiveAvgPool2d(int size)
      Returns an adaptive average pooling layer.
      Parameters:
      size - the output size.
      Returns:
      an adaptive average pooling layer.

    • batchNorm1d

      static BatchNorm1dLayer batchNorm1d(int in)
      Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another. Centering and scaling the intermediate tensors has a number of beneficial effects, such as allowing higher learning rates without exploding/vanishing gradients.
      Parameters:
      in - the number of input features.
      Returns:
      a normalization layer.

    • batchNorm1d

      static BatchNorm1dLayer batchNorm1d(int in, double eps, double momentum, boolean affine)
      Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another. Centering and scaling the intermediate tensors has a number of beneficial effects, such as allowing higher learning rates without exploding/vanishing gradients.
      Parameters:
      in - the number of input features.
      eps - a value added to the denominator for numerical stability.
      momentum - the value used for the running_mean and running_var computation. Can be set to 0.0 for cumulative moving average (i.e. simple average).
      affine - when set to true, this layer has learnable affine parameters.
      Returns:
      a normalization layer.

    • batchNorm2d

      static BatchNorm2dLayer batchNorm2d(int in)
      Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another. Centering and scaling the intermediate tensors has a number of beneficial effects, such as allowing higher learning rates without exploding/vanishing gradients.
      Parameters:
      in - the number of input features.
      Returns:
      a normalization layer.

    • batchNorm2d

      static BatchNorm2dLayer batchNorm2d(int in, double eps, double momentum, boolean affine)
      Returns a normalization layer that re-centers and normalizes the output of one layer before feeding it to another. Centering and scaling the intermediate tensors has a number of beneficial effects, such as allowing higher learning rates without exploding/vanishing gradients.
      Parameters:
      in - the number of input features.
      eps - a value added to the denominator for numerical stability.
      momentum - the value used for the running_mean and running_var computation. Can be set to 0.0 for cumulative moving average (i.e. simple average).
      affine - when set to true, this layer has learnable affine parameters.
      Returns:
      a normalization layer.

    • dropout

      static DropoutLayer dropout(double p)
      Returns a dropout layer that randomly zeroes some of the elements of the input tensor with probability p during training. The zeroed elements are chosen independently for each forward call and are sampled from a Bernoulli distribution. Each channel will be zeroed out independently on every forward call.

      This has proven to be an effective technique for regularization and preventing the co-adaptation of neurons as described in the paper "Improving Neural Networks by Preventing Co-adaptation of Feature Detectors".

      Parameters:
      p - the probability of an element to be zeroed.
      Returns:
      a dropout layer.

    • embedding

      static EmbeddingLayer embedding(int numTokens, int dim)
      Returns an embedding layer that is a simple lookup table that stores embeddings of a fixed dictionary and size.

      This layer is often used to store word embeddings and retrieve them using indices. The input to the module is a list of indices, and the output is the corresponding word embeddings.

      Parameters:
      numTokens - the size of the dictionary of embeddings.
      dim - the size of each embedding vector.
      Returns:
      a dropout layer.

    • embedding

      static EmbeddingLayer embedding(int numTokens, int dim, double alpha)
      Returns an embedding layer that is a simple lookup table that stores embeddings of a fixed dictionary and size.

      This layer is often used to store word embeddings and retrieve them using indices. The input to the module is a list of indices, and the output is the corresponding word embeddings.

      Parameters:
      numTokens - the size of the dictionary of embeddings.
      dim - the size of each embedding vector.
      alpha - optional scaling factor.
      Returns:
      a dropout layer.