Package smile.llm.llama

Class Transformer

java.lang.Object

smile.deep.layer.LayerBlock

smile.llm.llama.Transformer

All Implemented Interfaces:

Function<Tensor,Tensor>, Layer

public class Transformer
extends LayerBlock

The Transformer model. It consists of token embeddings, stacked Transformer blocks, and the final output layer. This model can be used for various natural language processing tasks, such as language modeling or text generation.

Field Summary

Fields inherited from class smile.deep.layer.LayerBlock

device, dtype, module

Constructor Summary

Constructors

Constructor	Description
Transformer(ModelArgs args, Device device)	Constructor.

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor tokens)	Forward propagation (or forward pass) through the layer.
Tensor	forward(Tensor tokens, int startPos)	Forward pass through the model.

Methods inherited from class smile.deep.layer.LayerBlock

add, add, asTorch, device, dtype, eval, isTraining, load, save, to, to, toString, train

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.function.Function

andThen, compose

Methods inherited from interface smile.deep.layer.Layer

apply

Constructor Details

Transformer

public Transformer(ModelArgs args,
 Device device)

Constructor.

Parameters:

args - the model configuration parameters.

device - the compute device.

Method Details

forward

public Tensor forward(Tensor tokens,
 int startPos)

Forward pass through the model.

Parameters:

tokens - the input token indices.

startPos - the starting position for attention caching.

Returns:

the output tensor.

forward

public Tensor forward(Tensor tokens)

Description copied from interface: Layer

Forward propagation (or forward pass) through the layer.

Parameters:

tokens - the input tensor.

Returns:

the output tensor.