Package smile.llm.llama

Class Attention

java.lang.Object

smile.llm.llama.Attention

public class Attention
extends Object

Multi-head attention. It caches key and value information, applying rotary embeddings, and performing linear transformations.

Constructor Summary

Constructors

Constructor	Description
Attention(ModelArgs args)	Constructor.

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor x, int startPos, Tensor cis, Tensor mask)	Forward pass through the attention module.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Attention

public Attention(ModelArgs args)

Constructor.

Parameters:

args - the model configuration parameters.

Method Details

forward

public Tensor forward(Tensor x,
 int startPos,
 Tensor cis,
 Tensor mask)

Forward pass through the attention module.

Parameters:

x - the input tensor.

startPos - the starting position for attention caching.

cis - the precomputed frequency tensor.

mask - the attention mask tensor.

Returns:

the output tensor.