cn.smartjavaai.translation.entity

类 BatchTensorList

java.lang.Object

cn.smartjavaai.translation.entity.BatchTensorList

直接已知子类:

GreedyBatchTensorList

public abstract class BatchTensorList
extends Object

BatchTensorList represents a search state, and the NDArrays inside are updated in each iteration of the autoregressive loop.

It is a struct consisting of NDArrays, whose first dimension is batch, and also contains sequence dimension (whose position in tensor's shape is specified by seqDimOrder). The SeqBatcher batch operations will operate on these two dimensions.

方法概要

限定符和类型	方法和说明
abstract BatchTensorList	fromList(ai.djl.ndarray.NDList inputList, long[] seqDimOrder) Constructs a new BatchTensorList instance from the serialized version of the batch tensors.
abstract ai.djl.ndarray.NDList	getList() Returns the serialized version of the BatchTensorList.
ai.djl.ndarray.NDArray	getPastAttentionMask() Returns the value of the pastAttentionMask.
ai.djl.ndarray.NDList	getPastKeyValues() Returns the value of the pastKeyValues.
ai.djl.ndarray.NDArray	getPastOutputIds() Returns the value of the pastOutputIds.
long[]	getSeqDimOrder() Returns the sequence dimension order which specifies where the sequence dimension is in a tensor's shape.
void	setPastAttentionMask(ai.djl.ndarray.NDArray pastAttentionMask) Sets the attention mask.
void	setPastKeyValues(ai.djl.ndarray.NDList pastKeyValues) Sets the kv cache.
void	setPastOutputIds(ai.djl.ndarray.NDArray pastOutputIds) Sets the past output token ids.
void	setSeqDimOrder(long[] seqDimOrder) Sets the sequence dimension order which specifies where the sequence dimension is in a tensor's shape.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

方法详细资料

fromList

public abstract BatchTensorList fromList(ai.djl.ndarray.NDList inputList,
                                         long[] seqDimOrder)

Constructs a new BatchTensorList instance from the serialized version of the batch tensors.

The pastOutputIds has to be the first in the output list.

参数:

inputList - the serialized version of the batch tensors

seqDimOrder - the sequence dimension order that specifies where the sequence dimension is in a tensor's shape

返回:

BatchTensorList

getList

public abstract ai.djl.ndarray.NDList getList()

Returns the serialized version of the BatchTensorList. The pastOutputIds has to be the first in the output list.

返回:

the NDList that contains the serialized BatchTensorList

getSeqDimOrder

public long[] getSeqDimOrder()

Returns the sequence dimension order which specifies where the sequence dimension is in a tensor's shape.

返回:

the sequence dimension order which specifies where the sequence dimension is in a tensor's shape

getPastOutputIds

public ai.djl.ndarray.NDArray getPastOutputIds()

Returns the value of the pastOutputIds.

返回:

the value of pastOutputIds

setPastOutputIds

public void setPastOutputIds(ai.djl.ndarray.NDArray pastOutputIds)

Sets the past output token ids.

参数:

pastOutputIds - the past output token ids

getPastAttentionMask

public ai.djl.ndarray.NDArray getPastAttentionMask()

Returns the value of the pastAttentionMask.

返回:

the value of pastAttentionMask

setPastAttentionMask

public void setPastAttentionMask(ai.djl.ndarray.NDArray pastAttentionMask)

Sets the attention mask.

参数:

pastAttentionMask - the attention mask

getPastKeyValues

public ai.djl.ndarray.NDList getPastKeyValues()

Returns the value of the pastKeyValues.

返回:

the value of pastKeyValues

setPastKeyValues

public void setPastKeyValues(ai.djl.ndarray.NDList pastKeyValues)

Sets the kv cache.

参数:

pastKeyValues - the kv cache

setSeqDimOrder

public void setSeqDimOrder(long[] seqDimOrder)

Sets the sequence dimension order which specifies where the sequence dimension is in a tensor's shape.

参数:

seqDimOrder - the sequence dimension order which specifies where the sequence dimension is in a tensor's shape