Skip to main content

@elizaos/core v1.0.0-beta.51 / TokenizeTextParams

Interface: TokenizeTextParams

Parameters for tokenizing text, i.e., converting a string into a sequence of numerical tokens. This is a common preprocessing step for many language models. This structure is used with AgentRuntime.useModel when the modelType is ModelType.TEXT_TOKENIZER_ENCODE.

Extends​

Properties​

prompt​

prompt: string

The input string to be tokenized.

Defined in​

packages/core/src/types.ts:1353


modelType​

modelType: string

The model type to use for tokenization, which determines the tokenizer algorithm and vocabulary.

Defined in​

packages/core/src/types.ts:1355


runtime​

runtime: IAgentRuntime

The agent runtime for accessing services and utilities

Inherited from​

BaseModelParams.runtime

Defined in​

packages/core/src/types.ts:1625