@elizaos/core v1.0.0-beta.51 / TokenizeTextParams
Interface: TokenizeTextParams
Parameters for tokenizing text, i.e., converting a string into a sequence of numerical tokens.
This is a common preprocessing step for many language models.
This structure is used with AgentRuntime.useModel when the modelType is ModelType.TEXT_TOKENIZER_ENCODE.
Extends​
Properties​
prompt​
prompt:
string
The input string to be tokenized.
Defined in​
packages/core/src/types.ts:1353
modelType​
modelType:
string
The model type to use for tokenization, which determines the tokenizer algorithm and vocabulary.
Defined in​
packages/core/src/types.ts:1355
runtime​
runtime:
IAgentRuntime
The agent runtime for accessing services and utilities