@elizaos/core v1.0.6 / v2 / TokenizeTextParams
Interface: TokenizeTextParams
Parameters for tokenizing text, i.e., converting a string into a sequence of numerical tokens.
This is a common preprocessing step for many language models.
This structure is used with AgentRuntime.useModel when the modelType is ModelType.TEXT_TOKENIZER_ENCODE.
Properties​
prompt​
prompt:
string
The input string to be tokenized.
Defined in​
packages/core/src/specs/v2/types.ts:1402
modelType​
modelType:
string
The model type to use for tokenization, which determines the tokenizer algorithm and vocabulary.