Tokenization

By giving a text input, generate a tokenized output of token IDs.

post/serverless/v1/tokenize

Request

Base URL
https://api.friendli.ai
Auth
HTTP bearer
TypeScript
https://skmtc.dev/friendli/apis/friendli-suite-api-reference/docs/serverless/v1/tokenize/post.ts
A function with typed inputs. Its response is checked against the schema.

X-Friendli-Teamstring nullable

ID of team to run requests as (optional parameter).

modelstring required

Code of the model to use. See available model list.

promptstring required

Input text prompt to tokenize.

{
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "prompt": "What is generative AI?"
}

Successfully tokenized the text.

tokensinteger[] required

A list of token IDs.

No recorded changes to this operation across all 1 revision of this API.