Text Embeddings Inference

Tokenize inputs

post/tokenize

Request

The document declares no server URL.
Auth
none declared
TypeScript
https://skmtc.dev/oracle/apis/text-embeddings-inference/docs/tokenize/post.ts
A function with typed inputs. Its response is checked against the schema.

Request body

add_special_tokensboolean

prompt_namestring nullable

The name of the prompt that should be used by for encoding. If not set, no prompt will be applied.

Must be a key in the sentence-transformers configuration prompts dictionary.

For example if prompt_name is "query" and the prompts is {"query": "query: ", ...}, then the sentence "What is the capital of France?" will be encoded as "query: What is the capital of France?" because the prompt text will be prepended before any text to encode.

Example request

{
  "prompt_name": "null"
}

Response

Tokenized ids

Example response

[
  [
    {
      "id": 0,
      "special": false,
      "start": 0,
      "stop": 2,
      "text": "test"
    }
  ]
]

Changes

No recorded changes to this operation across all 1 revision of this API.