Sign in
model

Run a model with specified tools and messages.

This endpoint runs a model with the specified tools and messages.

post/api/v1/{model_id}/run

Path parameters

model_idstring required

The ID of the model to run.

Query parameters

waitboolean

Whether to wait for the response or not.

Request body

modelstring required
streamboolean required
temperaturenumber nullable

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or temperature but not both.

top_pnumber nullable

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or temperature but not both.

ninteger nullable

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

max_tokensinteger nullable

The maximum number of tokens that can be generated in the chat completion.

The total length of input tokens and generated tokens is limited by the model's context length. Example Python code for counting tokens.

Example request

{
  "temperature": 1,
  "top_p": 1,
  "n": 1
}

Response

Successful response

idstring
objectstring
createdinteger
modelstring

Changes