MImistral
batch

Create Batch Job

Changed on

Create a new batch job, it will be queued for processing.

post/v1/batch/jobs

Request

Request body

input_filesstring[] nullable

A list of .jsonl files for batch inference. Each line must be a JSON object with a body field containing the request payload:

{"custom_id": "0", "body": {"max_tokens": 100, "messages": [{"role": "user", "content": "What is the best French cheese?"}]}}
{"custom_id": "1", "body": {"max_tokens": 100, "messages": [{"role": "user", "content": "What is the best French wine?"}]}}
endpoint'/v1/chat/completions' | '/v1/embeddings' | '/v1/fim/completions' | '/v1/moderations' | '/v1/chat/moderations' | '/v1/ocr' | '/v1/classifications' | '/v1/chat/classifications' | '/v1/conversations' | '/v1/audio/transcriptions' required
modelstring nullable

The model to be used for batch inference.

agent_idstring nullable

In case you want to use a specific agent from the deprecated agents api for batch inference, you can specify the agent ID here.

metadataobject nullable

The metadata of your choice to be associated with the batch inference job.

timeout_hoursinteger

The timeout in hours for the batch inference job.

Example request

{
  "model": "mistral-small-latest"
}

Response

OK

idstring required
object'batch'
input_filesstring[] required
metadataobject nullable
endpointstring required
modelstring nullable
agent_idstring nullable
output_filestring uuid nullable
error_filestring uuid nullable
outputsobject[] nullable
status'QUEUED' | 'RUNNING' | 'SUCCESS' | 'FAILED' | 'TIMEOUT_EXCEEDED' | 'CANCELLATION_REQUESTED' | 'CANCELLED' required
created_atinteger required
total_requestsinteger required
completed_requestsinteger required
succeeded_requestsinteger required
failed_requestsinteger required
started_atinteger nullable
completed_atinteger nullable

Changes

  • 933c4ebdcd72Revision changes
    • warning

      the request property's max was set to 168.00

    • warning

      the request property's min was set to 1.00

  • ff8a7389b7a4Revision changes
    • info

      the response property const value batch was added for the status

    • info

      removed the batch enum value from the response property for the response status