service_tier'default' | 'priority' | 'flex'
fail_fastboolean
If true, the request is rejected immediately with HTTP 429 when the model has no spare capacity, instead of waiting in the queue. Opt-in; the default (false) keeps standard queueing behavior.
modelsstring[] nullable
Ordered list of up to 4 fallback models. The request is attempted on each model in order: when a model rejects it for lack of capacity (HTTP 429 model-busy / flex no-capacity), the next model is tried server-side. The first model that accepts serves the request; the response's model field and billing reflect that model, at that model's pricing. Models before the last are attempted without queueing (as if fail_fast were set); the last model honors the request's own fail_fast value. When models is set, the model field is ignored. Entries must be plain model names (no deploy_id:, custom_hostport, or :revision specifiers); duplicate entries are ignored, keeping the first occurrence.
instructionsstring nullable
max_output_tokensinteger nullable
max_tool_callsinteger nullable
temperaturenumber nullable
top_logprobsinteger nullable
parallel_tool_callsboolean nullable
previous_response_idstring nullable
truncation'auto' | 'disabled' nullable
repetition_penaltynumber nullable
stop_token_idsinteger[] nullable
chat_template_kwargsobject nullable
continue_final_messageboolean nullable
ignore_eosboolean nullable
prompt_cache_keystring nullable