Sign in
Large Language Models

Update an LLM

Updates an existing LLM's configuration. This endpoint supports partial updates. Provide only the fields you want to change. Only the name field is immutable.

The platform tests the updated LLM before saving it to ensure the credentials are valid.

Updatable fields:

  • description - LLM description
  • type - LLM type (openai-compatible, vertex-ai, etc.)
  • model - Model identifier
  • uri - API endpoint
  • auth - Authentication credentials (including service account key_json)
  • headers - Additional HTTP headers (for openai-compatible and anthropic types)
  • enabled - Whether the LLM is enabled
  • capabilities - Model capabilities (image support, context limit, tool calling)

Immutable fields:

  • id - System-generated identifier
  • name - LLM name

Built-in LLMs (system-provided models) cannot be updated.

patch/v2/llms/{llm_id}

Path parameters

llm_idstring required

The ID of the LLM to update.

Headers

Request-Timeoutinteger

The platform makes a best effort to complete the request in the specified seconds, or it times out.

Request-Timeout-Millisinteger

The platform makes a best effort to complete the request in the specified milliseconds, or it times out.

Request body

OR
OR
OR

Example request

{
  "auth": {
    "type": "bearer",
    "token": "abcdef......"
  },
  "idle_timeout_seconds": 300,
  "test_model_parameters": {
    "max_tokens": 512
  }
}

Response

The updated LLM.

idstring required

The ID of the LLM.

namestring required

Name of the LLM.

descriptionstring

The description of the LLM.

enabledboolean

Indicates whether the LLM is enabled.

defaultboolean

Whether this is the default LLM. Queries use the default LLM when they do not specify a generator.

ownership'platform' | 'customer'

Indicates whether the LLM is provided by the platform or created by the customer. Platform LLMs are pre-configured and cannot be modified or deleted. Customer LLMs are created and managed by the customer.

type'openai-compatible' | 'openai-responses' | 'vertex-ai' | 'anthropic' required

The LLM provider type, identifying which authentication and configuration apply. New values may be added as providers are onboarded; clients should ignore values they do not recognize rather than fail.

modelstring

Provider-specific model identifier (e.g. gpt-4o, claude-3-5-sonnet-20241022, gemini-2.5-flash).

uristring uri

The API endpoint URI configured for this LLM.

headersobject

Additional HTTP headers configured for requests to the LLM API. Not applicable to vertex-ai.

idle_timeout_secondsinteger nullable

The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.

requests_per_secondinteger

The maximum number of requests per second for this LLM. The platform omits this field when the LLM has no limit.

Example response

{
  "id": "llm_1021844",
  "name": "Claude 3.7 Sonnet",
  "description": "The Anthropic Claude 3.7 Sonnet model.",
  "ownership": "platform",
  "type": "openai-compatible",
  "idle_timeout_seconds": 300,
  "auth": {
    "type": "bearer",
    "token": "abcdef......"
  }
}

Changes

Changed in 2 of the 7 revisions of this API.110

  • 0d51a9e3a0b315See the full diff
    • ▲

      the response's property type changed from string to no type for status

      response-property-type-changed

    • ○

      added to the response property allOf list for the response status

      response-property-all-of-added

    • ○

      removed the anthropic enum value from the response property for the response status

      response-property-enum-value-removed

    • ○

      removed the openai-compatible enum value from the response property for the response status

      response-property-enum-value-removed

    • ○

      removed the openai-responses enum value from the response property for the response status

      response-property-enum-value-removed

    • ○

      removed the vertex-ai enum value from the response property for the response status

      response-property-enum-value-removed

    This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • fca567a46b3a5See the full diff
    • ○

      added the new optional request property ////

      new-optional-request-property

    • ○

      added the new optional request property ////////

      new-optional-request-property

    • ○

      added the new optional request property ////////

      new-optional-request-property

    • ○

      added the new optional request property ////

      new-optional-request-property

    • ○

      added the optional property to the response with the status

      response-optional-property-added