Sign in
Large Language Models

Get an LLM

Returns the details of a Large Language Model (LLM) configured within the platform. The details include the name, description, model type, API endpoint, and authentication method.

Use this endpoint to verify model configurations and confirm connectivity details.

Authentication methods

The LLM's authentication is configured either as a Bearer token or as custom header-based authentication.

get/v2/llms/{llm_id}

Path parameters

llm_idstring required

The name of the LLM to retrieve.

Headers

Request-Timeoutinteger

The platform makes a best effort to complete the request in the specified seconds, or it times out.

Request-Timeout-Millisinteger

The platform makes a best effort to complete the request in the specified milliseconds, or it times out.

Response

The LLM details.

idstring required

The ID of the LLM.

namestring required

Name of the LLM.

descriptionstring

The description of the LLM.

enabledboolean

Indicates whether the LLM is enabled.

defaultboolean

Whether this is the default LLM. Queries use the default LLM when they do not specify a generator.

ownership'platform' | 'customer'

Indicates whether the LLM is provided by the platform or created by the customer. Platform LLMs are pre-configured and cannot be modified or deleted. Customer LLMs are created and managed by the customer.

type'openai-compatible' | 'openai-responses' | 'vertex-ai' | 'anthropic' required

The LLM provider type, identifying which authentication and configuration apply. New values may be added as providers are onboarded; clients should ignore values they do not recognize rather than fail.

modelstring

Provider-specific model identifier (e.g. gpt-4o, claude-3-5-sonnet-20241022, gemini-2.5-flash).

uristring uri

The API endpoint URI configured for this LLM.

headersobject

Additional HTTP headers configured for requests to the LLM API. Not applicable to vertex-ai.

idle_timeout_secondsinteger nullable

The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.

requests_per_secondinteger

The maximum number of requests per second for this LLM. The platform omits this field when the LLM has no limit.

input_tokens_per_minuteinteger

The maximum number of input (prompt) tokens per minute for this LLM, as reported by the provider and including cached prompt tokens. The platform omits this field when the LLM has no such limit.

output_tokens_per_minuteinteger

The maximum number of output tokens (completion plus reasoning) per minute for this LLM. The platform omits this field when the LLM has no such limit.

input_tokens_per_dayinteger

The maximum number of input tokens per UTC calendar day for this LLM. The platform omits this field when the LLM has no such limit.

output_tokens_per_dayinteger

The maximum number of output tokens per UTC calendar day for this LLM. The platform omits this field when the LLM has no such limit.

input_tokens_per_monthinteger

The maximum number of input tokens per UTC calendar month for this LLM. The platform omits this field when the LLM has no such limit.

output_tokens_per_monthinteger

The maximum number of output tokens per UTC calendar month for this LLM. The platform omits this field when the LLM has no such limit.

Example response

{
  "id": "llm_1021844",
  "name": "Claude 3.7 Sonnet",
  "description": "The Anthropic Claude 3.7 Sonnet model.",
  "ownership": "platform",
  "type": "openai-compatible",
  "idle_timeout_seconds": 300,
  "input_tokens_per_minute": 200000,
  "output_tokens_per_minute": 50000,
  "input_tokens_per_day": 2000000,
  "output_tokens_per_day": 500000,
  "input_tokens_per_month": 50000000,
  "output_tokens_per_month": 10000000,
  "auth": {
    "type": "bearer",
    "token": "abcdef......"
  }
}

Changes

Changed in 3 of the 13 revisions of this API.114

  • 3f4075adad3d8See the full diff
    • ○

      added the optional property ///// to the response with the status

      response-optional-property-added

    • ○

      added the optional property ///// to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

    • ○

      added the optional property to the response with the status

      response-optional-property-added

  • 0d51a9e3a0b315See the full diff
    • ▲

      the response's property type changed from string to no type for status

      response-property-type-changed

    • ○

      added to the response property allOf list for the response status

      response-property-all-of-added

    • ○

      removed the anthropic enum value from the response property for the response status

      response-property-enum-value-removed

    • ○

      removed the openai-compatible enum value from the response property for the response status

      response-property-enum-value-removed

    • ○

      removed the openai-responses enum value from the response property for the response status

      response-property-enum-value-removed

    • ○

      removed the vertex-ai enum value from the response property for the response status

      response-property-enum-value-removed

    This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • fca567a46b3a1See the full diff
    • ○

      added the optional property to the response with the status

      response-optional-property-added