Sign in
Large Language Models

List LLMs

Lists LLMs that you can use with query and chat endpoints. A query does not specify the LLM directly. Instead, it uses a generation_preset_name. The generation_preset_name in generation parameters corresponds to the name property of the generation presets that /v2/generation_presets lists.

get/v2/llms

Query parameters

filterstring

A regular expression to match names and descriptions of the LLMs.

limitinteger

The maximum number of results to return in the list.

page_keystring

The opaque cursor returned in metadata.page_key of the previous response. Omit it to fetch the first page.

Headers

Request-Timeoutinteger

The platform makes a best effort to complete the request in the specified seconds, or it times out.

Request-Timeout-Millisinteger

The platform makes a best effort to complete the request in the specified milliseconds, or it times out.

Response

List of LLMs.

Example response

{
  "llms": [
    {
      "id": "llm_1021844",
      "name": "Claude 3.7 Sonnet",
      "description": "The Anthropic Claude 3.7 Sonnet model.",
      "ownership": "platform",
      "type": "openai-compatible",
      "idle_timeout_seconds": 300,
      "auth": {
        "type": "bearer",
        "token": "abcdef......"
      }
    }
  ],
  "metadata": {
    "page_key": "eyJvZmZzZXQiOjF9"
  }
}

Changes

Changed in 2 of the 7 revisions of this API.16

  • 0d51a9e3a0b315See the full diff
    • ▲

      the // response's property type changed from string to no type for status

      response-property-type-changed

    • ○

      added to the // response property allOf list for the response status

      response-property-all-of-added

    • ○

      removed the anthropic enum value from the // response property for the response status

      response-property-enum-value-removed

    • ○

      removed the openai-compatible enum value from the // response property for the response status

      response-property-enum-value-removed

    • ○

      removed the openai-responses enum value from the // response property for the response status

      response-property-enum-value-removed

    • ○

      removed the vertex-ai enum value from the // response property for the response status

      response-property-enum-value-removed

    This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • fca567a46b3a1See the full diff
    • ○

      added the optional property // to the response with the status

      response-optional-property-added