---
title: "OpenAI-compatible chat endpoint"
method: POST
path: "/v1/chat/completions"
---

# OpenAI-compatible chat endpoint

`POST /v1/chat/completions`

## Request

- The document declares no server URL.
- Auth: none declared
- TypeScript: <https://skmtc.dev/nvidia/apis/nvidia-nim-for-vlms/docs/v1/chat/completions/post.ts> (a function with typed inputs; its response is checked against the schema)

## Headers

- `content-type` string, nullable

## Request body

- NIMLLMChatCompletionRequest
  - `messages` NIMLLMChatCompletionMessage[], required: A list of messages comprising the conversation so far.
    - `role` 'assistant' | 'user' | 'system' | 'tool' | 'function', required
    - `content` union, required: The contents of the message. <br>To pass images (only with role=`user`): <br> - When content is a string, image can be passed together with the text with `img` HTML tags that wraps an image URL (`<img src="{url}" />`), base64 encoded image data (`<img src="data:image/{format};base64,{base64encodedimage}" />`), or an NVCF asset ID (`<img src="data:image/{format};asset_id,{asset_id}" />`) when the container is hosted in NVCF and the payload exceeds 200KB. or http link (`https:{image_link}`) <br> - When content is a list of objects, images can be passed as objects with type=`image_url`. <br> - In both cases, images can be PNG, JPG or JPEG. <br>To pass videos (only with role=`user`): <br> - When content is a string, video can be passed together with the text with `video` HTML tags that wraps a video URL (`<video src="{url}" />`), base64 encoded video data (`video src="data:video/{format};base64,{base64encodedvideo}" />`), or an NVCF asset ID (`<video src="data:video/{format};asset_id,{asset_id}" />`), or http link (`https:{video_link}`) when the container is hosted in NVCF and the payload exceeds 200KB. <br> - When content is a list of objects, images can be passed as objects with type=`video_url` or `rtsp`. For rtsp streams, an extra parameter `duration` needs to be set as the length of the stream in seconds to be sampled. <br> - In both cases, videos can be H.264, H.265, AV1, JPEG and MJPEG formats. <br>For `system` and `assistant` roles, the object list format is not supported.
      - string
      - union[]
        - union
          - NIMVLMChatCompletionContentPartImage
            - `image` string, required
            - `type` 'image', required
          - NIMVLMChatCompletionContentPartText
            - `text` string, required
            - `type` 'text', required
          - NIMVLMChatCompletionContentPartVideo
            - `video` string
            - `start_offset` number, nullable
            - `duration` number, nullable
            - `type` 'video'
          - NIMVLMChatCompletionContentPartVideoStreaming
            - `video_url` VideoURL (unresolved $ref)
            - `connect_timeout` number, nullable
            - `duration` number
            - `fps` number, nullable
            - `type` 'rtsp'
    - `tool_call_id` string, nullable: The id of the tool call.
    - `name` string, nullable: The name of the tool called.
    - `tool_calls` ToolCall[], nullable: The tool(s) called by the model.
      - `index` integer, nullable
      - `id` string
      - `type` 'function'
      - `function` FunctionCall, required
        - `name` string, nullable
        - `arguments` string, nullable
  - `model` string, required: The model to use.
  - `frequency_penalty` number, nullable: Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
  - `logit_bias` object, nullable: Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect varies based on the model, but values between -1 and 1 should decrease or increase likelihood of selection; values at the ends of the range, such as -100 or 100 should result in a ban or exclusive selection of the relevant token.
  - `logprobs` boolean, nullable: Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the `content` of `message`.
  - `top_logprobs` integer, nullable: An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. `logprobs` must be set to `true` if this parameter is used.
  - `max_tokens` integer, nullable: The maximum number of tokens that can be generated.
  - `n` integer, nullable: How many completions to generate for each prompt.
  - `presence_penalty` number, nullable: Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
  - `response_format` ResponseFormat
    - `type` 'text' | 'json_object', required: Must be one of `text` or `json_object`.
  - `seed` integer, nullable: Changing the seed produces a different response with similar characteristics. Fixing the seed reproduces the same results if all other parameters are also kept constant.
  - `stop` union: Sequences where the API will stop generating further tokens.
    - string
    - string[]
  - `stream` boolean, nullable: If set, partial message deltas are sent, like in ChatGPT. Tokens are sent as data-only [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Event_stream_format) as they become available, with the stream terminated by a `data: [DONE]`
  - `stream_options` StreamOptions
    - `include_usage` boolean, nullable
    - `continuous_usage_stats` boolean, nullable
  - `temperature` number, nullable: What sampling temperature to use, between 0 and 2. Higher values such as 0.8 make the output more random, while lower values such as 0.2 make it more focused and deterministic.
  - `top_p` number, nullable: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with `top_p` probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. NVIDIA recommends that you alter this option or `temperature` but not both.
  - `tools` ChatCompletionToolsParam[], nullable
    - `type` 'function'
    - `function` FunctionDefinition, required
      - `name` string, required
      - `description` string, nullable
      - `parameters` object, nullable
  - `tool_choice` union: Controls which (if any) tool is called by the model.
    - 'none' | 'auto' | 'required'
    - ChatCompletionNamedToolChoiceParam
      - `function` ChatCompletionNamedFunction, required
        - `name` string, required
      - `type` 'function'
  - `parallel_tool_calls` boolean, nullable
  - `user` string, nullable
  - `min_p` number
  - `prompt_logprobs` integer, nullable
  - `echo` boolean, nullable: This is currently unsupported. If true, the previous message is emitted again before the current message if they belong to the same role.
  - `add_special_tokens` boolean: If true, special tokens (e.g. BOS) will be added to the prompt on top of what is added by the chat template. For most models, the chat template takes care of adding the special tokens so this should be set to false (as is the default).
  - `documents` object[], nullable: A list of dicts representing documents that will be accessible to the model if it is performing RAG (retrieval-augmented generation). If the template does not support RAG, this argument will have no effect. We recommend that each document should be a dict containing "title" and "text" keys.
  - `chat_template` string, nullable: A Jinja template to use for this conversion. As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one.
  - `chat_template_kwargs` object, nullable: Additional kwargs to pass to the template renderer. Will be accessible by the chat template.
  - `guided_whitespace_pattern` string, nullable: If specified, will override the default whitespace pattern for guided json decoding.
  - `image` string, nullable: The image as a string or null.
  - `nvext` NVExt
    - `ignore_eos` boolean, nullable: Whether to ignore End of Sequence (EOS) tokens.
    - `repetition_penalty` number, nullable: How much to penalize tokens based on how frequently they occur in the text. A value of 1 means no penalty, while values larger than 1 discourage and values smaller encourage.
    - `top_k` integer: How many of the top tokens to sample from. Must be -1 or greater than or equal to 1, and cannot be null. If not set, then the default is -1 which disables `top_k` sampling (greedy).
    - `guided_choice` string[], nullable: If specified, the output is exactly one of the choices.
    - `guided_json` union: If specified, the output follows the JSON schema. Can be a string, an object, a reference to a BaseModel, or null.
      - string
      - object
      - BaseModel
    - `guided_regex` string, nullable: If specified, the output follows the regex pattern. Can be a string or null.
    - `guided_grammar` string, nullable: If specified, the output follows the context-free grammar. Can be a string or null.
  - `num_frames_per_inference` integer, nullable: Number of frames to sample from the video or stream. They will be the input to model.
  - `media_response_type` 'none' | 'image' | 'video', nullable: The return format if there are media file outputs such as images or videos. `none` means to not return these files; `base64` means to return them as base64 string in response `content`, such as `The response content is... <img src="data:image/png;base64,{image_b64}" />`; `nvcf` means to store them in NVCF large storage directory

## Response `200`

Successful Response

- union
  - NIMLLMChatCompletionResponse
    - `id` string
    - `object` 'chat.completion'
    - `created` integer
    - `model` string, required
    - `choices` ChatCompletionResponseChoice[], required
      - `index` integer, required
      - `message` ChatMessage, required
        - `role` string, required
        - `content` string, nullable
        - `tool_calls` ToolCall[], nullable
          - `index` integer, nullable
          - `id` string
          - `type` 'function'
          - `function` FunctionCall, required
            - `name` string, nullable
            - `arguments` string, nullable
      - `logprobs` ChatCompletionLogProbs
        - `content` ChatCompletionLogProbsContent[], nullable
          - `token` string, required
          - `logprob` number
          - `bytes` integer[], nullable
          - `top_logprobs` ChatCompletionLogProb[]
            - `token` string, required
            - `logprob` number
            - `bytes` integer[], nullable
      - `finish_reason` 'stop' | 'length' | 'tool_calls', nullable
      - `stop_reason` union
        - integer
        - string
    - `usage` UsageInfo, required
      - `prompt_tokens` integer
      - `total_tokens` integer
      - `completion_tokens` integer, nullable
    - `prompt_logprobs` object[], nullable
    - `data` NIMLLMMediaObject[], nullable: If specified, returns the media data generated by chat completion.
      - `type` string, required: The media type, such as image/jpeg, video/mp4.
      - `b64_json` string, nullable: The base64-encoded JSON of the generated media.
      - `url` string, nullable: The URL of the generated media, if response_format is url (default).
      - `revised_prompt` string, nullable: The prompt that was used to generate the media, if there was any revision to the prompt.
  - NIMLLMChatCompletionStreamResponse
    - `id` string
    - `object` 'chat.completion.chunk'
    - `created` integer
    - `model` string, required
    - `choices` ChatCompletionResponseStreamChoice[], required
      - `index` integer, required
      - `delta` DeltaMessage, required
        - `role` string, nullable
        - `content` string, nullable
        - `tool_calls` ToolCall[], nullable
          - `index` integer, nullable
          - `id` string
          - `type` 'function'
          - `function` FunctionCall, required
            - `name` string, nullable
            - `arguments` string, nullable
      - `logprobs` ChatCompletionLogProbs
        - `content` ChatCompletionLogProbsContent[], nullable
          - `token` string, required
          - `logprob` number
          - `bytes` integer[], nullable
          - `top_logprobs` ChatCompletionLogProb[]
            - `token` string, required
            - `logprob` number
            - `bytes` integer[], nullable
      - `finish_reason` 'stop' | 'length' | 'tool_calls', nullable
      - `stop_reason` union
        - integer
        - string
    - `usage` UsageInfo
      - `prompt_tokens` integer
      - `total_tokens` integer
      - `completion_tokens` integer, nullable
    - `data` NIMLLMMediaObject[], nullable: If specified, returns the media data generated by chat completion.
      - `type` string, required: The media type, such as image/jpeg, video/mp4.
      - `b64_json` string, nullable: The base64-encoded JSON of the generated media.
      - `url` string, nullable: The URL of the generated media, if response_format is url (default).
      - `revised_prompt` string, nullable: The prompt that was used to generate the media, if there was any revision to the prompt.

## Other responses

- `400`: Received an invalid request possibly containing unsupported or out-of-range parameter values.
- `404`: The requested model does not exist.
- `415`: Unsupported Media Type
- `422`: Validation Error
- `500`: Internal Server Error

## Changes

- **2025-05-17** `4227e796ee0b`: 2 breaking, 2 warning
  - the `messages/items/content/anyOf[subschema #2]/items/anyOf[subschema #3: NIMVLMChatCompletionContentPartVideo]/video` request property type changed from `object` to `string`
  - the `messages/items/content/anyOf[subschema #2]/items/anyOf[subschema #4: NIMVLMChatCompletionContentPartVideoStreaming]/video_url` request property type changed from `object` to no type
  - removed the request property `messages/items/content/anyOf[subschema #2]/items/anyOf[subschema #3: NIMVLMChatCompletionContentPartVideo]/video/url`
  - removed the request property `messages/items/content/anyOf[subschema #2]/items/anyOf[subschema #4: NIMVLMChatCompletionContentPartVideoStreaming]/video_url/url`
- **2025-05-17** `23cf55a1da8a`: 3 breaking, 2 warning, 1 info
  - added the new required request property `messages/items/content/anyOf[subschema #2]/items/anyOf[subschema #1: NIMVLMChatCompletionContentPartImage]/image`
  - the `type` request property const value changed from `image_url` to `image`
  - the `type` request property const value changed from `video_url` to `video`
  - removed the request property `messages/items/content/anyOf[subschema #2]/items/anyOf[subschema #1: NIMVLMChatCompletionContentPartImage]/image_url`
  - …2 more
- …earlier changes not shown

[Operation history](https://skmtc.dev/nvidia/apis/nvidia-nim-for-vlms/changes/v1/chat/completions/post.md)

---

[API](https://skmtc.dev/nvidia/apis/nvidia-nim-for-vlms.md) - [All operations](https://skmtc.dev/nvidia/apis/nvidia-nim-for-vlms/llms.txt) - [OpenAPI document](https://skmtc.dev/nvidia/apis/nvidia-nim-for-vlms/revisions/a7f196bb2b7e?raw)