---
title: "Generate Chat Completions"
method: POST
path: "/api/v3/inference/chat/completions"
tags: ["Inference"]
---

# Generate Chat Completions

`POST /api/v3/inference/chat/completions`

## Request

- Base URL: `https://api.nugen.in`
- URL: `https://api.nugen.in/api/v3/inference/chat/completions`
- Auth: HTTP bearer

Generate conversational responses with streaming support and multimodal capabilities.


This endpoint generates chat completions using language models, supporting both text-only conversations and vision inputs. Includes support for function calling, streaming responses, and automatic conversation tracking.


**Request Body:**

- `model`: Model ID (required). Use your aligned model id (e.g. `model_01kmqm4nrn9fw6r`); base text models (e.g. `model_01kmqm4nrn9fw6r`) also work if you have not aligned a model yet. For reserved capacity pass `ptm_id:model_id`
- `messages`: Array of message objects (required, minimum 1), each containing:
  - `role`: Message role (`system`, `user`, or `assistant`)
  - `content`: Text string OR array of content objects (for multimodal)
    - Text: `{"type": "text", "text": "your message"}`
    - Image URL: `{"type": "image_url", "image_url": {"url": "https://..."}}`
    - Image Base64: `{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}`
  - `name` (optional): Author name (a-z, A-Z, 0-9, underscores, max 64 chars)
- `max_tokens` (optional): Maximum tokens to generate in completion
- `prompt_truncate_len` (optional): Size to truncate chat prompts (default: 1500)
- `temperature` (optional): Sampling temperature between 0 and 2 (default: 1)
- `stream` (optional): Stream the response as server-sent events (default: `true`). Set `stream: false` for one JSON response.
  > **Tip:** For prompts where you expect a long output (e.g., `max_tokens > 500`), keep streaming on to avoid HTTP 504 gateway timeouts on slow or large responses.
- `tools` (optional): List of tools/functions available to the model
- `tool_choice` (optional): Tool selection mode (`auto`, `none`, or specific tool)
- `top_p` (optional): Nucleus sampling parameter
- `top_k` (optional): Top-k sampling parameter
- `n` (optional): Number of completions to generate (default: 1)
- `reasoning` (optional): Reasoning configuration object with:
  - `effort`: Reasoning effort level (`xhigh`, `high`, `medium`, `low`, `minimal`, `none`)
  - `max_tokens`: Token limit for reasoning
  - `exclude`: Set true to exclude reasoning tokens from response
  - `enabled`: Enable reasoning with default parameters


**Optional Headers:**

- `X-Session-ID`: Session identifier for multi-turn conversation tracking


**Returns:**

**Non-streaming mode** -
- `id`: Unique identifier for the response
- `object`: Object type (always `chat.completion`)
- `created`: Unix timestamp when completion was created
- `model`: Model used for chat completion
- `choices`: List of completion choices, each containing:
  - `index`: Index of the choice
  - `message`: Response message with:
    - `role`: Role of the author (always `assistant`)
    - `content`: Generated response text
    - `tool_calls` (optional): Tool calls made by the model (for function calling)
  - `finish_reason`: Reason model stopped (`stop` for natural stop, `length` if max tokens reached)
- `usage`: Token usage statistics:
  - `prompt_tokens`: Number of tokens in the prompt
  - `completion_tokens`: Number of tokens generated
  - `total_tokens`: Total tokens used (prompt + completion)
- `confidence_score` (optional): Confidence score, 0-100, on aligned models; null on base models

**Streaming mode** - `ChatCompletionChunk` stream with:
- `id`: Unique identifier
- `created`: Timestamp
- `model`: Model ID
- `choices`: List of chunk choices with:
  - `index`: ChatCompletionChoice index
  - `delta`: Delta content with `role` and `content`
  - `finish_reason`: Reason for stopping (only in final chunk)
- `usage` (optional): Only present in final chunk


**Example Request (Text Chat):**

```json
POST /api/v3/inference/chat/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ],
  "max_tokens": 500,
  "temperature": 0.7,
  "stream": false
}
```


**Example Response (Text Chat):**

```json
{
  "id": "nugen-abc123",
  "object": "chat.completion",
  "created": 1704123600.0,
  "model": "model_01kmqm4nrn9fw6r",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 8,
    "total_tokens": 28
  },
  "confidence_score": 89.5454
}
```


**Example Request (Using Domain-Aligned Model):**

```json
POST /api/v3/inference/chat/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "messages": [
    {
      "role": "user",
      "content": "How do I return a product?"
    }
  ],
  "max_tokens": 500,
  "temperature": 0.7,
  "stream": false
}
```


**Example Response (Domain-Aligned Model):**

```json
{
  "id": "nugen-abc456",
  "object": "chat.completion",
  "created": 1704123700.0,
  "model": "model_01kmqm4nrn9fw6r",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "To return a product, please visit our returns portal within 30 days of purchase with your order number and receipt."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 25,
    "total_tokens": 40
  },
  "confidence_score": 95.8
}
```

**How to get your aligned model ID:**
1. Create an alignment project using `POST /api/v3/alignment-projects/create`
2. Poll `GET /api/v3/alignment-projects/{id}/status` until status is `READY`
3. Use the `model_id` field from the response (e.g., `"model_01kmqm4nrn9fw6r"`)
4. Deploy the model using `POST /api/v3/models/{model_id}/deployment` (optional - for evaluation/production use)
5. Use the `model_id` in the `model` parameter for inference requests


**Example Request (Vision - Image URL):**

```json
POST /api/v3/inference/chat/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Describe what you see in this image."
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/image.jpg"
          }
        }
      ]
    }
  ],
  "max_tokens": 1000,
  "stream": false
}
```


**Example Request (Vision - Base64 Image):**

```json
POST /api/v3/inference/chat/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What's in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD..."
          }
        }
      ]
    }
  ],
  "stream": false
}
```


**Example Request (Function Calling):**

```json
POST /api/v3/inference/chat/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "messages": [
    {"role": "user", "content": "What is the status of INC-4821?"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_incident",
        "description": "Get incident status",
        "parameters": {
          "type": "object",
          "properties": {
            "incident_id": {"type": "string"}
          }
        }
      }
    }
  ],
  "tool_choice": "auto",
  "stream": false
}
```


**Example Request (Streaming):**

```json
POST /api/v3/inference/chat/completions
Headers: {"Authorization": "Bearer <api_key>"}

{
  "model": "model_01kmqm4nrn9fw6r",
  "messages": [{"role": "user", "content": "Tell me a story"}],
  "stream": true
}
```


**Example Response (Streaming):**

```
data: {"id":"nugen-abc123","created":1704123600.0,"model":"model_01kmqm4nrn9fw6r","choices":[{"index":0,"delta":{"role":"assistant","content":"Once"},"finish_reason":null}]}

data: {"id":"nugen-abc123","created":1704123600.0,"model":"model_01kmqm4nrn9fw6r","choices":[{"index":0,"delta":{"content":" upon"},"finish_reason":null}]}

data: {"id":"nugen-abc123","created":1704123600.0,"model":"model_01kmqm4nrn9fw6r","choices":[{"index":0,"delta":{"content":" a time"},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15}}

data: [DONE]
```


**Provisioned throughput:**

Reserve inference capacity for production workloads. Pass your provisioned id in `model` as `ptm_id:model_id` in place of the plain `model_id`. The request and response surface is identical, so your application code stays as it is once capacity is reserved. Available for aligned models on the Enterprise Edition.

**Notes:**

- Supports both text-only and vision/multimodal inputs depending on the model
- **Using Domain-Aligned Models**: Pass the `model_id` from your completed alignment project as the `model` parameter. Get your aligned model IDs from `GET /api/v3/alignment-projects/{id}/status` or `GET /api/v3/models/aligned`
- Function calling enables models to invoke external tools via `tools` and `tool_choice` parameters
- `confidence_score` (0-100) comes from inference-time alignment: uncertainty is quantified across the generation trajectory, flagging any drift from your domain. Available on aligned models only; always null for base models. In streaming mode, confidence arrives as its own `{"object": "confidence", ...}` events interleaved with content chunks, one per generated span, rather than a single value on the final chunk
- For streaming, usage statistics are included only in the final chunk
- Conversations are automatically saved to S3 if enabled in user settings
- Include `X-Session-ID` header for multi-turn conversation tracking
- Use `prompt_truncate_len` to control context window usage for long conversations
- The `reasoning` parameter enables advanced reasoning capabilities for supported models

## Headers

- `X-Session-ID` string, nullable: Session identifier for multi-turn conversation tracking

## Request body

- ChatCompletionRequest: A chat completion request in the OpenAI format.
  - `model` string, required: Use your aligned model id (e.g. model_01kmqm4nrn9fw6r). The same aligned id serves chat completions, completions, embeddings and reranking.
  - `messages` ChatCompletionRequestMessage[], required: A list of messages comprising the conversation so far.
    - `role` 'system' | 'user' | 'assistant', required: The role of the messages author. One of `system`, `user`, or `assistant`.
    - `content` union, required: The contents of the message. `content` is required for all messages
      - string
      - object[]
    - `name` string, nullable: The name of the author of this message. May contain a-z, A-Z, 0-9, and underscores, with a maximum length of 64 characters.
  - `max_tokens` integer, nullable: The maximum number of tokens to generate in the completion.
  - `prompt_truncate_len` integer, nullable: The size to which to truncate chat prompts.
  - `temperature` number, nullable: What sampling temperature to use, between 0 and 2.
  - `stream` boolean, nullable: Stream partial progress as server-sent events. Defaults to true. Set false for one JSON response.
  - `tools` object[], nullable: List of tools/functions
  - `tool_choice` union: 'auto', 'none', or specific tool
    - string
    - object
  - `top_p` number, nullable: Nucleus sampling
  - `top_k` integer, nullable: Top-k sampling
  - `n` integer, nullable: Number of completions
  - `reasoning` ChatCompletionReasoning: Reasoning settings of a chat completion: whether reasoning was on, its token budget and its effort level.
    - `effort` 'xhigh' | 'high' | 'medium' | 'low' | 'minimal' | 'none', nullable: Reasoning effort level (OpenAI-style)
    - `max_tokens` integer, nullable: Specific token limit for reasoning (Anthropic-style)
    - `exclude` boolean, nullable: Set to true to exclude reasoning tokens from response
    - `enabled` boolean, nullable: Enable reasoning with default parameters
  - `stream_options` object, nullable: Options for streaming responses, e.g., {'include_usage': true}

## Response `200`

Streaming chat completion responses or complete response depending on stream parameter

- ChatCompletionResponse: A chat completion response in the OpenAI format, with a confidence score on aligned models.
  - `id` string, required: A unique identifier of the response.
  - `object` 'chat.completion': The object type, which is always "chat.completion".
  - `created` number, required: The Unix time in seconds when the response was generated.
  - `model` string, required: The model used for the chat completion.
  - `choices` ChatCompletionChoice[], required: The list of chat completion choices.
    - `index` integer, required: The index of the chat completion choice.
    - `message` ChatCompletionResponseMessage, required: The message the model generated.
      - `role` 'system' | 'user' | 'assistant', required: The role of the author of this message.
      - `content` string, nullable: The contents of the message.
    - `finish_reason` 'stop' | 'length', required: The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.
  - `confidence_score` number, nullable, required: Confidence Score from Domain-Aligned AI
  - `usage` ChatCompletionUsage: Token counts for a chat completion.
    - `prompt_tokens` integer, required: The number of tokens in the prompt.
    - `completion_tokens` integer, required: The number of tokens in the generated completion.
    - `total_tokens` integer, required: The total number of tokens used in the request (prompt + completion).

## Other responses

- `401`: Invalid API key
- `403`: Authorization header missing or not a Bearer token
- `422`: Validation Error

---

[API](https://skmtc.dev/nugen/apis/nugen-intelligence-api.md) - [All operations](https://skmtc.dev/nugen/apis/nugen-intelligence-api/llms.txt) - [OpenAPI document](https://skmtc.dev/nugen/apis/nugen-intelligence-api/revisions/62baa7960a4a?raw)