--- title: "Create eval run" method: POST path: "/evals/{eval_id}/runs" tags: ["Evals"] --- # Create eval run `POST /evals/{eval_id}/runs` ## Request - Base URL: `https://api.openai.com/v1` - URL: `https://api.openai.com/v1/evals/{eval_id}/runs` - Auth: HTTP bearer Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation. ## Path parameters - `eval_id` string, required ## Request body - CreateEvalRunRequest - `name` string: The name of the run. - `metadata` Metadata, nullable: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `data_source` union, required: Details about the run's data source. - object: A JsonlRunDataSource object with that specifies a JSONL file that matches the eval - `type` 'jsonl', required: The type of data source. Always `jsonl`. - `source` union, required: Determines what populates the `item` namespace in the data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - `item` object, required - `sample` object - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - object: A CompletionsRunDataSource object describing a model sampling configuration. - `type` 'completions', required: The type of run data source. Always `completions`. - `input_messages` union: Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required: The type of input messages. Always `template`. - `template` union[], required: A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - union - … - object - `type` 'item_reference', required: The type of input messages. Always `item_reference`. - `item_reference` string, required: A reference to a variable in the `item` namespace. Ie, "item.input_trajectory" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number: A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer: The maximum number of tokens in the generated output. - `top_p` number: An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer: A seed value to initialize the randomness, during sampling. - `response_format` union: An object specifying the format that the model must output. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - ResponseFormatText: Default response format. Used to generate text responses. - `type` 'text', required: The type of response format being defined. Always `text`. - ResponseFormatJsonSchema: JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs). - `type` 'json_schema', required: The type of response format being defined. Always `json_schema`. - `json_schema` object, required: Structured Outputs configuration options, including a JSON Schema. - … - ResponseFormatJsonObject: JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - `type` 'json_object', required: The type of response format being defined. Always `json_object`. - `tools` ChatCompletionTool[]: A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported. - `type` 'function', required: The type of the tool. Currently, only `function` is supported. - `function` FunctionObject, required - `description` string: A description of what the function does, used by the model to choose when and how to call the function. - `name` string, required: The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64. - `parameters` FunctionParameters: The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://developers.openai.com/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict` boolean, nullable: Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://developers.openai.com/api/docs/guides/function-calling). - `model` string: The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required: Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - `item` object, required - `sample` object - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - EvalStoredCompletionsSource: A StoredCompletionsRunDataSource configuration describing a set of filters - `type` 'stored_completions', required: The type of source. Always `stored_completions`. - `metadata` Metadata, nullable: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `model` string, nullable: An optional model to filter by (e.g., 'gpt-6-astra'). - `created_after` integer, nullable: An optional Unix timestamp to filter items created after this time. - `created_before` integer, nullable: An optional Unix timestamp to filter items created before this time. - `limit` integer, nullable: An optional maximum number of items to return. - object: A ResponsesRunDataSource object describing a model sampling configuration. - `type` 'responses', required: The type of run data source. Always `responses`. - `input_messages` union: Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required: The type of input messages. Always `template`. - `template` union[], required: A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - union - … - object - `type` 'item_reference', required: The type of input messages. Always `item_reference`. - `item_reference` string, required: A reference to a variable in the `item` namespace. Ie, "item.name" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number: A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer: The maximum number of tokens in the generated output. - `top_p` number: An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer: A seed value to initialize the randomness, during sampling. - `tools` Tool[]: An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the model's capabilities, like [web search](https://developers.openai.com/api/docs/guides/tools-web-search) or [file search](https://developers.openai.com/api/docs/guides/tools-file-search). Learn more about [built-in tools](https://developers.openai.com/api/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about [function calling](https://developers.openai.com/api/docs/guides/function-calling). - union: A tool that can be used to generate a response. - FunctionTool: Defines a function in your own code the model can choose to call. Learn more about [function calling](https://developers.openai.com/api/docs/guides/function-calling). - … - FileSearchTool: A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://developers.openai.com/api/docs/guides/tools-file-search). - … - ComputerTool: A tool that controls a virtual computer. Learn more about the [computer tool](https://developers.openai.com/api/docs/guides/tools-computer-use). - … - ComputerUsePreviewTool: A tool that controls a virtual computer. Learn more about the [computer tool](https://developers.openai.com/api/docs/guides/tools-computer-use). - … - WebSearchTool: Search the Internet for sources related to the prompt. Learn more about the [web search tool](https://developers.openai.com/api/docs/guides/tools-web-search). - … - MCPTool: Give the model access to additional tools via remote Model Context Protocol (MCP) servers. [Learn more about MCP](https://developers.openai.com/api/docs/guides/tools-connectors-mcp). - … - CodeInterpreterTool: A tool that runs Python code to help generate a response to a prompt. - … - ProgrammaticToolCallingParam - … - ImageGenTool: A tool that generates images using the GPT image models. - … - LocalShellToolParam: A tool that allows the model to execute shell commands in a local environment. - … - FunctionShellToolParam: A tool that allows the model to execute shell commands. - … - CustomToolParam: A custom tool that processes input using a specified format. Learn more about [custom tools](https://developers.openai.com/api/docs/guides/function-calling#custom-tools) - … - NamespaceToolParam: Groups function/custom tools under a shared namespace. - … - ToolSearchToolParam: Hosted or BYOT tool search configuration for deferred tools. - … - WebSearchPreviewTool: This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](https://developers.openai.com/api/docs/guides/tools-web-search). - … - ApplyPatchToolParam: Allows the assistant to create, delete, or update files using unified diffs. - … - `text` object: Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - [Text inputs and outputs](https://developers.openai.com/api/docs/guides/text) - [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs) - `format` union: An object specifying the format that the model must output. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. **Not recommended for gpt-4o and newer models:** Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - ResponseFormatText: Default response format. Used to generate text responses. - … - TextResponseFormatJsonSchema: JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs). - … - ResponseFormatJsonObject: JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - … - `model` string: The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required: Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - `item` object, required - `sample` object - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - EvalResponsesSource: A EvalResponsesSource object describing a run data source configuration. - `type` 'responses', required: The type of run data source. Always `responses`. - `metadata` object, nullable: Metadata filter for the responses. This is a query parameter used to select responses. - `model` string, nullable: The name of the model to find responses for. This is a query parameter used to select responses. - `instructions_search` string, nullable: Optional string to search the 'instructions' field. This is a query parameter used to select responses. - `created_after` integer, nullable: Only include items created after this timestamp (inclusive). This is a query parameter used to select responses. - `created_before` integer, nullable: Only include items created before this timestamp (inclusive). This is a query parameter used to select responses. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number, nullable: Sampling temperature. This is a query parameter used to select responses. - `top_p` number, nullable: Nucleus sampling parameter. This is a query parameter used to select responses. - `users` string[], nullable: List of user identifiers. This is a query parameter used to select responses. - `tools` string[], nullable: List of tool names. This is a query parameter used to select responses. ## Response `201` Successfully created a run for the evaluation - EvalRun: A schema representing an evaluation run. - `object` 'eval.run', required: The type of the object. Always "eval.run". - `id` string, required: Unique identifier for the evaluation run. - `eval_id` string, required: The identifier of the associated evaluation. - `status` string, required: The status of the evaluation run. - `model` string, nullable, required: The model that is evaluated, if applicable. - `name` string, nullable, required: The name of the evaluation run. - `created_at` integer, required: Unix timestamp (in seconds) when the evaluation run was created. - `report_url` string, uri, required: The URL to the rendered evaluation run report on the UI dashboard. - `result_counts` object, required: Counters summarizing the outcomes of the evaluation run. - `total` integer, required: Total number of executed output items. - `errored` integer, required: Number of output items that resulted in an error. - `failed` integer, required: Number of output items that failed to pass the evaluation. - `passed` integer, required: Number of output items that passed the evaluation. - `per_model_usage` object[], nullable, required: Usage statistics for each model during the evaluation run. - `model_name` string, required: The name of the model. - `invocation_count` integer, required: The number of invocations. - `prompt_tokens` integer, required: The number of prompt tokens used. - `completion_tokens` integer, required: The number of completion tokens generated. - `total_tokens` integer, required: The total number of tokens used. - `cached_tokens` integer, required: The number of tokens retrieved from cache. - `per_testing_criteria_results` object[], nullable, required: Results per testing criteria applied during the evaluation run. - `testing_criteria` string, required: A description of the testing criteria. - `passed` integer, required: Number of tests passed for this criteria. - `failed` integer, required: Number of tests failed for this criteria. - `data_source` union, required: Information about the run's data source. - object: A JsonlRunDataSource object with that specifies a JSONL file that matches the eval - `type` 'jsonl', required: The type of data source. Always `jsonl`. - `source` union, required: Determines what populates the `item` namespace in the data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - `item` object, required - `sample` object - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - object: A CompletionsRunDataSource object describing a model sampling configuration. - `type` 'completions', required: The type of run data source. Always `completions`. - `input_messages` union: Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required: The type of input messages. Always `template`. - `template` union[], required: A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - union - … - object - `type` 'item_reference', required: The type of input messages. Always `item_reference`. - `item_reference` string, required: A reference to a variable in the `item` namespace. Ie, "item.input_trajectory" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number: A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer: The maximum number of tokens in the generated output. - `top_p` number: An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer: A seed value to initialize the randomness, during sampling. - `response_format` union: An object specifying the format that the model must output. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - ResponseFormatText: Default response format. Used to generate text responses. - `type` 'text', required: The type of response format being defined. Always `text`. - ResponseFormatJsonSchema: JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs). - `type` 'json_schema', required: The type of response format being defined. Always `json_schema`. - `json_schema` object, required: Structured Outputs configuration options, including a JSON Schema. - … - ResponseFormatJsonObject: JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - `type` 'json_object', required: The type of response format being defined. Always `json_object`. - `tools` ChatCompletionTool[]: A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported. - `type` 'function', required: The type of the tool. Currently, only `function` is supported. - `function` FunctionObject, required - `description` string: A description of what the function does, used by the model to choose when and how to call the function. - `name` string, required: The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64. - `parameters` FunctionParameters: The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://developers.openai.com/api/docs/guides/function-calling) for examples, and the [JSON Schema reference](https://json-schema.org/understanding-json-schema/) for documentation about the format. Omitting `parameters` defines a function with an empty parameter list. - `strict` boolean, nullable: Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the `parameters` field. Only a subset of JSON Schema is supported when `strict` is `true`. Learn more about Structured Outputs in the [function calling guide](https://developers.openai.com/api/docs/guides/function-calling). - `model` string: The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required: Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - `item` object, required - `sample` object - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - EvalStoredCompletionsSource: A StoredCompletionsRunDataSource configuration describing a set of filters - `type` 'stored_completions', required: The type of source. Always `stored_completions`. - `metadata` Metadata, nullable: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `model` string, nullable: An optional model to filter by (e.g., 'gpt-6-astra'). - `created_after` integer, nullable: An optional Unix timestamp to filter items created after this time. - `created_before` integer, nullable: An optional Unix timestamp to filter items created before this time. - `limit` integer, nullable: An optional maximum number of items to return. - object: A ResponsesRunDataSource object describing a model sampling configuration. - `type` 'responses', required: The type of run data source. Always `responses`. - `input_messages` union: Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required: The type of input messages. Always `template`. - `template` union[], required: A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - union - … - object - `type` 'item_reference', required: The type of input messages. Always `item_reference`. - `item_reference` string, required: A reference to a variable in the `item` namespace. Ie, "item.name" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number: A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer: The maximum number of tokens in the generated output. - `top_p` number: An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer: A seed value to initialize the randomness, during sampling. - `tools` Tool[]: An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the model's capabilities, like [web search](https://developers.openai.com/api/docs/guides/tools-web-search) or [file search](https://developers.openai.com/api/docs/guides/tools-file-search). Learn more about [built-in tools](https://developers.openai.com/api/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about [function calling](https://developers.openai.com/api/docs/guides/function-calling). - union: A tool that can be used to generate a response. - FunctionTool: Defines a function in your own code the model can choose to call. Learn more about [function calling](https://developers.openai.com/api/docs/guides/function-calling). - … - FileSearchTool: A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://developers.openai.com/api/docs/guides/tools-file-search). - … - ComputerTool: A tool that controls a virtual computer. Learn more about the [computer tool](https://developers.openai.com/api/docs/guides/tools-computer-use). - … - ComputerUsePreviewTool: A tool that controls a virtual computer. Learn more about the [computer tool](https://developers.openai.com/api/docs/guides/tools-computer-use). - … - WebSearchTool: Search the Internet for sources related to the prompt. Learn more about the [web search tool](https://developers.openai.com/api/docs/guides/tools-web-search). - … - MCPTool: Give the model access to additional tools via remote Model Context Protocol (MCP) servers. [Learn more about MCP](https://developers.openai.com/api/docs/guides/tools-connectors-mcp). - … - CodeInterpreterTool: A tool that runs Python code to help generate a response to a prompt. - … - ProgrammaticToolCallingParam - … - ImageGenTool: A tool that generates images using the GPT image models. - … - LocalShellToolParam: A tool that allows the model to execute shell commands in a local environment. - … - FunctionShellToolParam: A tool that allows the model to execute shell commands. - … - CustomToolParam: A custom tool that processes input using a specified format. Learn more about [custom tools](https://developers.openai.com/api/docs/guides/function-calling#custom-tools) - … - NamespaceToolParam: Groups function/custom tools under a shared namespace. - … - ToolSearchToolParam: Hosted or BYOT tool search configuration for deferred tools. - … - WebSearchPreviewTool: This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](https://developers.openai.com/api/docs/guides/tools-web-search). - … - ApplyPatchToolParam: Allows the assistant to create, delete, or update files using unified diffs. - … - `text` object: Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - [Text inputs and outputs](https://developers.openai.com/api/docs/guides/text) - [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs) - `format` union: An object specifying the format that the model must output. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. **Not recommended for gpt-4o and newer models:** Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - ResponseFormatText: Default response format. Used to generate text responses. - … - TextResponseFormatJsonSchema: JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs). - … - ResponseFormatJsonObject: JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - … - `model` string: The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required: Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - `item` object, required - `sample` object - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - EvalResponsesSource: A EvalResponsesSource object describing a run data source configuration. - `type` 'responses', required: The type of run data source. Always `responses`. - `metadata` object, nullable: Metadata filter for the responses. This is a query parameter used to select responses. - `model` string, nullable: The name of the model to find responses for. This is a query parameter used to select responses. - `instructions_search` string, nullable: Optional string to search the 'instructions' field. This is a query parameter used to select responses. - `created_after` integer, nullable: Only include items created after this timestamp (inclusive). This is a query parameter used to select responses. - `created_before` integer, nullable: Only include items created before this timestamp (inclusive). This is a query parameter used to select responses. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number, nullable: Sampling temperature. This is a query parameter used to select responses. - `top_p` number, nullable: Nucleus sampling parameter. This is a query parameter used to select responses. - `users` string[], nullable: List of user identifiers. This is a query parameter used to select responses. - `tools` string[], nullable: List of tool names. This is a query parameter used to select responses. - `metadata` Metadata, nullable, required: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `error` EvalApiError, required: An object representing an error response from the Eval API. - `code` string, required: The error code. - `message` string, required: The error message. ## Other responses - `400`: Bad request (for example, missing eval object) - `404`: The evaluation or a referenced resource was not found. - `429`: The request was rejected because a rate limit was exceeded. - `500`: The evaluation run could not be created. ## Changes - **2026-10-26** `410e6429083a`: 14 info - added the new optional request property `data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #1: Input message]/content/oneOf[subschema #2: Input item content list]/items/oneOf[subschema #1: Input text]/prompt_cache_breakpoint/ttl` - added the new optional request property `data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #1: Input message]/content/oneOf[subschema #2: Input item content list]/items/oneOf[subschema #2: Input image]/prompt_cache_breakpoint/ttl` - added the new optional request property `data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #1: Input message]/content/oneOf[subschema #2: Input item content list]/items/oneOf[subschema #3: Input file]/prompt_cache_breakpoint/ttl` - added the new optional request property `data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl` - …10 more - …earlier changes not shown [Operation history](https://skmtc.dev/omnistack-sh/apis/openai-api/changes/evals/:eval_id/runs/post.md) --- [API](https://skmtc.dev/omnistack-sh/apis/openai-api.md) - [All operations](https://skmtc.dev/omnistack-sh/apis/openai-api/llms.txt) - [OpenAPI document](https://skmtc.dev/omnistack-sh/apis/openai-api/revisions/0a26f104d35f?raw)