--- title: "Get eval runs" method: GET path: "/evals/{eval_id}/runs" tags: ["Evals"] --- # Get eval runs `GET /evals/{eval_id}/runs` ## Request - Base URL: `https://api.openai.com/v1` - URL: `https://api.openai.com/v1/evals/{eval_id}/runs` - Auth: HTTP bearer Get a list of runs for an evaluation. ## Path parameters - `eval_id` string, required ## Query parameters - `after` string - `limit` integer - `order` 'asc' | 'desc' - `status` 'queued' | 'in_progress' | 'completed' | 'canceled' | 'failed' ## Response `200` A list of runs for the evaluation - EvalRunList: An object representing a list of runs for an evaluation. - `object` 'list', required: The type of this object. It is always set to "list". - `data` EvalRun[], required: An array of eval run objects. - `object` 'eval.run', required: The type of the object. Always "eval.run". - `id` string, required: Unique identifier for the evaluation run. - `eval_id` string, required: The identifier of the associated evaluation. - `status` string, required: The status of the evaluation run. - `model` string, nullable, required: The model that is evaluated, if applicable. - `name` string, nullable, required: The name of the evaluation run. - `created_at` integer, required: Unix timestamp (in seconds) when the evaluation run was created. - `report_url` string, uri, required: The URL to the rendered evaluation run report on the UI dashboard. - `result_counts` object, required: Counters summarizing the outcomes of the evaluation run. - `total` integer, required: Total number of executed output items. - `errored` integer, required: Number of output items that resulted in an error. - `failed` integer, required: Number of output items that failed to pass the evaluation. - `passed` integer, required: Number of output items that passed the evaluation. - `per_model_usage` object[], nullable, required: Usage statistics for each model during the evaluation run. - `model_name` string, required: The name of the model. - `invocation_count` integer, required: The number of invocations. - `prompt_tokens` integer, required: The number of prompt tokens used. - `completion_tokens` integer, required: The number of completion tokens generated. - `total_tokens` integer, required: The total number of tokens used. - `cached_tokens` integer, required: The number of tokens retrieved from cache. - `per_testing_criteria_results` object[], nullable, required: Results per testing criteria applied during the evaluation run. - `testing_criteria` string, required: A description of the testing criteria. - `passed` integer, required: Number of tests passed for this criteria. - `failed` integer, required: Number of tests failed for this criteria. - `data_source` union, required: Information about the run's data source. - object: A JsonlRunDataSource object with that specifies a JSONL file that matches the eval - `type` 'jsonl', required: The type of data source. Always `jsonl`. - `source` union, required: Determines what populates the `item` namespace in the data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - … - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - object: A CompletionsRunDataSource object describing a model sampling configuration. - `type` 'completions', required: The type of run data source. Always `completions`. - `input_messages` union: Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required: The type of input messages. Always `template`. - `template` union[], required: A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - … - object - `type` 'item_reference', required: The type of input messages. Always `item_reference`. - `item_reference` string, required: A reference to a variable in the `item` namespace. Ie, "item.input_trajectory" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number: A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer: The maximum number of tokens in the generated output. - `top_p` number: An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer: A seed value to initialize the randomness, during sampling. - `response_format` union: An object specifying the format that the model must output. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - ResponseFormatText: Default response format. Used to generate text responses. - … - ResponseFormatJsonSchema: JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs). - … - ResponseFormatJsonObject: JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - … - `tools` ChatCompletionTool[]: A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported. - `type` 'function', required: The type of the tool. Currently, only `function` is supported. - `function` FunctionObject, required - … - `model` string: The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required: Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - … - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - EvalStoredCompletionsSource: A StoredCompletionsRunDataSource configuration describing a set of filters - `type` 'stored_completions', required: The type of source. Always `stored_completions`. - `metadata` Metadata, nullable: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `model` string, nullable: An optional model to filter by (e.g., 'gpt-6-astra'). - `created_after` integer, nullable: An optional Unix timestamp to filter items created after this time. - `created_before` integer, nullable: An optional Unix timestamp to filter items created before this time. - `limit` integer, nullable: An optional maximum number of items to return. - object: A ResponsesRunDataSource object describing a model sampling configuration. - `type` 'responses', required: The type of run data source. Always `responses`. - `input_messages` union: Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required: The type of input messages. Always `template`. - `template` union[], required: A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - … - object - `type` 'item_reference', required: The type of input messages. Always `item_reference`. - `item_reference` string, required: A reference to a variable in the `item` namespace. Ie, "item.name" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number: A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer: The maximum number of tokens in the generated output. - `top_p` number: An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer: A seed value to initialize the randomness, during sampling. - `tools` Tool[]: An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the model's capabilities, like [web search](https://developers.openai.com/api/docs/guides/tools-web-search) or [file search](https://developers.openai.com/api/docs/guides/tools-file-search). Learn more about [built-in tools](https://developers.openai.com/api/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about [function calling](https://developers.openai.com/api/docs/guides/function-calling). - union: A tool that can be used to generate a response. - … - `text` object: Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - [Text inputs and outputs](https://developers.openai.com/api/docs/guides/text) - [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs) - `format` union: An object specifying the format that the model must output. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. **Not recommended for gpt-4o and newer models:** Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - … - `model` string: The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required: Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required: The type of jsonl source. Always `file_content`. - `content` object[], required: The content of the jsonl file. - … - EvalJsonlFileIdSource - `type` 'file_id', required: The type of jsonl source. Always `file_id`. - `id` string, required: The identifier of the file. - EvalResponsesSource: A EvalResponsesSource object describing a run data source configuration. - `type` 'responses', required: The type of run data source. Always `responses`. - `metadata` object, nullable: Metadata filter for the responses. This is a query parameter used to select responses. - `model` string, nullable: The name of the model to find responses for. This is a query parameter used to select responses. - `instructions_search` string, nullable: Optional string to search the 'instructions' field. This is a query parameter used to select responses. - `created_after` integer, nullable: Only include items created after this timestamp (inclusive). This is a query parameter used to select responses. - `created_before` integer, nullable: Only include items created before this timestamp (inclusive). This is a query parameter used to select responses. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable: Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) for model-specific support. - `temperature` number, nullable: Sampling temperature. This is a query parameter used to select responses. - `top_p` number, nullable: Nucleus sampling parameter. This is a query parameter used to select responses. - `users` string[], nullable: List of user identifiers. This is a query parameter used to select responses. - `tools` string[], nullable: List of tool names. This is a query parameter used to select responses. - `metadata` Metadata, nullable, required: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `error` EvalApiError, required: An object representing an error response from the Eval API. - `code` string, required: The error code. - `message` string, required: The error message. - `first_id` string, required: The identifier of the first eval run in the data array. - `last_id` string, required: The identifier of the last eval run in the data array. - `has_more` boolean, required: Indicates whether there are more evals available. ## Other responses - `400`: The run list parameters are invalid. - `404`: The evaluation or the run specified by `after` was not found. - `429`: The request was rejected because a rate limit was exceeded. ## Changes - **2026-10-26** `410e6429083a`: 7 info - added the optional property `data/items/data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #1: Input message]/content/oneOf[subschema #2: Input item content list]/items/oneOf[subschema #1: Input text]/prompt_cache_breakpoint/ttl` to the response with the `200` status - added the optional property `data/items/data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #1: Input message]/content/oneOf[subschema #2: Input item content list]/items/oneOf[subschema #2: Input image]/prompt_cache_breakpoint/ttl` to the response with the `200` status - added the optional property `data/items/data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #1: Input message]/content/oneOf[subschema #2: Input item content list]/items/oneOf[subschema #3: Input file]/prompt_cache_breakpoint/ttl` to the response with the `200` status - added the optional property `data/items/data_source/oneOf[subschema #2: CompletionsRunDataSource]/input_messages/oneOf[subschema #1: TemplateInputMessages]/template/items/oneOf[subschema #2: Eval message object]/content/oneOf[subschema #1: Eval content item]/oneOf[subschema #2: Input text]/prompt_cache_breakpoint/ttl` to the response with the `200` status - …3 more - …earlier changes not shown [Operation history](https://skmtc.dev/omnistack-sh/apis/openai-api/changes/evals/:eval_id/runs/get.md) --- [API](https://skmtc.dev/omnistack-sh/apis/openai-api.md) - [All operations](https://skmtc.dev/omnistack-sh/apis/openai-api/llms.txt) - [OpenAPI document](https://skmtc.dev/omnistack-sh/apis/openai-api/revisions/0a26f104d35f?raw)