---
title: "Create a model comparison"
method: POST
path: "/v3/model-comparisons"
tags: ["Function Accuracy"]
---

# Create a model comparison

`POST /v3/model-comparisons`

**Compare several function versions on one dataset.**

Scores a saved Golden Data Set against each entry's function version (each as
an eval-score run) and reports per-entry accuracy, latency, and cost, plus
lift of every entry against the baseline (the first entry). Entries may span
different functions and different versions.

Runs asynchronously. The response carries a `comparisonID`; poll
`GET /v3/model-comparisons/{comparisonID}` until `status` is `complete`.

## Request body

- ModelComparisonCreateRequestV3 — Request body for `POST /v3/model-comparisons`. It carries no comparison configuration: creating a comparison dispatches extractions, and how their output is scored is decided on `GET /v3/model-comparisons/{comparisonID}` (`matchMode` / `orderMatching`).
  - `name` string — Optional human-facing name.
  - `datasetID` string, required — Saved Golden Data Set (`gds_…`) to score against. It must have an `input`-role column (the input files) and a `corrected`-role column (the expected JSON).
  - `entries` ModelComparisonEntryInputV3[], required — Between 2 and 3 distinct function versions to compare — the same `(function, version)` twice is rejected. Each entry scores the whole dataset, so the count is capped to bound fan-out. The dataset's input / corrected / schema columns are resolved by column role. When the dataset carries a `schema` column, every entry is scored against that per-row ground-truth schema instead of each function's own schema — so the comparison stays fair as schemas evolve.
    - `functionName` string, required — Name of the function to score. Must be extract / transform / analyze.
    - `functionVersionNum` integer — Version number to score; omit to use the function's current version.
    - `label` string — Optional display label (defaults to `name@vN`).

## Response `202`

The request has been accepted for processing, but processing has not yet completed.

- ModelComparisonCreateResponseV3 — Returned by `POST /v3/model-comparisons`.
  - `comparisonID` string, required
  - `status` 'pending' | 'running' | 'complete' | 'partial' | 'cancelled' | 'error', required — Rolled-up status of a comparison, derived from its entries' scoring runs.

---

[API](https://skmtc.dev/bem-team/apis/bem-api.md) · [All operations](https://skmtc.dev/bem-team/apis/bem-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/bem-team/bem-api/revisions/a6e4624117c6/schema)