---
title: "Judge Disagreements"
method: GET
path: "/api/v1/judges/disagreements/{ranking_id}"
tags: ["LLM Judges"]
---

# Judge Disagreements

`GET /api/v1/judges/disagreements/{ranking_id}`

## Request

- Base URL: `https://app.aimrank.io` (the document declares no server; this is the origin it was published from)
- URL: `https://app.aimrank.io/api/v1/judges/disagreements/{ranking_id}`
- Auth: none declared

LLM-vs-human alignment for one judge in a single call.

Computes the human majority winner per matchup, the judge's pick per
matchup, Cohen κ over the overlap, a Wilson 95% CI on the agreement rate,
and the top matchups where the judge disagrees with the human majority.

Cohen κ on the a/b-relative encoding (did the rater pick the lower-sorted
entity). Returns null when a rater's marginal is degenerate (e.g. everyone
always prefers the stronger entity) — in that case agreement_pct + the
Wilson 95% CI are the primary alignment signal.

## Path parameters

- `ranking_id` string, required

## Query parameters

- `judge_id` string
- `limit` integer

## Response `200`

Successful Response

- unknown

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/aimrank/apis/aimrank-api.md) · [All operations](https://skmtc.dev/aimrank/apis/aimrank-api/llms.txt) · [OpenAPI document](https://skmtc.dev/aimrank/apis/aimrank-api/revisions/3574d0aff459?raw)