---
title: "Judge Agreement"
method: GET
path: "/api/v1/judges/agreement/{ranking_id}"
tags: ["LLM Judges"]
---

# Judge Agreement

`GET /api/v1/judges/agreement/{ranking_id}`

## Request

- Base URL: `https://app.aimrank.io` (the document declares no server; this is the origin it was published from)
- URL: `https://app.aimrank.io/api/v1/judges/agreement/{ranking_id}`
- Auth: none declared

Compute Cohen's κ between each LLM judge and the humans on this ranking.

Requires that matchups have been voted on by BOTH humans and the given
judge. Used by customers to decide whether a judge is trustworthy enough
to handle routine matchups without human review.

## Path parameters

- `ranking_id` string, required

## Response `200`

Successful Response

- unknown

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/aimrank/apis/aimrank-api.md) · [All operations](https://skmtc.dev/aimrank/apis/aimrank-api/llms.txt) · [OpenAPI document](https://skmtc.dev/aimrank/apis/aimrank-api/revisions/3574d0aff459?raw)