LLM Judges

Judge Agreement

Compute Cohen's κ between each LLM judge and the humans on this ranking.

Requires that matchups have been voted on by BOTH humans and the given judge. Used by customers to decide whether a judge is trustworthy enough to handle routine matchups without human review.

get/api/v1/judges/agreement/{ranking_id}

Request

  • Base URL: https://app.aimrank.io (the document declares no server; this is the origin it was published from)
  • URL: https://app.aimrank.io/api/v1/judges/agreement/{ranking_id}
  • Auth: none declared

Path parameters

ranking_idstring required

Response

Successful Response

unknown required

Changes

No recorded changes to this operation across all 1 revision of this API.