Analytics

Get Per Annotator Agreement

Per-annotator κ vs the majority-vote consensus.

Surfaces noisy / low-quality annotators so the ranking owner can drop them from RLHF exports. Owner-only — exposes per-voter quality which is sensitive. Returns annotators sorted worst-first (lowest κ).

min_pairs excludes annotators with too few overlapping pairs to estimate κ stably (default 3).

get/api/v1/analytics/rankings/{ranking_id}/per-annotator-agreement

Request

  • Base URL: https://app.aimrank.io (the document declares no server; this is the origin it was published from)
  • URL: https://app.aimrank.io/api/v1/analytics/rankings/{ranking_id}/per-annotator-agreement
  • Auth: none declared

Path parameters

ranking_idstring required

Query parameters

min_pairsinteger
limitinteger

Response

Successful Response

unknown required

Changes

No recorded changes to this operation across all 1 revision of this API.