Judge Prompt Synthesis

Post Validate Judge Prompt

Score a synthesised judge against its source ranking's human votes.

Owner-only (the validation runs N Bedrock calls — only the artifact owner gets to spend the quota). Tier-gated: each pair = 1 judge- quota unit.

Default: 30 pairs sampled with replacement from pairs where ≥60% of humans agreed. Returns the full report with accuracy + Wilson 95% CI + Cohen κ vs humans + per-subset breakdown + cost.

post/api/v1/judge-prompts/{prompt_id}/validate

Request

  • Base URL: https://app.aimrank.io (the document declares no server; this is the origin it was published from)
  • URL: https://app.aimrank.io/api/v1/judge-prompts/{prompt_id}/validate
  • Auth: none declared

Path parameters

prompt_idstring required

Request body

n_pairsinteger
judge_model_idstring nullable
random_seedinteger
validation_ranking_idstring nullable

Optional override — validate the prompt against a DIFFERENT ranking's human votes (transfer test). Defaults to the artifact's source ranking.

Response

Successful Response

unknown required

Changes

No recorded changes to this operation across all 1 revision of this API.