Leaderboard

Creates a run that evaluates an explicit list of matchup configurations, each with its own response count.

Answers 202 once the configurations are validated and the run is queued; it then starts behind whatever else is queued on the leaderboard, and its matchups are resolved at that point. Poll GET /leaderboard/{leaderboardId}/runs for its status. Every id is verified before the 202, but a configuration that resolves to no matchups is only found at start and is skipped; the run fails when all of them do. At most 2000 configurations, 1000000 responses and 10000 referenced ids per request.

post/leaderboard/{leaderboardId}/runs/custom

Request

  • Base URL
    https://api.rapidata.ai
  • URL
    https://api.rapidata.ai/leaderboard/{leaderboardId}/runs/custom
  • Auth
    OpenID Connect (https://auth.rapidata.ai/.well-known/openid-configuration, scopes: openid, profile, email, offline_access, api)

Path parameters

leaderboardIdstring required

The id of the leaderboard to run the matchups on.

Request body

namestring required

The display name of the run.

seedinteger nullable

Optional seed for deterministic matchup selection. Reuse it across a benchmark's leaderboards to align them on the same image pairs; omit for random selection.

Response

Accepted

runIdstring required

The id of the queued run.

Changes