Benchmark

Creates one run per targeted leaderboard of the benchmark, all evaluating the same matchups.

Changed on

Answers 202 with one entry per created run and the seed they all share, so every leaderboard evaluates identical image pairs; a seed is generated when none is supplied. Without leaderboardIds, hidden and excluded-from-overall leaderboards are left out; listed leaderboards run even when hidden or excluded, and ids not on the benchmark are a 400. See POST /leaderboard/{leaderboardId}/runs/custom for how configurations are validated and resolved.

post/benchmark/{benchmarkId}/runs/custom

Request

  • Base URL
    https://api.rapidata.ai
  • URL
    https://api.rapidata.ai/benchmark/{benchmarkId}/runs/custom
  • Auth
    OpenID Connect (https://auth.rapidata.ai/.well-known/openid-configuration, scopes: openid, profile, email, offline_access, api)

Path parameters

benchmarkIdstring required

The id of the benchmark whose leaderboards should run the matchups.

Request body

namestring required

The display name given to every created run.

seedinteger nullable

Optional seed for deterministic matchup selection, applied to every created run. Omit to have one generated, which still keeps the runs aligned with each other.

leaderboardIdsstring[] nullable

Optional leaderboards of the benchmark to run on, hidden or excluded ones included. Omit or leave empty to target every visible leaderboard included in the benchmark overall.

Response

Accepted

seedinteger required

The seed every created run shares.

Changes