Leaderboard

Creates a new leaderboard.

Changed on

displayInstruction is presentation-only: consumers render it instead of instruction when set, falling back to instruction when null. It never affects run creation, vote recording or ranking, and supplying it requires platform-admin.

post/leaderboard

Request

  • Base URL
    https://api.rapidata.ai
  • URL
    https://api.rapidata.ai/leaderboard
  • Auth
    OpenID Connect (https://auth.rapidata.ai/.well-known/openid-configuration, scopes: openid, profile, email, offline_access, api)

Request body

benchmarkIdstring nullable

If the leaderboard should be added to a preexisting benchmark, the benchmark id.

benchmarkNamestring nullable

When no BenchmarkId is provided, the name of the new benchmark to create.

benchmarkDescriptionstring nullable

Optional plain-text credit for the people or sources behind the benchmark created via BenchmarkName. Limited to 2000 characters; ignored when BenchmarkId is set.

namestring required

The name of the leaderboard.

instructionstring required

The instruction datapoints will be matched up against.

displayInstructionstring nullable

Optional text to display instead of instruction. Presentation-only; it never changes what annotators are asked or how results are scored. Leave unset to display instruction itself.

showPromptboolean required

Indicates if the prompt is shown on the rapids.

showPromptAssetboolean

Whether the prompt asset should be shown on the rapids.

responseBudgetinteger nullable

Total amount of responses that get collected per run.

minResponsesinteger nullable

The minimum amount of responses that need to be collected per comparison.

isInversedboolean

Whether the results should be inversed, selecting the worse model.

audienceIdstring nullable

Optional audience id; defaults to the global audience when unset.

priorityinteger

Optional priority override applied to every run's job; unset uses the default.

weightnumber double

Multiplier applied to this leaderboard's match results when they contribute to the parent benchmark's overall / combined scoreboards. Defaults to 1.0; unbounded.

scoreShiftnumber double

Additive offset applied to displayed scores on this leaderboard's scoreboard; defaults to 0.

scoreScalenumber double

Multiplicative factor applied to displayed scores on this leaderboard's scoreboard, relative to the Bradley-Terry center. Defaults to 1.0; must be strictly positive.

isHiddenboolean

Whether the leaderboard should be created hidden.

isExcludedFromOverallboolean

Whether this leaderboard is excluded from its benchmark's overall / combined scoreboard, making it a private experiment of the owning organization. Defaults to false.

voteAggregation'AllVotes' | 'MajorityVote'
includedTagsstring[]

Prompt tag values to restrict matchups to; only prompts carrying at least one are used. Empty means no restriction.

excludedTagsstring[]

Prompt tag values the leaderboard never creates matchups for; takes precedence over includedTags, even when another tag is included.

skipInitialRunboolean

Set to true to skip the initial standings run. New participants are still compared against the whole existing field and boost runs still work; there is just no run collecting the first responses. Defaults to false.

Response

OK

idstring required

The id of the created leaderboard.

benchmarkIdstring required

The id of the benchmark the leaderboard belongs to.

showPromptboolean required

Whether the prompt is shown on the rapids.

showPromptAssetboolean required

Whether the prompt asset is shown on the rapids.

responseBudgetinteger required

The response budget per run.

minResponsesinteger required

The minimum responses per comparison.

isInversedboolean required

Whether the leaderboard results are inversed.

voteAggregation'AllVotes' | 'MajorityVote' required

Changes

  • ebc7fbc31114Revision changes
    • info

      added the new optional request property

    • info

      added the new optional request property

  • 9da534659e5fRevision changes
    • info

      the security scope api was added to the endpoint's security scheme OpenIdConnect