Benchmark

Queries a benchmark's prompt ratings — a page of prompts, each with its ranked entries and the pairwise win matrix over them.

Changed on

Filters apply to the underlying votes with the standings query's field set. A rating is an entry's standing against the other entries on its prompt, so a participant filter narrows the entries but does not refit, and a prompt with no matching votes yields no entries. groupBy decides what competes: per participant, a participant's images become one entry rated from their pooled wins, and comparisons between its own images are excluded from the rating, both matrices, and responseCount. Grouping never changes which prompts the page holds.

get/benchmark/{benchmarkId}/prompt-ratings/query

Request

  • Base URL
    https://api.rapidata.ai
  • URL
    https://api.rapidata.ai/benchmark/{benchmarkId}/prompt-ratings/query
  • Auth
    OpenID Connect (https://auth.rapidata.ai/.well-known/openid-configuration, scopes: openid, profile, email, offline_access, api)

Path parameters

benchmarkIdstring required

The id of the benchmark.

Query parameters

useWeightedScoringboolean

Whether to weight ratings by annotator reliability.

groupBy'Image' | 'Participant'

What each rated entry of a prompt represents.

What each rated entry represents.

pageinteger

The 1-based page index.

page_sizeinteger

The number of items per page.

sortstring[]

Sort fields. Prefix with - for descending order (e.g. -created_at).

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by country.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by language.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by gender.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by age_bucket.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by occupation.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by tags.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by participant_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by leaderboard_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by run_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by prompt_identifier.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by voted_at.

logic'and' | 'or'

How to combine the field filters: "and" (default) requires every filter to match, "or" requires any of them to match.

Response

OK

totalinteger required

The number of prompts the filter leaves rankable — the paging total.

pageinteger required

The 1-based page index.

pageSizeinteger required

The page size in prompts.

Changes

  • 21c16ab2e7abRevision changes
    • info

      added the optional property //////////////// to the response with the status

    • info

      added the optional property ///////////////// to the response with the status

  • ebc7fbc31114Revision changes
    • breaking

      the ////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the ////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the ////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the ////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the /////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the /////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the /////////////////////// response's property format changed from date-time to no format for status

    • breaking

      the /////////////////////// response's property format changed from date-time to no format for status

    • info

      the ////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the ////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the ////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the ////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the /////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the /////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the /////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status

    • info

      the /////////////////////// response's property pattern ^-?(\d+\.)?\d{2}:\d{2}:\d{2}(\.\d{1,7})?$ was added for the status