Benchmark
SampleGeneration

Starts an asynchronous sample generation run.

Changed on

In target mode the run queues only the samples that are missing, counting both what already exists and what an earlier run has not finished producing yet. A repeat call over an unchanged benchmark therefore queues nothing and completes immediately. Returns 403 unless the caller's organization has been granted the Faucets capability.

post/benchmark/{benchmarkId}/sample-generation

Request

  • Base URL
    https://api.rapidata.ai
  • URL
    https://api.rapidata.ai/benchmark/{benchmarkId}/sample-generation
  • Auth
    OpenID Connect (https://auth.rapidata.ai/.well-known/openid-configuration, scopes: openid, profile, email, offline_access, api)

Path parameters

benchmarkIdstring required

The benchmark whose participants should run their faucets.

Request body

mode'Additive' | 'Target'
samplesPerPromptinteger required

How many samples to request per matching prompt (1 to 16).

participantIdsstring[] nullable

Restrict generation to specific participants. Defaults to all participants in the benchmark that have a configured faucet.

promptIdentifiersstring[] nullable

Restrict generation to specific prompt identifiers. Defaults to every prompt in the benchmark.

tagsstring[] nullable

Restrict generation to prompts having at least one of these tags.

Response

OK

idstring required

The id of the created sample generation request.

totalCountinteger required

The total number of generation items that were queued.

queuedSampleCountinteger required

How many samples the queued items will produce in total. In target mode each item carries only its own shortfall, so this is not simply totalCount times the per-prompt count.

skippedParticipantIdsstring[] required

Participants that matched the selection but were skipped because they have no faucet configured. Empty when explicit participant ids were supplied.

satisfiedParticipantIdsstring[] required

Participants nothing was queued for because they already hold the requested count. Distinct from SkippedParticipantIds, which is about participants that cannot generate at all. Always empty in additive mode.

Changes

  • ebc7fbc31114Revision changes
    • info

      added the new optional request property

    • info

      added the required property to the response with the status

    • info

      added the required property to the response with the status

  • 9da534659e5fRevision changes
    • info

      the security scope api was added to the endpoint's security scheme OpenIdConnect