Benchmark

Create a Benchmark.

Changed on

Create a Benchmark with a set of Scenarios.

post/v1/benchmarks

Request

  • Base URL: https://api.runloop.ai
  • URL: https://api.runloop.ai/v1/benchmarks
  • Auth: HTTP bearer

Request body

namestring required

The unique name of the Benchmark.

scenario_idsstring[] nullable

The Scenario IDs that make up the Benchmark.

metadataobject nullable

User defined metadata to attach to the benchmark.

required_environment_variablesstring[] nullable

Environment variables required to run the benchmark. If any required variables are not supplied, the benchmark will fail to start.

required_secret_namesstring[]

Secrets required to run the benchmark with (environment variable name will be mapped to the your user secret by name). If any of these secrets are not provided or the mapping is incorrect, the benchmark will fail to start.

attributionstring nullable

Attribution information for the benchmark.

descriptionstring nullable

Detailed description of the benchmark.

Response

OK

idstring required

The ID of the Benchmark.

namestring required

The name of the Benchmark.

scenarioIdsstring[] required

List of Scenario IDs that make up the benchmark.

metadataobject required

User defined metadata to attach to the benchmark for organization.

required_environment_variablesstring[]

Required environment variables used to run the benchmark. If any required environment variables are missing, the benchmark will fail to start.

required_secret_namesstring[]

Required secrets used to run the benchmark. If any required secrets are missing, the benchmark will fail to start.

is_publicboolean

Whether this benchmark is public.

attributionstring

Attribution information for the benchmark.

descriptionstring

Detailed description of the benchmark.

status'active' | 'archived' required

Changes

  • d3ddb452754cRevision changes
    • breaking

      the request property became not nullable

    • breaking

      the request property became not nullable

    • breaking

      the request property became not nullable

    • breaking

      the request property became not nullable

    • breaking

      the request property became not nullable

  • c33fa67077f6Revision changes
    • info

      added the required property to the response with the status