Benchmark

[Beta] Create a BenchmarkJob.

Changed on

[Beta] Create a BenchmarkJob that runs a set of scenarios entirely on runloop.

post/v1/benchmark_jobs

Request

  • Base URL: https://api.runloop.ai
  • URL: https://api.runloop.ai/v1/benchmark_jobs
  • Auth: HTTP bearer

Request body

namestring nullable

The name of the BenchmarkJob. If not provided, name will be generated based on target dataset.

Response

OK

idstring required

The ID of the BenchmarkJob.

namestring required

The unique name of the BenchmarkJob.

state'initializing' | 'queued' | 'running' | 'completed' | 'failed' | 'cancelled' | 'timeout' required
failure_reasonstring nullable

Failure reason if job failed.

create_time_msinteger required

Timestamp when job was created (Unix milliseconds).

Changes

  • d3ddb452754cRevision changes
    • breaking

      the request property became not nullable

    • breaking

      the request property became not nullable

    • breaking

      removed from the request property oneOf list

    • breaking

      the //// response's property type changed from object to no type for status

    • breaking

      the response's property type changed from object to no type for status

    • breaking

      removed the required property ///// from the response with the status

    • breaking

      removed the required property ///// from the response with the status

    • breaking

      removed the required property / from the response with the status

    • breaking

      removed the required property / from the response with the status

    • warning

      removed the optional property / from the response with the status

    • info

      added subschema #2 to the request property anyOf list

    • info

      removed discriminator from request property

    • info

      added subschema #2 to the //// response property anyOf list for the response status

    • info

      added subschema #2 to the response property anyOf list for the response status

    • info

      added subschema #2 to the response property anyOf list for the response status

    • info

      added subschema #2 to the // response property anyOf list for the response status

    • info

      removed discriminator from // response property for the response status

    • info

      removed discriminator from response property for the response status

    • info

      removed from the // response property oneOf list for the response status

    • info

      removed from the response property oneOf list for the response status

  • c33fa67077f6Revision changes
    • breaking

      the response property //// became nullable for the status

    • breaking

      the response property //// became optional for the status