Sign in
Evals

Creates an eval run.

post/experimental/eval-runs-v2

Headers

Authorizationstring

Request body

workspaceNamestring required
scenariosobject required

Per-scenario tarballs and fixtures, keyed by scenario path

evalRunFixturesobject
skipForcedContextActivationboolean
skipScoringboolean
agentstring
modelstring
scorerAgentstring
scorerModelstring
runsinteger
skipSolvedboolean
runQualityCheckboolean
labelstring

Response

Default Response

Changes

Changed in 4 of the 55 revisions of this API.18

  • b777eddaa9134See the full diff
    • ○

      added subschema #3 to the / request property anyOf list

      request-property-any-of-added

    • ○

      request property deprecated

      request-property-deprecated

    • ○

      request property // deprecated

      request-property-deprecated

    • ○

      request property // deprecated

      request-property-deprecated

  • 050f3217cf2c11See the full diff
    • ▲

      removed subschema #1 subschema #2 subschema #3 subschema #4 from the //// request property anyOf list

      request-property-any-of-removed

    • ○

      added subschema #1 subschema #2 to the //// request property anyOf list

      request-property-any-of-added

  • ffb03b5d287f2See the full diff
    • ○

      added the new optional request property

      new-optional-request-property

    • ○

      the request property became optional

      request-property-became-optional

  • 62bb86f8aa901See the full diff
    • ○

      the request property default value changed from false to true

      request-property-default-value-changed