AYaymara-ai
Eval Runs

Run Eval

Changed on

Run the eval with the provided responses.

Args: eval_run_data (EvalRunRequest): Data for the eval run, including responses. workspace_uuid (str, optional): UUID of the workspace. Defaults to None. is_sandbox (bool, optional): Whether to run in sandbox mode. Defaults to False.

Returns: EvalRunResult: The result of the eval run after scoring the responses.

Raises: AymaraAPIError: If the organization is missing or the request is invalid.

Example: POST /api/eval-runs/-/score-responses { "eval_uuid": "...", "responses": [...] }

post/v2/eval-runs/-/score-responses

Request

Query parameters

workspace_uuidstring
is_sandboxboolean

Request body

eval_uuidstring required

Unique identifier for the eval.

eval_run_uuidstring nullable

Unique identifier for the eval run, if any.

namestring nullable

Name of the eval run, if any (defaults to the eval name + timestamp).

ai_descriptionstring nullable

Description of the AI for this run, if any.

continue_threadboolean nullable

Whether to continue the thread after this run.

Response

OK

eval_run_uuidstring required

Unique identifier for the eval run.

eval_uuidstring required

Unique identifier for the eval.

namestring nullable

Name of the eval run, if any (defaults to the eval name + timestamp).

status'created' | 'processing' | 'finished' | 'failed' required

Resource status.

created_atstring date-time required

Timestamp when the eval run was created.

updated_atstring date-time required

Timestamp when the eval run was last updated.

ai_descriptionstring nullable

Description of the AI for this run, if any.

is_sandboxboolean

Indicates if the eval run is sandboxed (default: False).

workspace_uuidstring nullable

UUID of the associated workspace, if any.

pass_ratenumber nullable

Pass rate for the eval run, if any.

num_promptsinteger nullable

Number of prompts in the eval run, if any.

num_responses_scoredinteger nullable

Number of responses scored in the eval run, if any.

Changes

  • 0cb64ef11adfRevision changes
    • breaking

      removed the required property //////// from the response with the status

    • breaking

      removed the required property ////////// from the response with the status

  • cd49adceebf8Revision changes
    • info

      added subschema #1 subschema #4 to the // response property anyOf list for the response status

    • info

      removed subschema #1 subschema #4 from the // response property anyOf list for the response status