test_framework

Retrieve a test result

Changed on

success indicates whether the run passed its quality evaluation. A run with success=false either failed to connect (check error_message) or failed an evaluation check (check evaluation). success_rate is the percentage of runs with success=true.

The runs field is a dict keyed by run ID string (e.g. {"86368": {...}, "86367": {...}}), not an array — unlike the list endpoint. Each run also has a call_id STRING field (the provider's call identifier) — that is a data field, NOT an endpoint ID; do not pass it where run_id is expected.

get/test_framework/v2/results/{id}/

Request

  • Base URL
    https://api.cekura.ai
  • URL
    https://api.cekura.ai/test_framework/v2/results/{id}/
  • Auth
    • API key in header X-CEKURA-API-KEY
    • or HTTP bearer
    • or HTTP bearer
    • or API key in header X-VOCERA-SHAREABLE-LINK-TOKEN
    • or HTTP bearer

Path parameters

idinteger required

A unique integer value identifying this result.

Response

idinteger
namestring

Name of the result Example: "Test Result 1"

agentinteger required
agent_namestring

Name of the agent associated with this result

status'running' | 'completed' | 'failed' | 'pending' | 'in_progress' | 'evaluating' | 'in_queue' | 'timeout' | 'cancelled' | 'scaling_up'

Current status of the result

  • running - Running
  • completed - Completed
  • failed - Failed
  • pending - Pending
  • in_progress - In Progress
  • evaluating - Evaluating
  • in_queue - In Queue
  • timeout - Timeout
  • cancelled - Cancelled
  • scaling_up - Scaling Up
met_expected_outcome_countstring

Number of runs that fully met their expected outcomes with a score of 100

total_expected_outcome_countstring

Total number of runs that had expected outcomes defined

success_ratenumber double

Success rate of the test runs

run_as_textboolean

Whether this test was run in text mode instead of voice mode Example: true or false

is_cronjobstring

Whether this result was created by a scheduled cronjob

created_viastring nullable

How this result was started: user (dashboard), api, mcp, cli, sdk, schedule or agent (the Cekura agent). Null when unknown.

runsobject

Run objects keyed by run ID string (e.g. {"12345": {...}}). The list endpoint returns runs as an array of summaries instead.

overall_evaluationunknown
total_durationstring

Total duration of the test runs for this result Example: 22:30

total_runs_countstring

Total number of test runs associated with this result Example: 10

completed_runs_countstring

Number of test runs that have completed successfully Example: 10

success_runs_countstring

Number of test runs that were marked as successful Example: 10

failed_runs_countstring

Number of test runs that failed or encountered errors Example: 10

connected_runsstring

List of run IDs that got connected successfully (have transcript data). Returns empty list if no connected runs.

not_connected_runsstring

List of run IDs that never connected (no transcript data). Returns empty list if every run connected.

failed_infrastructure_runsstring

List of run IDs that failed infrastructure issues metric (score=0). Returns null if metric not found, empty list if metric exists but no failures.

failed_workflow_runsstring

List of run IDs that failed expected outcome metric (score=0). Returns null if metric not found, empty list if metric exists but no failures.

successful_callsstring

List of run IDs that completed successfully. Returns empty list if no successful runs.

scenariosstring

List of scenario names used in the test runs for this result Example: ```

[
    {
        "id": 123,
        "name": "Scenario 1"
    },
    {
        "id": 456,
        "name": "Scenario 2"
    }
]
critical_categoriesstring

List of critical categories for this result Example:

[
    {
      "id": 2950,
      "name": "Pronunciation Analysis",
      "eval_type": "continuous_qualitative",
      "simulation_enabled": true,
      "observability_enabled": true
    },
    {
      "id": 3284,
      "name": "Latency",
      "eval_type": "numeric",
      "simulation_enabled": true,
      "observability_enabled": false
    },
    {
      "id": 3295,
      "name": "Detect Silence in Conversation",
      "eval_type": "binary",
      "simulation_enabled": true,
      "observability_enabled": true
    }
  ]
metricsstring
runs_by_tagsstring
latency_datastring
failed_reasonsunknown
ai_summaryunknown
ai_summary_statusstring

Generation state of the async AI summary, derived so the UI can show a reliable state instead of guessing from an empty ai_summary. completed = summary populated; skipped_no_failures = no failed runs, nothing to summarize; pending = there are failed runs but the summary isn't ready yet (still generating).

next_stepsunknown
path_coverageunknown
performance_metricsstring

Per-rubric-rule performance breakdown. One entry per rule in the project's rubric_config: metric name, aggregated value across the result's runs, and whether that aggregate meets the rule's conditions.

run_typestring nullable

Run Execution Type

run_settingsunknown
created_atstring date-time

Timestamp when this test result was created Example: 2021-01-01 00:00:00

updated_atstring date-time

Timestamp when this test result was last updated Example: 2021-01-01 00:00:00

Changes

    • breaking

      added subschema #2 to the response property oneOf list for the response status

    • breaking

      the response's property type changed from string to no type for status

    • info

      api operation id results-retrieve removed and replaced with results-retrieve_3

    • info

      the endpoint scheme security embedded_session was added to the API

    • info

      the endpoint scheme security oauth2 was added to the API

    • info

      the endpoint scheme security shareable_link was added to the API

    • info

      the endpoint scheme security supabase_session was added to the API

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      added the optional property to the response with the status

    • info

      the response property became required for the status