Eval Jobs

Start Eval Job

Queues a durable eval job and returns immediately with its polling ID.

For prompt evals, provide explicit targets to override the configured prompt targets, or omit targets to use the organization's configured scheduled prompt targets. For API Performance evals, explicit targets or models override the stored configuration; when omitted, Lightsage uses configured scheduled targets or legacy configured models. For eval runs with source_type set to custom_evals, custom_eval_ids identify the evals to run, while targets specify the text models or coding-agent harness and model combinations used for the run. Omit custom_eval_ids to use the configured eval set.

For API Performance source types backed by a source (rest_api, sdk, or cli), provide source_id from api-performance sources list, or omit it when operation_ids, eval_ids, or the requested source_type identify a single source. operation_ids are Lightsage API Performance operation IDs, not OpenAPI operationId strings.

Every run is queued through the durable job queue. The response id is the eval_run_id to pass to GET /v1/eval-runs/{eval_run_id}. When Lightsage can determine the runnable work, the response also includes dispatch_id and initial queued progress for the child jobs. If neither the request nor the saved configuration identifies runnable work, the request fails with 400. If the organization lacks the estimated credits, it fails with 402 before dispatching work. An environment containing an empty or placeholder credential also fails with 400 before any evaluation is dispatched.

post/v1/eval-runs

Request

  • Base URL
    https://api.lightsage.com
  • Auth
    API key in header X-Lightsage-Api-Key

Request body

OR

Response

Successful Response

idstring required

Job ID for polling; not a result ID from GET /v1/api-performance/runs.

job_idstring required

The parent job ID. It currently contains the same polling ID as id.

type'prompts' | 'api_performance' required
source_type'prompts' | 'rest_api' | 'sdk' | 'cli' | 'custom_evals' required
stagestring nullable

Current lifecycle stage, including retrying while interrupted child jobs wait to be reclaimed.

dispatch_idstring nullable
estimated_costnumber nullable

Estimated credits for the run, worked out when it was queued. Credits actually used can differ by agent and model. Null when no estimate is available.

eval_countinteger nullable
model_countinteger nullable
errorstring nullable
created_atstring nullable
started_atstring nullable
completed_atstring nullable

Can be set while status is still summarizing; use status to detect completion.

Changes

No operation changes recorded.