USusesapient
Evals

Create Eval

Creates an eval and saves its eval configuration. This endpoint does not start an eval job. To run it, call POST /v1/eval-runs with source_type set to custom_evals, include the returned ID in custom_eval_ids, and provide targets or use saved target configuration.

post/v1/api-performance/custom-evals

Request

  • Base URL
    https://api.lightsage.com
  • Auth
    API key in header X-Lightsage-Api-Key

Request body

promptstring required

Workflow the eval should test.

prompt_templatestring

Rendered into prompt before storing. Variables: {{company_name}}, {{company_domain}}.

category_namestring
descriptionstring

Display description for the eval. Judging criteria belong in eval.graders.

tagstring
personastring

Persona whose workflow or perspective this eval represents.

competitorstring

Competitor ID; renders the prompt against that competitor.

prompt_competitor_groupstring

Base eval ID this competitor variant belongs to.

Response

Successful Response

idstring required
promptstring required

Rendered from prompt_template when one is set.

prompt_templatestring nullable

Variables: {{company_name}}, {{company_domain}}.

descriptionstring nullable

Display description for the eval; it does not affect grading.

tagstring nullable
personastring nullable

Persona whose workflow or perspective this eval represents.

competitorstring nullable

Null unless this is a competitor variant.

prompt_competitorstring nullable
prompt_competitor_groupstring nullable

Base eval of this variant; null on the base itself.

categoryobject nullable

Carries id, name, and slug.

created_atstring nullable
updated_atstring nullable

Changes

No operation changes recorded.