---
title: "Create Eval"
method: POST
path: "/v1/api-performance/custom-evals"
tags: ["Evals"]
---

# Create Eval

`POST /v1/api-performance/custom-evals`

## Request

- Base URL: `https://api.lightsage.com`
- URL: `https://api.lightsage.com/v1/api-performance/custom-evals`
- Auth: API key in header `X-Lightsage-Api-Key`

Creates an eval and saves its eval configuration. This endpoint does not start an eval job. To run it, call `POST /v1/eval-runs` with `source_type` set to `custom_evals`, include the returned ID in `custom_eval_ids`, and provide `targets` or use saved target configuration.

## Request body

- PublicCustomEvalRequest
  - `prompt` string, required: Workflow the eval should test.
  - `prompt_template` string: Rendered into prompt before storing. Variables: {{company_name}}, {{company_domain}}.
  - `category_name` string
  - `description` string: Display description for the eval. Judging criteria belong in `eval.graders`.
  - `tag` string
  - `persona` string: Persona whose workflow or perspective this eval represents.
  - `competitor` string: Competitor ID; renders the prompt against that competitor.
  - `prompt_competitor_group` string: Base eval ID this competitor variant belongs to.
  - `eval` PublicEvalDefinitionUpdate
    - `custom_prompt` string: Overrides the stored prompt.
    - `expected_behavior` string: Deprecated compatibility alias for a judge criterion. Prefer `graders`; this value updates the primary judge grader.
    - `graders` PublicGraderDefinition[], nullable: Complete grader definitions used to score the eval. Supplying this field replaces the stored grader array.
      - `name` string, required
      - `type` string, required
      - `needle` string, nullable
      - `needles` string[], nullable
      - `pattern` string, nullable
      - `case_sensitive` boolean
      - `criteria` string, nullable
      - `weight` number
      - `blocking` boolean
    - `starting_project_id` string: ID from GET /v1/api-performance/starter-projects. Empty string clears it.
    - `docs_mode` 'default' | 'include' | 'exclude': include adds the organization's docs to the agent's context; exclude withholds them.
    - `include_env_vars` boolean: Names only, never values.
    - `env_profile_ids` string[], nullable: IDs from env_profiles in GET /v1/api-performance/config. Empty list clears the selection.
    - `model_ids` string[], nullable: IDs from GET /v1/api-performance/targets. Empty list falls back to the config-level models.
    - `compare_skills` boolean: Run an additional comparison with skill context enabled.
    - `skills_mode` 'none' | 'all' | 'selected': selected uses skill_ids; all uses every skill; none supplies no skill context.
    - `skill_ids` string[], nullable: IDs from GET /v1/api-performance/skills; used when skills_mode is selected.
    - `mcp_enabled` boolean
    - `search_grounding` 'disabled' | 'enabled' | 'default': Requested web-retrieval behavior for an eval run.
    - `grounding_enabled` boolean: Allow native web search and URL retrieval tools during agent runs. When false, configured brand docs are also withheld.
    - `run_context` object, nullable
    - `enabled` boolean

## Response `200`

Successful Response

- PublicCustomEval
  - `id` string, required
  - `prompt` string, required: Rendered from `prompt_template` when one is set.
  - `prompt_template` string, nullable: Variables: {{company_name}}, {{company_domain}}.
  - `description` string, nullable: Display description for the eval; it does not affect grading.
  - `tag` string, nullable
  - `persona` string, nullable: Persona whose workflow or perspective this eval represents.
  - `competitor` string, nullable: Null unless this is a competitor variant.
  - `prompt_competitor` string, nullable
  - `prompt_competitor_group` string, nullable: Base eval of this variant; null on the base itself.
  - `category` object, nullable: Carries `id`, `name`, and `slug`.
  - `eval` PublicEvalDefinition
    - `id` string, required
    - `operation_id` string, required: For a user-created eval, equals the eval ID when `source_id` is null.
    - `source_id` string, nullable: Null for a user-created eval configuration.
    - `source` PublicApiPerformanceSourceRef
      - `id` string, required
      - `type` string, nullable: The source type: `rest_api`, `python_sdk`, `node_sdk`, or `cli`.
      - `label` string, nullable
    - `eval_type` string, required
    - `prompt` string, nullable
    - `custom_prompt` string, nullable: Overrides `prompt` when set.
    - `expected_behavior` string, nullable: Deprecated compatibility view of the primary judge criterion. Use `graders` for the canonical configuration.
    - `graders` PublicGraderDefinition[]: Canonical grader definitions used to score the eval.
      - `name` string, required
      - `type` string, required
      - `needle` string, nullable
      - `needles` string[], nullable
      - `pattern` string, nullable
      - `case_sensitive` boolean
      - `criteria` string, nullable
      - `weight` number
      - `blocking` boolean
    - `starting_project_id` string, nullable
    - `docs_mode` 'default' | 'include' | 'exclude': `include` adds the organization's docs to the agent's context; `exclude` withholds them.
    - `include_env_vars` boolean: Names only, never values.
    - `env_profile_ids` string[]
    - `model_ids` string[]: Empty means the config-level models are used.
    - `compare_skills` boolean: Whether to run an additional comparison with skill context enabled.
    - `skills_mode` 'none' | 'all' | 'selected': `selected` uses `skill_ids`; `all` uses every skill; `none` supplies no skill context.
    - `skills_enabled` boolean
    - `skill_ids` string[]
    - `mcp_enabled` boolean
    - `search_grounding` 'disabled' | 'enabled' | 'default': Requested web-retrieval behavior for an eval run.
    - `grounding_enabled` boolean: Whether native web search and URL retrieval tools are available to the agent. When false, configured brand docs are also withheld.
    - `run_context` object
    - `enabled` boolean
  - `created_at` string, nullable
  - `updated_at` string, nullable

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/usesapient/apis/sapient-public-api.md) - [All operations](https://skmtc.dev/usesapient/apis/sapient-public-api/llms.txt) - [OpenAPI document](https://skmtc.dev/usesapient/apis/sapient-public-api/revisions/14890dfe867e?raw)