---
title: "Create Evaluation Pipeline"
method: POST
path: "/reports"
tags: ["reports"]
---

# Create Evaluation Pipeline

`POST /reports`

## Request body

- CreateEvaluationPipelineRequest
  - `dataset_group_id` integer, required — ID of the dataset group to use.
  - `name` string, nullable — Name for the pipeline. Auto-generated if omitted.
  - `folder_id` integer, nullable — Folder ID for organization.
  - `dataset_version_number` integer, nullable — Specific dataset version. Uses latest published version if omitted.
  - `columns` EvaluationColumnDefinition[], nullable — Evaluation columns to add to the pipeline.
    - `column_type` 'PROMPT_TEMPLATE' | 'CODE_EXECUTION' | 'ENDPOINT' | 'WORKFLOW' | 'MCP' | 'HUMAN' | 'CONVERSATION_SIMULATOR' | 'LLM_ASSERTION' | 'AI_DATA_EXTRACTION' | 'COMPARE' | 'CONTAINS' | 'REGEX' | 'COSINE_SIMILARITY' | 'ABSOLUTE_NUMERIC_DISTANCE' | 'JSON_PATH' | 'XML_PATH' | 'REGEX_EXTRACTION' | 'PARSE_VALUE' | 'VARIABLE' | 'ASSERT_VALID' | 'COALESCE' | 'COMBINE_COLUMNS' | 'COUNT' | 'MATH_OPERATOR' | 'MIN_MAX', required — Type of evaluation column to create. Does not include COMPOSITION, which is Tables-only.
    - `name` string, required — Display name for the column. Must be unique within the pipeline.
    - `configuration` object, required — Column-type-specific configuration. See Node & Column Types for details.
    - `position` integer, nullable — 1-based position in the pipeline. Auto-assigned if omitted.
    - `is_part_of_score` boolean — When true, PromptLayer includes this column in built-in scoring by averaging its values.
  - `score_configuration` ScoreConfiguration — Custom scoring configuration. The code receives a data variable containing row dictionaries and must return an object with at least a score key from 0 to 100.
    - `code` string, required — Python or JavaScript code used to calculate the score.
    - `code_language` 'PYTHON' | 'JAVASCRIPT' — Language used by the scoring code.
  - `external_ids` ExternalId[] — External ID mappings to attach to the pipeline.
    - `source` string, required — The external system or namespace that owns the ID.
    - `external_id` string, required — The identifier for this entity in the external system.

## Response `201`

Evaluation pipeline created.

- CreateEvaluationPipelineResponse
  - `success` true, required
  - `report_id` integer, required — ID of the created evaluation pipeline.
  - `report_columns` ReportColumnSummary[] — Columns created on the pipeline.
    - `id` integer
    - `name` string
    - `column_type` 'PROMPT_TEMPLATE' | 'CODE_EXECUTION' | 'ENDPOINT' | 'WORKFLOW' | 'MCP' | 'HUMAN' | 'CONVERSATION_SIMULATOR' | 'LLM_ASSERTION' | 'AI_DATA_EXTRACTION' | 'COMPARE' | 'CONTAINS' | 'REGEX' | 'COSINE_SIMILARITY' | 'ABSOLUTE_NUMERIC_DISTANCE' | 'JSON_PATH' | 'XML_PATH' | 'REGEX_EXTRACTION' | 'PARSE_VALUE' | 'VARIABLE' | 'ASSERT_VALID' | 'COALESCE' | 'COMBINE_COLUMNS' | 'COUNT' | 'MATH_OPERATOR' | 'MIN_MAX' | 'DATASET'
    - `position` integer
    - `configuration` object
  - `external_ids` ExternalId[], required — External ID mappings attached to the pipeline.
    - `source` string, required — The external system or namespace that owns the ID.
    - `external_id` string, required — The identifier for this entity in the external system.

## Other responses

- `400` — Bad Request - Invalid parameters, draft dataset version, or no datasets in group
- `401` — Unauthorized - missing or invalid API key.
- `403` — Forbidden - API key does not have access to the requested resource.
- `404` — Not found - requested resource does not exist or is not accessible.
- `409` — External ID conflict
- `422` — Validation error - request parameters or body are invalid.

---

[API](https://skmtc.dev/promptlayer/apis/promptlayer-api.md) · [All operations](https://skmtc.dev/promptlayer/apis/promptlayer-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/promptlayer/promptlayer-api/revisions/a23093a0a506/schema)