---
title: "Validate Llm Scorer Dataset"
method: POST
path: "/v2/scorers/llm/validate/dataset"
tags: ["data", "prompts", "rows"]
---

# Validate Llm Scorer Dataset

`POST /v2/scorers/llm/validate/dataset`

## Request

- Base URL: `https://api.galileo.ai`
- URL: `https://api.galileo.ai/v2/scorers/llm/validate/dataset`
- Auth: one of:
  - API key in header `Galileo-API-Key`
  - API key in header `Splunk-AO-API-Key`
  - OAuth 2
    - Password
    - Token URL: `https://api.galileo.ai/login`
  - HTTP basic

## Request body

- ValidateLLMScorerDatasetRequest: Request to validate a new LLM scorer against a dataset.
  - `query` string, required
  - `response` string, required
  - `chain_poll_template` ChainPollTemplate, required: Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt.
    - `metric_system_prompt` string, nullable: System prompt for the metric.
    - `metric_description` string, nullable: Description of what the metric should do.
    - `value_field_name` string: Field name to look for in the chainpoll response, for the rating.
    - `explanation_field_name` string: Field name to look for in the chainpoll response, for the explanation.
    - `template` string, required: Chainpoll prompt template.
    - `metric_few_shot_examples` FewShotExample[]: Few-shot examples for the metric.
      - `generation_prompt_and_response` string, required
      - `evaluating_response` string, required
    - `response_schema` object, nullable: Response schema for the output
  - `scorer_configuration` GeneratedScorerConfiguration, required
    - `model_alias` string
    - `num_judges` integer
    - `output_type` 'boolean' | 'categorical' | 'count' | 'discrete' | 'freeform' | 'percentage' | 'multilabel' | 'retrieved_chunk_list_boolean' | 'boolean_multilabel': Enumeration of output types.
    - `scoreable_node_types` string[]: Types of nodes that can be scored by this scorer.
    - `cot_enabled` boolean: Whether chain of thought is enabled for this scorer.
    - `ground_truth` boolean: Whether ground truth is enabled for this scorer.
    - `multimodal_capabilities` MultimodalCapability[], nullable: Multimodal capabilities required by this scorer.
  - `normalized_input` union[], nullable: Optional multimodal content parts. When set, replaces the text-only query/response formatting in the validation job so that file content is passed through to the LLM.
    - union
      - TextContentPart: A text segment within a message.
        - `type` 'text'
        - `text` string, required
      - FileContentPart: Reference to a file associated with this message. The file_id can be resolved via the ``files`` dict returned on trace/span detail responses, which contains metadata such as modality, MIME type, and a presigned download URL.
        - `type` 'file'
        - `file_id` string, uuid4, required
  - `user_prompt` string, required
  - `dataset_id` string, uuid4, required
  - `dataset_version_index` integer, nullable
  - `limit` integer: Maximum number of dataset rows to process.
  - `starting_token` integer, nullable: Pagination offset into dataset rows.
  - `sort` object, nullable: Optional sort configuration for dataset rows.

## Response `200`

Successful Response

- ValidateLLMScorerDatasetResponse
  - `metrics_experiment_id` string, uuid4, required
  - `project_id` string, uuid4, required

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/galileo/apis/galileo-api.md) - [All operations](https://skmtc.dev/galileo/apis/galileo-api/llms.txt) - [OpenAPI document](https://skmtc.dev/galileo/apis/galileo-api/revisions/441b96da8c30?raw)