Evaluations

Create a dataset

Creates a project-scoped dataset — a named collection of test cases an eval runs an agent against. Names are unique per project.

Datasets are operator-owned fixtures. The platform's content purge never deletes or mutates a dataset item, so erasing a generation cannot silently stop a test suite from being runnable.

post/v1/projects/{project_id}/datasets

Request

  • Base URL: https://api.naturali.ai
  • URL: https://api.naturali.ai/v1/projects/{project_id}/datasets
  • Auth: one of:
    • HTTP bearer
    • OAuth 2 (authorization code, authorization URL https://api.naturali.ai/authorize, token URL https://api.naturali.ai/token, scopes: mcp:access)

Request body

namestring required

Unique name within the project

descriptionstring nullable

What this suite covers

Example request

{
  "name": "billing-regressions",
  "description": "Questions the billing agent regressed on in Q2"
}

Response

Dataset created successfully

idstring
project_idstring
namestring
descriptionstring nullable
created_atstring date-time
updated_atstring date-time

Example response

{
  "id": "dset_V1StGXR8Z5jdHi6B"
}

Changes

No recorded changes to this operation across all 1 revision of this API.