Evaluations

Add a dataset item

Adds one test case. input is replayed verbatim as the generation's messages, so it must be a non-empty array of { role, content }.

post/v1/projects/{project_id}/datasets/{dataset_id}/items

Request

  • Base URL: https://api.naturali.ai
  • URL: https://api.naturali.ai/v1/projects/{project_id}/datasets/{dataset_id}/items
  • Auth: one of:
    • HTTP bearer
    • OAuth 2 (authorization code, authorization URL https://api.naturali.ai/authorize, token URL https://api.naturali.ai/token, scopes: mcp:access)

Path parameters

dataset_idstring required
Example:dset_V1StGXR8Z5jdHi6B

Dataset ID

Request body

expected_outputstring nullable

Reference answer for exact_match / embedding_similarity / llm_judge scorers

metadataNullableMetadataBag nullable

A MetadataBag on a request field where null is meaningful: a full-replacement update that clears the bag. A record whose bag is unset leaves the field out.

Example request

{
  "input": [
    {
      "role": "user",
      "content": "When is my invoice issued?"
    }
  ],
  "expected_output": "Your invoice is issued on the first of each month.",
  "metadata": {
    "author": "John",
    "revision": 2
  }
}

Response

Dataset item created successfully

idstring
dataset_idstring
expected_outputstring nullable
metadataMetadataBag

Caller-owned annotations on a resource, stored as the object they were written as: the types a value was written with are the types a read returns, so a filter can ask an ordering question about a number. Unlike other body fields, keys are stored and returned verbatim in the casing supplied — they are not converted between snake_case and camelCase.

No key is reserved, and that is the point: every piece of state the platform owns lives in its own typed column, so nothing written here reaches platform state. The platform never reads the bag — it is not an IAM context, not a policy input and not part of a prompt — which is what separates it from a tag bag.

source_generation_idstring nullable

The generation this item was curated from. A curated item is a deliberate fixture: erasing the source generation neither deletes nor mutates it.

created_atstring date-time
updated_atstring date-time

Example response

{
  "id": "dsit_V1StGXR8Z5jdHi6B",
  "dataset_id": "dset_V1StGXR8Z5jdHi6B",
  "input": [
    {
      "role": "user",
      "content": "When is my invoice issued?"
    }
  ],
  "metadata": {
    "author": "John",
    "revision": 2
  },
  "source_generation_id": "gen_V1StGXR8Z5jdHi6B"
}

Changes

No recorded changes to this operation across all 1 revision of this API.