Sign in
Evals

Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and a datasource. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](/docs/guides/evals).

post/evals

Request body

namestring

The name of the evaluation.

metadataMetadata nullable

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

Response

OK

object'eval' required

The object type.

idstring required

Unique identifier for the evaluation.

namestring required

The name of the evaluation.

created_atinteger required

The Unix timestamp (in seconds) for when the eval was created.

metadataMetadata nullable required

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

Example response

{
  "name": "Chatbot effectiveness Evaluation"
}

Changes