evals

Create Eval

Changed on

Create a new evaluation.

Model-based routing (for multi-account support):

  • Pass model via header: x-litellm-model: gpt-4-account-1
  • Pass model via query: ?model=gpt-4-account-1
  • Pass model via body: {"model": "gpt-4-account-1"}

Example usage:

curl -X POST "http://localhost:4000/v1/evals"       -H "Authorization: Bearer your-key"       -H "Content-Type: application/json"       -d '{
    "name": "Test Eval",
    "data_source_config": {"type": "file", "file_id": "file-abc123"},
    "testing_criteria": {"graders": [{"type": "llm_as_judge"}]}
  }'

Returns: Eval object with id, status, timestamps, etc.

post/v1/evals

Request

  • Base URL
    https://api.flock.io (the document declares no server; this is the origin it was published from)
  • URL
    https://api.flock.io/v1/evals
  • Auth
    API key in header x-litellm-api-key

Query parameters

custom_llm_providerstring nullable

Response

Successful Response

created_atinteger required
data_source_configobject required
idstring required
metadataobject nullable
namestring nullable
objectstring
testing_criteriaobject[] required
updated_atinteger nullable

Changes