LLM Observability

Aggregate LLM Observability experimentation

Execute an analytics aggregation over LLM Observability experimentation data. Use this endpoint to compute metrics (for example average eval scores) grouped by fields such as span_id or experiment_id.

At least one compute definition and one index must be provided.

post/api/v2/llm-obs/v1/experimentation/analytics

Request

Base URL
https://api.datadoghq.com
URL
https://api.datadoghq.com/api/v2/llm-obs/v1/experimentation/analytics
Auth
API key in header DD-API-KEY and API key in header DD-APPLICATION-KEY

Request body

Example request

{
  "data": {
    "attributes": {
      "aggregate": {
        "compute": [
          {
            "metric": "score_value",
            "name": "avg_faithfulness"
          }
        ],
        "group_by": [
          {
            "field": "span_id"
          }
        ],
        "indexes": [
          "experiment-evals"
        ],
        "limit": 1000,
        "search": {
          "query": "@experiment_id:3fd6b5e0-8910-4b1c-a7d0-5b84de329012"
        },
        "time": {
          "from": 1705312200000,
          "to": 1705315800000
        }
      }
    },
    "type": "experimentation"
  }
}

Response

Example response

{
  "data": {
    "attributes": {
      "hit_count": 1500,
      "result": {
        "values": [
          {
            "by": {
              "span_id": "span-7a1b2c3d"
            },
            "metrics": {
              "score_value": 0.85
            }
          }
        ]
      }
    },
    "id": "00000000-0000-0000-0000-000000000001",
    "type": "experimentation"
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.