LAlangchain-ai
datasets

Fetch experiment runs for dataset examples

Changed on

Returns a paginated page of dataset examples with runs from the requested experiments. Response uses the canonical {items, next_cursor} envelope.

Self-hosted deployments require LangSmith v0.16 or later.

post/api/v2/datasets/{dataset_id}/experiment-runs

Request

  • Base URL
    https://api.smith.langchain.com
  • URL
    https://api.smith.langchain.com/api/v2/datasets/{dataset_id}/experiment-runs
  • Auth
    • API key in header X-API-Key and API key in header X-Tenant-Id
    • or HTTP bearer and API key in header X-Tenant-Id

Path parameters

dataset_idstring required

Dataset ID

Request body

comparative_experiment_idstring

comparative_experiment_id scopes pairwise-annotation feedback (optional).

cursorstring

cursor is the opaque string from a previous response's next_cursor. Absent for the first page.

example_idsstring[]

example_ids optionally restricts the page to these dataset example UUIDs (max 1000).

experiment_idsstring[]

experiment_ids lists the experiment (tracing session) UUIDs to query. Required, non-empty.

filtersobject

filters maps a project (session) UUID string to a list of filter expressions (optional).

page_sizeinteger

page_size is the maximum number of examples to return. Defaults to 20, max 100.

selectsQueryRunSelectField[]

selects lists which run properties to include. Omitted => only id. Tokens mirror /v2/runs/query.

Response

OK

next_cursorstring

Example response

{
  "items": [
    {
      "created_at": "2024-01-15T10:30:00.000Z",
      "dataset_id": "0190a1b2-c3d4-7ef0-a5b6-6ea3a82e9328",
      "id": "018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327",
      "modified_at": "2024-01-15T10:30:00.000Z",
      "runs": [
        {
          "app_path": "/app/chains/chat.py:invoke",
          "completion_cost": 0.0003,
          "completion_tokens": 150,
          "dotted_order": "20240115T103000000000Z018e4c7ea9fb7ef0a5b66ea3a82e9327.",
          "end_time": "2024-01-15T10:30:01.500Z",
          "error": "context deadline exceeded",
          "events": [
            {
              "name": "new_token",
              "time": "2024-01-15T10:30:00.312Z"
            }
          ],
          "first_token_time": "2024-01-15T10:30:00.312Z",
          "id": "018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327",
          "is_in_dataset": true,
          "is_root": true,
          "last_queued_at": "2024-01-15T10:31:00Z",
          "latency_seconds": 1.523,
          "ls_user_id": "f6a7b8c9-d0e1-4f2a-3b4c-5d6e7f8a9b0c",
          "name": "ChatOpenAI",
          "parent_run_ids": [
            "018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327",
            "a1b2c3d4-e5f6-4a5b-8c9d-0e1f2a3b4c5d"
          ],
          "price_model_id": "e5f6a7b8-c9d0-4e1f-2a3b-4c5d6e7f8a9b",
          "project_id": "018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327",
          "prompt_cost": 0.0002,
          "prompt_tokens": 200,
          "reference_dataset_id": "c3d4e5f6-a7b8-4c5d-0e1f-2a3b4c5d6e7f",
          "reference_example_id": "b2c3d4e5-f6a7-4b5c-9d0e-1f2a3b4c5d6e",
          "share_url": "https://smith.langchain.com/public/4f7a1b2c-8d9e-4a0b-9c1d-2e3f4a5b6c7d/r",
          "start_time": "2024-01-15T10:30:00.000Z",
          "tags": [
            "production",
            "gpt-4"
          ],
          "thread_id": "d4e5f6a7-b8c9-4d5e-1f2a-3b4c5d6e7f8a",
          "total_cost": 0.000525,
          "total_tokens": 350,
          "trace_id": "018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327"
        }
      ]
    }
  ]
}

Changes