---
title: "Diff Benchmark Tasks"
method: POST
path: "/api/v1/benchmarks/{bench_id}/tasks/diff"
tags: ["benchmarks"]
---

# Diff Benchmark Tasks

`POST /api/v1/benchmarks/{bench_id}/tasks/diff`

## Request

- Base URL: `https://api.trajectory.ai` (the document declares no server; this is the origin it was published from)
- URL: `https://api.trajectory.ai/api/v1/benchmarks/{bench_id}/tasks/diff`
- Auth: none declared

Split submitted tasks into new tasks, tasks already stored with identical content, and updates whose names are stored with different content. Lets a client upload only what is missing or changed.

## Path parameters

- `bench_id` string, required

## Request body

- BenchmarkTaskDiffRequest
  - `manifest` BenchmarkSpec, required
    - `name` string, required
    - `description` string
    - `visibility` 'public' | 'private'
    - `runtime` RuntimeSpec
      - `runtime_id` string, nullable
      - `source` ImageSpec
        - `dockerfile_path` string, nullable
        - `image_ref` string, nullable
    - `tasks` TaskSpec[], required
      - `name` string, required
      - `id` string, nullable
      - `split` 'train' | 'test'
      - `input_messages` BenchmarkMessage[]: Messages stored on the task. For SDK evaluation and training, your agent script must pass messages to the model; this field does not do that automatically.
        - `role` string, required
        - `content` string, nullable
        - `tool_calls` ToolCall[]
          - `name` string, required
          - `arguments` object, required
          - `id` string, nullable
          - `raw_string` string, nullable
          - `argument_encoding` 'json' | 'xml'
          - `is_parsed_successfully` boolean
          - `parser_error_msg` string, nullable
        - `tool_response` ToolResponse
          - `id` string, required
          - `name` string, required
          - `arguments` object, required
          - `response` unknown
          - `error` string, nullable
          - `raw_name_args` string, nullable
          - `metadata` ToolResponseMetadata
            - `generated_tool_info` GeneratedToolInfo
              - …
          - `duration_seconds` number, nullable
          - `is_env_done` boolean
        - `tool_definitions` ToolDefinition[]
          - `name` string, required
          - `description` string, required
          - `parameters` object, required
        - `usage` MessageUsage
          - `prompt_tokens` integer
          - `completion_tokens` integer
          - `total_tokens` integer
        - `finish_reason` string, nullable
        - `metadata` object, nullable
        - `reasoning` string, nullable
        - `reasoning_details` unknown
        - `tokens` integer[], nullable
        - `token_masks` integer[], nullable
        - `logprobs` number[], nullable
        - `trainable_status` 'trainable' | 'not_trainable' | 'superseded' | 'summarization_boundary'
      - `reference_trajectory_id` string, nullable
      - `image` unknown
      - `runtime` RuntimeSpec
        - `runtime_id` string, nullable
        - `source` ImageSpec
          - `dockerfile_path` string, nullable
          - `image_ref` string, nullable
      - `run_command` string, required: For SDK evaluation and training, the command to run your agent in the sandbox, e.g. `python /app/agent.py`; include the executable, not only its arguments. Your script must call the model, run any tools, log a reward, and finalize the trajectory before exiting.
      - `env_vars` object: Keys must match [A-Za-z_][A-Za-z0-9_]* and cannot start with TRAJECTORY_ or use MODEL_ENDPOINT_ID, MODEL_ENDPOINT_ACCESS_TOKEN, or MODEL_ENDPOINT_URL.
      - `env_resources` EnvResources
        - `docker_engine` boolean: Require a Docker-capable sandbox. The user image must provide Docker and the harness; the harness owns nested containers and grading.
        - `cpus` number, nullable
        - `memory_mb` integer, nullable
        - `gpus` integer
        - `network_mode` 'no-network' | 'public' | 'allowlist'
        - `allowed_hosts` string[]
        - `agent_user` string, nullable
        - `env` object
        - `setup_commands` array[]
          - string[]
        - `mcp_servers` McpServerSpec[]
          - `name` string, required
          - `transport` 'sse' | 'streamable-http' | 'stdio'
          - `url` string, nullable
          - `command` string, nullable
          - `args` string[]
        - `allow_internet` boolean, nullable
      - `spec` object
      - `task_spec_uri` string, nullable
      - `tags` string[]: Up to 20 distinct, non-empty task tags of at most 64 characters each.
  - `artifacts` object

## Response `200`

Successful Response

- BenchmarkTaskDiffResponse
  - `bench_id` string, required
  - `new` string[], required
  - `present` string[], required
  - `updated` string[], required
  - `task_count` integer, required

## Changes

- **2026-10-31** `7b218d495d5c`: 1 warning
  - removed the request property `manifest/family`
- **2026-10-07** `db059a2d3fb0`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/trajectory/apis/trajectory-api/changes/api/v1/benchmarks/:bench_id/tasks/diff/post.md)

---

[API](https://skmtc.dev/trajectory/apis/trajectory-api.md) - [All operations](https://skmtc.dev/trajectory/apis/trajectory-api/llms.txt) - [OpenAPI document](https://skmtc.dev/trajectory/apis/trajectory-api/revisions/5b5a8ae4b4a4?raw)