---
title: "Update proving scenario"
method: PUT
path: "/v3/proving/scenarios/{scenario_id}"
tags: ["v3", "proving"]
---

# Update proving scenario

`PUT /v3/proving/scenarios/{scenario_id}`

## Request

- Base URL: `https://api.mentium.io` (the document declares no server; this is the origin it was published from)
- URL: `https://api.mentium.io/v3/proving/scenarios/{scenario_id}`
- Auth: HTTP bearer

## Path parameters

- `scenario_id` string, uuid, required

## Headers

- `mentium-workspace-id` string, nullable

## Request body

- ProvingScenarioUpdate
  - `severity` 'p0_safety' | 'p1_correctness' | 'p2_quality'
  - `name` string, nullable: Scenario name
  - `description` string, nullable: Scenario description
  - `node_type` 'conversation-agent' | 'ai-agent', nullable: Send together with node_config.
  - `node_config` object, nullable
  - `prefix_transcript_turns` ProvingTranscriptTurn[], nullable: Prefix transcript turns. May start with either a user or agent turn. Must end with an agent turn (latest response).
    - `type` 'transcript_turn'
    - `transcript_turn_idx` integer: 0-indexed index of ProvingScenario.prefix_transcript_turns and ProvingRun.transcript_turns. 0 for introductory agent turn, 1 for first user turn, 2 for second agent turn, etc. Used by backend and frontend for ordering turns.
    - `role` 'user' | 'agent', required
    - `transcript_turn_items` union[]
      - union
        - ProvingTranscriptTurnItemMessage
          - `type` 'transcript_turn_item_message'
          - `transcript_turn_item_idx` integer: 0-indexed index of ProvingTranscriptTurn.transcript_turn_items. Used by backend and frontend for ordering turn items.
          - `message_content` string, required
        - ProvingTranscriptTurnItemToolCall
          - `type` 'transcript_turn_item_tool_call'
          - `transcript_turn_item_idx` integer: 0-indexed index of ProvingTranscriptTurn.transcript_turn_items. Used by backend and frontend for ordering turn items.
          - `tool_call_id` string, required
          - `tool_call_name` string, required
          - `tool_call_params` object, required
          - `tool_call_result` object, required
  - `mock_tool_calls` ProvingMockToolCallInput[], nullable: Tool call mocks for replay during proving runs
    - `type` 'mock_tool_call'
    - `mock_tool_call_idx` integer: 0-indexed index of ProvingScenario.mock_tool_calls. 0 for first mock tool call, 1 for second mock tool call, etc. Used by backend and frontend for ordering tool calls. Tool calls can be repeated with the same name and parameters and can only be consumed once so needed to be ordered for priority. E.g. first tool call to delete a database record must give success result but second tool call to delete the same record must give failure result.
    - `transcript_turn_idx` integer, nullable: 0-indexed index of ProvingScenario.prefix_transcript_turns. None for a transcript-level, 0 for agent introductory turn, 1 for first user turn, 2 for second agent turn, etc.
    - `tool_call_name` string, required
    - `tool_call_params` ProvingMockToolCallParamItem[]
      - `type` 'mock_tool_call_param_item'
      - `mock_tool_call_param_item_idx` integer: 0-indexed index of ProvingMockToolCall.tool_call_params. Not used by backend since tool call parameters are functionally unordered. Used by frontend for ordering tool call parameters in the UI for consistency.
      - `tool_call_param_path` union[], required
        - union
          - string
          - integer
      - `tool_call_param_value` union, required
        - string
        - integer
        - number
        - boolean
      - `match_method` 'exact_match' | 'one_of' | 'contains' | 'in_range', required
      - `case_sensitive` boolean
    - `tool_call_result` object, required
  - `evaluations` ProvingEvaluationInput[], nullable: Evaluation criteria for agent responses
    - `type` 'evaluation'
    - `evaluation_idx` integer: 0-indexed index of ProvingScenario.evaluations. Used by backend and frontend for ordering evaluations. Evaluations are ran sequentially. E.g. agent must call google search tool with terms 'weather in San Francisco' (evaluation item 0) and 'weather in New York' (evaluation item 1) and then must respond about temerature and rain chances (evaluation item 2).
    - `name` string, required
    - `transcript_turn_idx` integer, nullable: 0-indexed index of ProvingScenario.prefix_transcript_turns. Used by backend and frontend for ordering evaluations. None for a transcript-level and must run for all turns serialized and passed to the LLM at once, 0 for agent introductory turn, 1 for first user turn, 2 for second agent turn, etc. Transcript-level evaluation limited to only LLM evaluation since message and tool call evaluation at transcript level does not make sense.
    - `operator` 'not' | 'and' | 'or', required
    - `evaluation_items` union[]
      - union
        - ProvingEvaluationItemMessage
          - `type` 'evaluation_item_message'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item must run for each of the turn items and pass for one of them to pass the evaluation.
          - `name` string, required
          - `expected_message_content` string, required
          - `match_method` 'exact_match' | 'one_of' | 'contains', required
          - `case_sensitive` boolean
        - ProvingEvaluationItemToolCallInput
          - `type` 'evaluation_item_tool_call'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item must run for each of the turn items and pass for one of them to pass the evaluation.
          - `name` string, required
          - `tool_call_name` string, required
          - `expected_tool_call_params` ProvingEvaluationItemToolCallParamItem[]
            - `type` 'evaluation_item_tool_call_param_item'
            - `evaluation_item_tool_call_param_item_idx` integer: 0-indexed index of ProvingEvaluationItemToolCall.expected_tool_call_params. Not used by backend since evaluation item tool call parameters are unordered. Used by frontend for ordering evaluation item tool call parameters in the UI.
            - `name` string, required
            - `tool_call_param_path` union[], required
              - …
            - `tool_call_param_expected_value` union, required
              - …
            - `match_method` 'exact_match' | 'one_of' | 'contains' | 'in_range', required
            - `case_sensitive` boolean
        - ProvingEvaluationItemLLM
          - `type` 'evaluation_item_llm'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item must run for all turn items serialized and passed to the LLM at once and pass for one of them to pass the evaluation.
          - `name` string, required
          - `expected_turn_observation` string, required: Natural language expected observation in the turn
          - `expected_turn_observation_criteria` string[]: Natural-language criteria, each judged met or not; empty judges expected_turn_observation alone.
          - `criteria_pass_fraction` number: Fraction of the criteria that must be met. Ignored when there are no criteria.
        - ProvingEvaluationItemSafetyInvariant
          - `type` 'evaluation_item_safety_invariant'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item runs once over the whole conversation.
          - `name` string, required
          - `invariant_id` 'quotes_only_known_figures' | 'names_only_allowed_figures' | 'never_names_figures', required
          - `invariant_params` object: What the scenario tells the invariant, such as figures it may speak

## Response `200`

Successful Response

- ProvingScenarioResponse
  - `id` string, required
  - `phone_call_id` string, nullable
  - `ai_agent_session_id` string, nullable
  - `is_system` boolean: System-owned; immutable through this API
  - `severity` 'p0_safety' | 'p1_correctness' | 'p2_quality'
  - `name` string, required
  - `description` string, nullable
  - `node_type` 'conversation-agent' | 'ai-agent'
  - `node_config` object
  - `prefix_transcript_turns` ProvingTranscriptTurn[]
    - `type` 'transcript_turn'
    - `transcript_turn_idx` integer: 0-indexed index of ProvingScenario.prefix_transcript_turns and ProvingRun.transcript_turns. 0 for introductory agent turn, 1 for first user turn, 2 for second agent turn, etc. Used by backend and frontend for ordering turns.
    - `role` 'user' | 'agent', required
    - `transcript_turn_items` union[]
      - union
        - ProvingTranscriptTurnItemMessage
          - `type` 'transcript_turn_item_message'
          - `transcript_turn_item_idx` integer: 0-indexed index of ProvingTranscriptTurn.transcript_turn_items. Used by backend and frontend for ordering turn items.
          - `message_content` string, required
        - ProvingTranscriptTurnItemToolCall
          - `type` 'transcript_turn_item_tool_call'
          - `transcript_turn_item_idx` integer: 0-indexed index of ProvingTranscriptTurn.transcript_turn_items. Used by backend and frontend for ordering turn items.
          - `tool_call_id` string, required
          - `tool_call_name` string, required
          - `tool_call_params` object, required
          - `tool_call_result` object, required
  - `mock_tool_calls` ProvingMockToolCallOutput[]
    - `type` 'mock_tool_call'
    - `mock_tool_call_idx` integer: 0-indexed index of ProvingScenario.mock_tool_calls. 0 for first mock tool call, 1 for second mock tool call, etc. Used by backend and frontend for ordering tool calls. Tool calls can be repeated with the same name and parameters and can only be consumed once so needed to be ordered for priority. E.g. first tool call to delete a database record must give success result but second tool call to delete the same record must give failure result.
    - `transcript_turn_idx` integer, nullable: 0-indexed index of ProvingScenario.prefix_transcript_turns. None for a transcript-level, 0 for agent introductory turn, 1 for first user turn, 2 for second agent turn, etc.
    - `tool_call_name` string, required
    - `tool_call_params` ProvingMockToolCallParamItem[]
      - `type` 'mock_tool_call_param_item'
      - `mock_tool_call_param_item_idx` integer: 0-indexed index of ProvingMockToolCall.tool_call_params. Not used by backend since tool call parameters are functionally unordered. Used by frontend for ordering tool call parameters in the UI for consistency.
      - `tool_call_param_path` union[], required
        - union
          - string
          - integer
      - `tool_call_param_value` union, required
        - string
        - integer
        - number
        - boolean
      - `match_method` 'exact_match' | 'one_of' | 'contains' | 'in_range', required
      - `case_sensitive` boolean
    - `tool_call_result` object, required
  - `evaluations` ProvingEvaluationOutput[]
    - `type` 'evaluation'
    - `evaluation_idx` integer: 0-indexed index of ProvingScenario.evaluations. Used by backend and frontend for ordering evaluations. Evaluations are ran sequentially. E.g. agent must call google search tool with terms 'weather in San Francisco' (evaluation item 0) and 'weather in New York' (evaluation item 1) and then must respond about temerature and rain chances (evaluation item 2).
    - `name` string, required
    - `transcript_turn_idx` integer, nullable: 0-indexed index of ProvingScenario.prefix_transcript_turns. Used by backend and frontend for ordering evaluations. None for a transcript-level and must run for all turns serialized and passed to the LLM at once, 0 for agent introductory turn, 1 for first user turn, 2 for second agent turn, etc. Transcript-level evaluation limited to only LLM evaluation since message and tool call evaluation at transcript level does not make sense.
    - `operator` 'not' | 'and' | 'or', required
    - `evaluation_items` union[]
      - union
        - ProvingEvaluationItemMessage
          - `type` 'evaluation_item_message'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item must run for each of the turn items and pass for one of them to pass the evaluation.
          - `name` string, required
          - `expected_message_content` string, required
          - `match_method` 'exact_match' | 'one_of' | 'contains', required
          - `case_sensitive` boolean
        - ProvingEvaluationItemToolCallOutput
          - `type` 'evaluation_item_tool_call'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item must run for each of the turn items and pass for one of them to pass the evaluation.
          - `name` string, required
          - `tool_call_name` string, required
          - `expected_tool_call_params` ProvingEvaluationItemToolCallParamItem[]
            - `type` 'evaluation_item_tool_call_param_item'
            - `evaluation_item_tool_call_param_item_idx` integer: 0-indexed index of ProvingEvaluationItemToolCall.expected_tool_call_params. Not used by backend since evaluation item tool call parameters are unordered. Used by frontend for ordering evaluation item tool call parameters in the UI.
            - `name` string, required
            - `tool_call_param_path` union[], required
              - …
            - `tool_call_param_expected_value` union, required
              - …
            - `match_method` 'exact_match' | 'one_of' | 'contains' | 'in_range', required
            - `case_sensitive` boolean
        - ProvingEvaluationItemLLM
          - `type` 'evaluation_item_llm'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item must run for all turn items serialized and passed to the LLM at once and pass for one of them to pass the evaluation.
          - `name` string, required
          - `expected_turn_observation` string, required: Natural language expected observation in the turn
          - `expected_turn_observation_criteria` string[]: Natural-language criteria, each judged met or not; empty judges expected_turn_observation alone.
          - `criteria_pass_fraction` number: Fraction of the criteria that must be met. Ignored when there are no criteria.
        - ProvingEvaluationItemSafetyInvariant
          - `type` 'evaluation_item_safety_invariant'
          - `evaluation_item_idx` integer: 0-indexed index of ProvingEvaluation.evaluation_items. Not used by backend since evaluation items are unordered. Used by frontend for ordering evaluation items in the UI. Evaluation item runs once over the whole conversation.
          - `name` string, required
          - `invariant_id` 'quotes_only_known_figures' | 'names_only_allowed_figures' | 'never_names_figures', required
          - `invariant_params` object: What the scenario tells the invariant, such as figures it may speak
  - `linked_agent_count` integer
  - `definition_version` integer
  - `definition_digest` string: A fingerprint of what the test is now; a run whose snapshot carries a different one ran an older version of the test
  - `mentium_created_at` string, date-time, required
  - `mentium_updated_at` string, date-time, required

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/mentium/apis/mentium-api.md) - [All operations](https://skmtc.dev/mentium/apis/mentium-api/llms.txt) - [OpenAPI document](https://skmtc.dev/mentium/apis/mentium-api/revisions/b4ed8e59432e?raw)