LAlangchain-ai
run

Create Decision Model Rule

Changed on

Create a decision-model evaluator (Jev, SemIf or OpenAI Decisions) from its prompt and questions, and attach it to a project or dataset. Each question's key becomes a feedback key.

post/api/v1/runs/rules/decision-models

Request

  • Base URL
    https://api.smith.langchain.com
  • URL
    https://api.smith.langchain.com/api/v1/runs/rules/decision-models
  • Auth
    • API key in header X-API-Key
    • or API key in header X-Tenant-Id
    • or HTTP bearer

Request body

display_namestring required
session_idstring uuid nullable

The tracing project to evaluate.

dataset_idstring uuid nullable

The dataset whose experiments to evaluate.

template_format'mustache' | 'f-string'
variable_mappingobject nullable

Maps the prompt's variables to run fields.

questionsobject required

The questions, keyed by feedback key, in the order to ask them.

provider'typesafe' | 'typesafe_compatible' | 'openai_decisions'

typesafe for Jev and SemIf, typesafe_compatible for a TypeSafe-compatible endpoint, or openai_decisions.

modelstring nullable

Defaults to jev-latest for typesafe and gpt-6-luna for openai_decisions. Use semif-qwen3.5-4b for SemIf.

base_urlstring nullable

Call this endpoint directly instead of through the LangSmith gateway, with the workspace's TYPESAFE_API_KEY or OPENAI_API_KEY secret. Required for typesafe_compatible.

model_headersobject nullable

Headers sent with each model call. They are saved in the evaluator's prompt as given, so do not put API keys here.

timeout_secondsinteger nullable

Seconds to wait for each model call. openai_decisions only.

max_retriesinteger nullable

How many times to retry a failed model call. openai_decisions only.

playground_settings_idstring uuid nullable

A saved model configuration to start from: its model, endpoint, key and settings apply unless set here.

sampling_ratenumber

The fraction of runs to evaluate.

filterstring nullable

Only evaluate runs that match this filter.

Response

Successful Response

idstring uuid required
tenant_idstring uuid required
is_enabledboolean
session_idstring uuid nullable
session_namestring nullable
dataset_idstring uuid nullable
dataset_namestring nullable
display_namestring required
sampling_ratenumber required
filterstring nullable
trace_filterstring nullable
tree_filterstring nullable
add_to_annotation_queue_idstring uuid nullable
add_to_annotation_queue_namestring nullable
add_to_dataset_idstring uuid nullable
add_to_dataset_namestring nullable
add_to_dataset_prefer_correctionboolean
corrections_dataset_idstring uuid nullable
use_corrections_datasetboolean
num_few_shot_examplesinteger nullable
extend_onlyboolean
is_managed_evaluatorboolean
is_tracing_disabledboolean
extend_evaluator_trace_retentionboolean nullable
extend_dataset_trace_retentionboolean nullable
extend_annotation_queue_trace_retentionboolean nullable
extend_webhook_trace_retentionboolean nullable
include_extended_statsboolean
created_atstring date-time required
updated_atstring date-time required
backfill_fromstring date-time nullable
backfill_idstring uuid nullable
backfill_statusstring nullable
backfill_progressnumber nullable
backfill_errorstring nullable
backfill_completed_atstring date-time nullable
transientboolean
evaluator_versioninteger required
evaluator_idstring uuid nullable
evaluator_namestring nullable
evaluator_typestring nullable
alignment_annotation_queue_idstring uuid nullable
group_by'thread_id' | 'trajectory_id' nullable
trace_countinteger nullable
spend_usdnumber nullable

Changes