---
title: "Get a custom evaluator configuration"
method: GET
path: "/api/unstable/llm-obs/config/evaluators/custom/{eval_name}"
tags: ["LLM Observability"]
---

# Get a custom evaluator configuration

`GET /api/unstable/llm-obs/config/evaluators/custom/{eval_name}`

Retrieve a custom LLM Observability evaluator configuration by its name.

## Path parameters

- `eval_name` string, required

## Response `200`

OK

- LLMObsCustomEvalConfigResponse — Response containing a custom LLM Observability evaluator configuration.
  - `data` LLMObsCustomEvalConfigData, required — Data object for a custom LLM Observability evaluator configuration.
    - `attributes` LLMObsCustomEvalConfigAttributes, required — Attributes of a custom LLM Observability evaluator configuration.
      - `category` string — Category of the evaluator.
      - `created_at` string, date-time, required — Timestamp when the evaluator configuration was created.
      - `created_by` LLMObsCustomEvalConfigUser — A Datadog user associated with a custom evaluator configuration.
        - `email` string — Email address of the user.
      - `eval_name` string, required — Name of the custom evaluator.
      - `last_updated_by` LLMObsCustomEvalConfigUser — A Datadog user associated with a custom evaluator configuration.
        - `email` string — Email address of the user.
      - `llm_judge_config` LLMObsCustomEvalConfigLLMJudgeConfig — LLM judge configuration for a custom evaluator.
        - `assessment_criteria` LLMObsCustomEvalConfigAssessmentCriteria — Criteria used to assess the pass/fail result of a custom evaluator.
          - `max_threshold` number, double, nullable — Maximum numeric threshold for a passing result.
          - `min_threshold` number, double, nullable — Minimum numeric threshold for a passing result.
          - `pass_values` string[], nullable — Specific output values considered as a passing result.
          - `pass_when` boolean, nullable — When true, a boolean output of true is treated as passing.
        - `inference_params` LLMObsCustomEvalConfigInferenceParams, required — LLM inference parameters for a custom evaluator.
          - `frequency_penalty` number, double — Frequency penalty to reduce repetition.
          - `max_tokens` integer — Maximum number of tokens to generate.
          - `presence_penalty` number, double — Presence penalty to reduce repetition.
          - `temperature` number, double — Sampling temperature for the LLM.
          - `top_k` integer — Top-k sampling parameter.
          - `top_p` number, double — Top-p (nucleus) sampling parameter.
        - `last_used_library_prompt_template_name` string, nullable — Name of the last library prompt template used.
        - `modified_library_prompt_template` boolean, nullable — Whether the library prompt template was modified.
        - `output_schema` object, nullable — JSON schema describing the expected output format of the LLM judge.
        - `parsing_type` 'structured_output' | 'json' — Output parsing type for a custom LLM judge evaluator.
        - `prompt_template` LLMObsCustomEvalConfigPromptMessage[] — List of messages forming the LLM judge prompt template.
          - `content` string — Text content of the message.
          - `contents` LLMObsCustomEvalConfigPromptContent[] — Multi-part content blocks for the message.
            - `type` string, required — Content block type.
            - `value` LLMObsCustomEvalConfigPromptContentValue, required — Value of a prompt message content block.
              - …
          - `role` string, required — Role of the message author.
      - `llm_provider` LLMObsCustomEvalConfigLLMProvider — LLM provider configuration for a custom evaluator.
        - `bedrock` LLMObsCustomEvalConfigBedrockOptions — AWS Bedrock-specific options for LLM provider configuration.
          - `region` string — AWS region for Bedrock.
        - `integration_account_id` string — Integration account identifier.
        - `integration_provider` 'openai' | 'amazon-bedrock' | 'anthropic' | 'azure-openai' | 'vertex-ai' | 'llm-proxy' — Name of the LLM integration provider.
        - `model_name` string — Name of the LLM model.
        - `vertex_ai` LLMObsCustomEvalConfigVertexAIOptions — Google Vertex AI-specific options for LLM provider configuration.
          - `location` string — Google Cloud region.
          - `project` string — Google Cloud project ID.
      - `target` LLMObsCustomEvalConfigTarget — Target application configuration for a custom evaluator.
        - `application_name` string, required — Name of the ML application this evaluator targets.
        - `enabled` boolean, required — Whether the evaluator is active for the target application.
        - `eval_scope` 'span' | 'trace' | 'session' — Scope at which to evaluate spans.
        - `filter` string, nullable — Filter expression to select which spans to evaluate.
        - `root_spans_only` boolean, nullable — When true, only root spans are evaluated.
        - `sampling_percentage` number, double, nullable — Percentage of traces to evaluate. Must be greater than 0 and at most 100.
      - `updated_at` string, date-time, required — Timestamp when the evaluator configuration was last updated.
    - `id` string, required — Unique name identifier of the evaluator configuration.
    - `type` 'evaluator_config', required — Type of the custom LLM Observability evaluator configuration resource.

## Other responses

- `400` — Bad Request
- `401` — Unauthorized
- `403` — Forbidden
- `404` — Not Found
- `429` — Too many requests

---

[API](https://skmtc.dev/datadog/apis/api-v2.md) · [All operations](https://skmtc.dev/datadog/apis/api-v2/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/datadog/api-v2/revisions/da68bf029e4c/schema)