Custom Metric Templates

Admin Create Bluejay Template

Changed on

Create a Bluejay-authored template (the library seeding mechanism).

post/v1/admin/metric-templates

Request

  • Base URL
    https://api.getbluejay.ai
  • URL
    https://api.getbluejay.ai/v1/admin/metric-templates
  • Auth
    API key in header X-API-Key

Headers

X-Admin-Secretstring nullable
X-API-Keystring required

API key required to authenticate requests.

Request body

bluejay_as_code_idstring uuid nullable

Stable code-addressable identifier

agent_idinteger nullable

ID of the agent this metric belongs to (deprecated, use agent_ids instead)

agent_idsinteger[] nullable

List of agent IDs to associate this metric with

namestring required

Name of the custom metric

promptstring nullable

LLM-judge system prompt (required for llm_judge; null for other types)

summarystring nullable

Short plain-English description of the metric (LLM-generated)

metric_typestring

Specific metric kind (llm_judge, tool_call, trace_span, audio_*, vocal_fry); legacy alias: type

settingsobject

Type-specific config (e.g. tool_call: {tool_names: [...]}; trace_span: {span_names: [...], condition: contains|does_not_contain})

eval_segment'agent' | 'user' | 'all'

Which side of the conversation is evaluated

scope'observability' | 'simulations' | 'all'

Which pipeline runs this metric

response_type'pass_fail' | 'yes_no' | 'qualitative' | 'quantitative' | 'json' | 'enum' | 'int' | 'float' | 'boolean' | 'tool_call' required

Enum representing the possible response types for custom metrics.

min_valuenumber nullable

Minimum value for quantitative metrics

max_valuenumber nullable

Maximum value for quantitative metrics

categorystring nullable

Category for organizing metrics

tagsstring[] nullable

Tags for categorizing the metric

scoring_guidancestring nullable

Guidance on how to score this metric

eval_modality'AUDIO' | 'TEXT' | 'AUTO'

Enum representing the evaluation route of a call.

modelstring nullable

Model to use for text evaluation of the metric

audio_modelstring nullable

Model to use for audio evaluation of the metric

temperaturenumber nullable

Temperature for evaluating the metric

enum_optionsstring[] nullable

Options for enum metrics

tool_namesstring[] nullable

Expected tool names for tool_call metrics

json_schemaobject nullable

JSON Schema describing the object the judge must return (required for json metrics)

allow_not_applicableboolean

Whether this metric allows 'not applicable' responses.

template_idstring uuid nullable

Library template this metric was created from

Response

Successful Response

idstring required
created_atstring date-time required
namestring required
promptstring nullable
descriptionstring nullable
summarystring nullable
metric_typestring
eval_methodstring
eval_segmentstring
scopestring
settingsobject
response_type'pass_fail' | 'yes_no' | 'qualitative' | 'quantitative' | 'json' | 'enum' | 'int' | 'float' | 'boolean' | 'tool_call' required

Enum representing the possible response types for custom metrics.

min_valuenumber nullable
max_valuenumber nullable
categorystring nullable
tagsstring[]
scoring_guidancestring nullable
eval_modality'AUDIO' | 'TEXT' | 'AUTO'

Enum representing the evaluation route of a call.

modelstring nullable
audio_modelstring nullable
temperaturenumber nullable
enum_optionsstring[] nullable
tool_namesstring[] nullable
json_schemaobject nullable
allow_not_applicableboolean

Changes