---
title: "Get Evidence"
method: GET
path: "/models/release-decisions/evidence"
tags: ["models"]
---

# Get Evidence

`GET /models/release-decisions/evidence`

## Request

- Base URL: `https://api.os3robotics.com` (the document declares no server; this is the origin it was published from)
- URL: `https://api.os3robotics.com/models/release-decisions/evidence`
- Auth: one of:
  - HTTP bearer
  - HTTP bearer

An adapter version's task comparisons and the evidence each still waits for before it
can be decided.

## Query parameters

- `adapter_version_id` string, uuid, required

## Response `200`

Successful Response

- Evidence[]
  - `desired_job_id` string, uuid, required
  - `task` string, required
  - `candidate_eval_id` string, uuid, nullable, required
  - `decision_id` string, uuid, nullable, required
  - `safety_eval_id` string, uuid, nullable, required
  - `general_eval_id` string, uuid, nullable, required
  - `outstanding` EvidenceRun[], required
    - `kind` 'hard_rule_suite' | 'general_suite', required
    - `spec` EvalSpecOutput, required: An eval job: candidate vs baseline, judged pairwise on the held-out cases of an eval set. With `environment` (an RL task dataset version of the workspace), each side instead runs the version's held-out test tasks as episodes of its task environment, graded by the environment's hidden checks; the eval set then names the task the comparison is for. With `env_revision` (a bridged environment of the environment hub), likewise on its test split, a public one a retention and benchmark probe. With `purpose` `development` (OS3's optimizer agent's own development check of a run, or an RL algorithm selection's comparison), on the set's dev cases instead of its held-out ones (with `environment`: the version's dev tasks), and never release evidence. Filed as a desired job (POST /desired-jobs).
      - `kind` 'eval'
      - `eval_set_id` string, uuid, required
      - `baseline` VariantOutput, required: One side of an eval job: `ref` names it on its eval run (candidate_ref), `model` is the model the gateway routes to, and `system`, when set, replaces the cases' system prompt. `upstream`, when set, names your gateway upstream that answers this side, so a frontier vendor can be compared with a specialist served on OS3 GPUs; unset, the eval job's own judge key's upstream answers it. `caller:<vendor>` names the provider API your app called with its own key, which OS3 answers on its own account of that provider. `gpu`, when set, says the side is served on OS3 GPUs of that shape (`<family>:<count>`, as JobSpec.gpu: the replica of the endpoint behind its upstream), so the eval job prices it from the GPU time it measures, not a vendor list price.
        - `ref` string, required
        - `model` string, required
        - `system` string, nullable
        - `upstream` string, nullable
        - `gpu` string, nullable
        - `adapter_version_id` string, uuid, nullable
        - `serving` ServingConfig: How an llm_vllm endpoint's engine serves its base and the adapters it loads, beyond the base's own serving: vLLM v0.30.0's online FP8 quantization of the base's linear weights at load time (`--quantization`; H100s run both schemes as W8A8; the adapters stay bf16), an FP8 KV cache (`--kv-cache-dtype`), speculative decoding (n-gram, the base's MTP drafter at another depth, or none; each replaces the drafter a base serves with by default), the batch size (`--max-num-seqs`, else the base's), the LoRA slots (`--max-loras`, at most the base's: fewer slots decode faster, while a batch holding more adapters than slots defers the rest), a step's token budget (`--max-num-batched-tokens`: 512, 1,024 or 2,048) and the requests the engine runs at once (`--max-num-active-seqs`, at most the batch size). Unset fields are the engine's defaults and are left out of the configuration's JSON. It sets at least one field: the default serving is no configuration.
          - `quantization` 'fp8_per_tensor' | 'fp8_per_block', nullable
          - `kv_cache_dtype` 'fp8', nullable
          - `speculative` union
            - NgramSpeculation: vLLM's n-gram speculative decoding (prompt lookup, `--speculative-config`): drafts are the tokens that followed the last `prompt_lookup_max`..`prompt_lookup_min` tokens earlier in the request, so no draft model is loaded, and it works with any base and with LoRA (vLLM v0.30.0; unset `prompt_lookup_min`: the engine's, `prompt_lookup_max`).
              - …
            - MtpSpeculation: vLLM's MTP speculative decoding (`--speculative-config`) with the base's own drafter (Gemma 4's assistant model), which drafts from the base's hidden states, so it drafts for every adapter too. `model` at `revision` is that drafter, and no other is accepted; the replica downloads it from the Hub as it does the base.
              - …
            - NoSpeculation: No speculative decoding, where the base drafts with its own drafter by default; refused for a base without one (its default serving already drafts nothing).
              - …
          - `max_num_seqs` integer, nullable
          - `max_loras` integer, nullable
          - `max_num_batched_tokens` 512 | 1024 | 2048, nullable
          - `max_num_active_seqs` integer, nullable
        - `route` Route: A routed variant's router: the fitted router ``policy`` (its ref, and the sha256 of its record, which names its weights) sends a case to the variant's own model and upstream when its probability that this model wins over ``weak`` is at least ``threshold``, else to ``weak`` on ``weak_upstream`` (unset: the eval job's judge key's upstream). ``embedding_upstream`` is the upstream the router's fit embedded its prompts on, where a canary serving the router embeds each request's (unset: the serving key's own upstream); the eval job embeds each case there before its answer, so the router's call counts in its timing and cost (unset: the variant's own upstream).
          - `policy` string, required
          - `record_sha256` string, required
          - `threshold` number, required
          - `weak` string, required
          - `weak_upstream` string, nullable
          - `embedding_upstream` string, nullable
        - `auto` AutoOutput
          - `decider` 'rules' | 'mf' | 'jev' | 'llm', required
          - `decider_id` string, nullable
          - `policy` string, nullable
          - `record_sha256` string, nullable
          - `threshold` number, nullable
          - `embedding_upstream` string, nullable
          - `options` AutoOptionOutput[], required
            - `model` string, required
            - `effort` string, nullable
            - `upstream` string, nullable
            - `kind` 'incumbent' | 'model' | 'catalog' | 'specialist', required
            - `tools` boolean
            - `image_input` boolean
            - `context_tokens` integer, nullable
            - `usd_per_1m` string, nullable
            - `description` string, required
            - `decision_id` string, uuid, nullable
        - `sampling` object, nullable
        - `effort` string, nullable
        - `room` integer, nullable
      - `candidate` VariantOutput, required: One side of an eval job: `ref` names it on its eval run (candidate_ref), `model` is the model the gateway routes to, and `system`, when set, replaces the cases' system prompt. `upstream`, when set, names your gateway upstream that answers this side, so a frontier vendor can be compared with a specialist served on OS3 GPUs; unset, the eval job's own judge key's upstream answers it. `caller:<vendor>` names the provider API your app called with its own key, which OS3 answers on its own account of that provider. `gpu`, when set, says the side is served on OS3 GPUs of that shape (`<family>:<count>`, as JobSpec.gpu: the replica of the endpoint behind its upstream), so the eval job prices it from the GPU time it measures, not a vendor list price.
        - `ref` string, required
        - `model` string, required
        - `system` string, nullable
        - `upstream` string, nullable
        - `gpu` string, nullable
        - `adapter_version_id` string, uuid, nullable
        - `serving` ServingConfig: How an llm_vllm endpoint's engine serves its base and the adapters it loads, beyond the base's own serving: vLLM v0.30.0's online FP8 quantization of the base's linear weights at load time (`--quantization`; H100s run both schemes as W8A8; the adapters stay bf16), an FP8 KV cache (`--kv-cache-dtype`), speculative decoding (n-gram, the base's MTP drafter at another depth, or none; each replaces the drafter a base serves with by default), the batch size (`--max-num-seqs`, else the base's), the LoRA slots (`--max-loras`, at most the base's: fewer slots decode faster, while a batch holding more adapters than slots defers the rest), a step's token budget (`--max-num-batched-tokens`: 512, 1,024 or 2,048) and the requests the engine runs at once (`--max-num-active-seqs`, at most the batch size). Unset fields are the engine's defaults and are left out of the configuration's JSON. It sets at least one field: the default serving is no configuration.
          - `quantization` 'fp8_per_tensor' | 'fp8_per_block', nullable
          - `kv_cache_dtype` 'fp8', nullable
          - `speculative` union
            - NgramSpeculation: vLLM's n-gram speculative decoding (prompt lookup, `--speculative-config`): drafts are the tokens that followed the last `prompt_lookup_max`..`prompt_lookup_min` tokens earlier in the request, so no draft model is loaded, and it works with any base and with LoRA (vLLM v0.30.0; unset `prompt_lookup_min`: the engine's, `prompt_lookup_max`).
              - …
            - MtpSpeculation: vLLM's MTP speculative decoding (`--speculative-config`) with the base's own drafter (Gemma 4's assistant model), which drafts from the base's hidden states, so it drafts for every adapter too. `model` at `revision` is that drafter, and no other is accepted; the replica downloads it from the Hub as it does the base.
              - …
            - NoSpeculation: No speculative decoding, where the base drafts with its own drafter by default; refused for a base without one (its default serving already drafts nothing).
              - …
          - `max_num_seqs` integer, nullable
          - `max_loras` integer, nullable
          - `max_num_batched_tokens` 512 | 1024 | 2048, nullable
          - `max_num_active_seqs` integer, nullable
        - `route` Route: A routed variant's router: the fitted router ``policy`` (its ref, and the sha256 of its record, which names its weights) sends a case to the variant's own model and upstream when its probability that this model wins over ``weak`` is at least ``threshold``, else to ``weak`` on ``weak_upstream`` (unset: the eval job's judge key's upstream). ``embedding_upstream`` is the upstream the router's fit embedded its prompts on, where a canary serving the router embeds each request's (unset: the serving key's own upstream); the eval job embeds each case there before its answer, so the router's call counts in its timing and cost (unset: the variant's own upstream).
          - `policy` string, required
          - `record_sha256` string, required
          - `threshold` number, required
          - `weak` string, required
          - `weak_upstream` string, nullable
          - `embedding_upstream` string, nullable
        - `auto` AutoOutput
          - `decider` 'rules' | 'mf' | 'jev' | 'llm', required
          - `decider_id` string, nullable
          - `policy` string, nullable
          - `record_sha256` string, nullable
          - `threshold` number, nullable
          - `embedding_upstream` string, nullable
          - `options` AutoOptionOutput[], required
            - `model` string, required
            - `effort` string, nullable
            - `upstream` string, nullable
            - `kind` 'incumbent' | 'model' | 'catalog' | 'specialist', required
            - `tools` boolean
            - `image_input` boolean
            - `context_tokens` integer, nullable
            - `usd_per_1m` string, nullable
            - `description` string, required
            - `decision_id` string, uuid, nullable
        - `sampling` object, nullable
        - `effort` string, nullable
        - `room` integer, nullable
      - `environment` string, uuid, nullable
      - `env_revision` string, uuid, nullable
      - `purpose` 'release' | 'development'
      - `label_enums` boolean
      - `cases` integer[], nullable
      - `judge` string, nullable
    - `desired_job_id` string, uuid, nullable, required
    - `blocked` string, nullable
    - `idempotency_key` string, nullable

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/os3robotics/apis/os3-api.md) - [All operations](https://skmtc.dev/os3robotics/apis/os3-api/llms.txt) - [OpenAPI document](https://skmtc.dev/os3robotics/apis/os3-api/revisions/af32cd4f14bb?raw)