rollouts

Submit rollouts: specs x seeds under one judge

Submit ROLLOUTS: one episode per spec per seed, in an environment, under a judge, ending in a verdict (the manual's Rollouts section, gman://docs/llms.txt). The platform owns each rollout's identity (a digest of environment, runner, spec, seed, judge, policy, parent state and reproducibility class), its lineage, and its provenance-stamped verdict, and never interprets your spec. Two kinds: task (your container serving the environment, leased episodes from inside via $GMN_METADATA_URL/v1/tasks/lease and declaring with /v1/rollout/verdicts; catalog containers mjx-rollout and openenv-rollout) and sandbox (a fork per rollout of a baked env-... snapshot, driven and graded by your harness from OUTSIDE; nothing inside a sandbox may declare a verdict). Under profile eval, identical identities return their stored verdict unbilled (memoization, workspace-scoped); under train, memoization is off. Anything that changes the result and is not in your environment, spec, seed, judge or policy makes two different rollouts one identity.

post/preview/rollouts

Request

  • Base URL: https://autoresearch.sfcompute.com
  • URL: https://autoresearch.sfcompute.com/preview/rollouts
  • Auth: HTTP bearer

Query parameters

workspacestring

Act in this workspace (its slug). Omitted: the token's bound workspace, then the organization's default. Never ambient - there is no switch-workspace mutation on this door.

Request body

chipstring nullable

The runner's chip for a task-kind submit ("h100", "cpu-2"). Default: the catalog container's, else h100. Ignored for sandbox kind.

commandstring nullable

The runner command (default: the image's). Task kind only.

egressstring nullable

Egress for an openenv bake: "open" (default) or "none".

envobject

Environment variables for the runner (a policy endpoint's key rides here, sealed, and never reaches a sandbox). Task kind only.

environmentunknown required
evaluationstring nullable

The evaluation to attach to (created if new; refused if sealed or owned by another workspace).

gpustring nullable

Alias of chip.

groupinteger nullable

Attested group of N children from one state (sandbox kind: N forks of the snapshot; requires --no-memo under eval). Seeds must number N.

idempotency_keystring nullable

Replay protection: the same key returns the first submit's evaluation rather than creating a second set.

judgestring required

The judge, pinned: "goal-region@v1", or "swe-tests@sha256:<digest>" to bind the label to code, or "upstream:<name>@<image digest>" for an environment's own reward. Reports never average rows whose judges differ.

labelsunknown
lease_batchinteger nullable

Rollouts one lease hands the runner (default 32; up to 8192 for a vectorized simulator; openenv pools default 16, max 64 - each granted rollout forks a sandbox).

max_duration_minutesinteger nullable

The runner attempt's max_duration_minutes (default 240). Task kind.

memoboolean nullable

Memoization on/off (default follows the profile; true with "train" is refused). A memo hit returns the stored verdict, unbilled.

missionstring nullable

The mission this work rolls up into.

openboolean nullable

Leave the evaluation open so add_rollouts can append later.

policystring nullable

The policy label, part of the identity. REQUIRED for a sandbox-kind eval submit with memoization on: the model driving the sandbox is outside the envelope, so without it the next checkpoint would be served this one's verdicts. What we cannot see, we cannot tell apart.

profilestring nullable

"eval" (default: memoization on, seeded-stochastic) or "train" (memoization off - a memoized training sample collapses your sample diversity - and unrepeatable).

repro_classstring nullable

"bit-exact", "seeded-stochastic" or "unrepeatable". Never defaults to bit-exact: we stamp the class you declare, and qualify_rollout is how you find out whether it is true.

runnersinteger nullable

Machines that may work the pool at once (default 4, max 32); started as the backlog warrants, so a ceiling on spend, not a reservation.

seedsunknown
specunknown
state_expires_afterstring nullable

Expiry for states these rollouts declare ("30d", "never"). Default: the org's snapshot default, else never.

Response

The capability's JSON result.

objectstring

The resource type, when the result is one resource.

Changes

No recorded changes to this operation across all 1 revision of this API.