---
title: "Create Generation Job"
method: POST
path: "/v1/generate"
tags: ["generate"]
---

# Create Generation Job

`POST /v1/generate`

## Request

- Base URL: `https://api.pioneer.ai` (the document declares no server; this is the origin it was published from)
- URL: `https://api.pioneer.ai/v1/generate`
- Auth: one of:
  - API key in header `X-API-Key`
  - HTTP bearer
- TypeScript: <https://skmtc.dev/pioneer/apis/brain-api/docs/v1/generate/post.ts> (a function with typed inputs; its response is checked against the schema)

Create an async dataset generation job.

Returns immediately with a ``job_id``. Poll ``GET /generate/jobs/{job_id}``
for status and results.

The ``task_type`` field in the request body determines what kind of dataset
is generated:
- ``ner``: Named-entity recognition dataset. Requires ``labels``.
- ``classification``: Text classification dataset. Requires ``labels``.
- ``custom``: Free-form prompt-based dataset. Requires ``prompt``.
- ``decoder``: Instruction-tuning (chat format) dataset.
  Requires ``domain_description``.

Dispatches via SQS with idempotency deduplication; falls back to an
in-process background task if SQS is unavailable.

Args:
    request: Incoming HTTP request (used by SlowAPI rate limiter).
    generation_request: Unified generation parameters including task_type.
    auth: Authenticated user context.
    is_seed: Whether this is a small seed dataset for UI preview.
    synthesis_session_id: Optional synthesis log session UUID for resume.

Returns:
    GenerateAsyncResponse with ``job_id`` for polling and initial status.

Raises:
    HTTPException: 422 if required task-specific fields are missing.

## Query parameters

- `is_seed` boolean: Whether this is a seed generation
- `synthesis_session_id` string, nullable: Synthesis log session ID for resume support

## Request body

- GenerateRequest: Unified async generation request for all task types. Use ``task_type`` to specify what kind of dataset to generate. All generation goes through SQS with BackgroundTasks fallback. Task-specific required fields: - ``ner``: requires ``labels`` - ``classification``: requires ``labels`` - ``custom``: requires ``prompt`` - ``decoder``: requires ``domain_description`` - ``records``: requires ``fields`` - ``fields``: requires ``input_fields`` and ``output_fields``
  - `task_type` 'ner' | 'classification' | 'custom' | 'decoder' | 'records' | 'fields', required: Type of generation task
  - `dataset_name` string, required: Name for the generated dataset
  - `num_examples` integer: Number of examples to generate
  - `domain_description` string, nullable: Domain description for generation context (required for decoder)
  - `temperature` number: Generation temperature
  - `quality` string, nullable: Generation quality: 'light', 'medium', or 'heavy'
  - `generation_profile` 'auto' | 'fast' | 'balanced' | 'quality': Queue/runtime execution profile. 'auto' selects a task-aware default.
  - `session_id` string, nullable: Session ID for log streaming
  - `config_num_examples` integer: Number of examples to use for config generation
  - `seed` integer, nullable: Random seed for reproducibility
  - `labels` string[], nullable: Entity/classification labels (required for ner and classification)
  - `classified_examples` union[], nullable: Seed examples for generation. NER: {text, entities: [[span, label], ...], feedback?}. Classification: {text, label, feedback?}.
    - union
      - NERClassifiedExample: NER example with optional user feedback for improving generation.
        - `text` string, required: Example text
        - `entities` array[], required: List of (entity_text, entity_type) tuples
          - unknown[]
            - unknown
        - `feedback` 'positive' | 'negative', nullable: User feedback: positive (upvote) or negative (downvote)
      - ClassifiedExample: Example with optional user feedback for improving generation.
        - `text` string, required: Example text
        - `label` string, required: Example label
        - `feedback` 'positive' | 'negative', nullable: User feedback: positive (upvote) or negative (downvote)
  - `multi_label` boolean: Enable multi-label classification (classification only)
  - `class_balance` object, nullable: Optional class distribution map (classification only)
  - `batch_size` integer: Batch size for generation API calls
  - `negative_ratio` integer, nullable: Percentage of rows with no labels/entities (negative samples). Applies to ner, and to classification only when multi_label=True — a no-op for single-label classification, which already expresses 'none of these apply' via its own label. For ner, note bench_use_cases.py's weekly quality benchmark separately caps the empty-entity row ratio at ner_empty_entity_row_ratio_max (0.35 by default); values above ~35 risk a dataset the platform's own quality gate rejects.
  - `fields` RecordField[], nullable: Field definitions for structured records (required for records)
    - `name` string, required
    - `type` string
    - `description` string, nullable
    - `allowed_values` string[], nullable
  - `input_fields` RecordField[], nullable: Input field definitions (required for fields)
    - `name` string, required
    - `type` string
    - `description` string, nullable
    - `allowed_values` string[], nullable
  - `output_fields` RecordField[], nullable: Output field definitions (required for fields)
    - `name` string, required
    - `type` string
    - `description` string, nullable
    - `allowed_values` string[], nullable
  - `prompt` string, nullable: Natural language prompt describing the task (required for custom)
  - `output_format` object, nullable: Expected output schema for custom generation
  - `infer_output_format` boolean: Infer output format from prompt when output_format is absent
  - `instruction` string, nullable: System instruction for decoder chat format
  - `include_reasoning_trace` boolean: Whether to include reasoning traces (<think> blocks) in generated outputs for decoder tasks. Ignored when task_type is not 'decoder'.
  - `reasoning_effort` string, nullable: Reasoning effort for the underlying LLM (e.g., 'low', 'medium', 'high')
  - `constraints` ConstraintRequest[], nullable: Custom constraints applied to all generated examples
    - `description` string, required: Constraint description
    - `choices` string[], nullable: Optional list of choices for this constraint
    - `weights` number[], nullable: Optional weights for choices (must align with choices length)
    - `probability` number, nullable: Optional probability (0.0-1.0) for this constraint
  - `multiplicator` MultiplicatorRequest: Multiplicator for balanced dataset distribution
    - `prompt` string, required: Multiplicator prompt (e.g., 'The sentiment should be')
    - `choices` string[], required: List of choices to balance across
  - `use_meta_synthesizer` boolean: Use meta-synthesis to auto-generate diversity criteria
  - `min_criteria` integer: Minimum diversity criteria to generate
  - `target_choices` integer: Number of choices to expand for diversity
  - `project_id` string, uuid, nullable: Project ID to assign the dataset to
  - `type` string: Dataset type tag: 'training', 'evaluation', or 'split'
  - `visibility` string: Dataset visibility: 'private' or 'public'
  - `split_ratio` SplitRatioConfig: Split ratio configuration for train/eval split datasets. Values are decimals (e.g. 0.8 for 80% training, 0.2 for 20% evaluation).
    - `training` number: Fraction of data for training (0.05–0.95)
    - `evaluation` number: Fraction of data for evaluation (0.05–0.95)
  - `output_style` 'prose' | 'code' | 'mixed', nullable: Decoder-only formatting policy: 'prose', 'code', or 'mixed'. Rejected when task_type is not decoder.

## Response `202`

Successful Response

- GenerateAsyncResponse: Response from async generation job creation (HTTP 202). Returned immediately when a background generation job is started. The job_id (which is the dataset_id) can be used to poll for status.
  - `job_id` string, required: Job ID (same as dataset_id) for polling status
  - `status` string: Initial job status: queued or generating
  - `dataset_name` string, required: Name of the dataset being generated
  - `task_type` string, required: Generation task type: ner, classification, or custom
  - `is_seed` boolean: Whether this is a seed generation job
  - `message` string: Human-readable status message

## Other responses

- `422`: Validation Error

## Changes

- **2026-09-24** `0e5e86e2bae5`: 1 info
  - endpoint added

[Operation history](https://skmtc.dev/pioneer/apis/brain-api/changes/v1/generate/post.md)

---

[API](https://skmtc.dev/pioneer/apis/brain-api.md) - [All operations](https://skmtc.dev/pioneer/apis/brain-api/llms.txt) - [OpenAPI document](https://skmtc.dev/pioneer/apis/brain-api/revisions/0e5e86e2bae5?raw)