Batch

Get a batch job and its progress

Returns the job with live progress counters. Cheap to call: progress is read from sharded counters, not by counting items. Poll until status is completed, failed or cancelled; a reasonable interval is 2-5 s for jobs under ~1,000 items and 10-30 s for larger ones. If open is true the job will never finish on its own — call POST /v1/batch/{batchID}/close when you are done appending.

get/v1/batch/{batchID}

Request

  • Base URL: https://api.spicrawl.com
  • URL: https://api.spicrawl.com/v1/batch/{batchID}
  • Auth: HTTP bearer

Response

The job.

idstring required

Job id (26-character ULID). Use it in every /v1/batch/{batchID} path.

project_idstring required

Project the submitting API key belongs to.

namestring

The name you submitted. Absent when none.

status'queued' | 'running' | 'paused' | 'cancelling' | 'completed' | 'failed' | 'cancelled' required

Job lifecycle. queued → running → completed | failed; cancel moves a live job to cancelling, then the worker moves it to cancelled once in-flight items drain. paused is an operator state (no public endpoint pauses). Terminal: completed, failed, cancelled — stop polling. A terminal job returns to queued after batchRetry.

openboolean

Present and true while the job accepts appended items; it will not complete until you close it. Absent means closed.

status_urlstring required

Relative path to poll: /v1/batch/{id}.

results_urlstring required

Relative path of the JSONL results: /v1/batch/{id}/results.

paramsobject required

The job-level scrape parameters as stored (your top-level BatchScrapeParams, without url), plus two keys the API sets and you cannot — credit_budget_micro, the run ceiling the worker enforces, and default_proxy_tier (direct or datacenter), the exit an item that sets no proxy, premium_proxy, proxy_country or sticky_key takes, decided at submission the way /v1/scrape decides it. Always present, possibly only those two.

total_itemsinteger required

Number of items in the job. Fixed at submission unless items are appended to an open job.

concurrencyinteger required

Effective per-job concurrency after clamping.

priorityinteger required

Effective priority.

max_attemptsinteger required

Effective attempts per item.

failure_thresholdinteger

Failed-item count that aborts the job. Absent when not set.

estimated_creditsinteger required

Credits RESERVED against the monthly allowance at submission — the dearest outcome of every item (whole credits, rounded down). It is a hold, not a charge; credits held for items that fail, are cancelled or skipped are returned to the monthly allowance within about a minute of the item finishing, by a background settler. Succeeded items keep their charge. Not increased by appends. Appended items are reserved when they are added, and the append's own total is its X-Request-Cost. Retrying failed items (POST /v1/batch/{id}/retry) holds their price again first and is refused with 402 if the allowance cannot cover it.

estimated_credits_microinteger required

Same as estimated_credits in micro-credits.

submitted_atstring date-time required

When the job was accepted.

started_atstring date-time

When the first item started. Absent until then.

finished_atstring date-time

When the job reached a terminal status. Cleared again by batchRetry.

cancel_requested_atstring date-time

When a cancel was accepted. The job may still be cancelling after this while in-flight items drain.

results_expire_atstring date-time

Results deadline: submitted_at + 72 h by default. After it, GET .../results answers 410. Download before this time.

error_codestring

Machine code of a job-level failure (for example the job was aborted). Absent otherwise.

error_messagestring

Human-readable job-level failure reason.

warningsstring[]

Decisions made on your behalf at submission — a clamped concurrency, or items not yet handed to the queue (do NOT resubmit in that case). Present only on the create response.

Example response

{
  "id": "01J9Z6T3W9E21T5TZARVJRVN5C",
  "project_id": "01J8X2M4C7Q1H5V9P3R6T8W0YB",
  "name": "catalog-2026-09-22",
  "status": "running",
  "status_url": "/v1/batch/01J9Z6T3W9E21T5TZARVJRVN5C",
  "results_url": "/v1/batch/01J9Z6T3W9E21T5TZARVJRVN5C/results",
  "params": {
    "js_render": true,
    "proxy_country": "us",
    "credit_budget_micro": 8000000
  },
  "total_items": 2,
  "concurrency": 20,
  "priority": 100,
  "max_attempts": 3,
  "estimated_credits": 8,
  "estimated_credits_micro": 8000000,
  "progress": {
    "total": 2,
    "completed": 1,
    "succeeded": 1,
    "failed": 0,
    "cancelled": 0,
    "skipped": 0,
    "remaining": 1,
    "percent_complete": 50,
    "credits_charged": 4,
    "credits_charged_micro": 4000000,
    "bytes": 326000
  },
  "submitted_at": "2026-09-22T06:41:12Z",
  "started_at": "2026-09-22T06:41:13Z",
  "results_expire_at": "2026-09-25T06:41:12Z"
}

Changes

No recorded changes to this operation across all 1 revision of this API.