--- title: "Submit a batch" method: POST path: "/batch/submit" tags: ["Batch"] --- # Submit a batch `POST /batch/submit` Scrape a list of URLs or crawl a site asynchronously. Send an Idempotency-Key header to make retries safe: resubmitting with the same key returns the batch the first attempt created instead of creating and charging a second one. ## Headers - `Idempotency-Key` string — Any string unique to this submission, at most 200 characters. Retries with the same key return the original batch; reusing a key with a different body fails with IDEMPOTENCY_KEY_CONFLICT. ## Request body - BatchSubmitRequest — Create an asynchronous scraping job. - `input` union, required — Choose a URL list or a site crawl. - object — Scrape a list of pages. - `type` 'scrape', required — Scrape the pages in `data.urls`. - `data` union, required — Pages to scrape and their output format. - object — Scrape the listed pages as Markdown. - `type` 'markdown', required — Return page content as Markdown. - `urls` object[], required — Pages to scrape. Maximum 25000. - `url` string, required — Page URL to scrape. - `itemId` string — Your ID for this page, returned with its result. The same URL can use different IDs. - `meta` object — Custom JSON returned unchanged with this page result. - `options` object — Options for Markdown output. - `includeLinks` boolean — Include links in the Markdown. - `includeImages` boolean — Include image references in the Markdown. - `shortenBase64Images` boolean — Shorten inline base64 image data. - `useMainContentOnly` boolean — Return the main content without navigation or footers. - `pdf` object — PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range. - … - `includeSelectors` string[], nullable — CSS selectors. When provided, only matching subtrees (and their descendants) are kept and everything else is dropped. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored), metadata still comes from the full document, and crawl link discovery still uses the full document. - `excludeSelectors` string[], nullable — CSS selectors to remove from the result. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored). - `waitForMs` integer — Browser wait time in milliseconds after initial page load. Only applies to pages that need a browser render; most batch pages are fetched without one. Min: 0. Max: 15000 (15 seconds). - `settleAnimations` boolean — Wait briefly for CSS and transition animations to settle before extraction, on pages that render in a browser. - `country` 'ad' | 'ae' | 'af' | 'ag' | 'ai' | 'al' | 'am' | 'ao' | 'ar' | 'at' | 'au' | 'aw' | 'az' | 'ba' | 'bb' | 'bd' | 'be' | 'bf' | 'bg' | 'bh' | 'bi' | 'bj' | 'bm' | 'bn' | 'bo' | 'bq' | 'br' | 'bs' | 'bw' | 'by' | 'bz' | 'ca' | 'cd' | 'cf' | 'cg' | 'ch' | 'ci' | 'cl' | 'cm' | 'cn' | 'co' | 'cr' | 'cv' | 'cw' | 'cy' | 'cz' | 'de' | 'dj' | 'dk' | 'dm' | 'do' | 'dz' | 'ec' | 'ee' | 'eg' | 'es' | 'et' | 'fi' | 'fj' | 'fr' | 'ga' | 'gb' | 'gd' | 'ge' | 'gf' | 'gg' | 'gh' | 'gm' | 'gn' | 'gp' | 'gq' | 'gr' | 'gt' | 'gu' | 'gw' | 'gy' | 'hk' | 'hn' | 'hr' | 'ht' | 'hu' | 'id' | 'ie' | 'il' | 'im' | 'in' | 'iq' | 'ir' | 'is' | 'it' | 'je' | 'jm' | 'jo' | 'jp' | 'ke' | 'kg' | 'kh' | 'kn' | 'kr' | 'kw' | 'ky' | 'kz' | 'la' | 'lb' | 'lc' | 'lk' | 'lr' | 'ls' | 'lt' | 'lu' | 'lv' | 'ly' | 'ma' | 'mc' | 'md' | 'me' | 'mf' | 'mg' | 'mk' | 'ml' | 'mm' | 'mn' | 'mo' | 'mq' | 'mr' | 'mt' | 'mu' | 'mv' | 'mw' | 'mx' | 'my' | 'mz' | 'na' | 'nc' | 'ne' | 'ng' | 'ni' | 'nl' | 'no' | 'np' | 'nz' | 'om' | 'pa' | 'pe' | 'pf' | 'pg' | 'ph' | 'pk' | 'pl' | 'pr' | 'ps' | 'pt' | 'py' | 'qa' | 're' | 'ro' | 'rs' | 'ru' | 'rw' | 'sa' | 'sc' | 'sd' | 'se' | 'sg' | 'si' | 'sk' | 'sl' | 'sm' | 'sn' | 'so' | 'sr' | 'ss' | 'st' | 'sv' | 'sx' | 'sy' | 'sz' | 'tc' | 'td' | 'tg' | 'th' | 'tj' | 'tl' | 'tm' | 'tn' | 'tr' | 'tt' | 'tw' | 'tz' | 'ua' | 'ug' | 'us' | 'uy' | 'uz' | 'vc' | 've' | 'vg' | 'vi' | 'vn' | 'ye' | 'yt' | 'za' | 'zm' | 'zw' — Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country. - `maxAgeMs` integer, nullable — Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh. - object — Scrape the listed pages as HTML. - `type` 'html', required — Return page content as HTML. - `urls` object[], required — Pages to scrape. Maximum 25000. - `url` string, required — Page URL to scrape. - `itemId` string — Your ID for this page, returned with its result. The same URL can use different IDs. - `meta` object — Custom JSON returned unchanged with this page result. - `options` object — Options for HTML output. - `useMainContentOnly` boolean — Return the main content without navigation or footers. - `pdf` object — PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range. - … - `includeSelectors` string[], nullable — CSS selectors. When provided, only matching subtrees (and their descendants) are kept and everything else is dropped. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored), metadata still comes from the full document, and crawl link discovery still uses the full document. - `excludeSelectors` string[], nullable — CSS selectors to remove from the result. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored). - `waitForMs` integer — Browser wait time in milliseconds after initial page load. Only applies to pages that need a browser render; most batch pages are fetched without one. Min: 0. Max: 15000 (15 seconds). - `settleAnimations` boolean — Wait briefly for CSS and transition animations to settle before extraction, on pages that render in a browser. - `country` 'ad' | 'ae' | 'af' | 'ag' | 'ai' | 'al' | 'am' | 'ao' | 'ar' | 'at' | 'au' | 'aw' | 'az' | 'ba' | 'bb' | 'bd' | 'be' | 'bf' | 'bg' | 'bh' | 'bi' | 'bj' | 'bm' | 'bn' | 'bo' | 'bq' | 'br' | 'bs' | 'bw' | 'by' | 'bz' | 'ca' | 'cd' | 'cf' | 'cg' | 'ch' | 'ci' | 'cl' | 'cm' | 'cn' | 'co' | 'cr' | 'cv' | 'cw' | 'cy' | 'cz' | 'de' | 'dj' | 'dk' | 'dm' | 'do' | 'dz' | 'ec' | 'ee' | 'eg' | 'es' | 'et' | 'fi' | 'fj' | 'fr' | 'ga' | 'gb' | 'gd' | 'ge' | 'gf' | 'gg' | 'gh' | 'gm' | 'gn' | 'gp' | 'gq' | 'gr' | 'gt' | 'gu' | 'gw' | 'gy' | 'hk' | 'hn' | 'hr' | 'ht' | 'hu' | 'id' | 'ie' | 'il' | 'im' | 'in' | 'iq' | 'ir' | 'is' | 'it' | 'je' | 'jm' | 'jo' | 'jp' | 'ke' | 'kg' | 'kh' | 'kn' | 'kr' | 'kw' | 'ky' | 'kz' | 'la' | 'lb' | 'lc' | 'lk' | 'lr' | 'ls' | 'lt' | 'lu' | 'lv' | 'ly' | 'ma' | 'mc' | 'md' | 'me' | 'mf' | 'mg' | 'mk' | 'ml' | 'mm' | 'mn' | 'mo' | 'mq' | 'mr' | 'mt' | 'mu' | 'mv' | 'mw' | 'mx' | 'my' | 'mz' | 'na' | 'nc' | 'ne' | 'ng' | 'ni' | 'nl' | 'no' | 'np' | 'nz' | 'om' | 'pa' | 'pe' | 'pf' | 'pg' | 'ph' | 'pk' | 'pl' | 'pr' | 'ps' | 'pt' | 'py' | 'qa' | 're' | 'ro' | 'rs' | 'ru' | 'rw' | 'sa' | 'sc' | 'sd' | 'se' | 'sg' | 'si' | 'sk' | 'sl' | 'sm' | 'sn' | 'so' | 'sr' | 'ss' | 'st' | 'sv' | 'sx' | 'sy' | 'sz' | 'tc' | 'td' | 'tg' | 'th' | 'tj' | 'tl' | 'tm' | 'tn' | 'tr' | 'tt' | 'tw' | 'tz' | 'ua' | 'ug' | 'us' | 'uy' | 'uz' | 'vc' | 've' | 'vg' | 'vi' | 'vn' | 'ye' | 'yt' | 'za' | 'zm' | 'zw' — Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country. - `maxAgeMs` integer, nullable — Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh. - object — Discover and scrape pages from a URL or sitemap. - `type` 'crawl', required — Discover and scrape pages from `data.source`. - `data` union, required — Crawl source and output format. - object — Crawl pages and return Markdown. - `type` 'markdown', required — Return page content as Markdown. - `source` union, required — How to find pages to crawl. - object — Discover pages by following links from one URL. - … - object — Scrape the pages listed in a domain's sitemap. Links on those pages are not followed. - … - `options` object — Options for Markdown output. - `includeLinks` boolean — Include links in the Markdown. - `includeImages` boolean — Include image references in the Markdown. - `shortenBase64Images` boolean — Shorten inline base64 image data. - `useMainContentOnly` boolean — Return the main content without navigation or footers. - `pdf` object — PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range. - … - `includeSelectors` string[], nullable — CSS selectors. When provided, only matching subtrees (and their descendants) are kept and everything else is dropped. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored), metadata still comes from the full document, and crawl link discovery still uses the full document. - `excludeSelectors` string[], nullable — CSS selectors to remove from the result. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored). - `waitForMs` integer — Browser wait time in milliseconds after initial page load. Only applies to pages that need a browser render; most batch pages are fetched without one. Min: 0. Max: 15000 (15 seconds). - `settleAnimations` boolean — Wait briefly for CSS and transition animations to settle before extraction, on pages that render in a browser. - `country` 'ad' | 'ae' | 'af' | 'ag' | 'ai' | 'al' | 'am' | 'ao' | 'ar' | 'at' | 'au' | 'aw' | 'az' | 'ba' | 'bb' | 'bd' | 'be' | 'bf' | 'bg' | 'bh' | 'bi' | 'bj' | 'bm' | 'bn' | 'bo' | 'bq' | 'br' | 'bs' | 'bw' | 'by' | 'bz' | 'ca' | 'cd' | 'cf' | 'cg' | 'ch' | 'ci' | 'cl' | 'cm' | 'cn' | 'co' | 'cr' | 'cv' | 'cw' | 'cy' | 'cz' | 'de' | 'dj' | 'dk' | 'dm' | 'do' | 'dz' | 'ec' | 'ee' | 'eg' | 'es' | 'et' | 'fi' | 'fj' | 'fr' | 'ga' | 'gb' | 'gd' | 'ge' | 'gf' | 'gg' | 'gh' | 'gm' | 'gn' | 'gp' | 'gq' | 'gr' | 'gt' | 'gu' | 'gw' | 'gy' | 'hk' | 'hn' | 'hr' | 'ht' | 'hu' | 'id' | 'ie' | 'il' | 'im' | 'in' | 'iq' | 'ir' | 'is' | 'it' | 'je' | 'jm' | 'jo' | 'jp' | 'ke' | 'kg' | 'kh' | 'kn' | 'kr' | 'kw' | 'ky' | 'kz' | 'la' | 'lb' | 'lc' | 'lk' | 'lr' | 'ls' | 'lt' | 'lu' | 'lv' | 'ly' | 'ma' | 'mc' | 'md' | 'me' | 'mf' | 'mg' | 'mk' | 'ml' | 'mm' | 'mn' | 'mo' | 'mq' | 'mr' | 'mt' | 'mu' | 'mv' | 'mw' | 'mx' | 'my' | 'mz' | 'na' | 'nc' | 'ne' | 'ng' | 'ni' | 'nl' | 'no' | 'np' | 'nz' | 'om' | 'pa' | 'pe' | 'pf' | 'pg' | 'ph' | 'pk' | 'pl' | 'pr' | 'ps' | 'pt' | 'py' | 'qa' | 're' | 'ro' | 'rs' | 'ru' | 'rw' | 'sa' | 'sc' | 'sd' | 'se' | 'sg' | 'si' | 'sk' | 'sl' | 'sm' | 'sn' | 'so' | 'sr' | 'ss' | 'st' | 'sv' | 'sx' | 'sy' | 'sz' | 'tc' | 'td' | 'tg' | 'th' | 'tj' | 'tl' | 'tm' | 'tn' | 'tr' | 'tt' | 'tw' | 'tz' | 'ua' | 'ug' | 'us' | 'uy' | 'uz' | 'vc' | 've' | 'vg' | 'vi' | 'vn' | 'ye' | 'yt' | 'za' | 'zm' | 'zw' — Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country. - `maxAgeMs` integer, nullable — Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh. - object — Crawl pages and return HTML. - `type` 'html', required — Return page content as HTML. - `source` union, required — How to find pages to crawl. - object — Discover pages by following links from one URL. - … - object — Scrape the pages listed in a domain's sitemap. Links on those pages are not followed. - … - `options` object — Options for HTML output. - `useMainContentOnly` boolean — Return the main content without navigation or footers. - `pdf` object — PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range. - … - `includeSelectors` string[], nullable — CSS selectors. When provided, only matching subtrees (and their descendants) are kept and everything else is dropped. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored), metadata still comes from the full document, and crawl link discovery still uses the full document. - `excludeSelectors` string[], nullable — CSS selectors to remove from the result. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Selector-filtered pages are always fetched fresh (maxAgeMs is ignored). - `waitForMs` integer — Browser wait time in milliseconds after initial page load. Only applies to pages that need a browser render; most batch pages are fetched without one. Min: 0. Max: 15000 (15 seconds). - `settleAnimations` boolean — Wait briefly for CSS and transition animations to settle before extraction, on pages that render in a browser. - `country` 'ad' | 'ae' | 'af' | 'ag' | 'ai' | 'al' | 'am' | 'ao' | 'ar' | 'at' | 'au' | 'aw' | 'az' | 'ba' | 'bb' | 'bd' | 'be' | 'bf' | 'bg' | 'bh' | 'bi' | 'bj' | 'bm' | 'bn' | 'bo' | 'bq' | 'br' | 'bs' | 'bw' | 'by' | 'bz' | 'ca' | 'cd' | 'cf' | 'cg' | 'ch' | 'ci' | 'cl' | 'cm' | 'cn' | 'co' | 'cr' | 'cv' | 'cw' | 'cy' | 'cz' | 'de' | 'dj' | 'dk' | 'dm' | 'do' | 'dz' | 'ec' | 'ee' | 'eg' | 'es' | 'et' | 'fi' | 'fj' | 'fr' | 'ga' | 'gb' | 'gd' | 'ge' | 'gf' | 'gg' | 'gh' | 'gm' | 'gn' | 'gp' | 'gq' | 'gr' | 'gt' | 'gu' | 'gw' | 'gy' | 'hk' | 'hn' | 'hr' | 'ht' | 'hu' | 'id' | 'ie' | 'il' | 'im' | 'in' | 'iq' | 'ir' | 'is' | 'it' | 'je' | 'jm' | 'jo' | 'jp' | 'ke' | 'kg' | 'kh' | 'kn' | 'kr' | 'kw' | 'ky' | 'kz' | 'la' | 'lb' | 'lc' | 'lk' | 'lr' | 'ls' | 'lt' | 'lu' | 'lv' | 'ly' | 'ma' | 'mc' | 'md' | 'me' | 'mf' | 'mg' | 'mk' | 'ml' | 'mm' | 'mn' | 'mo' | 'mq' | 'mr' | 'mt' | 'mu' | 'mv' | 'mw' | 'mx' | 'my' | 'mz' | 'na' | 'nc' | 'ne' | 'ng' | 'ni' | 'nl' | 'no' | 'np' | 'nz' | 'om' | 'pa' | 'pe' | 'pf' | 'pg' | 'ph' | 'pk' | 'pl' | 'pr' | 'ps' | 'pt' | 'py' | 'qa' | 're' | 'ro' | 'rs' | 'ru' | 'rw' | 'sa' | 'sc' | 'sd' | 'se' | 'sg' | 'si' | 'sk' | 'sl' | 'sm' | 'sn' | 'so' | 'sr' | 'ss' | 'st' | 'sv' | 'sx' | 'sy' | 'sz' | 'tc' | 'td' | 'tg' | 'th' | 'tj' | 'tl' | 'tm' | 'tn' | 'tr' | 'tt' | 'tw' | 'tz' | 'ua' | 'ug' | 'us' | 'uy' | 'uz' | 'vc' | 've' | 'vg' | 'vi' | 'vn' | 'ye' | 'yt' | 'za' | 'zm' | 'zw' — Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country. - `maxAgeMs` integer, nullable — Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh. - `webhookUrl` string — URL notified when the batch finishes. - `tags` string[] — Tags stored on the batch. Filter the batch list by them later. ## Response `202` Batch created - object — Details returned when a batch is created. - `id` string, required — Batch ID used to retrieve or cancel the job. - `status` 'queued' | 'running' | 'cancelling' | 'completed' | 'cancelled' | 'failed', required — Current state. `completed`, `cancelled`, and `failed` are final. - `mode` 'scrape' | 'crawl', required — How pages are selected. - `type` 'markdown' | 'html', required — Output format. - `tags` string[], required — Tags stored on the batch at submission. - `input` object, required — Submission counts. - `submitted` integer, required — Pages submitted before validation. For a crawl, the page limit. - `accepted` integer, required — Pages accepted, or the crawl page limit. Credits are reserved for this count. - `duplicates` integer, required — Duplicate URL and `itemId` pairs skipped. Always 0 for crawls. - `invalid` integer, required — Pages rejected during validation. - `progress` object, required — Current processing counts. Use `status` to check completion. - `succeeded` integer, required — Pages scraped successfully. - `failed` integer, required — Pages that could not be scraped. - `pending` integer, required — Accepted pages not yet attempted. Always 0 once the batch completes; a crawl can finish under its page limit when the site has no more reachable pages. - `credits` object, required — Reserved and used credits. - `estimated` integer, required — Credits reserved when the batch was accepted. - `charged` integer, required — Credits used by successful pages. - `timing` object, required - `created_at` string, required — When the batch was created. - `started_at` string, nullable, required — When processing started. Null while queued. - `completed_at` string, nullable, required — When processing finished. Null while active. - `errors` BatchErrorCount[], required — Page failures grouped by error code. - `code` string, required — Error code for these failures. - `count` integer, required — Pages that failed with this code. - `error` BatchNullableError, required — Why the batch failed. - `code` string, required — Batch error code. - `message` string, required — Batch error message. - `results` object, nullable, required — Download links available when the batch finishes. GET /batch/{batch_id}/results serves the same records as paginated JSON. - `expires_at` string, required — When the download URLs expire. - `files` object[], required — Result files. Order is not guaranteed. - `url` string, required — Temporary URL for a gzipped NDJSON file. - `items` integer, required — Results in this file. - `bytes` integer, required — Compressed file size in bytes. - `key_metadata` KeyMetadata — Metadata about the API key used for the request. Included in every response whenever a valid API key is provided, even when the response status is not 200. - `credits_consumed` integer, required — The number of credits consumed by this request. - `credits_remaining` integer, required — The number of credits remaining for your organization after this request. - `invalid_urls` object[], required — Rejected URLs, up to 100. These are not charged. - `url` string, required — Rejected URL. - `reason` string, required — Why it was rejected. - `webhook_secret` string — Webhook signing secret. Also returned by GET /batch/{batch_id}. ## Other responses - `400` — Bad request - `401` — Unauthorized - `403` — Concurrent batch limit reached (error_code BATCH_LIMIT_EXCEEDED) - `409` — Idempotency-Key already used with a different request body (error_code IDEMPOTENCY_KEY_CONFLICT) - `500` — Batch input could not be staged or queued ## Changes - **2026-07-31** `f5c503b03f98` — 1 info - endpoint added [Change history](https://skmtc.dev/context/apis/context-dev/changes/batch/submit/post.md) --- [API](https://skmtc.dev/context/apis/context-dev.md) · [All operations](https://skmtc.dev/context/apis/context-dev/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/context/context-dev/revisions/f5c503b03f98/schema)