---
title: "Scrape Bulk"
method: POST
path: "/scrape/bulk"
tags: ["scraping"]
---

# Scrape Bulk

`POST /scrape/bulk`

Bulk scrape multiple URLs with checkpointing

Requires permission: scrape:write

## Request body

- BulkScrapeRequest — Request model for bulk URL scraping
  - `urls` string[], required — List of URLs to scrape
  - `engines` ScrapeEngine[], required — List of engines to use
    - `engine_type` 'FLEET' | 'ZENROWS' | 'SCRAPINGBEE' | 'FLEET_ASYNC' | 'FLEET_WORKFLOW' | 'ASYNC_FLEET_STICKY', required
    - `timeout` integer, nullable — Timeout in milliseconds
    - `headers` object, nullable — Custom headers
    - `proxy` string, nullable — Proxy URL
    - `wait_for` string, nullable — CSS selector to wait for (Playwright only)
    - `screenshot` boolean, nullable — Take screenshot (Playwright only)
  - `crawl_id` string, required — Unique identifier for this crawl
  - `max_workers` integer — Maximum concurrent workers
  - `batch_size` integer — URLs per batch
  - `debug` boolean — Enable debug information

## Response `200`

Successful Response

- BulkScrapeResponse — Response model for bulk URL scraping
  - `crawl_id` string, required
  - `total_urls` integer, required
  - `total_pages_visited` integer, required
  - `success_count` integer, required
  - `failed_count` integer, required
  - `cached_count` integer, required
  - `save_failed_count` integer, required
  - `timestamp` string, required
  - `results` BulkScrapeResultsByStatus, required — Grouped results by status
    - `success` BulkScrapeResult[]
      - `url` string, required
      - `status` string, required
      - `status_code` integer, nullable
      - `composite_hash` string, nullable
      - `error` string, nullable
    - `cached` BulkScrapeResult[]
      - `url` string, required
      - `status` string, required
      - `status_code` integer, nullable
      - `composite_hash` string, nullable
      - `error` string, nullable
    - `failed` BulkScrapeResult[]
      - `url` string, required
      - `status` string, required
      - `status_code` integer, nullable
      - `composite_hash` string, nullable
      - `error` string, nullable
    - `save_failed` BulkScrapeResult[]
      - `url` string, required
      - `status` string, required
      - `status_code` integer, nullable
      - `composite_hash` string, nullable
      - `error` string, nullable
  - `debug_frame` object[], nullable

## Other responses

- `422` — Validation Error

## Changes

- **2026-02-17** `29225d7b3ea4` — 1 warning, 2 info
  - removed the request property `s3_bucket`
  - added the optional property `detail/items/ctx` to the response with the `422` status
  - added the optional property `detail/items/input` to the response with the `422` status
- **2025-09-18** `019c4555bd7f` — 1 info
  - added the new `ASYNC_FLEET_STICKY` enum value to the request property `engines/items/engine_type`
- **2025-08-12** `707c0eb9cee1` — 1 breaking, 2 info
  - removed the enum value `ASYNC_FLEET` of the request property `engines/items/engine_type`
  - added the new `FLEET_ASYNC` enum value to the request property `engines/items/engine_type`
  - added the new `FLEET_WORKFLOW` enum value to the request property `engines/items/engine_type`
- **2025-08-05** `4eb68ae85080` — 4 breaking, 4 info
  - removed the enum value `async_fleet` of the request property `engines/items/engine_type`
  - removed the enum value `fleet` of the request property `engines/items/engine_type`
  - removed the enum value `scrapingbee` of the request property `engines/items/engine_type`
  - removed the enum value `zenrows` of the request property `engines/items/engine_type`
  - …4 more

[Change history](https://skmtc.dev/evrimai/apis/cartography-server/changes/scrape/bulk/post.md)

---

[API](https://skmtc.dev/evrimai/apis/cartography-server.md) · [All operations](https://skmtc.dev/evrimai/apis/cartography-server/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/evrimai/cartography-server/revisions/5ee44bc43d24/schema)