---
title: "Submit a bounded resumable web crawl"
method: POST
path: "/web/crawl"
tags: ["Retrieval"]
---

# Submit a bounded resumable web crawl

`POST /web/crawl`

## Request

- Base URL: `https://api.crawlora.net/api/v1`
- URL: `https://api.crawlora.net/api/v1/web/crawl`
- Auth: API key in header `x-api-key`

Accepts an owner-scoped durable crawl job. `max_credits` is required; the accepted request and active reader/source revisions are frozen for idempotent replay. A repeated matching idempotency key returns the original job without preparing another reservation.

## Request body

- CrawlretrievalCrawlRequestDoc
  - `fetch` CrawlretrievalCrawlFetchDoc
    - `max_age` integer
    - `only_main_content` boolean
    - `render` 'auto' | 'http'
    - `wait_for` integer
  - `idempotency_key` string, required
  - `limits` CrawlretrievalCrawlLimitsDoc, required
    - `max_browser_renders` integer
    - `max_credits` integer, required
    - `max_depth` integer
    - `max_pages` integer
    - `max_total_bytes` integer
    - `timeout_seconds` integer
  - `scope` CrawlretrievalCrawlScopeDoc
    - `exclude_paths` string[]
    - `include_paths` string[]
    - `include_subdomains` boolean
    - `query_policy` 'preserve' | 'drop_tracking'
  - `sitemap` 'include' | 'only' | 'skip'
  - `url` string, uri, required

## Response `202`

Durable crawl job accepted

- CrawlretrievalCrawlSubmissionEnvelope
  - `code` integer
  - `data` CrawlretrievalCrawlJobSubmissionDoc
    - `deadline_at` string, date-time
    - `job_id` string
    - `replayed` boolean
    - `results_url` string, uri
    - `state` 'queued' | 'running' | 'cancelling' | 'succeeded' | 'failed' | 'cancelled'
    - `status_url` string, uri
  - `msg` string

## Other responses

- `400`: Invalid request or idempotency conflict
- `401`: Authentication required
- `402`: Insufficient credits or daily budget
- `404`: Resource not found
- `429`: Rate or active job quota exceeded
- `500`: Internal configuration or storage error
- `503`: Required backend or source profile unavailable

---

[API](https://skmtc.dev/crawlora/apis/crawlora-api.md) · [All operations](https://skmtc.dev/crawlora/apis/crawlora-api/llms.txt) · [OpenAPI document](https://skmtc.dev/crawlora/apis/crawlora-api/revisions/028417aa35bb?raw)