---
title: "Create a crawl"
method: POST
path: "/v1/crawls"
tags: ["crawls"]
---

# Create a crawl

`POST /v1/crawls`

## Request

- Base URL: `https://api.olostep.com`
- URL: `https://api.olostep.com/v1/crawls`
- Auth: HTTP bearer

Starts crawling a site from a starting URL.

## Request body

- object
  - `start_url` string, uri, required: The URL to start crawling from.
  - `include_urls` string[]
  - `exclude_urls` string[]
  - `max_pages` integer
  - `max_depth` integer
  - `include_external` boolean
  - `include_subdomain` boolean
  - `search_query` string
  - `top_n` integer
  - `webhook` string: Webhook URL, or {endpoint} object.
  - `timeout` integer
  - `follow_robots_txt` boolean
  - `scrape_options` object
    - `formats` string[]: Output formats to request for each scraped page.
    - `parser` string

## Response `200`

Default Response

- object: A crawl resource.
  - `id` string, required
  - `object` 'crawl', required
  - `status` string, required
  - `created` number, required
  - `start_date` string
  - `start_url` string, required
  - `max_pages` number, nullable
  - `max_depth` number, nullable
  - `exclude_urls` string[]
  - `include_urls` string[]
  - `include_external` boolean
  - `search_query` string
  - `top_n` number
  - `current_depth` number
  - `pages_count` number
  - `webhook` string, nullable
  - `webhook_url` string, nullable
  - `follow_robots_txt` boolean
  - `credits_consumed` number, nullable
  - `cost_usd` number, nullable

## Other responses

- `400`: Default Response
- `401`: Default Response
- `429`: Default Response
- `502`: Default Response
- `504`: Default Response

---

[API](https://skmtc.dev/olostep/apis/olostep-api.md) · [All operations](https://skmtc.dev/olostep/apis/olostep-api/llms.txt) · [OpenAPI document](https://skmtc.dev/olostep/apis/olostep-api/revisions/13c766ab100c?raw)