---
title: "Crawl Website"
method: POST
path: "/api/v1/brands/{brand_id}/crawl"
tags: ["recommendations"]
---

# Crawl Website

`POST /api/v1/brands/{brand_id}/crawl`

## Request

- Base URL: `https://geo.sig.ai` (the document declares no server; this is the origin it was published from)
- URL: `https://geo.sig.ai/api/v1/brands/{brand_id}/crawl`
- Auth: HTTP bearer

Crawl a brand's website to collect content for analysis.

This endpoint starts an asynchronous task to:
1. Crawl all pages of the brand's website (up to max_pages)
2. Extract structured content (title, headings, body text, keywords, etc.)
3. Store content snapshots in the database for recommendation generation

Returns immediately with a task ID.

## Path parameters

- `brand_id` string, uuid, required

## Cookies

- `access_token` string, nullable

## Request body

- CrawlWebsiteRequest: Request to crawl a brand's website.
  - `max_pages` integer: Maximum number of pages to crawl (1-200)

## Response `202`

Successful Response

- CrawlTaskResponse: Response for async website crawling task.
  - `status` string, required: crawling, complete, failed
  - `task_id` string, required
  - `estimated_time_seconds` integer
  - `message` string, nullable
  - `pages_to_crawl` integer, nullable

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/sig/apis/ai-visibility-intelligence-api.md) · [All operations](https://skmtc.dev/sig/apis/ai-visibility-intelligence-api/llms.txt) · [OpenAPI document](https://skmtc.dev/sig/apis/ai-visibility-intelligence-api/revisions/3ec39f9be9e0?raw)