---
title: "Crawl a website for indexed search"
method: GET
path: "/integrations/web_crawler/index"
tags: ["Integrations", "Integrations"]
---

# Crawl a website for indexed search

`GET /integrations/web_crawler/index`

Recursively crawl a website to make it available for indexed search.

## Query parameters

- `url` string, required — The base URL of the website to crawl
- `max_depth` integer — Maximum depth of links to follow during crawling
- `limit` integer — Maximum number of pages to crawl in total

## Response `200`

Successful Response

- ResourceStatusResponse
  - `source` 'reddit' | 'notion' | 'slack' | 'google_calendar' | 'google_mail' | 'box' | 'dropbox' | 'github' | 'google_drive' | 'vault' | 'web_crawler' | 'trace' | 'microsoft_teams' | 'gmail_actions' | 'granola' | 'fathom' | 'fireflies' | 'linear' | 'hubspot' | 'salesforce' | 'coda' | 'lightfield' | 'gong', required
  - `resource_id` string, required
  - `status` 'pending' | 'processing' | 'completed' | 'failed' | 'pending_review' | 'skipped', required

## Other responses

- `422` — Validation Error

## Changes

> 37 revisions in range; 1 not diffed.

- **2026-06-17** `77d3e8d5e8a7` — 1 warning
  - added the new `gong` enum value to the `source` response property for the response status `200`
- **2026-05-29** `5485767cdcda` — 1 warning
  - added the new `fireflies` enum value to the `source` response property for the response status `200`
- **2026-05-20** `912f94dd47ba` — 1 warning
  - added the new `lightfield` enum value to the `source` response property for the response status `200`
- **2026-05-20** `d7c846c2cd79` — 1 warning
  - added the new `coda` enum value to the `source` response property for the response status `200`
- **2026-05-18** `6eb290df2d3b` — 2 warning
  - added the new `hubspot` enum value to the `source` response property for the response status `200`
  - added the new `salesforce` enum value to the `source` response property for the response status `200`

[Full history](https://skmtc.dev/hyperspell/apis/hyperspell-api/changes/integrations/web_crawler/index/get.md)

---

[API](https://skmtc.dev/hyperspell/apis/hyperspell-api.md) · [All operations](https://skmtc.dev/hyperspell/apis/hyperspell-api/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/hyperspell/hyperspell-api/revisions/5721acd588e6/schema)