---
title: "Attach a web source and crawl it now"
method: POST
path: "/voice/collections/{collection_id}/scrape-sources"
tags: ["Voice"]
---

# Attach a web source and crawl it now

`POST /voice/collections/{collection_id}/scrape-sources`

## Request

- Base URL: `https://be.graph8.com/api/v1`
- URL: `https://be.graph8.com/api/v1/voice/collections/{collection_id}/scrape-sources`
- Auth: HTTP bearer

Crawl a site into this collection, now and then on a schedule.

THE FIRST CRAWL STARTS IMMEDIATELY. It does not wait for the cadence, so
creating a source with `frequency: manual` still crawls once. Up to
`max_pages` pages are fetched and embedded, and both cost money - which is
why this is `billable` and voice guards it with its EXPENSIVE rate limit.

`frequency` is closed to voice's four cadences here. Voice itself takes a
bare string and only its scheduler notices an unknown one, so an invented
value there produces a source that quietly never re-crawls.

## Path parameters

- `collection_id` integer, required: Collection id.

## Headers

- `Authorization` string, nullable
- `X-Target-Org-Id` string, nullable

## Request body

- ScrapeSourceCreateRequest: A site to crawl into a collection on a schedule. CREATING ONE CRAWLS IMMEDIATELY - it does not wait for the first scheduled run. That is why this operation is `billable` rather than a plain write.
  - `url` string, required: Root URL. Everything under it is followed.
  - `frequency` 'daily' | 'weekly' | 'monthly' | 'manual': How often to re-crawl. `manual` never re-crawls.
  - `max_pages` integer: Ceiling, not a target - a small site only crawls what exists.

## Response `201`

Successful Response

- ApiResponseVoicePayload
  - `data` VoicePayload, required: One voice object - a collection, an agent, a route, a chat. Open-shaped: voice adds fields to these bodies on its own release cadence and a dropped-or-added key must not be a 500 in this service.
  - `pagination` PaginationMeta
    - `page` integer, required: Current page number (1-indexed)
    - `limit` integer, required: Items per page
    - `total` integer, required: Total number of items
    - `has_next` boolean, required: Whether there are more pages
    - `next_cursor` string, nullable: Opaque cursor for the next page; null when there are no more items. Pass it back as the `cursor` query param to fetch the following page without computing page numbers.
    - `total_available` integer, nullable: Search endpoints only: how many records really match, when known. Can be larger than `total`, which is capped at the 10,000 results that can be paged. For company search this is a distinct-company estimate. Null when unknown; absent outside search.
    - `total_is_lower_bound` boolean, nullable: Search endpoints only: true when more records match than `total` shows (the result set is larger than the 10,000 results that can be paged). Absent outside search.
    - `notes` string[], nullable: Search endpoints only: plain-language notes about the result, e.g. that more records match than can be paged, or that a reversed `between` range was put in order.

## Other responses

- `422`: Validation Error

---

[API](https://skmtc.dev/graph8/apis/graph8-developer-api.md) · [All operations](https://skmtc.dev/graph8/apis/graph8-developer-api/llms.txt) · [OpenAPI document](https://skmtc.dev/graph8/apis/graph8-developer-api/revisions/2e7cfa0ca921?raw)