tavily

Map a website's sitegraph

Returns the sitegraph of a website starting from a root URL: a list of discovered URLs without their full content. Cheaper and faster than crawl when you only need to enumerate pages (e.g. for sitemap discovery or coverage analysis). Use instructions to guide the mapper in natural language and the select_* / exclude_* filters (regex patterns, comma-separated) to narrow scope.

get/tavily/map

Request

  • Base URL: https://www.socialcrawl.dev/v1
  • URL: https://www.socialcrawl.dev/v1/tavily/map
  • Auth: API key in header x-api-key

Query parameters

urlstring required

Root URL to begin mapping.

max_depthinteger

Maximum link depth from the root URL. Defaults to 1.

max_breadthinteger

Maximum number of links followed per level (per page). Defaults to 20.

limitinteger

Total number of links the mapper will process before stopping. Defaults to 50.

instructionsstring

Natural-language instructions for the mapper (e.g. 'Find all pages related to API documentation').

select_pathsstring

Comma-separated regex patterns: only include URLs whose path matches.

select_domainsstring

Comma-separated regex patterns: only include URLs whose domain matches.

exclude_pathsstring

Comma-separated regex patterns: exclude URLs whose path matches.

exclude_domainsstring

Comma-separated regex patterns: exclude URLs whose domain matches.

allow_externalboolean

Whether to follow / return links to external domains. Defaults to true.

timeoutinteger

Maximum time in seconds (10-150).

categoriesstring

Comma-separated list of category hints to bias mapping toward.

Headers

Cache-Controlstring

Send no-cache to bypass the response cache and force a live fetch. Billed at the normal endpoint cost; the fresh result is written back to cache for the next caller. Only the no-cache directive triggers this. See the Response Schema guide for details.

Idempotency-Keystring

Optional UUID that makes the request safely retriable. The key is reserved atomically per account before billing. A concurrent retry receives 409 IDEMPOTENCY_IN_PROGRESS with a Retry-After of 1 second while the original is young, growing to a quarter of the time it has been running (at most 30); a crashed reservation expires after 5 minutes. A replay keeps the cached payload immutable except for billing metadata: credits_used becomes 0, idempotent_replay becomes true, and credits_remaining is refreshed to the current balance. A known current balance appears in both the body and X-Credits-Remaining header; no balance row resolves to 0. On a transient lookup failure, body credits_remaining is null and X-Credits-Remaining is omitted. Completed outcomes expire after 24 hours. If the claim store is unavailable, the request fails closed before billing. GET endpoints do not combine idempotent replay with Server-Sent Events: use Accept: application/json when sending this header. The unsupported key-plus-SSE combination is rejected before billing.

Response

Successful response

successboolean required

Whether the request succeeded

platformstring required

Platform name

endpointstring required

API endpoint path

credits_usedinteger required

Number of credits consumed

credits_remaininginteger nullable required

Current account balance, or null when this response could not establish one. Free cache hits always return null because they do not establish an authoritative post-request balance. An idempotency replay also returns null when its current-balance lookup fails. It is never a fabricated zero, so a zero here means the account really is empty. The authoritative reading is the one on a billed response, which comes straight from the deduction.

request_idstring required

Unique request identifier for support

cachedboolean required

Whether the response was served from cache

idempotent_replayboolean

True only when this response is an idempotency replay

Example response

{
  "success": true,
  "platform": "tavily",
  "endpoint": "/v1/tavily/map",
  "credits_used": 1,
  "credits_remaining": 4999,
  "request_id": "req-abc123",
  "idempotent_replay": true,
  "meta": {
    "hint": {
      "code": "search_multi",
      "message": "You searched \"matcha\" on 3 platforms one call at a time; /v1/search/multi returns each platform's own search rows in one call at the same price per platform, and a platform with no rows is not charged.",
      "method": "GET",
      "path": "/v1/search/multi?query=matcha&platforms=instagram,tiktok,youtube"
    }
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.