tavily

Extract clean content from one or more URLs

Pulls clean, AI-ready text out of one or more URLs via Tavily's extractor. Pass a single URL or a comma-separated list (max 20). Returns extracted content per URL alongside any URLs that failed extraction. Use extract_depth=advanced for harder pages (paywalls, JS-heavy SPAs).

get/tavily/extract

Request

  • Base URL: https://www.socialcrawl.dev/v1
  • URL: https://www.socialcrawl.dev/v1/tavily/extract
  • Auth: API key in header x-api-key

Query parameters

urlsstring required

Comma-separated list of URLs to extract (max 20).

extract_depth'basic' | 'advanced'

Extraction strategy. basic is the default and faster; advanced handles harder pages but takes longer.

format'markdown' | 'text'

Output format for the extracted content. markdown (default) preserves structure; text is plain.

include_imagesboolean

Include images extracted from each URL.

include_faviconboolean

Include the favicon URL for each page.

timeoutinteger

Per-URL timeout in seconds (1-60).

Headers

Cache-Controlstring

Send no-cache to bypass the response cache and force a live fetch. Billed at the normal endpoint cost; the fresh result is written back to cache for the next caller. Only the no-cache directive triggers this. See the Response Schema guide for details.

Idempotency-Keystring

Optional UUID that makes the request safely retriable. The key is reserved atomically per account before billing. A concurrent retry receives 409 IDEMPOTENCY_IN_PROGRESS with a Retry-After of 1 second while the original is young, growing to a quarter of the time it has been running (at most 30); a crashed reservation expires after 5 minutes. A replay keeps the cached payload immutable except for billing metadata: credits_used becomes 0, idempotent_replay becomes true, and credits_remaining is refreshed to the current balance. A known current balance appears in both the body and X-Credits-Remaining header; no balance row resolves to 0. On a transient lookup failure, body credits_remaining is null and X-Credits-Remaining is omitted. Completed outcomes expire after 24 hours. If the claim store is unavailable, the request fails closed before billing. GET endpoints do not combine idempotent replay with Server-Sent Events: use Accept: application/json when sending this header. The unsupported key-plus-SSE combination is rejected before billing.

Response

Successful response

successboolean required

Whether the request succeeded

platformstring required

Platform name

endpointstring required

API endpoint path

credits_usedinteger required

Number of credits consumed

credits_remaininginteger nullable required

Current account balance, or null when this response could not establish one. Free cache hits always return null because they do not establish an authoritative post-request balance. An idempotency replay also returns null when its current-balance lookup fails. It is never a fabricated zero, so a zero here means the account really is empty. The authoritative reading is the one on a billed response, which comes straight from the deduction.

request_idstring required

Unique request identifier for support

cachedboolean required

Whether the response was served from cache

idempotent_replayboolean

True only when this response is an idempotency replay

Example response

{
  "success": true,
  "platform": "tavily",
  "endpoint": "/v1/tavily/extract",
  "credits_used": 1,
  "credits_remaining": 4999,
  "request_id": "req-abc123",
  "idempotent_replay": true,
  "meta": {
    "hint": {
      "code": "search_multi",
      "message": "You searched \"matcha\" on 3 platforms one call at a time; /v1/search/multi returns each platform's own search rows in one call at the same price per platform, and a platform with no rows is not charged.",
      "method": "GET",
      "path": "/v1/search/multi?query=matcha&platforms=instagram,tiktok,youtube"
    }
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.