Web Scraping

Scrape URL

Scrape a web page and return its HTML content.

Send a URL and get back the rendered HTML, response headers, and metadata. Supports antibot bypass, JavaScript rendering, CAPTCHA solving, and residential proxies.

Minimal request: {"url": "https://example.com"}

post/api/v1/scrape

Request

  • Base URL: https://api.finedata.ai (the document declares no server; this is the origin it was published from)
  • URL: https://api.finedata.ai/api/v1/scrape
  • Auth: HTTP bearer

Headers

authorizationstring nullable
x-api-keystring nullable

Request body

urlstring required

Target URL to scrape

methodstring

HTTP method (GET, POST, PUT, DELETE)

headersobject nullable

Custom HTTP headers to send with the request

bodystring nullable

Request body (for POST/PUT requests)

timeoutinteger nullable

Request timeout in seconds. Increase for captcha solving (max 300)

max_retriesinteger nullable

Max automatic retry attempts on failure

auto_retryboolean

Automatically retry on failure (+1 token per retry)

use_antibotboolean

Browser-like TLS fingerprint (+2 tokens). Enabled by default

tls_profilestring nullable

TLS fingerprint profile: chrome136, chrome131, chrome124, firefox133, safari184, safari18_0, or 'vip' for premium rotation (vip:ios, vip:android, vip:windows, vip:mobile)

use_residentialboolean

Route through residential proxy instead of datacenter (+3 tokens)

use_ispboolean

Route through ISP proxy for clean IP reputation (+2 tokens)

use_mobileboolean

Route through mobile carrier proxy for maximum anonymity (+4 tokens)

proxy_stickyboolean

Keep the same proxy exit IP for the entire browser session. Required when a warmup request and the follow-up fetch must come from one IP, e.g. a site that ties a session cookie to the address that obtained it. Works with residential proxies and BYOP profiles (rotation_mode=sticky).

proxy_countrystring nullable

Request a proxy exit IP from a specific country (ISO-2 code, e.g. 'US', 'DE'). Applies to datacenter, residential, ISP and mobile proxies. Returns 422 if the country isn't currently supported for the resolved proxy type. Mutually exclusive with proxy_profile_id (BYOP).

proxy_profile_idinteger nullable

Use your own proxy list via a saved BYOP profile id (Bring Your Own Proxy). Mutually exclusive with use_residential / use_isp / use_mobile. Charges flat +1 token instead of the vendor surcharge. Requires team_s plan or admin override.

stealth_antibotboolean

Stronger browser emulation for pages that a plain request cannot load (+7 tokens)

stealth_antibot_headfulboolean

Full real browser with a visible display, for pages the headless engine cannot load (+25 tokens)

stealth_newboolean

Experimental next-generation stealth engine (+15 tokens)

stealth_premiumboolean

Premium stealth engine with full fingerprint emulation (+20 tokens)

stealth_premium_headfulboolean

Premium stealth engine in headful mode with GPU rendering (+30 tokens)

use_js_renderboolean

Render JavaScript with headless browser. Required for SPAs (+5 tokens)

js_wait_forstring

Page load strategy: 'networkidle', 'load', 'domcontentloaded', or 'selector:.css-selector'

js_scrollboolean

Scroll to bottom to trigger lazy-loaded content

solve_captchaboolean

Auto-detect and solve captchas: reCAPTCHA, hCaptcha, Turnstile (+10 tokens)

session_idstring nullable

Sticky session ID. All requests with the same ID share the same proxy IP. Max 64 chars

session_ttlinteger nullable

Session lifetime in seconds (default: 1800 = 30 min, max: 86400 = 24h)

formatsstring[] nullable

Output formats to include in response. Options: 'markdown', 'rawHtml', 'text', 'links', 'screenshot', 'csv', 'xlsx'. csv/xlsx export AI extraction results as tabular data (requires extract_prompt or extract_schema). If not specified, only 'body' (raw HTML) is returned for backward compatibility. Example: ['markdown', 'links']

only_main_contentboolean

Extract only the main content of the page (removes navigation, footer, sidebar, ads). Applies to 'markdown' and 'text' formats. Uses Mozilla Readability algorithm

raw_outputboolean

Return clean content directly instead of JSON wrapper. Requires exactly one format in 'formats' (e.g. formats=['markdown']). Response Content-Type matches the format: text/markdown, text/plain, text/html, image/png, text/csv, application/xlsx. Metadata is returned in response headers: X-Tokens-Used, X-Request-Id, X-Status-Code. Without 'formats', returns raw HTML as text/html.

extract_rulesobject nullable

CSS/XPath selectors for structured data extraction. Simple: {'title': 'h1', 'price': '.price'}. Advanced: {'links': {'selector': 'a.product', 'type': 'list', 'output': '@href'}}. Supported types: text (default), html, list. Output: text content or @attr

extract_schemaobject nullable

JSON Schema for AI-powered structured data extraction. The AI model analyzes page content and returns data matching your schema. Example: {'type': 'object', 'properties': {'products': {'type': 'array', 'items': {'type': 'object', 'properties': {'name': {'type': 'string'}, 'price': {'type': 'number'}}}}}} (+5 tokens)

extract_promptstring nullable

Natural language instruction for AI-powered data extraction. Alternative to extract_schema. Example: 'Extract all product names, prices, and ratings from this page' (+5 tokens)

ai_content_mode'full' | 'main' nullable

Content mode for AI extraction (extract_schema/extract_prompt). 'full' — AI sees full page content including sidebars, structured data blocks (best for product pages, real estate listings, marketplaces). 'main' — AI sees only main content extracted via Readability algorithm (best for articles, blog posts — fewer tokens, less noise). Default: 'full'

Example request

{
  "url": "https://example.com"
}

Response

JSON response (default) or raw content when raw_output=true

successboolean required

Whether the request completed successfully

status_codeinteger required

HTTP status code of the target page

headersobject required

Response headers from the target page

bodystring required

Raw HTML or text content of the page (kept for backward compatibility)

metaobject

Request metadata: url, resolved_url, elapsed_ms, proxy_country

tokens_usedinteger

Total tokens consumed for this request

captcha_detectedboolean

Whether a captcha was detected on the page

captcha_typestring nullable

Captcha type if detected: recaptcha_v2, recaptcha_v3, hcaptcha, turnstile

captcha_solvedboolean

Whether the captcha was solved successfully

Example response

{
  "body": "<!DOCTYPE html><html>...</html>",
  "captcha_detected": false,
  "captcha_solved": false,
  "data": {
    "markdown": "# Example Page\n\nThis is the page content...",
    "metadata": {
      "description": "An example page",
      "title": "Example"
    }
  },
  "headers": {
    "content-type": "text/html; charset=utf-8"
  },
  "meta": {
    "elapsed_ms": 1250,
    "resolved_url": "https://example.com/",
    "url": "https://example.com"
  },
  "status_code": 200,
  "success": true,
  "tokens_used": 3
}

Changes

No recorded changes to this operation across all 1 revision of this API.