WebScraper

Fetch any URL and extract structured data including title, meta description, keywords, favicon, social links, all links on the page, emails, phone numbers, images, body text, and detected languages. Meta keywords are always a JSON array of strings: content="php,,laravel" returns data.keywords=["php","laravel"]. Empty entries are omitted, remaining order and duplicates are preserved, and missing or all-empty metadata returns []. Set javascript=true for client-rendered pages. In JSON responses, url is the requested URL (with a protocol added if omitted), and final_url is the destination after redirects. Relative links, image URLs, and favicons are resolved against final_url, preserving non-default ports. Query-only references keep the page path: ?page=2 at https://example.com/catalog/page?old=1 resolves to https://example.com/catalog/page?page=2. Relative dot segments are normalized: /catalog/../item becomes /item and /catalog/./item becomes /catalog/item. Reference queries and fragments are preserved. Default HTTP(S) ports may be omitted from resolved relative URLs. Only HTTP(S) destinations are returned; absolute HTTP(S) references are left unchanged. If the destination URL is unavailable, the URL used for the fetch is returned. Returns the site's HTTP status code separately so you can distinguish site errors (404, 500) from infrastructure errors. Use response_type=markdown to get only the page content as clean markdown.

get/web-scraper

Request

  • Base URL: https://scrappa.co/api
  • URL: https://scrappa.co/api/web-scraper
  • Auth: API key in header X-API-KEY

Query parameters

urlstring required

The URL to scrape (must include protocol, e.g., https://example.com)

include_htmlboolean nullable

Set to true to include the raw HTML in the response. Default: false. Ignored when response_type=markdown

javascriptboolean nullable

Set to true to execute page JavaScript before extracting content. Rendering uses Obscura first and can fall back to Cloudflare Browser Rendering if navigation fails. Default: false

response_type'json' | 'markdown' | 'null' nullable

Set to "markdown" to return only the page content as clean markdown text. When set to markdown, only the markdown content is returned (not JSON). Default: json

Response

string required

Changes

No recorded changes to this operation across all 3 revisions of this API.