SmartScraper

Start Smartscraper

post/v1/smartscraper

Request body

website_urlstring nullable

The URL of the website to scrape (optional for Smartscraper)

render_heavy_jsboolean

Whether to render heavy JS

mockboolean

stealthboolean

brandingboolean

streamboolean

Whether to return streaming response

country_codestring nullable

The country code to use for the scrape

wait_msinteger

The number of milliseconds to wait before scraping the website

user_promptstring required

website_htmlstring nullable

HTML content, maximum size 2MB

website_markdownstring nullable

Markdown content, maximum size 2MB

output_schemaobject nullable

headersobject nullable

Optional headers to send with the request, including cookies and user agent

number_of_scrollsinteger

Number of scroll operations to perform for infinite scrolling (0 = disabled, range: 1-100). Values between 1-9 will be automatically set to 10.

total_pagesinteger

Total number of pages to scrape if pagination enabled

stepsstring[] nullable

List of steps to actions to perform on the website

cookiesobject nullable

Cookies to send with the request

plain_textboolean

webhook_urlstring nullable

Webhook URL to send the job result to

offsetinteger

Offset to start scraping from

is_multipage_childboolean

Whether this is a multi-page child request

Example request

{
  "user_prompt": "Extract info about the company",
  "website_html": "<html><body><h1>Title</h1><p>Content</p></body></html>",
  "website_markdown": "# Title\n\nThis is some content in markdown format.",
  "headers": {
    "Cookie": "cookie1=value1; cookie2=value2",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
  }
}

Response

Successful Response

request_idstring required

status'queued' | 'processing' | 'completed' | 'failed' required