SCscrapebadger

Scrape URL

Scrape a webpage and return its content as HTML, Markdown, or plain text.

post/v1/web/scrape

Request

Request body

urlstring required

The URL to scrape. Must be a valid HTTP or HTTPS URL.

engine'auto' | 'browser'

Scraping engine tier to use.

format'html' | 'markdown' | 'text'

Output format for the scraped content.

render_jsboolean

Force JavaScript rendering.

wait_forstring

CSS selector or XPath to wait for before extracting.

wait_timeoutinteger

Max wait time in ms for wait_for selector.

wait_after_loadinteger

Additional ms to wait after page load.

js_scenarioobject[]

Browser actions to perform before extracting.

session_idstring

Persist cookies and state across requests.

retry_countinteger

Max retry attempts on blocking detection.

retry_on_blockboolean

Auto-retry on blocking page detection.

countrystring

ISO 3166-1 alpha-2 country code for proxy geo-targeting.

custom_headersobject

Additional HTTP headers for the target request.

screenshotboolean

Capture a full-page PNG screenshot.

videoboolean

Record browser session as animated GIF (+3 credits).

anti_botboolean

Attempt anti-bot bypass when blocking detected.

escalateboolean

Allow auto-escalation to stronger engines.

max_costinteger

Maximum credits budget for this request.

ai_extractboolean

Run AI extraction on scraped content.

ai_promptstring

Natural language instruction for AI extraction.

Response

Successful scrape

successboolean
urlstring
status_codeinteger
contentstring
formatstring
engine_usedstring
credits_usedinteger
duration_msinteger
retries_usedinteger
content_lengthinteger
screenshot_urlstring nullable
video_urlstring nullable
headersobject
blocking_detectedboolean
blocking_detailsobject nullable
anti_bot_solvedboolean
solver_usedstring nullable
ai_extractionunknown
ai_modelstring nullable
ai_errorstring nullable

Changes

No recorded changes to this operation across all 1 revision of this API.