scrape

Scrape

post/v1/scrape

Request

  • Base URL: https://api.snoopscan.com
  • URL: https://api.snoopscan.com/v1/scrape
  • Auth: none declared

Headers

authorizationstring nullable

Request body

onlyMainContentboolean

Return the article and drop navigation, headers, footers and cookie banners. Turn it off to keep the whole page.

includeTagsstring[]

CSS selectors to keep even when they sit outside the main content.

excludeTagsstring[]

CSS selectors to drop before the page is read.

maxAgeinteger

Serve a cached copy if one was taken within this many milliseconds. A cache hit costs nothing. Set 0 to force a fresh fetch.

storeInCacheboolean

Keep this response in the cache so a later request can be served free.

waitForinteger

Milliseconds to wait after the page loads before reading it, for content that loads late. Renders the page in a browser, and is billed as a browser page.

timeoutinteger

Give up on this request after this many milliseconds.

captchaHandling'auto' | 'off'

Click a challenge checkbox once if one appears. Set off to never click.

captchaEvidenceboolean

Return screenshots from before and after an automatic checkbox click.

headersobject

Extra headers to send. User-Agent and other browser-identity headers are set for you and cannot be changed.

mobileboolean

Read the page as a phone would see it.

proxy'none' | 'datacenter' | 'residential' | 'mobile' | 'auto'
blockAssetsboolean

Skip images, fonts and media when the page is rendered in a browser. Faster.

removeBase64Imagesboolean

Strip inline base64 images out of the returned markdown and HTML.

parsersstring[]

Which document parsers may run, for example pdf.

urlstring required

The absolute http or https URL to read.

Response

Successful Response

object required

Changes

No recorded changes to this operation across all 1 revision of this API.