Sign in
Web Scraping

Scrape Markdown

Scrapes the given URL into LLM usable Markdown.

get/web/scrape/markdown

Query parameters

urlstring uri required

Full URL to scrape into LLM usable Markdown (must include http:// or https:// protocol)

includeLinksboolean

Preserve hyperlinks in Markdown output

includeImagesboolean

Include image references in Markdown output

shortenBase64Imagesboolean

Shorten base64-encoded image data in the Markdown output

useMainContentOnlyboolean

Extract only the main content of the page, excluding headers, footers, sidebars, and navigation

shouldParseboolean

When true, PDF URLs are fetched and parsed. When false, PDF URLs are skipped and a 400 WEBSITE_ACCESS_ERROR is returned.

startinteger

First 1-based PDF page to parse. When omitted, parsing starts at the first page.

endinteger

Last 1-based PDF page to parse. When omitted, parsing ends at the last page. Must be greater than or equal to start when both are provided.

PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.

includeFramesboolean

When true, the contents of iframes are rendered to Markdown.

includeSelectorsstring[]

CSS selectors. When provided, only matching HTML subtrees (and their descendants) are kept before conversion to Markdown. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]".

excludeSelectorsstring[]

CSS selectors to remove before conversion to Markdown. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Examples: "nav", "footer", ".ad-banner", "[aria-hidden=true]".

maxAgeMsinteger

Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.

waitForMsinteger

Optional browser wait time in milliseconds after initial page load before converting the page to Markdown. Min: 0. Max: 30000 (30 seconds).

settleAnimationsboolean

When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.

headersHeaderMap

Optional outbound HTTP headers forwarded only to the target URL, sent as deep-object query params such as headers[X-Custom]=value. When provided, caching is bypassed: the result is neither read from nor written to cache.

country'ad' | 'ae' | 'af' | 'ag' | 'ai' | 'al' | 'am' | 'ao' | 'ar' | 'at' | 'au' | 'aw' | 'az' | 'ba' | 'bb' | 'bd' | 'be' | 'bf' | 'bg' | 'bh' | 'bi' | 'bj' | 'bm' | 'bn' | 'bo' | 'bq' | 'br' | 'bs' | 'bw' | 'by' | 'bz' | 'ca' | 'cd' | 'cf' | 'cg' | 'ch' | 'ci' | 'cl' | 'cm' | 'cn' | 'co' | 'cr' | 'cv' | 'cw' | 'cy' | 'cz' | 'de' | 'dj' | 'dk' | 'dm' | 'do' | 'dz' | 'ec' | 'ee' | 'eg' | 'es' | 'et' | 'fi' | 'fj' | 'fr' | 'ga' | 'gb' | 'gd' | 'ge' | 'gf' | 'gg' | 'gh' | 'gm' | 'gn' | 'gp' | 'gq' | 'gr' | 'gt' | 'gu' | 'gw' | 'gy' | 'hk' | 'hn' | 'hr' | 'ht' | 'hu' | 'id' | 'ie' | 'il' | 'im' | 'in' | 'iq' | 'ir' | 'is' | 'it' | 'je' | 'jm' | 'jo' | 'jp' | 'ke' | 'kg' | 'kh' | 'kn' | 'kr' | 'kw' | 'ky' | 'kz' | 'la' | 'lb' | 'lc' | 'lk' | 'lr' | 'ls' | 'lt' | 'lu' | 'lv' | 'ly' | 'ma' | 'mc' | 'md' | 'me' | 'mf' | 'mg' | 'mk' | 'ml' | 'mm' | 'mn' | 'mo' | 'mq' | 'mr' | 'mt' | 'mu' | 'mv' | 'mw' | 'mx' | 'my' | 'mz' | 'na' | 'nc' | 'ne' | 'ng' | 'ni' | 'nl' | 'no' | 'np' | 'nz' | 'om' | 'pa' | 'pe' | 'pf' | 'pg' | 'ph' | 'pk' | 'pl' | 'pr' | 'ps' | 'pt' | 'py' | 'qa' | 're' | 'ro' | 'rs' | 'ru' | 'rw' | 'sa' | 'sc' | 'sd' | 'se' | 'sg' | 'si' | 'sk' | 'sl' | 'sm' | 'sn' | 'so' | 'sr' | 'ss' | 'st' | 'sv' | 'sx' | 'sy' | 'sz' | 'tc' | 'td' | 'tg' | 'th' | 'tj' | 'tl' | 'tm' | 'tn' | 'tr' | 'tt' | 'tw' | 'tz' | 'ua' | 'ug' | 'us' | 'uy' | 'uz' | 'vc' | 've' | 'vg' | 'vi' | 'vn' | 'ye' | 'yt' | 'za' | 'zm' | 'zw'

Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country.

Example:de

Two-letter ISO 3166-1 alpha-2 country code for the website request location. When provided, Context.dev fetches the target page from that country.

timeoutMSinteger

Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).

Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).

Response

Successful response

successtrue required

Indicates success

markdownstring required

Page content converted to GitHub Flavored Markdown

urlstring required

The URL that was scraped

Changes