--- title: "Scrape Markdown" method: GET path: "/web/scrape/markdown" tags: ["Web Scraping"] --- # Scrape Markdown `GET /web/scrape/markdown` Scrapes the given URL into LLM usable Markdown. Inspect key_metadata on JSON responses from a recognized API key; use error_code to distinguish stable failure categories. ### Billing & errors | HTTP status | Billed? | Meaning | | --- | --- | --- | | 200 | Yes — 1 credit, or 2 credits with actions | Successful scrape, including a zero-length result when includeSelectors matched nothing | | 400 | No | Invalid input, skipped PDF, or the page could not be scraped | | 401 / 403 | No | Invalid/disabled key, insufficient permissions, or credits exhausted; inspect error_code | | 404 | No | Target page returned or fingerprinted as not found | | 408 | No | Request timed out | | 415 | No | Unsupported content type | | 429 | No | Per-minute rate limit exceeded; honor Retry-After | | 500 | No | Internal error | ## Query parameters - `url` string, uri, required — Full URL to scrape into LLM usable Markdown (must include http:// or https:// protocol) - `includeLinks` union — Preserve hyperlinks in Markdown output - boolean - 'true' | 'false' - `includeImages` union — Include image references in Markdown output - boolean - 'true' | 'false' - `shortenBase64Images` union — Shorten base64-encoded image data in the Markdown output - boolean - 'true' | 'false' - `useMainContentOnly` union — Extract only the main content of the page, excluding headers, footers, sidebars, and navigation - boolean - 'true' | 'false' - `pdf` object — PDF parsing controls. Use start/end to limit text extraction and embedded-image detection/OCR to an inclusive 1-based page range. - `shouldParse` union — When true, PDF URLs are fetched and parsed. When false, PDF URLs are skipped and a 400 WEBSITE_ACCESS_ERROR is returned. - boolean - 'true' | 'false' - `start` integer — First 1-based PDF page to parse. When omitted, parsing starts at the first page. - `end` integer — Last 1-based PDF page to parse. When omitted, parsing ends at the last page. Must be greater than or equal to start when both are provided. - `ocr` union — When true, detect and OCR images embedded in the selected PDF pages, inserting recognized text at each image's position in page reading order while preserving the PDF text layer. This is separate from automatic scanned-PDF OCR fallback. - boolean - 'true' | 'false' - `includeFrames` union — When true, the contents of iframes are rendered to Markdown. - boolean - 'true' | 'false' - `includeSelectors` string[], nullable — CSS selectors. When provided, only matching HTML subtrees (and their descendants) are kept before conversion to Markdown. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]". - `excludeSelectors` string[], nullable — CSS selectors to remove before conversion to Markdown. Applied after includeSelectors. Exclusion takes precedence: an element matching both is removed. Examples: "nav", "footer", ".ad-banner", "[aria-hidden=true]". - `maxAgeMs` integer, nullable — Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh. - `waitForMs` integer, nullable — Optional browser wait time in milliseconds after initial page load before converting the page to Markdown. Min: 0. Max: 30000 (30 seconds). - `settleAnimations` union — When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages. - boolean - 'true' | 'false' - `actions` WebScrapeAction[], nullable — Optional browser actions executed in array order after the page loads and before content is captured. Requires a paid plan. Send a JSON array in the query parameter. Maximum: 5 actions. - union — Browser action discriminated by `do`. Each variant exposes only its applicable fields. - WebScrapeWaitAction — Pause for a fixed number of milliseconds before continuing to the next action. - `do` 'wait', required - `timeMs` integer, required - WebScrapePerformAction — Resolve and perform one natural-language browser action. - `do` 'perform', required - `action` string, required - `headers` object — Optional outbound HTTP headers forwarded only to the target URL, sent as deep-object query params such as headers[X-Custom]=value. When provided, caching is bypassed: the result is neither read from nor written to cache. - `country` 'ad' | 'ae' | 'af' | 'ag' | 'ai' | 'al' | 'am' | 'ao' | 'ar' | 'at' | 'au' | 'aw' | 'az' | 'ba' | 'bb' | 'bd' | 'be' | 'bf' | 'bg' | 'bh' | 'bi' | 'bj' | 'bm' | 'bn' | 'bo' | 'bq' | 'br' | 'bs' | 'bw' | 'by' | 'bz' | 'ca' | 'cd' | 'cf' | 'cg' | 'ch' | 'ci' | 'cl' | 'cm' | 'cn' | 'co' | 'cr' | 'cv' | 'cw' | 'cy' | 'cz' | 'de' | 'dj' | 'dk' | 'dm' | 'do' | 'dz' | 'ec' | 'ee' | 'eg' | 'es' | 'et' | 'fi' | 'fj' | 'fr' | 'ga' | 'gb' | 'gd' | 'ge' | 'gf' | 'gg' | 'gh' | 'gm' | 'gn' | 'gp' | 'gq' | 'gr' | 'gt' | 'gu' | 'gw' | 'gy' | 'hk' | 'hn' | 'hr' | 'ht' | 'hu' | 'id' | 'ie' | 'il' | 'im' | 'in' | 'iq' | 'ir' | 'is' | 'it' | 'je' | 'jm' | 'jo' | 'jp' | 'ke' | 'kg' | 'kh' | 'kn' | 'kr' | 'kw' | 'ky' | 'kz' | 'la' | 'lb' | 'lc' | 'lk' | 'lr' | 'ls' | 'lt' | 'lu' | 'lv' | 'ly' | 'ma' | 'mc' | 'md' | 'me' | 'mf' | 'mg' | 'mk' | 'ml' | 'mm' | 'mn' | 'mo' | 'mq' | 'mr' | 'mt' | 'mu' | 'mv' | 'mw' | 'mx' | 'my' | 'mz' | 'na' | 'nc' | 'ne' | 'ng' | 'ni' | 'nl' | 'no' | 'np' | 'nz' | 'om' | 'pa' | 'pe' | 'pf' | 'pg' | 'ph' | 'pk' | 'pl' | 'pr' | 'ps' | 'pt' | 'py' | 'qa' | 're' | 'ro' | 'rs' | 'ru' | 'rw' | 'sa' | 'sc' | 'sd' | 'se' | 'sg' | 'si' | 'sk' | 'sl' | 'sm' | 'sn' | 'so' | 'sr' | 'ss' | 'st' | 'sv' | 'sx' | 'sy' | 'sz' | 'tc' | 'td' | 'tg' | 'th' | 'tj' | 'tl' | 'tm' | 'tn' | 'tr' | 'tt' | 'tw' | 'tz' | 'ua' | 'ug' | 'us' | 'uy' | 'uz' | 'vc' | 've' | 'vg' | 'vi' | 'vn' | 'ye' | 'yt' | 'za' | 'zm' | 'zw' — Two-letter ISO 3166-1 alpha-2 country code identifying a supported Context.dev residential proxy exit location. Must be one of Context.dev's supported countries. When provided, Context.dev fetches the target page from that country. - `timeoutMS` integer — Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes). - `zdr` 'enabled' | 'disabled' — Set to enabled to bypass shared caches and omit request and response content from retained usage logs. Requires zero data retention to be enabled for your organization (contact support@context.dev), otherwise the request fails with ZDR_NOT_ENABLED. Successful ZDR responses include X-Context-ZDR: true. - `tags` string[] — Optional tags for tracking usage. Up to 20 tags, each 1 to 50 characters. ## Response `200` Successful response - object - `success` true, required — Indicates success - `markdown` string, required — Page content converted to GitHub Flavored Markdown - `contentLength` integer, required — UTF-8 byte length of the returned Markdown. Use 0 to identify an empty result and compare small values against your workload's minimum useful-content threshold. - `url` string, required — The URL that was scraped - `metadata` PageMetadata, required — Metadata extracted from the scraped page HTML. - `sourceUrl` string, required — Original URL requested by the caller. - `finalUrl` string, required — Final URL scraped after redirects or scraper fallback, when known. Falls back to sourceUrl when unavailable. - `title` string — Best title extracted from the page. - `description` string — Best description extracted from standard, Open Graph, or Twitter metadata. - `language` string — Language extracted from html lang or language meta tags. - `keywords` string[] — Keywords extracted from the page's keywords meta tag. - `canonicalUrl` string — Resolved canonical URL, when present. - `author` string — Author metadata, when present. - `siteName` string — Site or application name from page metadata. - `image` string — Primary resolved preview image from Open Graph, Twitter, or image metadata. - `favicon` string — Resolved favicon URL, when present. - `publishedTime` string — Published timestamp/date from page metadata, when present. - `modifiedTime` string — Modified timestamp/date from page metadata, when present. - `robots` string — Robots meta directive, when present. - `openGraph` object — Open Graph metadata with the og: prefix removed and keys camel-cased. - `twitter` object — Twitter card metadata with the twitter: prefix removed and keys camel-cased. - `alternates` PageMetadataAlternate[] — Resolved alternate links from link rel=alternate tags. - `href` string, required — Resolved alternate URL. - `hreflang` string — Language or locale for the alternate URL, when present. - `type` string — Alternate resource MIME type, when present. - `title` string — Alternate resource title, when present. - `jsonLd` object[] — JSON-LD structured data blocks parsed from the page. - `additionalMeta` object — Additional non-social meta tags not promoted to top-level metadata fields. - `key_metadata` KeyMetadata — Metadata about the API key used for the request. Included in every response whenever a valid API key is provided, even when the response status is not 200. - `credits_consumed` integer, required — The number of credits consumed by this request. - `credits_remaining` integer, required — The number of credits remaining for your organization after this request. - `actionsApplied` object[] — One verified outcome per requested browser action, in request order. - `instruction` string, required - `status` 'applied' | 'failed' | 'skipped', required — Applied means the requested page state was visibly verified. Failed means it was not verified. Skipped means it was not attempted. - `method` string - `targetDescription` string - `completionEvidence` string — Visible page evidence used to verify an applied action. - `error` string - `durationMs` number - `actionsHtmlStale` boolean — True when an action was applied but the returned content could not be refreshed afterward. ## Other responses - `400` — Bad request - Invalid URL or failed to scrape - `401` — Unauthorized - Invalid or missing API key - `403` — Forbidden - Insufficient permissions or usage limit exceeded - `404` — Target page returned a 404 - `408` — Request timeout - `415` — Unsupported content type - the URL resolved to a content type that is not supported (e.g. an image, presentation, media, or archive). Supported types are HTML, XML, PDF, DOCX, DOC, XLSX, XLS, PPTX, PPT, and CSV. - `429` — Rate limit exceeded - `500` — Internal server error ## Changes - **2026-07-31** `f5c503b03f98` — 2 info - added the optional property `actionsApplied` to the response with the `200` status - added the optional property `actionsHtmlStale` to the response with the `200` status - **2026-07-26** `35f1d8d4ea26` — 1 info - added the new optional `query` request parameter `actions` - **2026-07-21** `3764d9c86493` — 2 breaking, 1 warning, 17 info - for the `query` request parameter `pdf`, default value was changed from `{"shouldParse":true}` to `{"shouldParse":true,"ocr":false}` - for the `query` request parameter `url`, the minLength was increased from `0` to `1` - for the `query` request parameter `pdf`, the type/format of property `shouldParse` was changed from `boolean`/`` to ``/`` - added the new optional `query` request parameter `tags` - …16 more - **2026-07-09** `de91b92d5fb7` — 2 info - added the new optional `query` request parameter `country` - added the new optional `query` request parameter `settleAnimations` - **2026-06-22** `4767df4f281f` — 3 info - added the non-success response with the status `404` - added the non-success response with the status `415` - added the required property `metadata` to the response with the `200` status [Change history](https://skmtc.dev/context/apis/context-dev/changes/web/scrape/markdown/get.md) --- [API](https://skmtc.dev/context/apis/context-dev.md) · [All operations](https://skmtc.dev/context/apis/context-dev/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/context/context-dev/revisions/f5c503b03f98/schema)