---
title: "Extract Structured Website Data"
method: POST
path: "/web/extract"
tags: ["Web Extraction"]
---

# Extract Structured Website Data

`POST /web/extract`

Crawl a website, use the provided JSON Schema and instructions to prioritize relevant internal links, and extract structured data from the selected pages.

## Request body

- object
  - `url` string, uri, required — The starting website URL to crawl and extract from. Must include http:// or https://.
  - `schema` object, required — JSON Schema for the returned data object. Image fields such as `image_urls` or `product_photos` automatically make page image references available to extraction, so product data and photos can be returned in one call. TypeScript Zod users can pass a JSON Schema generated from a Zod object; Python users can pass the equivalent JSON Schema object.
  - `instructions` string — Optional extraction guidance, such as which facts to prioritize or how to interpret fields in the schema.
  - `factCheck` boolean — When true, every returned value must be grounded in facts stated on the page; fields that cannot be supported by the page are returned as null/empty. When false (default), the model may make reasonable inferences and derivations from the page content (e.g. ideal customer, competitor analysis, recommendations) while keeping verifiable specifics (names, quotes, URLs, dates, metrics) faithful to the source.
  - `followSubdomains` boolean — When true, follow links on subdomains of the starting URL's domain.
  - `maxPages` integer — Maximum number of pages to analyze for extraction. Hard cap: 50. Defaults to 5.
  - `maxDepth` integer — Optional maximum link depth from the starting URL (0 = only the starting page). If omitted, there is no crawl depth limit.
  - `pdf` object
    - `shouldParse` boolean — When true, PDF pages are fetched and parsed. When false, PDF pages are skipped.
    - `start` integer — First 1-based PDF page to parse.
    - `end` integer — Last 1-based PDF page to parse. Must be greater than or equal to start when both are provided.
  - `includeFrames` boolean — When true, iframe contents are included in Markdown before extraction.
  - `maxAgeMs` integer — Return cached scrape results if a prior scrape for the same parameters is younger than this many milliseconds. Defaults to 7 days (604800000 ms).
  - `waitForMs` integer — Optional browser wait time in milliseconds after initial page load for each crawled page.
  - `settleAnimations` boolean — When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
  - `actions` WebScrapeAction[] — Optional browser actions executed in order on the requested page after it loads, before links are discovered or additional pages are crawled. Requires a paid plan. When actions are provided and stopAfterMs is omitted, the crawl budget defaults to 110000 ms.
    - union — Browser action discriminated by `do`. Each variant exposes only its applicable fields.
      - WebScrapeWaitAction — Pause for a fixed number of milliseconds before continuing to the next action.
        - `do` 'wait', required
        - `timeMs` integer, required
      - WebScrapePerformAction — Resolve and perform one natural-language browser action.
        - `do` 'perform', required
        - `action` string, required
      - WebScrapeScrollAction — Scroll the page or a selected scrollable container, waiting adaptively for content and dimensions to settle after each iteration.
        - `do` 'scroll', required
        - `direction` 'up' | 'down' | 'left' | 'right' — Direction to scroll. Defaults to down.
        - `amount` union — Pixels per scroll, one visible viewport, or the current scroll boundary. Defaults to viewport.
          - integer
          - 'viewport' | 'max'
        - `container` string — CSS selector for the first matching scroll container. Defaults to the page.
        - `maxScrolls` integer — Maximum scroll iterations. Stops early when scrolling and scrollable extent stop changing. Defaults to 1.
  - `stopAfterMs` integer — Soft time budget for the crawl in milliseconds. Min: 10000 (10s). Max: 110000 (110s). Defaults to 80000 (80s), or 110000 (110s) when browser actions are provided.
  - `timeoutMS` integer — Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
  - `tags` string[] — Optional tags for tracking usage. Up to 20 tags, each 1 to 50 characters.

## Response `200`

Successful response

- object
  - `status` string, required — Status of the response, e.g., 'ok'
  - `url` string, required — The starting URL that was analyzed
  - `urls_analyzed` string[], required — List of URLs whose Markdown was used for extraction
  - `data` object, required — Extracted data matching the request schema
  - `metadata` object, required
    - `numUrls` integer, required
    - `maxCrawlDepth` integer, required
    - `numSucceeded` integer, required
    - `numFailed` integer, required
    - `numSkipped` integer, required
    - `numBlocked` integer, required — Number of crawled pages excluded because they were anti-bot challenges, error pages, or parked-domain placeholders.
    - `actionsApplied` object[] — One verified outcome per requested browser action, in request order.
      - `instruction` string, required
      - `status` 'applied' | 'failed' | 'skipped', required — Applied means the requested page state was visibly verified. Failed means it was not verified. Skipped means it was not attempted.
      - `method` string
      - `targetDescription` string
      - `completionEvidence` string — Visible page evidence used to verify an applied action.
      - `error` string
      - `durationMs` number
  - `key_metadata` KeyMetadata — Metadata about the API key used for the request. Included in every response whenever a valid API key is provided, even when the response status is not 200.
    - `credits_consumed` integer, required — The number of credits consumed by this request.
    - `credits_remaining` integer, required — The number of credits remaining for your organization after this request.
  - `cache_metadata` CacheMetadata, required — Cache outcome for this response. Composite responses are hits only when every cache-controlled fetch contributing to the output was a hit; age_ms is the oldest contributing hit.
    - `status` 'hit' | 'miss' | 'zdr', required — Whether the response was served from cache, required fresh work, or honored zero-data-retention cache bypass.
    - `age_ms` integer, required — Age of the cached data in milliseconds. Zero for miss and zdr responses.

## Other responses

- `400` — Bad request - Invalid URL, schema, or inaccessible website
- `401` — Unauthorized - Invalid or missing API key
- `403` — Forbidden - Insufficient permissions or usage limit exceeded
- `408` — Request timeout
- `429` — Rate limit exceeded
- `500` — Internal server error

## Changes

> 26 revisions in range; 19 could not be searched.

- **2026-07-31** `f5c503b03f98` — 16 warning
  - added the new `BATCH_LIMIT_EXCEEDED` enum value to the `error_code` response property for the response status `401`
  - added the new `BATCH_LIMIT_EXCEEDED` enum value to the `error_code` response property for the response status `403`
  - added the new `BATCH_LIMIT_EXCEEDED` enum value to the `error_code` response property for the response status `408`
  - added the new `BATCH_LIMIT_EXCEEDED` enum value to the `error_code` response property for the response status `500`
  - …12 more
- **2026-07-26** `35f1d8d4ea26` — 8 warning
  - added the new `MONITOR_NO_WEBHOOK` enum value to the `error_code` response property for the response status `401`
  - added the new `MONITOR_NO_WEBHOOK` enum value to the `error_code` response property for the response status `403`
  - added the new `MONITOR_NO_WEBHOOK` enum value to the `error_code` response property for the response status `408`
  - added the new `MONITOR_NO_WEBHOOK` enum value to the `error_code` response property for the response status `500`
  - …4 more
- …earlier changes not shown

[Full history](https://skmtc.dev/context/apis/context-dev/changes/web/extract/post.md)

---

[API](https://skmtc.dev/context/apis/context-dev.md) · [All operations](https://skmtc.dev/context/apis/context-dev/llms.txt) · [OpenAPI document](https://skmtc-service-production.skmtc.workers.dev/v1/apis/context/context-dev/revisions/1febe5476266/schema)