---
title: "Scrape a web page"
method: POST
path: "/v1/data/web/scrape"
tags: ["Web: Scrape"]
---

# Scrape a web page

`POST /v1/data/web/scrape`

## Request

- Base URL: `https://api.superagnt.com`
- URL: `https://api.superagnt.com/v1/data/web/scrape`
- Auth: API key in header `Authorization`

Fetches a single URL and returns its content. Default is clean main-content `markdown` — cheapest and right for almost everything; just read the markdown. Reach for the pricier options only when the task truly needs them: the `json` format with a `schema` runs an LLM to extract structured fields (costs several credits per page — prefer scraping markdown and parsing it in code), and `proxy: enhanced` forces heavy anti-bot handling. Scrape only the specific pages you will actually use. Handles JavaScript-rendered pages.

## Request body

- object
  - `url` string, uri, required: The URL to scrape (must be a publicly reachable http(s) URL).
  - `formats` union[]: Output formats to return. Each entry is either a string preset or an object form. String presets: 'markdown' (default), 'summary', 'html', 'rawHtml', 'links', 'images'. Object forms include { type: 'json', schema, prompt } for structured extraction, { type: 'screenshot', fullPage, quality } for an image, and { type: 'changeTracking', modes } for diffs. Defaults to ['markdown'] when omitted.
    - union
      - 'markdown' | 'summary' | 'html' | 'rawHtml' | 'links' | 'images'
      - object
        - `type` 'json' | 'screenshot' | 'changeTracking', required
        - `schema` object: JSON schema describing the structured data to extract (for type 'json').
        - `prompt` string: Natural-language extraction prompt (for type 'json').
        - `fullPage` boolean: Capture the full scrollable page (for type 'screenshot').
        - `modes` string[]: Change-tracking modes (for type 'changeTracking').
  - `onlyMainContent` boolean: Return only the main content of the page, excluding navs, headers, and footers.
  - `includeTags` string[]: Only include content within these HTML tags / CSS selectors.
  - `excludeTags` string[]: Exclude content within these HTML tags / CSS selectors.
  - `maxAge` integer: Return cached content up to this age in milliseconds (faster, cheaper). Default 172800000 (2 days).
  - `waitFor` integer: Milliseconds to wait for the page to render before scraping.
  - `mobile` boolean: Emulate a mobile device.
  - `timeout` integer: Request timeout in milliseconds (1000-300000). Default 60000.
  - `parsers` unknown[]: PDF parsing configuration.
    - unknown
  - `proxy` 'basic' | 'enhanced' | 'auto': Proxy / anti-bot mode. 'auto' (default) escalates only on failure; 'enhanced' forces advanced anti-bot handling and costs more.
  - `location` object: Geographic proxy location and language settings.
    - `country` string: ISO 3166-1 alpha-2 country code.
    - `languages` string[]

## Response `200`

The scraped page content in the requested formats.

- object

## Other responses

- `400`: Bad request — missing `url` or invalid parameter.
- `401`: Unauthorized.
- `402`: Insufficient balance.
- `429`: Too many requests.
- `5XX`: Upstream error.

---

[API](https://skmtc.dev/superagnt/apis/superagnt-api.md) · [All operations](https://skmtc.dev/superagnt/apis/superagnt-api/llms.txt) · [OpenAPI document](https://skmtc.dev/superagnt/apis/superagnt-api/revisions/b2d8de30ccf0?raw)