---
title: "Scrape to HTML"
method: POST
path: "/v1/web/html"
tags: ["Scrape"]
---

# Scrape to HTML

`POST /v1/web/html`

## Request

- Base URL: `https://api.hydrafetch.com`
- URL: `https://api.hydrafetch.com/v1/web/html`
- Auth: API key in header `X-API-Key`

Fetch a URL and return its cleaned main-content HTML.

## Request body

- ScrapeFormatRequestDto
  - `url` string, required: The URL to scrape. Must be http(s); a bare domain such as example.com is read over https.
  - `maxAge` number: Serve from cache if a capture of this URL is younger than this many milliseconds. Omit for the default 24h window; 0 always fetches fresh. Capped at 7 days.
  - `cacheOnly` boolean: Only serve from cache. If there is no fresh cached copy, return 404 instead of fetching.
  - `storeInCache` boolean: Persist the capture for later re-extraction. Default true.
  - `preferStructure` boolean: Preserve document structure (headings, lists, tables) over prose density — good for marketing and service pages. Default off.
  - `async` boolean: Return a job id immediately instead of waiting for the result. Poll GET /v1/web/scrape/{id}. Opt in only: a synchronous call never degrades to a job id on its own, so you get a page or an error, not two response shapes.
  - `onlyMainContent` boolean: Return only the main content, dropping nav/boilerplate. Default true.
  - `includeTags` string[]: CSS selectors to keep. When set, only matching elements are considered.
  - `excludeTags` string[]: CSS selectors to strip before extraction.
  - `removeBase64Images` boolean: Strip inline base64 images from the output. Default true.
  - `blockAds` boolean: Remove common ad and tracking elements. Default true.
  - `includeLinks` boolean: Keep inline links in the markdown. By default links are kept wherever the page text carries them. Set true to keep them on every page, or false for the densest prose.
  - `waitFor` number: Extra milliseconds to let the page settle before capture.
  - `timeout` number: Overall time budget for the request, in milliseconds.
  - `location` LocationDto
    - `country` string: ISO 3166 alpha-2 country to fetch the page as if from.
    - `languages` string[]: Preferred content languages, most-preferred first.
  - `headers` object: Extra request headers to send when fetching the page.
  - `method` 'GET' | 'POST': HTTP method to fetch the URL with. Use POST for endpoints that only answer a POST, such as JSON and GraphQL APIs. Defaults to GET.
  - `body` string: Request body, sent verbatim. Only used when `method` is POST. Set the content type yourself via `headers`; nothing is assumed about the format.

## Response `200`

- ScrapeResponseDto
  - `data` WebScrapeDataDto, required
    - `url` string, required: The URL you requested.
    - `finalUrl` string, required: The final URL after any redirects.
    - `redirected` boolean, required: Whether the origin redirected: true when `finalUrl` differs from the URL you requested. Explicit so a page with no redirect is distinguishable from one whose redirect was not tracked.
    - `status` number, required: HTTP status of the fetched page.
    - `cached` boolean, required: Whether this result was served from cache.
    - `warning` string: Set when the page was returned with a caveat (e.g. partial content).
    - `metadata` WebScrapeMetadataDto, required
      - `title` object, nullable, required: The page title.
      - `structure` 'markdown' | 'plain', required: Whether the content carries markdown structure (headings, lists, emphasis) or is plain prose. `plain` is not an error — the default extractor optimises for capturing every word and wins on listing and forum pages. If you need the structure, re-request with `preferStructure: true`.
      - `pageType` string, required: Coarse page classification (e.g. article, listing, forum, docs).
      - `wordCount` number, required: Word count of the extracted main content.
      - `description` object, nullable, required: The page's own description (meta description / og:description).
      - `language` object, nullable, required: The language the page declares.
      - `author` object, nullable, required: The declared author.
      - `siteName` object, nullable, required: The declared site name.
      - `publishedTime` object, nullable, required: When the page says it was published (ISO 8601).
      - `modifiedTime` object, nullable, required: When the page says it was last updated (ISO 8601).
      - `image` object, nullable, required: The page's lead image (og:image).
    - `usage` WebUsageDto
      - `creditsUsed` number, required: Credits this call consumed. Charged only on success.
      - `creditsRemaining` number, required: Credits left in your workspace's balance after this call.
      - `freshness` 'cache' | 'fresh', required: Whether the result was served from cache or freshly fetched.
    - `quality` WebQualityDto
      - `confidence` number, required: How trustworthy the extraction is, from 0 to 1. High when independent checks corroborate substantial content; near zero when the page yielded almost nothing.
      - `complete` boolean, required: Whether the result captured the bulk of the content available on the page. False when the result looks thin or truncated.
      - `blocked` boolean, required: Whether the page appeared to be behind a challenge or bot wall.
    - `markdown` string: Clean Markdown of the main content. Returned when `markdown` is requested.
    - `html` string: Cleaned main-content HTML. Returned when `html` is requested.
    - `rawHtml` string: The unmodified page HTML. Returned when `rawHtml` is requested.
    - `links` WebLinksDto
      - `internal` string[], required: Links pointing to the same site.
      - `external` string[], required: Links pointing to other sites.
    - `structured` WebStructuredDataDto
      - `entities` WebStructuredEntityDto[], required: The page's own structured data, normalised into one deduplicated list of typed entities.
        - `type` string, required: The schema.org type of the entity.
        - `source` 'json-ld' | 'microdata' | 'rdfa' | 'opengraph', required: Which structured-data syntax the entity came from.
        - `properties` object, required: The entity's properties, as published.
      - `jsonLd` object[], required: Raw JSON-LD blocks, as found on the page.
      - `microdata` object[], required: Raw microdata items.
      - `opengraph` object[], required: Raw OpenGraph/Twitter-card tags.
      - `rdfa` object[], required: Raw RDFa items.
      - `appState` string[], required: Names of embedded framework app-state blocks detected on the page.
    - `summary` string: A concise factual summary. Returned when `summary` is requested (LLM-backed).
    - `json` object: Schema-shaped JSON. Returned when `json` is requested (LLM-backed).

## Other responses

- `201`
- `400`: The request body or query failed validation.
- `401`: The X-API-Key header is missing, malformed, or names a key that no longer exists.
- `402`: The workspace has no credits left for this request. Nothing was charged and nothing was fetched.
- `403`: The key is valid but the workspace may not make this request.
- `404`: No such job, or the id belongs to another workspace.
- `422`: The target site did not respond, or we could not read that page. This is a fact about that URL rather than a fault on our side, and retrying it will not change the answer. When details.retryAfterSeconds is present, asking again before then returns this same answer. Nothing was charged.
- `429`: The workspace exceeded its per-minute request limit. Read the RateLimit headers, or Retry-After on this response, and retry after the window resets.
- `500`: Unexpected server error. Nothing was charged.
- `503`: A dependency needed for the requested formats is unavailable. Safe to retry.

---

[API](https://skmtc.dev/hydrafetch/apis/hydrafetch-api.md) · [All operations](https://skmtc.dev/hydrafetch/apis/hydrafetch-api/llms.txt) · [OpenAPI document](https://skmtc.dev/hydrafetch/apis/hydrafetch-api/revisions/b28322984b4b?raw)