Extract

Extract structured data from URLs

Pull schema-shaped JSON out of one or many pages in a single call. An LLM maps each page onto your schema and/or prompt. Point at a single page, a list, or a crawl scope with a trailing /* wildcard; optionally let web search find extra source pages, return per-field confidence with the source passage behind each value, and merge everything into one deduplicated collection of entities. Charged per page that returns data.

post/v1/web/extract

Request

  • Base URL: https://api.hydrafetch.com
  • URL: https://api.hydrafetch.com/v1/web/extract
  • Auth: API key in header X-API-Key

Request body

urlsstring[] required

The pages to extract from. Each must be an http(s) URL. A trailing /* marks a crawl scope: every page discovered under that path is extracted and merged into the result.

schemaobject

JSON Schema describing the shape you want back. Optional if prompt is given; when both are present the schema fixes the field names and types while the prompt guides what to pull.

promptstring

Natural-language instruction for what to extract. Use with or instead of a schema.

preferStructureboolean

Preserve document structure (headings, lists, tables) over prose density when reading the page — good for listing and catalog pages. Default off.

enableWebSearchboolean

Pull in extra source pages by web-searching your prompt, to fill fields your URLs do not cover. Requires a prompt.

showSourcesboolean

Return the concrete list of URLs that were actually extracted, after any wildcard and web-search expansion. Default off.

showConfidenceboolean

For each field, return a confidence score and the exact source passage the value was drawn from. Default off.

mergeEntitiesboolean

Merge the per-page results into one deduplicated collection — one row per entity, with its contributing source URLs — instead of a separate result per page. Default off.

maxAgenumber

Reuse a recent capture of each page if it is younger than this many milliseconds. Omit or 0 to always fetch fresh. Capped at 7 days.

headersobject

Extra request headers to send when fetching each page, such as a cookie that picks the store or delivery area a site prices by.

Example request

{
  "urls": [
    "https://example.com/products/widget",
    "https://example.com/products/*"
  ],
  "prompt": "Pull the product name, price in USD, and whether it is in stock.",
  "location": {
    "country": "us",
    "languages": [
      "en-US",
      "en"
    ]
  },
  "headers": {
    "Cookie": "store_id=1234"
  }
}

Response

The extracted data.

Example response

{
  "data": {
    "results": [
      {
        "url": "https://example.com/products/widget"
      }
    ]
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.