Web Scraping

Scrape content from a webpage

post/v1/scrape

Request

Base URL: https://apipie.ai
URL: https://apipie.ai/v1/scrape
Auth: API key in header Authorization

Request body

urlstring required

URL to scrape content from

provider'brightdata' | 'valyu'

Scrape provider (brightdata or valyu)

format'raw' | 'parsed'

Format of the response (raw or parsed)

scrape_rendertrue | false

Force Javascript to fully render before scraping

scrape_lengthstring

Content length for extraction (valyu supports short, medium, large, max, or custom integer 1000-1000000)

Example request

{
  "url": "https://example.com/article",
  "provider": "valyu",
  "format": "parsed",
  "scrape_render": true,
  "scrape_length": "medium"
}

Response

Successful scrape with content

Example response

{
  "scrape_content": {
    "title": "Article Title",
    "textContent": "Full text content of the article...",
    "excerpt": "Brief excerpt from the article...",
    "hrefs": [
      "https://example.com/link1",
      "https://example.com/link2"
    ]
  },
  "usage": {
    "scrape_id": "nai-123e4567-e89b-12d3-a456-426614174000",
    "timestamp": 1711729795000,
    "requested_records": 1,
    "raw_length": 150000,
    "parsed_length": 25000,
    "cost": 0.015,
    "latencyMs": 2500
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.