Contents

Extract clean, structured content from URLs

Turn any web page into clean, structured data. Extract content from up to 10 URLs per request with optional AI-powered summaries, structured extraction via JSON schemas, and screenshot capture.

  • Synchronous (default): Returns extracted content directly
  • Async: Set async: true to process in the background and poll a job_id
  • Pay-per-success: Only charged for URLs that are successfully processed ($0.001/URL, +$0.001 with AI features). Failed URLs are free
  • Partial / total failure: A 206 is returned when some URLs fail; a 422 is returned when every URL fails
post/v1/contents

Request

  • Base URL: https://api.valyu.ai
  • URL: https://api.valyu.ai/v1/contents
  • Auth: API key in header X-API-Key

Request body

urlsstring[] required

URLs to extract content from. Must be valid HTTP/HTTPS URLs. Duplicates are silently removed.

max_price_dollarsnumber

Maximum budget in USD for this request. If estimated cost exceeds this, the request is rejected. Defaults to 2x the maximum possible cost.

extract_effort'auto' | 'normal' | 'high'

Controls how pages are rendered for extraction.

  • "auto" - Automatically selects the best method per URL (default)
  • "normal" - HTTP-only extraction, fastest
  • "high" - Full Chrome rendering, best for JavaScript-heavy pages
screenshotboolean

Capture a screenshot of each page. Screenshot URL returned in screenshot_url field of each result.

asyncboolean

Process URLs asynchronously in the background. Returns a job_id to poll via GET /v1/contents/jobs/{job_id}. Useful for slow, JavaScript-heavy pages.

webhook_urlstring uri

HTTPS URL to receive a webhook POST when async processing completes. Only used when async is true. The webhook payload is signed with HMAC-SHA256.

Example request

{
  "urls": [
    "https://en.wikipedia.org/wiki/Quantum_computing",
    "https://arxiv.org/abs/2401.00001"
  ],
  "max_price_dollars": 0.05,
  "webhook_url": "https://your-app.com/webhooks/valyu"
}

Response

All URLs processed successfully (synchronous).

successboolean required

Whether the extraction completed successfully.

errorstring nullable

Error message if some URLs failed. null on complete success.

tx_idstring required

Transaction ID for tracking.

urls_requestedinteger required

Number of deduplicated URLs submitted.

urls_processedinteger required

Number of URLs successfully processed.

urls_failedinteger required

Number of URLs that failed processing.

total_cost_dollarsnumber required

Total cost in USD (only successful URLs are charged).

total_charactersinteger required

Sum of all result content lengths.

Example response

{
  "success": true,
  "tx_id": "tx_a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "urls_requested": 3,
  "urls_processed": 3,
  "results": [
    {
      "url": "https://en.wikipedia.org/wiki/Quantum_computing",
      "status": "success",
      "title": "Quantum computing - Wikipedia",
      "description": "Quantum computing is a type of computation that harnesses quantum mechanical phenomena.",
      "source": "web",
      "price": 0.001,
      "length": 24500,
      "data_type": "unstructured",
      "source_type": "website",
      "publication_date": "2024-03-15",
      "id": "https://en.wikipedia.org/wiki/Quantum_computing",
      "image_url": {
        "0": "https://upload.wikimedia.org/wikipedia/commons/quantum.png"
      },
      "error": "Could not extract content from this URL. The page may be unavailable, require sign-in, or block automated access.",
      "doi": "10.26434/chemrxiv-2025-chn4j",
      "citation_count": 5
    }
  ],
  "total_cost_dollars": 0.003,
  "total_characters": 45230
}

Changes

No recorded changes to this operation across all 1 revision of this API.