Tools

scrape

Changed on Oct 30, 2026

Scrape a webpage and return the text content synchronously — the only endpoint here that doesn't return a run to poll. The response uses the request_status format.

Quick example

  curl -X POST https://api.riveterhq.com/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Credit costs

With proxy: 0.30 credits
Without proxy: 0.10 credits
From cache: 0.01 credits

Proxy usage

Scraping is not guaranteed to succeed without a proxy. Some websites may block requests or require specific geographic locations. To use a proxy, include proxy_country_code with a two-character country code (e.g. 'us', 'gb', 'de').

Caching

Recently scraped pages are cached to save credits (0.01 credits on a cache hit). Set skip_cache: true to always fetch fresh content.

post/scrape

Request

Base URL
https://api.riveterhq.com/v1
Auth
HTTP bearer
TypeScript
https://skmtc.dev/riveterhq/apis/riveter-api/docs/scrape/post.ts
A function with typed inputs. Its response is checked against the schema.

Request body

urlstring uri required

The URL to scrape

proxy_country_codestring

Optional two-character country code for proxy (e.g. 'us', 'gb', 'de')

skip_cacheboolean

Set to true to bypass cache and always fetch fresh content

estimate_costboolean

Validate and price the request without running or charging anything; the 200 response is the credit estimate (EstimateCostResult).

max_creditsnumber

Credit ceiling. Refused with 422 credit_cap_exceeded (nothing charged) when the estimate's maximum exceeds it.

Response

Webpage scraped successfully (or, with estimate_cost, the credit estimate only — minimum is the cache-hit price, maximum depends on proxy_country_code)

Example response

{
  "estimate_cost": true,
  "type": "enrichment",
  "credit_detail": {
    "estimate": {
      "minimum": 0,
      "maximum": 1920,
      "charged_upfront": false
    },
    "credits_used": 0,
    "credits_remaining": 5000,
    "sufficient_credits": true
  }
}

Changes

Oct 30, 2026dcb4d45ade1a Revision changes
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property ////////// to the response with the status
- info
  added the optional property ////////// to the response with the status
- info
  added the optional property ////////// to the response with the status
- info
  added the optional property ////////// to the response with the status
Oct 13, 2026029668894d33 Revision changes
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property /////// to the response with the status
- info
  added the optional property ////////// to the response with the status
- info
  added the optional property ////////// to the response with the status

Operation history (5 revisions)