Web Tasks

Follow a site's links and read it

Starts a crawl from url. Same fields as /scrape plus limit (pages, default 25, max 200), maxDepth (default 2) and the site filters. Returns a jobId. Returns a jobId for asynchronous work. A successful start is acceptance, not completion; poll the job status. Account credits can be refunded for unused work; x402 has no automatic on-chain refund.

post/tasks/v1/crawl

Request

  • Base URL: https://api.oassis.dev
  • URL: https://api.oassis.dev/tasks/v1/crawl
  • Auth: one of:
    • HTTP bearer
    • none

Request body

formatsstring[]

Outputs you want in the same response. controls is the map of what can be clicked; elements needs selectors; json needs json.prompt or json.schema.

selectorsstring[]

CSS selectors for the elements format.

maxAgeinteger

Accept an answer up to this many milliseconds old. A cache hit costs $0.0002 instead of the format price. Leave it out to force a fresh render.

binaryAs'base64'

How a screenshot or a PDF comes back. base64 is the only value today.

urlstring uri required

Page to process.

limitinteger

Pages this crawl may read, up to 200. Charged up front; what it does not read comes back.

maxDepthinteger

How far from the starting page to follow links. 0 reads only the starting page.

includeSubdomainsboolean

Treat subdomains of the same site as part of it.

includePathsstring[]

Keep only urls whose path contains one of these. Up to twenty.

excludePathsstring[]

Discard urls whose path contains one of these. Up to twenty. It wins over includePaths.

Response

The result.

Changes

No recorded changes to this operation across all 1 revision of this API.