Scrape a URL

Fetch one URL synchronously and return it as html, markdown, text, a PDF, or a JSON envelope with extracted data.

How a request is rendered (cheapest first). You do not choose: Spicrawl picks the cheapest path that can do what the request asks.

• Default (plain fetch, 1 credit): plain HTTP, no JavaScript, presenting a current browser TLS/HTTP2 fingerprint (impersonate, on by default and free; impersonate=false turns it off).
• js_render=true (JavaScript rendering, 3 credits): content is built by JavaScript, or you need wait/wait_for/block_resources/network_capture.
• Screenshots, response_format=pdf and actions use full-browser rendering (8 credits). stealth=true (full-browser rendering hardened against bot detection, 25 credits) is coming soon.
• mode=auto: unsure - escalates through the available rendering tiers, bills only the tier that worked.
• Managed proxy pools (premium_proxy) are coming soon; during the beta pass your own proxy with proxy.
• The engine and headless fields are deprecated and ignored. Browser-only flags on the fetch tier are a 400, never silently ignored.

Billing: failures cost 0 (errors, bot challenges, timeouts, target statuses other than 200/404/410 unless listed in allowed_status_codes, undelivered screenshots). A cache hit is billed at the same price as the fetch that stored it: the cache saves the fetch (latency and load on the target), not credits. Like any request, a hit is refused with 402 when the monthly allowance cannot cover its price. Check X-Credits-Charged.

Bot challenges: a challenge page attributed to a named vendor (Cloudflare, DataDome, PerimeterX, Akamai) is never returned as a success. A fetch through a managed pool exit that meets one is retried once on a different exit, free (GET/HEAD/OPTIONS only, not with sticky_key or session_id, never on your own proxy). When you bring your own proxy (residential works best) and send no mode=auto, a request whose page is still a challenge is escalated automatically to stronger full-browser rendering on the same exit, which works through the interstitial and returns the real page; there is no flag to set. This is how Cloudflare-protected pages are supported. Automatic handling on managed premium exits is coming soon. Success is not guaranteed for every page. Only the tier that returned the page is billed (up to 25 credits, or the first tier's price), and 0 if every tier was challenged (502 ERR::UPSTREAM::CHALLENGE). On the default shared pool a challenged page may return that error; retry with your own residential proxy. A transport error, a refusal with no vendor marker (such as a bare 403) or a non-billable status such as 404 does not escalate. Escalation is skipped for a non-GET method, response_format=pdf, a screenshot, actions, a proxy that is not http://, or a max_cost below the escalated price; an allowance that covers the first attempt but not the escalation runs the request without it. X-Engine reports the rendering tier that served, each abandoned tier adds an X-Warning: ESCALATED, and diagnostics.attempts lists every attempt (in the JSON envelope on a successful escalation). Batch items never escalate.

Status: the HTTP status is the platform's. A blocked or erroring site is usually still HTTP 200 - read envelope status or X-Target-Status for the site's answer. Errors are application/problem+json; retry only when retryable is true.

Not idempotent: there is no idempotency key, and each call bills again, whether it performs a new scrape or is served from cache. GET and POST accept the same parameters and behave identically.