Fetch (Alpha)

Fetch (Alpha)

Fetch a public HTTP(S) page and return LLM-ready content. respond_with selects content (default), markdown, html, text or frontmatter, or a '+' combination. For screenshots use /api/web/screenshot. Uses the existing Litescrape bearer key, call ledger and total timeout; page_timeout bounds page loading. Always fetches fresh. Cookies, scripts, viewport and markdown options are JSON-body-only; Alpha contract may change.

post/api/web/fetch

Request

  • Base URL: https://api.litescrape.com
  • URL: https://api.litescrape.com/api/web/fetch
  • Auth: HTTP bearer

Query parameters

timeoutnumber

Optional budget for the entire request in seconds, measured from gateway receipt until the response is ready to send. Includes admission, queueing and settlement. Must be greater than 0 and at most 90. On expiry, the call is not billed and returns 503 request_deadline_exceeded with retryable=true and a request ID. Omit to retain the standard server deadline.

Request body

urlstring required

Public HTTP(S) URL to capture, 1-8,192 characters. Include http:// or https://; bare domains are rejected. The host must be public, on port 80 or 443, without embedded credentials.

remove_overlayboolean nullable

Remove modal and cookie-banner overlays from the page.

detach_invisiblesboolean nullable

Drop display:none elements from the page.

set_cookiesstring[] nullable

POST only. 1-100 cookies in Set-Cookie syntax (name=value; Path=/; Domain=...), one line each, up to 4,096 characters.

user_agentstring nullable

User-Agent for the page request, 1-1,024 characters without control characters. Defaults to the browser's own.

inject_page_scriptstring[] nullable

POST only. 1-8 scripts (source, or a URL to a script), up to 8,192 characters each, run in the main frame.

inject_frame_scriptstring[] nullable

POST only. 1-8 scripts (source, or a URL to a script), up to 8,192 characters each, run in every frame.

page_timeoutinteger nullable

Seconds from 1 through 180 for page loading and browser operations. Cannot extend the total request budget set by timeout.

localestring nullable

Browser locale as a language tag, 2-64 characters. Defaults to the browser's own.

refererstring nullable

HTTP(S) URL sent as the Referer.

assert_status_codeinteger nullable

Expected origin HTTP status, 100-599. A different status is rejected; otherwise a rendered error page is a successful capture.

robots_txtstring nullable

User agent, 1-256 characters, to check against the site's robots.txt; disallowed URLs are rejected.

respond_timing'html' | 'visible-content' | 'mutation-idle' | 'resource-idle' | 'media-idle' | 'network-idle' nullable

Page readiness condition to reach before responding.

respond_withstring nullable

Output format: content (default; main content as Markdown, or the whole page when extraction keeps too little), markdown (whole page as Markdown), html, text, frontmatter (content plus front matter), or a '+' combination such as markdown+frontmatter. For screenshots use /api/web/screenshot.

with_images_summaryboolean nullable

Add an images list to the response.

retain_images'none' | 'all' | 'alt' nullable

Images to keep: all (default), alt (alt text only) or none.

retain_links'none' | 'all' | 'text' | 'gpt-oss' nullable

Links to keep: all (default), text (link text only), none, or gpt-oss (citation format with a links summary).

retain_media'none' | 'text' | 'link' | 'image' | 'html' nullable

How video, audio and embedded video appear: none, text, link (default), image or html.

preset'reader' | 'index' | 'research' | 'agent' | 'spider' nullable

Option preset. Fills only the options the request leaves unset.

keep_img_data_urlboolean nullable

Keep inline data: images in Markdown.

with_shadow_domboolean nullable

Include open shadow DOM content before extraction.

engine'auto' | 'browser' | 'curl' nullable

auto (default; lightweight HTTP first, a browser when needed), browser, or curl (no JavaScript).

token_budgetinteger nullable

Integer 1-10,000,000. Reject the page when its token count exceeds the budget.

markdown_chunking'true' | 'h1' | 'h2' | 'h3' | 'h4' | 'h5' | 'structured' | 's1' | 's2' | 's3' | 's4' | 's5' nullable

true, h1-h5, structured or s1-s5: split the Markdown and return chunks.

max_tokensinteger nullable

Integer 500-10,000,000. Trim content to this many tokens.

base'initial' | 'final' nullable

Resolve relative links against the requested URL (initial, default) or the final URL after redirects (final).

Example request

{
  "url": "https://example.com",
  "set_cookies": [
    "session=YOUR_SESSION; Path=/; Secure; HttpOnly"
  ],
  "page_timeout": 30,
  "locale": "en-US",
  "referer": "https://example.com",
  "assert_status_code": 200,
  "viewport": {
    "height": 900,
    "width": 1440
  },
  "robots_txt": "MyBot",
  "respond_timing": "network-idle",
  "respond_with": "markdown",
  "retain_images": "alt",
  "markdown": {
    "link_style": "referenced"
  }
}

Response

Successful capture; one call consumed even for a rendered origin error page.

urlstring required
titlestring required
descriptionstring required
contentstring

Markdown for content and markdown formats.

chunksstring[]
htmlstring
textstring
published_timestring
warningstring
metadataobject
externalobject

Changes

No recorded changes to this operation across all 1 revision of this API.