tools

Interact with a page before scraping — click, type, press keys, hover, choose a select option, scroll, navigate on, run JavaScript, or wait for dynamic content. Use for SPAs, login-gated content, or multi-step flows. Set browserOptions.stealth to run the chain in a stealth browser context. A click or form submit that loads another page is a navigation too: it gets its own navigations entry (trigger names the action type), and finalUrl is where the chain ended. Screenshots are taken by {type:"screenshot"} actions (and on error) and stored as crawlforge://screenshot/{actionId} resources.

post/tools/scrape_with_actions

Request

  • Base URL: https://www.crawlforge.dev/api/v1
  • URL: https://www.crawlforge.dev/api/v1/tools/scrape_with_actions
  • Auth: API key in header X-API-Key

Request body

urlstring required

The URL to scrape

captureIntermediateStatesboolean

Capture page state after each action: intermediateStates[] gets one entry per action (capturePoint is the action's 1-based position) with the url, title and the requested formats at that point

respect_robotsboolean

Respect the target site's robots.txt. Omitted, the compliant default (true) applies: a URL disallowed for CrawlForge is refused before the browser opens and no credits are charged, and every navigate action is checked the same way. Setting this to false is honoured, returns a warning in the response, and is recorded against your API key.

continueOnActionErrorboolean

Continue executing actions if one fails

maxRetriesnumber

Whole-chain retries on failure (0-3). A retry re-navigates to the starting URL and replays every action; each attempt is reported under attempts[]

screenshotOnErrorboolean

Capture screenshot when an error occurs

max_inline_charsnumber

Largest result returned inline, in characters of its JSON (1000-10000000; env CRAWLFORGE_MAX_INLINE_CHARS sets the default). Over it, the result is stored for 1 hour and the response carries a preview, a result_handle, total_chars and truncated: true; read the rest with read_result (1 credit).

Response

The tool's result; 5 credits charged (see x-credits-note and x-extra for add-ons).

successtrue required
dataobject required

The tool's result. Its shape is tool-specific and documented on the tool's docs page (x-docs-url on the operation).

credits_usedinteger required

Credits charged for this call: the operation's x-credits, plus any add-on x-credits-note or x-extra describes.

credits_remaininginteger required

The account's credit balance after this call.

processing_timeinteger required

Server-side processing time in milliseconds.

warningsstring[]

Non-fatal notes: a robots.txt override that was recorded, a format that could not be produced, a redaction that changed the payload.

Changes

No recorded changes to this operation across all 1 revision of this API.