---
title: "Extract and analyze all links from HTML content. A url is fetched once (15 s timeout; a 429 or 503 whose Retry-After is 10 s or less is retried once after that wait). A PDF or other binary returns 415 UNSUPPORTED_CONTENT_TYPE with next_step process_document; a JSON body returns no links and a warning. A bot wall is a BLOCKED error, unbilled; with escalate: true the same call renders the page once in a stealth browser and reads the links from that instead. An empty client-side shell succeeds with rendered: false and the warning \"page renders client-side; use scrape\"."
method: POST
path: "/tools/extract_links"
tags: ["tools"]
---

# Extract and analyze all links from HTML content. A url is fetched once (15 s timeout; a 429 or 503 whose Retry-After is 10 s or less is retried once after that wait). A PDF or other binary returns 415 UNSUPPORTED_CONTENT_TYPE with next_step process_document; a JSON body returns no links and a warning. A bot wall is a BLOCKED error, unbilled; with escalate: true the same call renders the page once in a stealth browser and reads the links from that instead. An empty client-side shell succeeds with rendered: false and the warning "page renders client-side; use scrape".

`POST /tools/extract_links`

## Request

- Base URL: `https://www.crawlforge.dev/api/v1`
- URL: `https://www.crawlforge.dev/api/v1/tools/extract_links`
- Auth: API key in header `X-API-Key`

## Request body

- union
  - object
    - `html` string, required: HTML content to extract links from
    - `url` string, uri: URL to fetch and extract links from
    - `base_url` string, uri: Base URL for resolving relative links
    - `filter_domains` string[]: Only include links from specified domains
    - `include_external` boolean: Include external links
    - `include_internal` boolean: Include internal links (same hostname as the page)
    - `filter_external` boolean: The MCP tool's name for outbound-only: true is the same as include_internal: false. An alias; when true it overrides include_internal.
    - `include_anchors` boolean: Include anchor links (#fragment)
    - `deduplicate` boolean: Remove duplicate links, compared without fragment or trailing slash; the first occurrence is kept
    - `include_metadata` boolean: Include each link's title and the domains count map. Every record always carries href, text, type, domain, rel and original_href — the shape the MCP tool returns. type is internal, external, anchor, other (mailto:, tel:, javascript: and other non-web schemes; href is null except for mailto: and tel:) or relative (html input with no base_url). A <base href> in the page is honoured unless base_url is given.
    - `respect_robots` boolean
    - `escalate` boolean: Retry a walled page in a stealth browser, in this same call (url input only). The plain fetch always runs first; only a named bot wall, a 403, 429 or 444, or an empty shell or error placeholder served as 2xx escalates — never a 404, a 5xx, a binary or a host whose Retry-After window is still open. The response carries escalated: true|false whenever this is set, plus stealth: { engine, vendor_detected } when the render ran. It costs the escalation add-on only when it ran. The stage needs the CrawlForge execution backend: where that is not configured it returns 503 TOOL_NOT_AVAILABLE and charges nothing. robots.txt is respected on the escalated path too.
    - `escalate_engine` 'playwright' | 'camoufox': Browser engine for the escalated render (camoufox only where installed on the backend). Ignored when no escalation runs.
    - `max_inline_chars` integer: Largest result returned inline, in characters of its JSON (1000-10000000; env CRAWLFORGE_MAX_INLINE_CHARS sets the default). Over it, the result is stored for 1 hour and the response carries a preview, a result_handle, total_chars and truncated: true; read the rest with read_result (1 credit).
  - object
    - `html` string: HTML content to extract links from
    - `url` string, uri, required: URL to fetch and extract links from
    - `base_url` string, uri: Base URL for resolving relative links
    - `filter_domains` string[]: Only include links from specified domains
    - `include_external` boolean: Include external links
    - `include_internal` boolean: Include internal links (same hostname as the page)
    - `filter_external` boolean: The MCP tool's name for outbound-only: true is the same as include_internal: false. An alias; when true it overrides include_internal.
    - `include_anchors` boolean: Include anchor links (#fragment)
    - `deduplicate` boolean: Remove duplicate links, compared without fragment or trailing slash; the first occurrence is kept
    - `include_metadata` boolean: Include each link's title and the domains count map. Every record always carries href, text, type, domain, rel and original_href — the shape the MCP tool returns. type is internal, external, anchor, other (mailto:, tel:, javascript: and other non-web schemes; href is null except for mailto: and tel:) or relative (html input with no base_url). A <base href> in the page is honoured unless base_url is given.
    - `respect_robots` boolean
    - `escalate` boolean: Retry a walled page in a stealth browser, in this same call (url input only). The plain fetch always runs first; only a named bot wall, a 403, 429 or 444, or an empty shell or error placeholder served as 2xx escalates — never a 404, a 5xx, a binary or a host whose Retry-After window is still open. The response carries escalated: true|false whenever this is set, plus stealth: { engine, vendor_detected } when the render ran. It costs the escalation add-on only when it ran. The stage needs the CrawlForge execution backend: where that is not configured it returns 503 TOOL_NOT_AVAILABLE and charges nothing. robots.txt is respected on the escalated path too.
    - `escalate_engine` 'playwright' | 'camoufox': Browser engine for the escalated render (camoufox only where installed on the backend). Ignored when no escalation runs.
    - `max_inline_chars` integer: Largest result returned inline, in characters of its JSON (1000-10000000; env CRAWLFORGE_MAX_INLINE_CHARS sets the default). Over it, the result is stored for 1 hour and the response carries a preview, a result_handle, total_chars and truncated: true; read the rest with read_result (1 credit).

## Response `200`

The tool's result; 1 credits charged (see x-credits-note and x-extra for add-ons).

- ToolSuccess: The envelope every tool returns on success. Credits are deducted only after the tool succeeded, so credits_used is the price of this call and an error response never charges.
  - `success` true, required
  - `data` object, required: The tool's result. Its shape is tool-specific and documented on the tool's docs page (x-docs-url on the operation).
  - `credits_used` integer, required: Credits charged for this call: the operation's x-credits, plus any add-on x-credits-note or x-extra describes.
  - `credits_remaining` integer, required: The account's credit balance after this call.
  - `processing_time` integer, required: Server-side processing time in milliseconds.
  - `warnings` string[]: Non-fatal notes: a robots.txt override that was recorded, a format that could not be produced, a redaction that changed the payload.

## Other responses

- `400`: VALIDATION_ERROR: the body does not match the request schema.
- `401`: MISSING_API_KEY or INVALID_API_KEY.
- `402`: INSUFFICIENT_CREDITS: the balance does not cover the tool's base price. X-Auto-Recharge-Triggered says whether an automatic recharge was started; retry once it lands.
- `429`: RATE_LIMIT_EXCEEDED: your plan's rate limit. Retry-After: 60.
- `500`: TOOL_ERROR, INTERNAL_ERROR or CREDIT_DEDUCTION_FAILED.
- `default`: Any other error. Which of 403, 404, 413, 422, 501, 502, 503 and 504 a tool can return depends on what it does (fetching a page, calling the execution backend, reading stored results); the code vocabulary is listed on the ErrorResponse schema.

---

[API](https://skmtc.dev/crawlforge/apis/crawlforge-rest-api.md) · [All operations](https://skmtc.dev/crawlforge/apis/crawlforge-rest-api/llms.txt) · [OpenAPI document](https://skmtc.dev/crawlforge/apis/crawlforge-rest-api/revisions/a5532fa8d5d5?raw)