Pages

Site Map

Discover a site's URLs without fetching page content. Reads robots.txt sitemap directives, sitemap.xml and nested sitemap indexes, falling back to homepage link extraction. Price: $0.004.

post/map

Request

  • Base URL: https://api.weblens.dev
  • URL: https://api.weblens.dev/map
  • Auth: API key in header Payment-Signature

Request body

urlstring uri required

The URL to process

limitnumber

Maximum URLs to return

includestring[]

Only URLs whose path+query contains one of these

excludestring[]

Skip URLs whose path+query contains one of these

timeoutnumber

Request timeout in milliseconds

Example request

{
  "url": "https://example.com",
  "limit": 1000
}

Response

Every URL on a site, from sitemaps and links — no page fetches.

urlstring
urlsstring[]

Discovered same-host URLs

totalnumber
source'sitemap' | 'links' | 'none'

Where the URLs came from

sitemapsCheckednumber

Sitemap documents fetched (including nested indexes)

mappedAtstring
requestIdstring

Changes

No recorded changes to this operation across all 1 revision of this API.