---
title: "The URLs of a site, from robots.txt sitemaps and the start page's links, optionally ranked by a search (1 credit; free when nothing is found)"
method: POST
path: "/v1/map"
---

# The URLs of a site, from robots.txt sitemaps and the start page's links, optionally ranked by a search (1 credit; free when nothing is found)

`POST /v1/map`

## Request

- Base URL: `https://api.serpkite.com`
- URL: `https://api.serpkite.com/v1/map`
- Auth: HTTP bearer

Reads the sitemaps listed in robots.txt (or `/sitemap.xml`), following sitemap
indexes; XML, gzip, RSS/Atom and plain-text sitemaps are understood. URLs are
cleaned (fragment and tracking parameters such as `utm_*`, `gclid` and `fbclid`
dropped), kept on the site, filtered by `include_paths` / `exclude_paths` and
de-duplicated (`www.`, scheme and trailing slash folded). `search` ranks the URLs
by their path words and titles (BM25) and drops the ones that don't match. Takes
at most about 30 seconds.

## Request body

- MapRequest
  - `url` string, uri, required: The site (any page of it; sitemaps are read from its origin)
  - `search` string: Keep only URLs relevant to these words, most relevant first
  - `limit` integer
  - `include_subdomains` boolean
  - `include_paths` string[]: Regular expressions matched against the URL path; keep only URLs matching one
  - `exclude_paths` string[]: Regular expressions matched against the URL path; drop URLs matching one
  - `ignore_query_parameters` boolean: Treat URLs that differ only in their query string as one (the first one found is kept)
  - `sitemap` 'include' | 'only' | 'skip': `include` sitemaps and the start page's links, `only` sitemaps, `skip` sitemaps (links only)

## Response `200`

OK

- MapResponse
  - `request` object, required
    - `endpoint` 'map'
    - `url` string
    - `search` string
    - `limit` integer
    - `sitemap` string
    - `include_subdomains` boolean
    - `include_paths` string[]
    - `exclude_paths` string[]
    - `ignore_query_parameters` boolean
  - `results` MapURL[], required
    - `url` string, required
    - `title` string
    - `lastmod` string: The sitemap's lastmod, as written
    - `source` 'sitemap' | 'page', required
  - `meta` object, required
    - `request_id` string, required
    - `credits_used` number, required
    - `latency_ms` integer
    - `count` integer, required

## Other responses

- `400`: Error
- `401`: Error
- `402`: Error
- `403`: Error
- `429`: Error
- `503`: Temporarily unavailable, not billed, safe to retry after `Retry-After`. Codes: `upstream_error` (the upstream page couldn't be fetched or parsed), `upstream_blocked` (the upstream is rate limiting us), `upstream_timeout` (the search took too long), `unavailable` (endpoint or dependency down). The API never sends 502 or 504.

---

[API](https://skmtc.dev/serpkite/apis/serpkite-serp-api.md) · [All operations](https://skmtc.dev/serpkite/apis/serpkite-serp-api/llms.txt) · [OpenAPI document](https://skmtc.dev/serpkite/apis/serpkite-serp-api/revisions/f2c6dd05cfc5?raw)