--- title: "The site's XML sitemap from a crawl, as signed download links" method: GET path: "/v1/sites/{site}/sitemap" tags: ["sitemaps"] --- # The site's XML sitemap from a crawl, as signed download links `GET /v1/sites/{site}/sitemap` ## Request - Base URL: `https://frontshelf.io/api` - URL: `https://frontshelf.io/api/v1/sites/{site}/sitemap` - Auth: HTTP bearer The site's XML sitemap, generated from a committed crawl, as signed download links valid 15 minutes, one per file. By default: 2xx HTML pages by crawl depth, without noindex, canonicalised or paginated URLs, each with lastmod, changefreq and priority; `options` widens or narrows it. Past 50,000 URLs or 49,000,000 characters it rolls over into several files plus sitemapindex.xml: `file_count` says how many, and each file's `hosted_url` is where it belongs on the site. THIS IS AN EXPORT, NOT A PUBLISH: download the files, commit them to the site so they are served at those addresses, and reference the sitemap (or the index) from robots.txt. Frontshelf never writes to the site. ## Path parameters - `site` string, required: Site ID (site_…), origin (https://example.com), or bare domain, resolved server-side; the response carries the canonical ID. ## Query parameters - `crawl_seq` integer: A committed crawl's sequence number; omit for the site's current accepted crawl. - `options` object: What the sitemap includes. Every field has its default, so omit it for the standard sitemap. An option this version does not offer (images, hreflang) is refused by name. - `include_2xx` boolean - `include_3xx` boolean - `include_4xx` boolean - `include_5xx` boolean - `include_no_response` boolean: Include URLs that never answered (response class 0: a timeout, a DNS failure, a robots refusal). - `include_html` boolean - `include_pdfs` boolean - `include_noindex_pages` boolean - `include_canonicalised` boolean - `include_paginated` boolean: Include URLs that carry a rel=prev, i.e. pages 2..N of a series. The first page of a series is always included. - `include_blocked_by_robots` boolean: Include 2xx pages the crawl marked blocked by robots.txt (only possible when the crawl was configured to ignore and report robots). - `include_last_modified` boolean: ``: the page's Last-Modified, else the generation date (yyyy-MM-dd, UTC). - `include_priority` boolean: ``: 1.0 at depth 0 down to 0.5 at depth 5 and beyond. - `include_change_frequency` boolean: ``, read as `change_frequency_mode` says. - `change_frequency_mode` 'server_response' | 'crawl_depth': server_response: daily while the page's Last-Modified is under 24 h old, monthly once it is older. crawl_depth: daily for every page. - `filename_prefix` string: What the files are called, and so where you host them: `sitemap` gives sitemap1.xml, sitemap2.xml, ... beside sitemapindex.xml. ## Response `200` The site's XML sitemap from a crawl, as signed download links - object - `summary` string, required: The answer in one or two sentences, with the numbers it rests on. - `warnings` string[], required: What limits this answer (a cap, a read that was cut short). Empty when nothing does. - `next_actions` object[], required: What to call next, best first. Follow them. - `capability` string, required: What to call next: a capability. - `arguments` object, required: Its arguments, complete; pass them as they are. - `instruction` string, required: What to do, written to you. - `provenance` object, required: The committed crawl the sitemap is generated from. - `crawl_seq` integer, nullable, required: The crawl the answer was read from; null when the site has none yet. - `data_freshness` string, required: When that crawl finished (ISO 8601) or `no completed crawl`; for a read of one committed crawl, `committed crawl (immutable snapshot)`, or `not committed ()` when nothing committed answered. - `crawl_seq` integer, required - `url_count` integer, required: `` entries across every file. - `file_count` integer, required: `` files, the index not counted. - `content_type` 'application/xml', required - `files` object[], required: The `` files. - `file` integer, required: 0 is sitemapindex.xml; 1 to N are the `` files. - `name` string, required: The file's name, and the name to host it under. - `urls` integer, required: `` entries in this file; the index carries none. - `url` string, required: The signed download link: fetch it within 15 minutes, no Authorization header. A secret: never log it. - `expires_at` string, required: When the link stops working (ISO 8601). - `hosted_url` string, required: Where the file belongs on the site ({origin}/{name}); the index points at these addresses. - `index` object, nullable, required: sitemapindex.xml, present only when there is more than one file. - `file` integer, required: 0 is sitemapindex.xml; 1 to N are the `` files. - `name` string, required: The file's name, and the name to host it under. - `urls` integer, required: `` entries in this file; the index carries none. - `url` string, required: The signed download link: fetch it within 15 minutes, no Authorization header. A secret: never log it. - `expires_at` string, required: When the link stops working (ISO 8601). - `hosted_url` string, required: Where the file belongs on the site ({origin}/{name}); the index points at these addresses. - `options` object, required: What was included: every option resolved, defaults filled in. - `include_2xx` boolean, required - `include_3xx` boolean, required - `include_4xx` boolean, required - `include_5xx` boolean, required - `include_no_response` boolean, required: Include URLs that never answered (response class 0: a timeout, a DNS failure, a robots refusal). - `include_html` boolean, required - `include_pdfs` boolean, required - `include_noindex_pages` boolean, required - `include_canonicalised` boolean, required - `include_paginated` boolean, required: Include URLs that carry a rel=prev, i.e. pages 2..N of a series. The first page of a series is always included. - `include_blocked_by_robots` boolean, required: Include 2xx pages the crawl marked blocked by robots.txt (only possible when the crawl was configured to ignore and report robots). - `include_last_modified` boolean, required: ``: the page's Last-Modified, else the generation date (yyyy-MM-dd, UTC). - `include_priority` boolean, required: ``: 1.0 at depth 0 down to 0.5 at depth 5 and beyond. - `include_change_frequency` boolean, required: ``, read as `change_frequency_mode` says. - `change_frequency_mode` 'server_response' | 'crawl_depth', required: server_response: daily while the page's Last-Modified is under 24 h old, monthly once it is older. crawl_depth: daily for every page. - `filename_prefix` string, required: What the files are called, and so where you host them: `sitemap` gives sitemap1.xml, sitemap2.xml, ... beside sitemapindex.xml. ## Other responses - `400`: VALIDATION_FAILED - `401`: UNAUTHENTICATED - `403`: SCOPE_DENIED - `404`: SITE_NOT_FOUND, NOT_FOUND - `409`: AMBIGUOUS_SITE, AGENT_STOPPED - `503`: DEPENDENCY_UNAVAILABLE --- [API](https://skmtc.dev/frontshelf/apis/frontshelf-api.md) · [All operations](https://skmtc.dev/frontshelf/apis/frontshelf-api/llms.txt) · [OpenAPI document](https://skmtc.dev/frontshelf/apis/frontshelf-api/revisions/ce3602d26c7c?raw)