---
title: "Sitemap and Robots.txt Doctor: Audit Crawl Files"
method: GET
path: "/v1/sitemap-robots-doctor"
tags: ["sitemap"]
---

# Sitemap and Robots.txt Doctor: Audit Crawl Files

`GET /v1/sitemap-robots-doctor`

## Request

- Base URL: `https://api.ambolt.dev`
- URL: `https://api.ambolt.dev/v1/sitemap-robots-doctor`
- Auth: none required

Audit a website's robots.txt and XML sitemaps in one call: user-agent groups and blocked paths, sitemap declarations, whether each sitemap loads, URL counts (indexes followed to 10 files), lastmod coverage and age, duplicates, off-host and non-HTTPS URLs, size-limit breaches, and a sample of listed URLs checked for status. Returns a list of concrete issues.

## Query parameters

- `site` string, required: Domain or URL of the site, for example example.com or https://example.com/blog.
- `sampleUrls` integer: How many listed URLs to fetch and check for status (0 to skip).
- `free` '1'

## Response `200`

Result (JSON). Every result names its source and the time it was read.

- object

## Other responses

- `400`: Invalid input; not charged.
- `402`: Payment required: 0.005 USD in USDC (x402 v2).

---

[API](https://skmtc.dev/ambolt/apis/ambolt-data-tools.md) · [All operations](https://skmtc.dev/ambolt/apis/ambolt-data-tools/llms.txt) · [OpenAPI document](https://skmtc.dev/ambolt/apis/ambolt-data-tools/revisions/8397885dd82f?raw)