---
title: "robots.txt + llms.txt checker"
method: GET
path: "/api/seo/robots-check"
tags: ["SEO & Publishing"]
---

# robots.txt + llms.txt checker

`GET /api/seo/robots-check`

## Request

- Base URL: `https://api.webbersites.com`
- URL: `https://api.webbersites.com/api/seo/robots-check`
- Auth: API key in header `X-PAYMENT`

robots.txt + llms.txt checker: crawler access verdicts for major search AND AI bots (Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot, CCBot, Google-Extended and more), declared sitemaps, syntax warnings, and llms.txt / llms-full.txt presence with structure summary. ?url=any page on the site

Price: $0.001 per call, paid in USDC via the x402 protocol (see the x402 security scheme).

## Query parameters

- `url` string, required

## Response `200`

Success

- object
  - `origin` string
  - `robots` object
  - `llms_txt` object

## Other responses

- `402`: Payment required — the response body lists accepted payment methods (amount, asset, network, pay-to address). Retry the request with a signed X-PAYMENT header.

---

[API](https://skmtc.dev/webbersites/apis/webbersites-x402-data-api.md) · [All operations](https://skmtc.dev/webbersites/apis/webbersites-x402-data-api/llms.txt) · [OpenAPI document](https://skmtc.dev/webbersites/apis/webbersites-x402-data-api/revisions/c14a76c67a94?raw)