Tools

Analyze sitemap structure of a domain

Changed on

Discover and analyze the sitemap structure of any public domain. Reads the conventional /sitemap.xml AND every Sitemap: directive robots.txt declares — a site publishing several sitemaps returns all of them, deduplicated (or pass an explicit URL to read just that one). Categorizes URLs by content type, validates structure, and surfaces freshness and depth-distribution insights with sample URLs per category. Optionally include the full URL list (off by default to keep payload bounded). Where a declared sitemap could not be retrieved, truncated is true and unreadSitemaps names it, so a total is never reported as complete over a corpus only partly read. Stateless live check — no project required, no data is persisted. Suitable for content-strategy audits, SEO discovery, and content-volume benchmarking.

post/v1/tools/sitemap-visualizer

Request

  • Base URL: https://api.competlab.com
  • URL: https://api.competlab.com/v1/tools/sitemap-visualizer
  • Auth: API key in header CL-API-Key

Request body

domainstring required

Target domain whose sitemap to analyze. Accepts a bare hostname or a full URL — normalized (lowercased, scheme and path stripped) before analysis.

sitemapUrlstring

Explicit sitemap URL to fetch directly, short-circuiting /sitemap.xml then robots.txt discovery. Must be a full http:// or https:// URL with a public hostname.

includeUrlsboolean

When true, include the full raw urls[] list in the response. Defaults to false, which returns a lean insights-only payload without the URL list.

Example request

{
  "domain": "stripe.com",
  "sitemapUrl": "https://stripe.com/sitemap.xml"
}

Response

Example response

{
  "item": {
    "domain": "competlab.com",
    "fetchedAt": "2026-05-16T14:30:00.000Z",
    "status": "ok",
    "source": "sitemap-xml",
    "sitemaps": [
      {
        "url": "https://competlab.com/sitemap.xml",
        "type": "urlset",
        "urlCount": 247,
        "actualUrlCount": 247
      }
    ],
    "unreadSitemaps": [
      "https://example.com/sitemap-archive.xml"
    ],
    "unreadSitemapCount": 2,
    "metrics": {
      "totalUrls": 247,
      "totalActualUrls": 247,
      "totalSitemaps": 3,
      "categoriesFound": 6,
      "maxDepth": 4,
      "avgDepth": 2.6
    },
    "urls": [
      {
        "url": "https://competlab.com/blog/competitive-intelligence-guide",
        "category": "blog",
        "depth": 2,
        "lastmod": "2026-04-12"
      }
    ],
    "categories": {
      "blog": {
        "count": 47,
        "percentage": 18.4
      },
      "docs": {
        "count": 47,
        "percentage": 18.4
      },
      "tools": {
        "count": 47,
        "percentage": 18.4
      },
      "landing": {
        "count": 47,
        "percentage": 18.4
      },
      "legal": {
        "count": 47,
        "percentage": 18.4
      },
      "case-studies": {
        "count": 47,
        "percentage": 18.4
      },
      "comparison": {
        "count": 47,
        "percentage": 18.4
      },
      "integrations": {
        "count": 47,
        "percentage": 18.4
      },
      "changelog": {
        "count": 47,
        "percentage": 18.4
      },
      "webinars": {
        "count": 47,
        "percentage": 18.4
      },
      "careers": {
        "count": 47,
        "percentage": 18.4
      },
      "programmatic": {
        "count": 47,
        "percentage": 18.4
      },
      "other": {
        "count": 47,
        "percentage": 18.4
      }
    },
    "validation": {
      "sitemapFound": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      },
      "validXml": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      },
      "nonEmpty": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      },
      "validIndex": {
        "passed": true,
        "message": "Found valid sitemap at /sitemap.xml"
      }
    },
    "insights": {
      "freshness": {
        "fresh": 32,
        "recent": 18,
        "aging": 41,
        "stale": 122,
        "noDate": 34
      },
      "depthDistribution": {
        "0": 1,
        "1": 24,
        "2": 87,
        "3": 92,
        "4+": 43
      },
      "stalePageCount": 163,
      "sampleUrlsByCategory": {
        "blog": [
          "https://competlab.com/blog/competitive-intelligence-guide",
          "https://competlab.com/blog/ai-visibility-tracking"
        ],
        "docs": [
          "https://competlab.com/docs/getting-started"
        ],
        "case-studies": [
          "https://competlab.com/case-studies/acme"
        ]
      },
      "summary": "Content leans towards landing (48% of URLs). Active comparison-page SEO (74 pages). Strong social proof (113 case studies)."
    }
  }
}

Changes

  • c9dadd2d3a30Revision changes
    • info

      added the non-success response with the status

  • b08f96c9300bRevision changes
    • warning

      added the new nothing_to_return enum value to the / response property for the response status

    • warning

      added the new paging_requires_compact_view enum value to the / response property for the response status

    • warning

      added the new paging_requires_summary enum value to the / response property for the response status