mcp

Enumerate unique subdomains seen in CT logs for a domain

Returns all unique subdomains of the given domain observed across Certificate Transparency logs, with the earliest first-seen timestamp and total cert count. first_seen is the exact minimum first-seen over ALL of the subdomain's indexed certificates (including precertificates), and last_seen the exact maximum — computed per returned page row, so mega-domains hitting the sample cap still get exact values for the subdomains actually returned. Useful for attack-surface discovery and subdomain enumeration.

Incremental diff (task #855): pass first_seen_after (an ISO-8601 datetime — typically your last poll time) to get ONLY subdomains first observed at or after that instant, so a scheduled poller fetches just the delta instead of re-walking the full set. The filter matches on the subdomain's exact MIN(first_seen). Ordering is unchanged and stable for pagination: popularity order in offset mode, subdomain-name ASC keyset in cursor mode (recommended for pollers — pages are stable against concurrent inserts). An empty window returns zero results quickly.

Cached totals (task #988 / GH #297): on a hot apex the default unfiltered enumeration is answered from a periodically refreshed summary. When it is, the response carries total_subdomains_as_of — the instant that count describes. Its absence means the total was counted at request time. Any filter (first_seen_after / since / as_of) and cursor pagination bypass the cache, so those totals are always live and never stamped. Compare two totals only when they carry the same stamp (or both carry none): a cached total can be SMALLER than a filtered subset counted live a moment later on a growing domain — two instants, not a filter that added rows.

Query budget & partial results (GH #192): each call runs under a fixed server-side query budget (~10s). For very high-cardinality apexes the scan is bounded; when the bound is hit the response sets results_partial: true and total_is_lower_bound: true — the page is ranked from a sample and total_subdomains is a floor, not an exact count. The default limit (25) is sized to fit the budget even on huge domains; keep limit at or below the default and page through results rather than requesting large pages. A call that blows the budget entirely returns HTTP 200 (never 504) with results_partial: true, total_is_lower_bound: true, an empty results list and a hint — do NOT retry the same call; lower limit or use cursor pagination (task #852). This implication runs one way only — see the two lower-bound causes below.

Two lower-bound causes, opposite remedies (GH #305): total_is_lower_bound: true means total_subdomains is a FLOOR, and it has TWO independent causes. lower_bound_reason names which one applies, so you never have to join keys to decide what to do: query_budget — the bounded scan was truncated; the data IS indexed and this page simply did not fit (results_partial is also true); remedy: repaginate with a smaller limit, or use cursor pagination. coverage_scope — the scan behind this inventory did not finish within the advertised product scope, did not establish what it reached (enumeration_scope=unknown), or never ran (no domain_backfills row); an identical call returns an IDENTICAL count, so do NOT retry. A zero total with this cause means "we cannot tell", not "none exist". Expired-only names on a completed active_only pass are product scope, not this cause. query_budget_and_coverage_scope — both; repaginating recovers the rest of the indexed page, but the total stays a floor. backfill_in_progress — an on-demand backfill is still running; the inventory is growing; re-query later (do not apply the coverage_scope do-not-retry rule to a live scan). query_budget_and_backfill_in_progress — truncated page plus a running backfill. cold_tier_unavailable — cold was configured but unreachable; retry later or treat as reduced scope. The key is OMITTED when the total is exact. results_partial and total_is_lower_bound are therefore NOT always co-set: a domain with six subdomains can return total_is_lower_bound: true beside results_partial: false.

Self-healing for cold domains (task #113 / GH #266): if CertIndex has not indexed this domain yet, the response contains zero results plus a backfill_status sentinel. When backfill_status == 'in_progress', wait retry_after_seconds seconds and call this tool again. Use eta_basis to understand the projection: 'live' = measured entry-throughput ETA; 'jobs_coarse' = approximate estimate from job-completion rate; 'unknown' = no projection — terminal_at_latest (ISO-8601 UTC) and seconds_until_terminal are then emitted as the concrete stop point. in_progress_over_budget (no retry_after_seconds) means the projection exceeds the interactive budget: do NOT poll in a loop; retry much later. Any other status means the result will not change on retry.

Progress reporting: while an on-demand backfill is in flight the response carries a top-level progress block, and the coverage block carries a subset of the SAME snapshot under coverage.progress. Both are rendered from one snapshot under one emission rule, so shared quantities always agree. Shard-scan counters that carry no signal are OMITTED, not reported as zeros: an ABSENT shard_scan_progress_pct / shard_scan_entries_scanned means 'not measurable yet', not 'no progress'. shard_scan_progress_pct falls back to the jobs ratio when the entry-level aggregate is unavailable, so it can never read 0.0 while shard_scan_jobs_done climbs, and shard_scan_status can never read pending next to a non-zero shard_scan_jobs_done.

Coverage confidence: responses carry an advisory coverage block ({confidence, basis, hint, as_of}). coverage.basis is EXACTLY one of the values below — never prefix-match domain_backfill_completed: that prefix spans one trustworthy verdict, two scope-limited ones and an empty-inventory one, so mapping the prefix onto 'the inventory is exhaustive' reads the WORST case as a confirmed-complete answer. Match the exact string. domain_backfill_completed_full (HIGH — a backfill completed having walked the FULL certificate-transparency history and found records; the inventory is exhaustive as far as CertIndex can enumerate), domain_backfill_completed_active_only (RETAINED — no longer emitted), domain_backfill_completed_active_since_ingestion (MEDIUM — the free pass enumerated names since CertIndex's ingestion horizon; coverage.total_is_lower_bound is true with lower_bound_reason: ingestion_horizon), domain_backfill_completed_scope_unknown (MEDIUM — the pass finished but did not establish what it reached: a lane truncated its enumeration, or the row predates scope recording; the inventory is a LOWER BOUND), domain_backfill_completed (RETIRED — no longer emitted, but payloads minted by an older fleet still carry it. It could not distinguish the three cases above, so do NOT read it as high-confidence exhaustive), domain_backfill_completed_empty (LOW — backfill completed but enumerated zero records; treat as UNKNOWN, NOT as a confirmed empty surface — absence of records is not evidence of absence of certificates), domain_backfill_never_ran (LOW — the row reads terminal with zero records but the scan never actually executed, i.e. a dispatch failure; weaker evidence still than completed-empty — re-trigger), domain_backfill_in_progress (LOW — backfill running, inventory growing; re-query later; total_is_lower_bound is true and lower_bound_reason is backfill_in_progress), domain_backfill_stalled (LOW — the backfill stopped advancing; TERMINAL, stop polling — an operator re-trigger or a paid deep-historical run forces a fresh attempt), domain_backfill_failed (LOW — historical backfill failed; the historical inventory may be incomplete, but live CT-log ingestion continues so newly issued certificates still appear), inherited_from_apex_backfill (confidence varies — no backfill for this exact name, but a backfill for the ancestor named in coverage.inherited_from covers it; backfills are enumerated apex-wide, so a subdomain inherits its apex's verdict rather than falling back to the global signal), global_active_coverage (confidence varies — no per-domain or ancestor backfill history; confidence derives from global index coverage), or no_signal (UNKNOWN — no signal available). The block also carries as_of (UTC timestamp of the verdict).

Completeness scope (GH #300 / #304 / #536): every completed-basis block, and the global_active_coverage / no_signal fallbacks for a name with no domain_backfills row, also carries coverage.enumeration_scope — full, active_only or unknown, i.e. what the pass that produced this inventory could actually reach — plus coverage.scope_limited (bool; true for anything but full) and coverage.total_is_lower_bound (bool). The NESTED coverage.total_is_lower_bound is true when the scan is incomplete WITHIN the advertised product scope (enumeration_scope=unknown, an unfinished pass, or no domain_backfills row — a never-scanned name). A zero total with total_is_lower_bound: true means "we cannot tell"; 0 with total_is_lower_bound: false means "none exist within the advertised enumeration scope". A completed active_only pass is exact for currently-valid names: missing expired-only names are outside this product (read enumeration_scope / scope_limited), not a coverage floor. The TOP-LEVEL field of the same name is the OR of that nested flag and this response's own row-scan cap, so the top-level flag can be true while the nested one is false — never the reverse.

Verdict precedence: the verdict tracks the live backfill record — it changes as the scan progresses — with fixed precedence: in-progress (any active scan always outranks completed-empty — a running scan and a completed-empty verdict carry opposite instructions, so they must never coexist) > completed (non-empty) > completed-with-follow-up-pass-running (with sub_verdicts: {active: completed, historical: in_progress} — never a silent high-to-low regression) > completed-empty (low, terminal only) > failed > global coverage. Field precedence when retry_after_seconds is present in the sentinel: the backfill is still running; coverage.confidence reflects the state of the data available NOW.

Provenance — direct CT vs crt.sh (task #1033): each result row carries reconciled_from_crtsh: false when at least one certificate naming this subdomain was observed directly in a CT-log crawl (the data is wider than crt.sh); true when every cert came via the crt.sh reconciliation path (stored log_id == 'crt.sh'); null (field absent on exclude-none routes) when provenance was not recorded for any cert naming this subdomain — pre-existing rows ingested before source tracking, or when the cached summary served the page. Treat null as unknown, never as either boolean. The envelope also carries sources — the set of distinct provenance categories present on this page: direct_ct (at least one subdomain was seen directly in a CT-log crawl), crt_sh (at least one subdomain was seen only via crt.sh reconciliation), unknown (at least one subdomain has no recorded provenance). sources is omitted when the page is empty.

get/mcp-api/get_subdomains/{domain}

Request

  • Base URL: https://api.ctindex.io
  • URL: https://api.ctindex.io/mcp-api/get_subdomains/{domain}
  • Auth: one of:
    • API key in header X-API-Key
    • HTTP bearer

Path parameters

domainstring required

Query parameters

pageinteger
limitinteger
cursorstring nullable

Opaque keyset cursor (task #635 / GH #263). Pass cursor='*' (the coalescing-proof start token) to start a cursor enumeration, then the next_cursor from each response until it is null (cursor-mode pages ALWAYS carry the next_cursor key; an explicit null marks the terminal page). Cursor pages are ordered by subdomain name ASC (an immutable key), so pages are stable against concurrent inserts — unlike the default popularity (cert_count) ordering, which is offset-only. In cursor mode limit may go up to the caller's tier ceiling (base 200 free, 1000 paid) instead of the offset-mode maximum of 50. When cursor is supplied, page is ignored. Omit cursor entirely for the classic offset pagination (unchanged). Legacy: cursor= (empty string) also starts enumeration but may be coalesced to null by some transports.

first_seen_afterstring

Only subdomains whose earliest observation (MIN first_seen over all their indexed certs) is >= this ISO-8601 datetime. Incremental-diff filter (task #855): a scheduled poller passes its last poll time to get only NEWLY observed subdomains instead of re-walking the full set. Composable with cursor pagination (cursor pages stay subdomain-name ASC keyset order). Also accepted under its REST name, since — the two are identical; supplying both with different values is an error.

sincestring

Alias of first_seen_after, spelled as the /v1 REST route spells it (GH #290). Accepted so an agent working from the REST reference cannot have its filter silently dropped.

as_ofstring

ISO-8601 datetime — reconstruct the subdomain inventory as it was observed at this past point in time: only certificates first seen at or before as_of are counted, and last_seen is capped at as_of. Inventory is observation-based (CT sightings), so this is 'what CertIndex knew then', not a statement about what hosts were live then. Twin of the /v1 as_of parameter (GH #290).

wait_for_backfillboolean

When false, cold domains return a 'not_indexed' sentinel immediately without triggering a background backfill. Default true preserves the existing self-healing behaviour.

historicalboolean

Deep-historical enumeration; requires paid backfill entitlement (402 historical_not_entitled otherwise).

Cookies

ctix_sessionstring nullable

Response

Successful Response

backfill_statusstring nullable
retry_after_secondsinteger nullable
hintstring nullable
progressobject nullable

Live backfill counters while an on-demand scan is in flight (same snapshot as coverage.progress). Omitted when no in-progress scan is attached to this inventory.

typical_eta_secondsinteger nullable
eta_secondsinteger nullable
eta_basisstring nullable
index_freshnessobject nullable
domainstring required
partialboolean

True when the hot Neon page was returned but the Tier-1 cold apex supplement did not complete (timeout, error, or budget). Hot results are still authoritative for this page.

cold_tierobject nullable

Tier-1 cold supplement status for this response. status is one of ok, omitted, timeout, or error; reason names the failure when present. Omitted on the happy path when cold was not consulted or succeeded.

total_subdomainsinteger nullable
total_is_lower_boundboolean

True when total_subdomains is a FLOOR rather than an exact count. Two independent causes, ORed — read lower_bound_reason to tell them apart:

  • query budget — the bounded row scan hit its cap (or the ~10s budget). The data IS indexed; this page did not fit. results_partial is also true. Remedy: repaginate with a smaller limit, or use cursor pagination.
  • coverage scope — the scan behind this inventory did not finish within the advertised product scope, did not establish what it reached (enumeration_scope=unknown), or never ran (no domain_backfills row). results_partial is FALSE. Remedy: none by retrying — an identical call returns an identical count. A zero total with this cause means "we cannot tell", not "none exist". Expired-only names on a completed active_only pass are product scope, not this cause.
  • backfill in progress — an on-demand backfill is still running (lower_bound_reason=backfill_in_progress). The inventory is growing. Remedy: re-query later. Distinct from coverage_scope so a caller does not apply "do not retry" to a live scan.
  • cold tier unavailable — cold was configured but unreachable (lower_bound_reason=cold_tier_unavailable). Retry later or treat as reduced scope.

The two are NOT co-set: a 6-subdomain domain can carry total_is_lower_bound: true with results_partial: false.

results_partialboolean

True only when THIS response's row scan was truncated by the server-side query budget — the page is ranked from a sample. It is NOT implied by total_is_lower_bound, which has a second, coverage-scope cause that leaves this field false.

lower_bound_reasonstring nullable

Why total_subdomains is a lower bound. query_budget = the bounded scan was truncated; the data is indexed, repaginate (smaller limit or cursor mode). coverage_scope = the scan did not finish within the advertised product scope, did not establish what it reached, or never ran (no domain_backfills row); retrying returns the identical count — do NOT retry. A zero total with this cause means "we cannot tell", not "none exist". Expired-only names on a completed active_only pass are product scope (enumeration_scope), not this cause. query_budget_and_coverage_scope = both; repaginating recovers the rest of the indexed page but the total stays a floor. backfill_in_progress = an on-demand backfill is still running; re-query later (do not treat as exact, and do not apply the coverage_scope "do not retry" rule). query_budget_and_backfill_in_progress = truncated page plus running backfill. cold_tier_unavailable = cold was configured but unreachable; retry later or treat as reduced scope. Absent when total_is_lower_bound is false.

has_moreboolean nullable
next_cursorstring nullable
coverageobject nullable
total_subdomains_as_ofstring date-time nullable

Instant that total_subdomains describes, when that count was served from the cached apex summary rather than counted for this request (ISO-8601 UTC). Absence is meaningful: when this field is missing, total_subdomains was computed at request time. Any filter (since / first_seen_after / as_of) and cursor pagination bypass the cache, so those answers are always live and never carry this field. Compare two totals only when they carry the same stamp (or both carry none): a cached total can be SMALLER than a filtered subset counted live a moment later on a growing domain — that is two different instants, not a filter that added rows.

sourcesstring[] nullable
enumeration_scopestring

Product enumeration scope: active (active-only / active-since-ingestion / unknown internal scopes) or full (completed deep-historical backfill).

historical_statestring

Deep-historical lifecycle: disabled | backfill_pending | backfill_in_progress | full | refresh_paused.

corpus_depth_daysinteger nullable

Days from the earliest indexed certificate observation on this apex to now; null when no corpus is indexed yet.

Changes

No recorded changes to this operation across all 1 revision of this API.