subdomains

Get Subdomains

List unique subdomains of domain observed in the CT index.

Returns each subdomain with its earliest first_seen, latest last_seen (most recent CT observation of any cert naming it) and the total number of distinct certificates linking to it. The apex domain itself is excluded — only entries of the form *.{domain} appear here.

Inventory is observation-based: presence means a certificate naming the subdomain was observed in CT logs. A stale last_seen means the name rolled out of our observation window — NOT that it was decommissioned.

Latency target (GH #286): p50 ≤ 500 ms across cardinality buckets (≈10 / ≈200 / ≈5,000 subdomains). Measured via live /health/stats by_endpoint. The default path uses a single bounded suffix-index scan over domain_certs (capped at SUBDOMAIN_SCAN_CAP, default 100 k rows) with a certificates join scoped to the returned page only — never a full cross-domain aggregate. The since and as_of filter paths are heavier and are excluded from this target.

Use the since parameter to poll incrementally: pass the first_seen value of the last subdomain you received to get only newly-discovered subdomains. Use as_of to reconstruct the inventory at a past point in time.

Cached totals (task #988 / GH #297): on a hot apex the default unfiltered enumeration is answered from a periodically refreshed summary. When it is, the response carries total_subdomains_as_of — the instant that count describes. Its absence means the total was counted at request time. Any filter (since / first_seen_after / as_of) and cursor pagination bypass the cache, so those totals are always live and never stamped. Compare two totals only when they carry the same stamp (or both carry none): a cached total can be SMALLER than a filtered subset counted live a moment later on a growing domain — two instants, not a filter that added rows.

Two lower-bound causes, opposite remedies (GH #305): total_is_lower_bound: true means total_subdomains is a FLOOR, and it has TWO independent causes. lower_bound_reason names which one applies, so a caller never has to join keys to decide what to do:

  • query_budget — the bounded scan was truncated; the data IS indexed and this page did not fit (results_partial is also true). Remedy: repaginate with a smaller limit, or use cursor pagination.
  • coverage_scope — the scan behind this inventory did not finish within the advertised product scope, did not establish what it reached (enumeration_scope=unknown), or never ran (no domain_backfills row). An identical call returns an IDENTICAL count, so do NOT retry. A zero total with this cause means "we cannot tell", not "none exist". Expired-only names on a completed active_only pass are product scope, not this cause.
  • query_budget_and_coverage_scope — both. Repaginating recovers the rest of the indexed page, but the total stays a floor.
  • backfill_in_progress — an on-demand backfill is still running; the inventory is growing. Remedy: re-query later. Distinct from coverage_scope so a caller does not apply "do not retry" to a live scan.
  • query_budget_and_backfill_in_progress — truncated page plus a running backfill. Repaginate for the rest of the indexed page; re-query later for newly found records.
  • cold_tier_unavailable — cold was configured but unreachable / timed out / errored. Retry later or treat as reduced scope.

The key is OMITTED when the total is exact. results_partial and total_is_lower_bound are therefore NOT always co-set: a domain with six subdomains can return total_is_lower_bound: true beside results_partial: false.

Progress reporting: while an on-demand backfill is in flight the response carries a top-level progress block, and the coverage block carries a subset of the SAME snapshot under coverage.progress. Both are rendered from one snapshot. Omitted when no scan is in flight.

Coverage confidence (GH #300 / #304): responses carry an advisory coverage block ({confidence, basis, hint, as_of}). coverage.basis is EXACTLY one of domain_backfill_completed_full (HIGH — a full CT-history walk completed and found records), domain_backfill_completed_active_only (RETAINED — no longer emitted), domain_backfill_completed_active_since_ingestion (MEDIUM — the free pass enumerated names since CertIndex's ingestion horizon; coverage.total_is_lower_bound is true with lower_bound_reason: ingestion_horizon), domain_backfill_completed_scope_unknown (MEDIUM — the pass did not establish what it reached; the inventory is a LOWER BOUND), domain_backfill_completed (RETIRED — no longer emitted, still carried by payloads minted by older fleets; do NOT read it as exhaustive), domain_backfill_completed_empty (LOW — completed with zero records; treat as UNKNOWN, not as a confirmed empty surface), domain_backfill_never_ran (LOW — terminal row whose scan never executed), domain_backfill_in_progress (LOW — re-query later), domain_backfill_stalled (LOW — TERMINAL, stop polling), domain_backfill_failed (LOW), inherited_from_apex_backfill (verdict inherited from the ancestor in coverage.inherited_from), global_active_coverage, or no_signal (UNKNOWN). Never prefix-match domain_backfill_completed — that prefix spans one trustworthy verdict, two scope-limited ones and an empty-inventory one, so mapping it onto "the inventory is exhaustive" reads the WORST case as a confirmed-complete answer.

Every completed-basis block, and the global_active_coverage / no_signal fallbacks for a name with no domain_backfills row (GH #536), also carries coverage.enumeration_scope (full | active_only | active_since_ingestion | unknown — what the pass could actually reach), coverage.scope_limited (bool; true for anything but full) and coverage.total_is_lower_bound (bool). The NESTED coverage.total_is_lower_bound is true when the scan is incomplete WITHIN the advertised product scope (enumeration_scope= unknown, an unfinished pass, or no domain_backfills row — a never-scanned name) OR when the completed pass was bounded by CertIndex's ingestion horizon (active_since_ingestion, GH #591). total_subdomains: 0 with total_is_lower_bound: true means "we cannot tell"; 0 with total_is_lower_bound: false means "none exist within the advertised enumeration scope". Missing expired-only names are product scope (read enumeration_scope / scope_limited), not a scan-incomplete floor. The TOP-LEVEL field of the same name is the OR of that nested flag and this response's own row-scan cap, so the top-level flag can be true while the nested one is false — never the reverse.

Provenance — direct CT vs crt.sh (task #1033): each result row carries reconciled_from_crtsh:

  • false — at least one certificate naming this subdomain was observed directly in a CT-log crawl (log_id != 'crt.sh'). Wider than crt.sh.
  • true — every cert came via the crt.sh reconciliation path (stored log_id == 'crt.sh').
  • null — provenance was not recorded for any cert naming this subdomain: pre-existing rows ingested before source tracking was added, or the cached apex summary served the page. Treat as unknown, never as either boolean.

The envelope also carries sources — the distinct provenance categories present on this page: direct_ct, crt_sh, unknown. sources is omitted when the page is empty.

get/v1/domains/{domain}/subdomains

Request

  • Base URL: https://api.ctindex.io
  • URL: https://api.ctindex.io/v1/domains/{domain}/subdomains
  • Auth: one of:
    • API key in header X-API-Key
    • HTTP bearer

Path parameters

domainstring required

Query parameters

pageinteger
limitinteger
cursorstring nullable

Opaque keyset cursor (task #615 / GH #263). Pass cursor='*' (the coalescing-proof start token) to start a cursor enumeration, then the next_cursor from each response until it is null (cursor-mode pages ALWAYS carry the next_cursor key; an explicit null marks the terminal page). Cursor pages are ordered by subdomain name ASC (an immutable key), so pages are stable against concurrent inserts — unlike the default popularity (cert_count) ordering, which is offset-only. When cursor is supplied, page is ignored. Omit cursor entirely for the classic offset pagination (unchanged). Legacy: cursor= (empty string) also starts enumeration but may be coalesced to null by some transports.

sincestring nullable

ISO-8601 datetime — only return subdomains first seen at or after this time. Useful for incremental polling: pass the max first_seen from your last response. Also accepted under its agent-surface name, first_seen_after — identical filter; supplying both with different values is an error (GH #290).

first_seen_afterstring nullable

Alias of since, spelled as the MCP tools spell it (GH #290). Accepted so a client moving between the REST and agent surfaces cannot have its filter silently dropped.

as_ofstring nullable

ISO-8601 datetime — reconstruct the subdomain inventory as it was observed at this past point in time: only certificates first seen at or before as_of are counted, and last_seen is capped at as_of. Inventory is observation-based (CT sightings), so this is 'what CertIndex knew then', not a statement about what hosts were live then.

historicalboolean

When true, request deep-historical enumeration for this apex (requires paid backfill entitlement). Returns HTTP 402 with code: historical_not_entitled when not entitled.

Cookies

ctix_sessionstring nullable

Response

Successful Response

domainstring required
partialboolean

True when the hot Neon page was returned but the Tier-1 cold apex supplement did not complete (timeout, error, or budget). Hot results are still authoritative for this page.

cold_tierobject nullable

Tier-1 cold supplement status for this response. status is one of ok, omitted, timeout, or error; reason names the failure when present. Omitted on the happy path when cold was not consulted or succeeded.

total_subdomainsinteger nullable
total_is_lower_boundboolean

True when total_subdomains is a FLOOR rather than an exact count. Two independent causes, ORed — read lower_bound_reason to tell them apart:

  • query budget — the bounded row scan hit its cap (or the ~10s budget). The data IS indexed; this page did not fit. results_partial is also true. Remedy: repaginate with a smaller limit, or use cursor pagination.
  • coverage scope — the scan behind this inventory did not finish within the advertised product scope, did not establish what it reached (enumeration_scope=unknown), or never ran (no domain_backfills row). results_partial is FALSE. Remedy: none by retrying — an identical call returns an identical count. A zero total with this cause means "we cannot tell", not "none exist". Expired-only names on a completed active_only pass are product scope, not this cause.
  • backfill in progress — an on-demand backfill is still running (lower_bound_reason=backfill_in_progress). The inventory is growing. Remedy: re-query later. Distinct from coverage_scope so a caller does not apply "do not retry" to a live scan.
  • cold tier unavailable — cold was configured but unreachable (lower_bound_reason=cold_tier_unavailable). Retry later or treat as reduced scope.

The two are NOT co-set: a 6-subdomain domain can carry total_is_lower_bound: true with results_partial: false.

results_partialboolean

True only when THIS response's row scan was truncated by the server-side query budget — the page is ranked from a sample. It is NOT implied by total_is_lower_bound, which has a second, coverage-scope cause that leaves this field false.

lower_bound_reasonstring nullable

Why total_subdomains is a lower bound. query_budget = the bounded scan was truncated; the data is indexed, repaginate (smaller limit or cursor mode). coverage_scope = the scan did not finish within the advertised product scope, did not establish what it reached, or never ran (no domain_backfills row); retrying returns the identical count — do NOT retry. A zero total with this cause means "we cannot tell", not "none exist". Expired-only names on a completed active_only pass are product scope (enumeration_scope), not this cause. query_budget_and_coverage_scope = both; repaginating recovers the rest of the indexed page but the total stays a floor. backfill_in_progress = an on-demand backfill is still running; re-query later (do not treat as exact, and do not apply the coverage_scope "do not retry" rule). query_budget_and_backfill_in_progress = truncated page plus running backfill. cold_tier_unavailable = cold was configured but unreachable; retry later or treat as reduced scope. Absent when total_is_lower_bound is false.

has_moreboolean nullable
next_cursorstring nullable
coverageobject nullable
progressobject nullable

Live backfill counters while an on-demand scan is in flight (same snapshot as coverage.progress). Omitted when no in-progress scan is attached to this inventory.

total_subdomains_as_ofstring date-time nullable

Instant that total_subdomains describes, when that count was served from the cached apex summary rather than counted for this request (ISO-8601 UTC). Absence is meaningful: when this field is missing, total_subdomains was computed at request time. Any filter (since / first_seen_after / as_of) and cursor pagination bypass the cache, so those answers are always live and never carry this field. Compare two totals only when they carry the same stamp (or both carry none): a cached total can be SMALLER than a filtered subset counted live a moment later on a growing domain — that is two different instants, not a filter that added rows.

sourcesstring[] nullable
enumeration_scopestring

Product enumeration scope: active (active-only / active-since-ingestion / unknown internal scopes) or full (completed deep-historical backfill).

historical_statestring

Deep-historical lifecycle: disabled | backfill_pending | backfill_in_progress | full | refresh_paused.

corpus_depth_daysinteger nullable

Days from the earliest indexed certificate observation on this apex to now; null when no corpus is indexed yet.

Changes

No recorded changes to this operation across all 1 revision of this API.