search

Planned multi-country news search: one query, localized and fanned out across two independent news indexes in a single call.

Plans your query into search angles, localizes each angle into the language of every requested country edition (one combined LLM call, fail-open), then fans out up to 12 parallel legs per engine and merges the articles into one deduplicated list. Two independent news indexes are available through engines: google (the default, 50 country editions) and bing (49 of those editions). They overlap very little in practice, so asking for both roughly doubles the distinct articles a query returns and is the way to get a text snippet on most rows: the google engine returns headline, publisher, link, image and an exact publish time but rarely a snippet, while the bing engine returns a description on effectively every row. Every article carries engine provenance and a published_at_precision of exact or approximate, because the two indexes do not report publish times with the same fidelity. Every leg reports truthful localization provenance: query_source says whether its keyword was translated, confirmed original, or a fallback_original after a localization failure. Supports streaming via Accept: text/event-stream (the plan_refined chunk always carries the expanded legs with per-leg provenance before any leg settles) and sync via Accept: application/json. Filters: publisher and from/to exact date windows (google engine only; they pin its legs to the primary source), time_range recency, depth per-leg article depth, sort result ordering, max_legs cost cap. Billing is metered and settles down to the work actually done; empty, skipped and failed legs bill 0.

get/search/news

Request

  • Base URL: https://www.socialcrawl.dev/v1
  • URL: https://www.socialcrawl.dev/v1/search/news
  • Auth: API key in header x-api-key

Query parameters

querystring required

News topic or question (1-500 chars). Boolean AND/OR/NOT and quoted phrases are supported; Google advanced operators (site:, intitle:, before:) are rejected.

countriesstring

CSV of ISO 3166-1 alpha-2 country codes to search (1-12 of the 50 supported editions, default US). Each country searches its news edition in its default language; duplicates collapse.

enginesstring

CSV of news indexes to search: google (default) and bing. Asking for both runs the same plan on each and merges the results, which roughly doubles distinct articles and adds a snippet to most rows. The bing engine serves 49 of the 50 editions (not CN) and is priced per article; publisher and from/to are google-only.

sort'relevance' | 'date'

Ordering of the merged articles: relevance (default, each engine's own ranking) or date (newest first, undated articles last).

time_range'day' | 'week' | 'month' | 'year'

Recency window (day | week | month | year, default day). Ignored when from/to is present.

fromstring

Lower publish-date bound, YYYY-MM-DD or Unix seconds. Pins every leg to the primary news source.

tostring

Upper publish-date bound, YYYY-MM-DD or Unix seconds. Pins every leg to the primary news source.

publisherstring

Bare publisher domain filter (e.g. bbc.com; no scheme or path). Pins every leg to the primary news source.

depthinteger

Articles requested per leg (10-100 in steps of 10, default 10). The google engine is priced per call, so depth is free there; the bing engine is priced per article and caps its own legs at 20.

max_legsinteger

Hard cap on billable legs (1-12, default 12). The upfront hold shrinks with it.

group'stories'

Optional. stories groups the articles that report the same event across editions, outlets and languages into data.stories[] ({ id, head_id, member_ids, size, outlets, outlet_count, countries, languages, first_seen, last_seen, judged }), where every id is an article's canonical_url (or url). items[] and its order are unchanged. data.story_grouping reports how many articles were judged and what it cost. An article that could not be judged is its own story with judged: false; if grouping is unavailable, stories is null and data._warnings says stories_unavailable. JSON responses only (ignored on the event stream). Grouping is on by default now (see judgments); group=stories asks for it explicitly with a longer time budget. Free.

judgments'on' | 'off'

Optional. on (default) or off. On, JSON responses also carry data.stories and data.story_grouping (see group), grouped under a short time budget; when grouping does not finish in time, stories is null and data._warnings says stories_pending (stories_unavailable when it failed). items[], their order and the price are unchanged. off returns the response without them.

Headers

Cache-Controlstring

Send no-cache to bypass the response cache and force a live fetch. Billed at the normal endpoint cost; the fresh result is written back to cache for the next caller. Only the no-cache directive triggers this. See the Response Schema guide for details.

Idempotency-Keystring

Optional UUID that makes the request safely retriable. The key is reserved atomically per account before billing. A concurrent retry receives 409 IDEMPOTENCY_IN_PROGRESS with a Retry-After of 1 second while the original is young, growing to a quarter of the time it has been running (at most 30); a crashed reservation expires after 5 minutes. A replay keeps the cached payload immutable except for billing metadata: credits_used becomes 0, idempotent_replay becomes true, and credits_remaining is refreshed to the current balance. A known current balance appears in both the body and X-Credits-Remaining header; no balance row resolves to 0. On a transient lookup failure, body credits_remaining is null and X-Credits-Remaining is omitted. Completed outcomes expire after 24 hours. If the claim store is unavailable, the request fails closed before billing. GET endpoints do not combine idempotent replay with Server-Sent Events: use Accept: application/json when sending this header. The unsupported key-plus-SSE combination is rejected before billing.

Response

Successful response

successboolean required

Whether the request succeeded

platformstring required

Platform name

endpointstring required

API endpoint path

credits_usedinteger required

Number of credits consumed

credits_remaininginteger nullable required

Current account balance, or null when this response could not establish one. Free cache hits always return null because they do not establish an authoritative post-request balance. An idempotency replay also returns null when its current-balance lookup fails. It is never a fabricated zero, so a zero here means the account really is empty. The authoritative reading is the one on a billed response, which comes straight from the deduction.

request_idstring required

Unique request identifier for support

cachedboolean required

Whether the response was served from cache

idempotent_replayboolean

True only when this response is an idempotency replay

Example response

{
  "success": true,
  "platform": "search",
  "endpoint": "/v1/search/news",
  "credits_used": 1,
  "credits_remaining": 4999,
  "request_id": "req-abc123",
  "idempotent_replay": true,
  "meta": {
    "hint": {
      "code": "search_multi",
      "message": "You searched \"matcha\" on 3 platforms one call at a time; /v1/search/multi returns each platform's own search rows in one call at the same price per platform, and a platform with no rows is not charged.",
      "method": "GET",
      "path": "/v1/search/multi?query=matcha&platforms=instagram,tiktok,youtube"
    }
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.