clusters

Extract values, parameters in a JSON body

Pulls values out of the page source of the sites in the cluster, part by part: domains [offset, offset+limit). Call again with offset = next_offset until it is null. Costs extraction points: a site found in the index costs as many points as values found, 0.1 if none; when the day's points run out the call stops early (stopped: extract_quota_exceeded). format=csv gives one row per site with X-Next-Offset and X-Points-Left headers.

post/v1/clusters/{id}/extract

Request

  • Base URL: https://api.publicwww.com
  • URL: https://api.publicwww.com/v1/clusters/{id}/extract
  • Auth: HTTP bearer

Path parameters

idinteger required

Cluster id, as in the list.

Request body

presetsstring[]
regexstring[]
offsetinteger
limitinteger
skip_emptyboolean
format'json' | 'xml' | 'csv'

Response

Extracted values.

clusterinteger required
namestring
sizeinteger
offsetinteger required
scannedinteger required

Domains gone through in this call.

in_indexinteger
with_matchesinteger
next_offsetinteger nullable required

Pass as offset to continue; null when the cluster is done.

regexstring[] required

Expressions applied, presets resolved, one column each.

points_usednumber required
points_leftinteger required
stopped'extract_quota_exceeded' | 'time_limit'

Changes

No recorded changes to this operation across all 1 revision of this API.