Scraping

Submit Scrape Request

Ask for a scrape. Returns a job to poll.

Scraping fetches pages from a third party and can cost credits depending on the adapter - check GET /scrape/adapters before running this in a loop.

post/scrape/requests

Request

  • Base URL: https://be.graph8.com/api/v1
  • URL: https://be.graph8.com/api/v1/scrape/requests
  • Auth: HTTP bearer

Headers

Authorizationstring nullable
X-Target-Org-Idstring nullable

Request body

requirementstring required

What to scrape, in plain English.

staging_table_idstring nullable

Staging table to read from and write into.

source_columnstring nullable

Column holding the values to scrape from.

seed_targetsstring[] nullable

Cold start: add these as rows in source_column before scraping.

row_idsinteger[] nullable

Limit the run to these rows. Omit it to run every row; an empty list is rejected.

adapterstring nullable

Skip the planner and use this capability directly.

rescrapeboolean

Re-run rows that already have a result.

paramsobject
previewboolean

Hold the run behind a 5-row preview. Leave it on for a first run.

Response

Successful Response

dataSettingsPayload required

An open envelope for a settings object.

Open because an ontology concept and an SSO config both grow fields as the product does, and on a settings read a dropped field reads as "not configured" rather than "we did not tell you".

Changes

No recorded changes to this operation across all 2 revisions of this API.