Sign in
workflows

Create Request

Make a request to temporal worker :param request: crawl request data :param token_data: api token :return: response

post/workflows/request/crawl

Request body

crawl_idstring required
urlstring required
enginesEngineType[] required
depthinteger
stealthboolean
camoboolean
absolute_onlyboolean
keep_externalboolean
visit_externalboolean
max_workersinteger
batch_sizeinteger
max_urlsinteger
proxy_urlstring nullable
proxy_usernamestring nullable
proxy_passwordstring nullable
teardownboolean
session_idstring nullable
agenticboolean
wait_until'domcontentloaded' | 'load' | 'networkidle' | 'commit'

When to consider page load complete for web scraping operations

Response

Successful Response

urlstring required
workflow_idstring required

Changes

Changed in 10 of the 16 revisions of this API.4219

  • 666e807fc6d82See the full diff
    • ○

      added the new optional request property proxy_password

      new-optional-request-property

    • ○

      added the new optional request property proxy_username

      new-optional-request-property

  • 29225d7b3ea412See the full diff
    • ●

      removed the request property bucket_name

      request-property-removed

    • ○

      added the optional property detail/items/ctx to the response with the 422 status

      response-optional-property-added

    • ○

      added the optional property detail/items/input to the response with the 422 status

      response-optional-property-added

  • 46e7585b14b511See the full diff
    • ▲

      removed cartography__models__WaitUntil from the wait_until request property anyOf list

      request-property-any-of-removed

    • ○

      added WaitUntil to the wait_until request property anyOf list

      request-property-any-of-added

    This revision also has 2 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • e0272b0f7e0a1See the full diff
    • ○

      added the new optional request property wait_until

      new-optional-request-property

    This revision also has 3 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • d7ed14ab37e81See the full diff
    • ●

      removed the request property wait_until

      request-property-removed

    This revision also has 3 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • e0272b0f7e0a1See the full diff
    • ○

      added the new optional request property wait_until

      new-optional-request-property

    This revision also has 3 changes that name no endpoint, such as unreferenced schemas being removed. See the revision's changelog

  • 019c4555bd7f5See the full diff
    • ○

      added the new optional request property agentic

      new-optional-request-property

    • ○

      added the new optional request property proxy_url

      new-optional-request-property

    • ○

      added the new optional request property session_id

      new-optional-request-property

    • ○

      added the new optional request property teardown

      new-optional-request-property

    • ○

      added the new ASYNC_FLEET_STICKY enum value to the request property engines/items/

      request-property-enum-value-added

  • 83beb71baf381See the full diff
    • ○

      added the new optional request property camo

      new-optional-request-property

  • 1e457ec993431See the full diff
    • ○

      added the new optional request property stealth

      new-optional-request-property

  • 707c0eb9cee135See the full diff
    • ▲

      request property engines/items/ was restricted to a list of enum values

      request-property-became-enum

    • ▲

      the engines request property's minItems was increased to 1

      request-property-min-items-increased

    • ▲

      the engines/items/ request property type/format changed from object/ to string/

      request-property-type-changed

    • ○

      added the new FLEET enum value to the request property engines/items/

      request-property-enum-value-added

    • ○

      added the new FLEET_ASYNC enum value to the request property engines/items/

      request-property-enum-value-added

    • ○

      added the new FLEET_WORKFLOW enum value to the request property engines/items/

      request-property-enum-value-added

    • ○

      added the new SCRAPINGBEE enum value to the request property engines/items/

      request-property-enum-value-added

    • ○

      added the new ZENROWS enum value to the request property engines/items/

      request-property-enum-value-added