Sources

Get source stream catalog

Changed on

Start discovering the data streams of a source. Poll the returned operation; its result (GET /operations/{operationid}/result) is a dto.DiscoverCatalogResponse in one format: available_streams + selected_streams for a split catalog, otherwise streams_config.

post/api/v1/project/{projectid}/sources/streams

Request

  • The document declares no server URL.
  • Auth
    none declared

Path parameters

projectidstring required

project id (default is 123)

Request body

configstring required
job_idinteger required
job_namestring required
max_discover_threadsinteger
namestring required
target_query_enginesstring[]

TargetQueryEngines become each stream's available_update_types; OLake stores none, so resend every discover.

typestring required
versionstring required

Example request

{
  "config": "{\"host\":\"localhost\",\"port\":5432}",
  "job_id": 1,
  "job_name": "my-sync-job",
  "max_discover_threads": 50,
  "name": "my-postgres-source",
  "target_query_engines": [
    "spark",
    "duckdb"
  ],
  "type": "postgres",
  "version": "v0.2.7"
}

Response

Accepted

Changes