Get Scrapes Metadata

get/list_scrapes

Request

Base URL
https://api.datafuel.dev
Auth
HTTP bearer

Query parameters

job_idstring nullable

Optional job ID to filter scrapes

markdownboolean

If true, return markdown data on top of the signed_url

ai_responseboolean

If true, return AI response data on top of the signed_url

urlstring nullable

If provided, filter scrapes by the given URL

Response

Successful Response

job_idstring required

The unique identifier for the scraping job that can contain multiple scrapes

scrape_idstring nullable

The unique identifier for the each URL scrape

scrape_status'pending' | 'failed' | 'success' required

scrape_urlstring required

The URL that was scraped

scrape_timestampstring required

The timestamp of when the scrape was created

signed_urlstring nullable

Pre-signed URL containing the scraping results in JSON format with structure: { "ai_response": "AI-generated analysis or structured data", "markdown": "Content formatted in markdown", "html": "Raw HTML content from the scraped page" }

ai_responsestring nullable

AI-generated analysis or structured data

markdownstring nullable

Content formatted in markdown

job_status'pending' | 'finished' required

Example response

[
  {
    "job_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
    "job_status": "finished",
    "scrape_id": "550e8400-e29b-41d4-a716-446655440000",
    "scrape_status": "success",
    "scrape_timestamp": "2024-03-15T10:30:00Z",
    "scrape_url": "https://example.com",
    "signed_url": "https://docyjgyvimrauivbukcp.supabase.co/storage/v1/object/sign/scrapes-data/..."
  }
]

Changes

No recorded changes to this operation across all 1 revision of this API.