This endpoint allows parsing the content on any page you specify and will return the structured content of the target page, including link URLs, anchors, headings, and textual content.
Request
- Base URL: https://api.dataforseo.com
- URL: https://api.dataforseo.com/v3/on_page/content_parsing/live
- Auth: HTTP basic
Request body
<em>URL of the content to parse</em><br><strong>required field</strong><br>URL of the page to parse<br>example:<br>https://www.fujielectric.com/
<em>custom user agent</em><br>optional field<br>custom user agent for crawling a website<br>example: <code>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36<br></code><br>default value: <code>Mozilla/5.0 (compatible; RSiteAuditor)</code>
<em>preset for browser screen parameters</em><br>optional field<br>if you use this field, you don't need to indicate <code>browser_screen_width</code>, <code>browser_screen_height</code>, <code>browser_screen_scale_factor</code><p> possible values:<br><code>desktop</code>, <code>mobile</code>, <code>tablet</code><p> <code>desktop</code> preset will apply the following values:<p> <code>browser_screen_width: 1920</code><br><code>browser_screen_height: 1080</code><br><code>browser_screen_scale_factor: 1</code><p> <code>mobile</code> preset will apply the following values:<p> <code>browser_screen_width: 390</code><br><code>browser_screen_height: 844</code><br><code>browser_screen_scale_factor: 3</code><p> <code>tablet</code> preset will apply the following values:<p> <code>browser_screen_width: 1024</code><br><code>browser_screen_height: 1366</code><br><code>browser_screen_scale_factor: 2</code><p> <strong>Note:</strong> to use this parameter, set <code>enable_javascript</code> or <code>enable_browser_rendering</code> to <code>true</code>
<em>browser screen width</em><br>optional field<br>you can set a custom browser screen width to perform audit for a particular device;<br>if you use this field, you don't need to indicate <code>browser_preset</code> as it will be ignored;<p> <strong>Note:</strong> to use this parameter, set <code>enable_javascript</code> or <code>enable_browser_rendering</code> to <code>true</code><p> minimum value, in pixels: <code>240</code><br>maximum value, in pixels: <code>9999</code>
<em>browser screen height</em><br>optional field<br>you can set a custom browser screen height to perform audit for a particular device;<br>if you use this field, you don't need to indicate <code>browser_preset</code> as it will be ignored;<p> <strong>Note:</strong> to use this parameter, set <code>enable_javascript</code> or <code>enable_browser_rendering</code> to <code>true</code><p> minimum value, in pixels: <code>240</code><br>maximum value, in pixels: <code>9999</code>
<em>browser screen scale factor</em><br>optional field<br>you can set a custom browser screen resolution ratio to perform audit for a particular device;<br>if you use this field, you don't need to indicate <code>browser_preset</code> as it will be ignored;<p> <strong>Note:</strong> to use this parameter, set <code>enable_javascript</code> or <code>enable_browser_rendering</code> to <code>true</code><p> minimum value: <code>0.5</code><br>maximum value: <code>3</code>
<em>store HTML of a crawled page</em><br>optional field<br>set to <code>true</code> if you want to get the HTML of the page using the <a href="/v3/on_page/raw_html/" target="_blank" rel="noopener noreferrer">OnPage Raw HTML endpoint</a><br>default value: <code>false</code>
<em>disable the cookie popup </em><br>optional field<br>set to <code>true</code> if you want to disable the popup requesting cookie consent from the user;<br>default value:<br><code>false</code>
<em>language header for accessing the website</em><br>optional field<br>all locale formats are supported (xx, xx-XX, xxx-XX, etc.)<br><strong>Note:</strong> if you do not specify this parameter, some websites may deny access; in this case, pages will be returned with the <code>"type":"broken</code> in the response array
<em>load javascript on a page</em><br>optional field<br>set to <code>true</code> if you want to load the scripts available on a page<br>default value: <code>false</code><br><strong>Note:</strong> if you use this parameter, additional charges will apply; learn more about the cost of tasks with this parameter <a href="https://dataforseo.com/help-center/cost-of-onpage-api-parameters" rel="noopener noreferrer" target="_blank">in our help article</a>; the cost can be calculated on the <a href="https://dataforseo.com/pricing/on-page/onpage-api" target="_blank" rel="noopener noreferrer">Pricing Page</a>
<em>emulate browser rendering to measure Core Web Vitals</em><br>optional field<br>by using this parameter you will be able to emulate a browser when loading a web page;<br><code>enable_browser_rendering</code> loads styles, images, fonts, animations, videos, and other resources on a page;<br>default value: <code>false</code><br>set to <code>true</code> to obtain Core Web Vitals (FID, CLS, LCP) metrics in the response;<br><strong>if you use this field, <code>enable_javascript</code>, and <code>load_resources</code> parameters must be set to <code>true</code></strong><br><strong>Note:</strong> if you use this parameter, additional charges will apply; learn more about the cost of tasks with this parameter <a href="https://dataforseo.com/help-center/cost-of-onpage-api-parameters" rel="noopener noreferrer" target="_blank">in our help article</a>; the cost can be calculated on the <a href="https://dataforseo.com/pricing/on-page/onpage-api" rel="noopener noreferrer" target="_blank">Pricing Page</a>
<em>enable XMLHttpRequest on a page</em><br>optional field<br>set to <code>true</code> if you want our crawler to request data from a web server using the XMLHttpRequest object<br>default value:<br><code>false</code><p> if you use this field, <code>enable_javascript</code> must be set to <code>true</code>;
<em>switch proxy pool</em><br>optional field<br>if <code>true</code>, additional proxy pools will be used to obtain the requested data;<br>the parameter can be used if a multitude of tasks is set simultaneously, resulting in occasional <code>rate-limit</code> and/or <code>site_unreachable</code> errors
<em>proxy pool</em><br>optional field<br>you can choose a location of the proxy pool that will be used to obtain the requested data;<br>the parameter can be used if page content is inaccessible in one of the locations, resulting in occasional <code>site_unreachable</code> errors<br>possible values: <code>us</code>, <code>de</code>
<em>return page content as markdown</em><br>optional field<br>if set to <code>true</code>, the markdown-formatted content of the page will be returned in the <code>page_as_markdown</code> field of the response;<br>default value: <code>false</code>
Example request
[
{
"url": "https://dataforseo.com/blog/a-versatile-alternative-to-google-trends-exploring-the-power-of-dataforseo-trends-api"
}
]Response
Successful operation
the current version of the API
general status code you can find the full list of the response codes here
general informational message you can find the full list of general informational messages here
total execution time, seconds
total tasks cost, USD
the number of tasks in the tasks array
the number of tasks in the tasks array returned with an error
Changes
- breaking
added to the
///////////////////////////////////response propertyoneOflist for the response status - breaking
added to the
//////////////////////////////response propertyoneOflist for the response status - info
removed from the
///////////////////////////////////response propertyoneOflist for the response status - info
removed from the
//////////////////////////////response propertyoneOflist for the response status
- breaking
- info
api operation id
OnPageContentParsingLiveremoved and replaced withContentParsingLive
- info