---
title: "Changes to Crawl Website & Scrape Markdown"
method: POST
path: "/web/crawl"
---

# Changes to Crawl Website & Scrape Markdown

`POST /web/crawl`

> Every recorded change to this operation, newest first.

## Timeline

Changed in 6 of 26 revisions.

- **2026-08-27** `1febe5476266`: 1 info
- **2026-08-19** `d63b1aad8ff3`: 1 info
- **2026-08-08** `182f532c2552`: 1 info
- **2026-07-21** `3764d9c86493`: 4 info
- **2026-07-09** `de91b92d5fb7`: 2 info
- **2026-06-22** `4767df4f281f`: 6 breaking, 3 info

## Changes

- **2026-08-27** `1febe5476266`: 1 info
  - added the required property `cache_metadata` to the response with the `200` status
- **2026-08-19** `d63b1aad8ff3`: 1 info
  - the `waitForMs` request property default value `3500` was added
- **2026-08-08** `182f532c2552`: 1 info
  - added the optional property `results/items/metadata/allOf[#/components/schemas/PageMetadata]/headings` to the response with the `200` status
- **2026-07-21** `3764d9c86493`: 4 info
  - added the new optional request property `pdf/ocr`
  - added the new optional request property `tags`
  - added the new optional request property `zdr`
  - the `pdf` request property default value changed from `{"shouldParse":true}` to `{"shouldParse":true,"ocr":false}`
- **2026-07-09** `de91b92d5fb7`: 2 info
  - added the new optional request property `country`
  - added the new optional request property `settleAnimations`
- **2026-06-22** `4767df4f281f`: 6 breaking, 3 info
  - the `results/items/metadata` response's property type changed from `object` to no type for status `200`
  - removed the required property `results/items/metadata/crawlDepth` from the response with the `200` status
  - removed the required property `results/items/metadata/statusCode` from the response with the `200` status
  - removed the required property `results/items/metadata/success` from the response with the `200` status
  - removed the required property `results/items/metadata/title` from the response with the `200` status
  - removed the required property `results/items/metadata/url` from the response with the `200` status
  - added the non-success response with the status `404`
  - added the non-success response with the status `415`
  - added `#/components/schemas/PageMetadata, subschema #2` to the `results/items/metadata` response property `allOf` list for the response status `200`

---

[Operation](https://skmtc.dev/context/apis/context-dev/docs/web/crawl/post.md) - [API](https://skmtc.dev/context/apis/context-dev.md) - [Page](https://skmtc.dev/context/apis/context-dev/changes/web/crawl/post)