Start a website crawl

POST 
/api/v1/chatbots/:chatbotId/crawls

Crawls one or more website URLs into the chatbot's knowledge base (embedded as searchable vectors), mirroring the dashboard website-crawl flow. Provide a single url or a urls array (max 100; the App batches into ≤10-per-request worker calls). Asynchronous — returns 202 with a crawlId + jobId; poll GET /crawls to track progress. Refuses a second in-flight crawl for the same domain + version (409). Requires admin-level chatbot access (owner / team admin / per-chatbot admin) — plain team editors cannot crawl via this API. Required scope: documents:write.

Request

Responses

202

Crawl accepted and dispatched.

400

Bad Request — invalid input

401

Unauthorized — missing or invalid API key

403

Forbidden — key lacks required scope

404

Not Found

409

A crawl for this domain + version is already in progress.

429

Too Many Requests — rate limit exceeded