Start a website crawl

POST 
/api/v1/chatbots/:chatbotId/crawls

Crawls one or more website URLs into the chatbot's knowledge base (embedded as searchable vectors), mirroring the dashboard website-crawl flow. Provide a single url or a urls array (max 100; the App batches into ≤10-per-request worker calls). Asynchronous — returns 202 with a crawlId + jobId; poll GET /crawls to track progress. Refuses a second in-flight crawl for the same domain + version (409). Requires admin-level chatbot access (owner / team admin / per-chatbot admin) — plain team editors cannot crawl via this API. Required scope: documents:write.

Request

Responses

202

Crawl accepted and dispatched.

400

Bad Request — invalid input

401

Unauthorized — missing or invalid API key

403

Forbidden — key lacks required scope

404

Not Found

409

A crawl for this domain + version is already in progress (code crawl_in_progress), or the site is being moved to another knowledge scope (code scope_move_in_progress). Retry once it finishes.

429

Too Many Requests — rate limit exceeded

503

The site's knowledge scope could not be determined right now. Safe to retry.