Publisher information

kbDrop web crawler

kbDropBot/1.0 (+https://kbdrop.io/crawler) fetches public pages only when a kbDrop user asks to build a private knowledge base from that site.

Publisher controls

The crawler follows RFC 9309 robots.txt rules, declared sitemaps, crawl delays, HTTP retry guidance, and strict page, depth, time, and byte limits. It does not bypass authentication, paywalls, CAPTCHAs, or anti-bot controls.

Safety and traffic

Requests use a recognizable user agent and are spaced out per site. Rendering runs isolated, and every navigation, redirect, and subresource is restricted to public HTTP(S) destinations.