Observed arrival · 2026-10-22
Baken’s focused web index
Baken describes a web index built by its own crawler, with each passage tied to the page it came from.
○Open
⊠Login
$Paid
†Ads
✦Pretty
●Pro
◎Niche
◉Human
⚑Risk
ƒJS
- For
- Webmasters reviewing BakenBot’s crawl behavior
- Worth noticing
- The site says BakenBot honors robots.txt and Retry-After, and removes stored material after a robots.txt opt-out.
Field notes
The crawler section gives operators details they can check in their own logs, including the bot’s user-agent, IPv4 and IPv6 addresses, and reverse-DNS name. Baken says the bot observes Crawl-delay and Retry-After, uses ETag or Last-Modified on later visits when available, and removes stored material after an opt-out. The homepage does not demonstrate the index’s search interface.
Observed signals
Read the marks
Editorial observations of this landing page, not a rating.
○OpenPublic substance visible
✦PrettyNotable craft visible
◎NicheUnusually specific use
One card from the complete issue