IndieCrawl asks who can crawl your site

Field notes

The interface accepts a domain or an exact page URL and organizes the inspection as a “crawl map.” It names three sources of crawler behavior—robots.txt, robots meta, and X-Robots-Tag—and links the interpretation to RFC 9309. The page also states that a check does not become public inventory and that lookup history is not stored, making the privacy constraint unusually visible for a small diagnostic utility.