Canicrawl, a census of the web’s AI doors

Field notes

Canicrawl compares crawler directives across named domains rather than treating “AI access” as a single yes-or-no property. Its displayed records distinguish allowed, restricted, blocked, and unknown states for six bots, while category filters cover areas from classifieds and ecommerce to government and news. The snapshot also separates the 652 sites it could read from 348 sites it could not, making coverage limitations visible instead of hiding them.