Web probe
DomainArrivalsBot
Domain Arrivals probes newly registered domain names once a day to understand what's appearing on the internet. This page explains what the probe does and how to opt out.
User-Agent
DomainArrivalsBot/1.0 (+https://www.domainarrivals.com/bot)
What it does
- Makes a single HTTP GET request to the root URL of a newly registered domain.
- Reads the page title, meta description, and visible text.
- For sites that appear framework or JavaScript-driven, it may make a second visit using a headless browser to render the page and read the content that loads client-side.
- Does not follow deep links, submit forms, or interact with the page beyond reading its content.
- Each domain is probed at most once per day, and only in the days immediately after registration.
- Results are used to identify interesting new sites for the Domain Arrivals daily issue.
What it does not do
- It does not crawl or index multiple pages of your site.
- It does not store personal data from your pages.
- It does not click, scroll, submit forms, or interact with your site beyond reading its content.
- It does not re-probe domains that are already established — only newly registered ones.
How to opt out
Add the following to your robots.txt file:
User-agent: DomainArrivalsBot Disallow: /
Or email dstaff4@gmail.com and we'll exclude your domain manually.
About Domain Arrivals
Domain Arrivals is a daily field guide to newly registered internet domains — a reading of what people are building, registering, and occasionally abandoning. The probe is how we discover what's alive.