AHXBot

AHXBot is the crawler behind AHX SEO Labs site audits. It identifies itself with this user agent:

Mozilla/5.0 (compatible; AHXBot/1.0; +https://seo.ahxlabs.com/bot)

When it visits

Only when someone who manages the site in Google Search Console starts an audit of it. AHX checks that access with Google before every audit. Audits of an uploaded list of URLs accept only URLs on that same site. AHXBot does not crawl the web on its own and does not build a search index.

How it behaves

It follows robots.txt. Rules for User-agent: AHXBot apply to it; without them it follows the rules for all crawlers (*).
It makes at most two requests at a time to a site, with a pause after each (about 2 to 3 requests a second at most), and waits up to 30 seconds for a page.
When a site answers 429 or 503 with Retry-After, it pauses, waits longer each time it is asked (up to 15 minutes), and stays slower for the rest of the audit.
It stops when a firewall asks for a browser check or a CAPTCHA, and does not try to get around it.
Besides the pages it audits, an audit makes a few test requests to the same site: about 20 to addresses that should not exist and to variants of real pages (to find soft 404s and duplicate URLs), up to 50 to follow internal redirects one step at a time, and one homepage request with the user agent of each AI crawler it reports on (for example OAI-SearchBot or PerplexityBot), so the report can say whether your firewall lets them in. Those requests come from the same address as AHXBot.
For accessibility checks it opens a small sample of the pages it already fetched (up to 25 per audit) in a headless browser with the AHXBot user agent, so the page's own scripts run. There it does not download images, video or fonts, and it blocks common analytics and advertising services. Analytics that a site runs itself, or on its server, may still record these visits.

Where it connects from

AHXBot connects from 15.252.79.75. The same list, for automatic updates, is at /bot/ips.json. If the address ever changes, both are updated first. A request that says AHXBot but comes from another address is not AHXBot.

Letting it through your firewall

You only need this if your firewall blocks or slows down audits you started. Match both the user agent and the address, never the user agent alone. In Cloudflare, a WAF custom rule:

(http.user_agent contains "AHXBot" and ip.src in {15.252.79.75})

Action Skip, choosing only what gets in the way of audits, usually rate limiting rules and Super Bot Fight Mode. Choose “all remaining custom rules” only if one of your own rules blocks AHXBot. Bot Fight Mode on Cloudflare's Free plan can't be skipped by a rule; turn it off while auditing. An exception like this lets AHXBot reach public pages without being challenged. It does not let it past logins or into anything private, and AHXBot keeps the pace described above.

If you can't allow by address, the site owner can create a scan key on the AHX site audit page instead: a random secret for that one site, sent as the X-AHX-Scan-Key header to that site only. A firewall rule must compare the header with that exact value (the rule AHX gives you does this); a rule that only checks the header is present would let anyone through. Creating a new key replaces the old one.

Blocking it

Add this to robots.txt:

User-agent: AHXBot Disallow: /

You can also block its address or user agent in your firewall. Questions: support@ahxlabs.com. More about AHX SEO Labs on the home page.