Anthropic
anthropic-ai— crawlers from Anthropic
Legacy user-agent still used in places.
anthropic-ai is an older user-agent string from Anthropic, kept around for backwards compatibility. New deployments use ClaudeBot, Claude-Web, or Claude-User. If you want to opt out of Anthropic training entirely, include anthropic-ai in your robots.txt alongside ClaudeBot.
Vendor
Anthropic
Category
Crawlers (training & indexing)
User-Agent
anthropic-ai
Documentation
robots.txt snippets
Disallow
User-agent: anthropic-ai Disallow: /
FAQ
What is anthropic-ai?
anthropic-ai is an older user-agent string from Anthropic, kept around for backwards compatibility. New deployments use ClaudeBot, Claude-Web, or Claude-User. If you want to opt out of Anthropic training entirely, include anthropic-ai in your robots.txt alongside ClaudeBot.
What is the user-agent string for anthropic-ai?
anthropic-ai identifies itself with the user-agent token "anthropic-ai". You can match it in robots.txt with "User-Agent: anthropic-ai" and route nginx / log-analyzer rules against that token.
How do I block anthropic-ai in robots.txt?
Add the following block to your /robots.txt — note that well-behaved bots honor this, but not every crawler does:
User-agent: anthropic-ai
Disallow: /
How can I check whether my site is ready for anthropic-ai?
Run a free check at https://agentics.page — it audits whether your robots.txt allows the right bots, whether you publish llms.txt and JSON-LD structured data, whether your content is server-rendered, and whether anthropic-ai can actually consume your site.
Is your domain ready for anthropic-ai?
agentics checks whether your robots.txt allows the right bots, your llms.txt is in shape, your JSON-LD and SSR content are visible, and whether anthropic-ai can actually use your domain.
Run free check →