The AI agents your domain should be ready for.
Curated list of the agents that crawl, browse, code, and send mail on behalf of your customers in 2026. Click any entry for a dedicated reference page with user-agent string, robots.txt snippets and vendor docs. Run a free check to see which of these can actually read, find, and act on your site.
Crawlers (training & indexing)
These bots crawl the open web to build training corpora or search indexes for AI assistants. Your robots.txt decides whether they may use your content.
Crawls the web for training ChatGPT models.
GPTBotIndexes pages for ChatGPT Search results.
OAI-SearchBotCrawler used to train Claude models.
ClaudeBotLegacy user-agent still used in places.
anthropic-aiOpt-out token for Bard/Gemini training data. Does not affect Search ranking.
Google-ExtendedCrawler used for TikTok/Doubao AI products.
BytespiderCrawls for Alexa and other Amazon AI services.
AmazonbotUsed for Meta AI training and content fetches.
Meta-ExternalAgentOpt-out token for Apple Intelligence training data.
Applebot-ExtendedOpen public web archive used by many LLM training datasets.
CCBotSearch-time browsing agents
These fetch your page on-demand when a user asks an assistant a question. They typically follow robots.txt and reveal a recognizable user-agent.
Fetches pages when a ChatGPT user follows a link or asks for a browse.
ChatGPT-UserOn-demand fetches from Claude during conversations.
Claude-Web, Claude-UserCrawls pages cited in Perplexity answers and search results.
PerplexityBotOn-demand fetches when a user asks Perplexity a question.
Perplexity-UserIndexer for You.com search and Pro answers.
YouBotUsed by Mistral's Le Chat browsing tools.
MistralAI-UserUsed to support DuckAssist answers.
DuckAssistBotCoding agents
Run on a developer's machine (or in CI). They open files, run shell commands, edit code. Often consume llms.txt and JSON-LD to understand a project.
Terminal coding agent from Anthropic. Pair with MCP servers for tool use.
VS-Code-based AI editor with agent mode and MCP support.
Open-source autonomous coding agent for VS Code.
Terminal pair-programming agent, git-native.
Agentic IDE with Cascade flow and MCP.
Cloud-hosted autonomous engineering agent.
Code-aware AI assistant with deep search integration.
Agentic mode in Copilot Chat with MCP support.
General-purpose assistants
End-user-facing assistants that browse, summarize, and call tools. Most ship MCP support; some expose their own user-agent for outbound fetches.
Browse + tools + MCP via Custom GPTs and Connectors.
First-class MCP client.
Browses via Google-Extended, tool use via Gemini API.
Citation-first answer engine, fetches pages live.
European AI assistant with web search and tools.
Realtime-leaning assistant integrated with X.
Bing-grounded assistant across Windows, M365, Edge.
Browser & computer-use agents
Drive a real browser (or computer) to perform multi-step tasks. They send standard browser user-agents and depend on accessible DOM + SSR content.
Multi-step browser agent driving Chromium.
Claude controlling a desktop via screenshots + actions.
Open-source library to build browser agents on top of any LLM.
Autonomous agent running real browser tasks.
Form-filling and workflow automation via vision-grounded browser control.
Personal browser agent platform.
MCP clients (Model Context Protocol)
These connect to your `.well-known/mcp.json` server and consume tools and resources. Publishing a working MCP server is the most direct way for agents to act on your behalf.
Reference MCP client; the original MCP host.
Native MCP server support in Responses API.
Reads MCP config from `.cursor/mcp.json`.
All ship MCP support out of the box.
MCP client built into the editor.
Outreach & sales agents
Send mail, schedule, and qualify on a domain's behalf. Their deliverability depends on your SPF / DKIM / DMARC / MTA-STS — exactly what agentics checks.
AI-powered cold-mail and inbox-warming platform.
Sales agent for personalized outbound at scale.
Sales engagement + AI-assisted prospecting.
End-to-end outbound stack with warmup, sending, and analytics.
AI-powered prospecting and enrichment workflows.
Are you actually ready for them?
Run a check against agentics.page and see your scores in 30 seconds: agent readiness, email deliverability, LLM grounding.
Run free check →