Concepts · plain language

What is an AI agent?

An AI agent is software that acts on someone's behalf— it visits websites, gathers content, books, codes, sends mail. The defining trait: it doesn't just answer, it does.

You already encounter agents every day, mostly without noticing. When ChatGPT answers a question with fresh web content, a browsing agent fetched that page seconds ago. When Claude's knowledge includes your company, a crawler like ClaudeBot read your site months earlier. When a developer's assistant fixes a bug, a coding agent edited the files.

For website owners this matters because agents are becoming a major share of your visitors. They decide — based on what they can read on your domain — whether you appear in AI answers, AI search, and agent-driven purchases. An agent that can't parse your site simply moves on.

The 48 agents that matter, in 6 families

Crawlers

GPTBot, ClaudeBot — build training corpora and search indexes

Browsing agents

fetch your page live at question time

Coding agents

Claude Code, Cursor — work in repositories

Browser agents

drive a real browser through multi-step tasks

MCP clients

connect to tools, data, and your MCP server

Outreach agents

send mail and schedule on a domain's behalf

Browse all 48 agents with robots.txt snippets →

FAQ

What is an AI agent in simple terms?
An AI agent is a program powered by an AI model that acts on someone's behalf instead of just answering questions. It visits websites, fills forms, books appointments, writes and runs code, or gathers content. The defining trait is action: an agent doesn't only talk, it does.
What kinds of AI agents exist?
The main families: crawlers (GPTBot, ClaudeBot — gather web content for training or search), search-time browsing agents (fetch your page live when a user asks a question), coding agents (Claude Code, Cursor — work in repositories), browser/computer-use agents (drive a real browser to complete tasks), MCP clients (connect to tools and data sources), and outreach agents (send mail and schedule on a domain's behalf).
How do AI agents identify themselves?
Well-behaved agents send a recognizable user-agent string with every request — for example "GPTBot" or "ClaudeBot". That string is what you target in your robots.txt to allow or block them. Some also publish official IP ranges and support verification standards like Web Bot Auth.
How do I control which agents access my website?
Through your robots.txt: each agent's user-agent token can be allowed or disallowed individually. Our agents directory lists the exact snippet for every major agent. Note that robots.txt is a convention — reputable agents honor it, bad scrapers may not.
What is the difference between an AI agent and a skill?
Agents act, skills are abilities. An assistant becomes an agent the moment it uses skills (MCP servers) to do things — browse, read files, query databases. See our agents-vs-skills explainer for the full picture.

Can agents read your website?

agentics checks free of charge whether the agents above can read, find, and trust your domain — robots.txt, llms.txt, JSON-LD, and email deliverability in one report.

Run free check →
What is an AI skill? →

The abilities you plug into an assistant.

Agents vs. skills →

The difference, side by side.