Meta
Meta-ExternalAgent— crawlers from Meta
Used for Meta AI training and content fetches.
Meta-ExternalAgent is Meta's crawler for training Llama models and powering the Meta AI assistant across Facebook, Instagram, and WhatsApp. Honors robots.txt. Allow if you want Meta AI to be able to reference your content; disallow to opt out of training.
Vendor
Meta
Category
Crawlers (training & indexing)
User-Agent
Meta-ExternalAgent
Documentation
robots.txt snippets
Allow
User-agent: Meta-ExternalAgent Allow: /
Disallow
User-agent: Meta-ExternalAgent Disallow: /
FAQ
What is Meta-ExternalAgent?
Meta-ExternalAgent is Meta's crawler for training Llama models and powering the Meta AI assistant across Facebook, Instagram, and WhatsApp. Honors robots.txt. Allow if you want Meta AI to be able to reference your content; disallow to opt out of training.
What is the user-agent string for Meta-ExternalAgent?
Meta-ExternalAgent identifies itself with the user-agent token "Meta-ExternalAgent". You can match it in robots.txt with "User-Agent: Meta-ExternalAgent" and route nginx / log-analyzer rules against that token.
How do I allow Meta-ExternalAgent in robots.txt?
Add the following block to your /robots.txt — this explicitly grants Meta-ExternalAgent access:
User-agent: Meta-ExternalAgent
Allow: /
How do I block Meta-ExternalAgent in robots.txt?
Add the following block to your /robots.txt — note that well-behaved bots honor this, but not every crawler does:
User-agent: Meta-ExternalAgent
Disallow: /
How can I check whether my site is ready for Meta-ExternalAgent?
Run a free check at https://agentics.page — it audits whether your robots.txt allows the right bots, whether you publish llms.txt and JSON-LD structured data, whether your content is server-rendered, and whether Meta-ExternalAgent can actually consume your site.
Is your domain ready for Meta-ExternalAgent?
agentics checks whether your robots.txt allows the right bots, your llms.txt is in shape, your JSON-LD and SSR content are visible, and whether Meta-ExternalAgent can actually use your domain.
Run free check →