Google
Google-Extended— crawlers from Google
Opt-out token for Bard/Gemini training data. Does not affect Search ranking.
Google-Extended is a separate robots.txt token that controls whether Google can use your content for training Gemini/Bard and the Vertex AI models. Critically, disallowing Google-Extended does NOT remove your site from Google Search — it only blocks training use. Most sites either allow both or allow Googlebot but disallow Google-Extended.
Vendor
Google
Category
Crawlers (training & indexing)
User-Agent
Google-Extended
Documentation
robots.txt snippets
Allow
User-agent: Google-Extended Allow: /
Disallow
User-agent: Google-Extended Disallow: /
FAQ
What is Google-Extended?
Google-Extended is a separate robots.txt token that controls whether Google can use your content for training Gemini/Bard and the Vertex AI models. Critically, disallowing Google-Extended does NOT remove your site from Google Search — it only blocks training use. Most sites either allow both or allow Googlebot but disallow Google-Extended.
What is the user-agent string for Google-Extended?
Google-Extended identifies itself with the user-agent token "Google-Extended". You can match it in robots.txt with "User-Agent: Google-Extended" and route nginx / log-analyzer rules against that token.
How do I allow Google-Extended in robots.txt?
Add the following block to your /robots.txt — this explicitly grants Google-Extended access:
User-agent: Google-Extended
Allow: /
How do I block Google-Extended in robots.txt?
Add the following block to your /robots.txt — note that well-behaved bots honor this, but not every crawler does:
User-agent: Google-Extended
Disallow: /
How can I check whether my site is ready for Google-Extended?
Run a free check at https://agentics.page — it audits whether your robots.txt allows the right bots, whether you publish llms.txt and JSON-LD structured data, whether your content is server-rendered, and whether Google-Extended can actually consume your site.
Is your domain ready for Google-Extended?
agentics checks whether your robots.txt allows the right bots, your llms.txt is in shape, your JSON-LD and SSR content are visible, and whether Google-Extended can actually use your domain.
Run free check →