OpenAI
OAI-SearchBot— crawlers from OpenAI
Indexes pages for ChatGPT Search results.
OAI-SearchBot builds the index used by ChatGPT Search to surface fresh web results inside ChatGPT. Disallowing it means ChatGPT Search will not show your domain even when it matches a query. Distinct from GPTBot (training) and ChatGPT-User (on-demand browse).
Vendor
OpenAI
Category
Crawlers (training & indexing)
User-Agent
OAI-SearchBot
Documentation
robots.txt snippets
Allow
User-agent: OAI-SearchBot Allow: /
Disallow
User-agent: OAI-SearchBot Disallow: /
FAQ
What is OAI-SearchBot?
OAI-SearchBot builds the index used by ChatGPT Search to surface fresh web results inside ChatGPT. Disallowing it means ChatGPT Search will not show your domain even when it matches a query. Distinct from GPTBot (training) and ChatGPT-User (on-demand browse).
What is the user-agent string for OAI-SearchBot?
OAI-SearchBot identifies itself with the user-agent token "OAI-SearchBot". You can match it in robots.txt with "User-Agent: OAI-SearchBot" and route nginx / log-analyzer rules against that token.
How do I allow OAI-SearchBot in robots.txt?
Add the following block to your /robots.txt — this explicitly grants OAI-SearchBot access:
User-agent: OAI-SearchBot
Allow: /
How do I block OAI-SearchBot in robots.txt?
Add the following block to your /robots.txt — note that well-behaved bots honor this, but not every crawler does:
User-agent: OAI-SearchBot
Disallow: /
How can I check whether my site is ready for OAI-SearchBot?
Run a free check at https://agentics.page — it audits whether your robots.txt allows the right bots, whether you publish llms.txt and JSON-LD structured data, whether your content is server-rendered, and whether OAI-SearchBot can actually consume your site.
Is your domain ready for OAI-SearchBot?
agentics checks whether your robots.txt allows the right bots, your llms.txt is in shape, your JSON-LD and SSR content are visible, and whether OAI-SearchBot can actually use your domain.
Run free check →