Unblock Google-Extended for Gemini visibility
Your robots.txt blocks Google-Extended, the token Google uses to control whether your content can feed Gemini and Google’s generative AI features. Blocking it doesn’t affect your classic Google Search rankings - but it does limit your presence in the AI experiences Google is steadily putting in front of search users.
What exactly is Google-Extended?
It’s not a separate crawler - Googlebot does the fetching either way. Google-Extended is a robots.txt control token: disallowing it tells Google “you may index this for Search, but don’t use it for Gemini training and grounding.” Many sites toggled it off during 2023’s AI backlash and forgot.
Should you allow it?
If you want your business represented in Gemini’s answers - and in AI-assisted results shown to billions of search users - yes. The trade-off arguments that apply to paywalled publishers rarely apply to businesses trying to be found.
How do you allow it?
Remove the disallow, or add an explicit group:
User-agent: Google-Extended
Allow: /Deploy to your web root and rescan - it’s usually a one-line deletion. CCBot rode along on most of the blocklists that caught this token, so glance at that rule too.
See where your site stands. The free scan takes about fifteen seconds and shows every fix.
Run a free AI visibility scan