Controlling Google's AI: how to use Google-Extended and the Search generative AI control

Most websites depend on attracting visitors through search engines. Especially in the new world of generative AI answers, they need to stay as visible as possible. This means that the options described below play no role for most website operators. There are, however, some very important exceptions, which we want to describe here. First, how it works in principle.

Contents

1. Google-Extended: foundational LLM training & grounding in Gemini apps
2. Search Generative AI Controls in Google Search Console
3. Combining Google-Extended & Search Generative AI Controls
4. Example strategy

1. Google-Extended: foundational LLM training & grounding in Gemini apps

Google-Extended is not a crawler of its own but a user agent token that can be used as a filter rule in robots.txt. Unlike classic robots.txt directives, which block crawling but do not retroactively remove pages already in the search index, Google-Extended acts as a subsequent usage filter for content Google has already crawled.

With it, you withdraw Google’s permission to use data for the following AI services:

• AI training of future Gemini LLMs: Takes effect from the next training run. Model generations that have already been trained, however, cannot retroactively “forget” what they have learned (no unlearning).
• Real-time grounding of AI answers: In the Gemini apps (such as the Gemini chatbot) and for Vertex AI APIs, the block takes effect immediately, because Google checks the permission live with every user query.

IMPORTANT: Using Google-Extended has no effect at all on indexing and ranking in classic Google Search.

A website can give Googlebot full access in order to rank in classic search and at the same time block Google-Extended in robots.txt, in order to prevent, quite generally, ending up in the training data of the Gemini LLMs and, quite explicitly, being used as an answer in the Gemini apps.

What Google-Extended does not prevent is real-time grounding in Google AI Overviews and Google AI Mode. For that, there are explicitly the Search Generative AI Controls in Google Search Console.

1.1 Global example for Google-Extended

robots.txt on www.example.com:

User-agent: *
Disallow:
User-agent: Google-Extended
Disallow: /

→ The content of www.example.com no longer flows into the training of future Gemini models and is no longer available to the Gemini apps and Vertex AI as a grounding source. Google AI Overviews and AI Mode are not affected: they populate their text answers and link sources via real-time grounding from the search index anyway. Classic search also remains unaffected: all user agents can still crawl everything and add it to their web indexes.

1.2 Local example for Google-Extended

robots.txt on www.example.com:

User-agent: *
Disallow:
User-agent: Google-Extended
Disallow: /news/

→ The content of the news section under www.example.com/news/ no longer flows into the training of future Gemini models and is no longer available to the Gemini apps and Vertex AI as a grounding source. All other areas of the website remain unaffected. Google AI Overviews and AI Mode are not affected either: they work via real-time grounding from the search index anyway. Classic search also remains fully accessible: all user agents can still crawl everything and add it to their web indexes, including all areas of the website.

2. Search Generative AI Controls in Google Search Console

Search Generative AI Control protects website content only from being displayed and linked in the generative AI search features and from being used as a grounding source there, but has no influence at all on the training of the Gemini language models.

IMPORTANT: The Search Generative AI Controls are new. Google initially introduced them only for UK properties, in response to a binding requirement from the British competition authority CMA. Since 31 August 2026 they have been rolled out globally.

2.1 Global example for the AI Controls

Search Generative AI Controls in GSC for www.example.com:

Exclude my site’s links and content from Search generative AI features

→ Google AI Overviews, AI Mode & AI features in Discover can no longer display the content at all or use it as a source.

2.2 Local example for the AI Controls

Search Generative AI Controls in GSC for www.example.com/news/:

Exclude Links and Content from my website for Generative AI Features in Google Search (for this, a separate property must be created in Google Search Console so that only this specific area of the website can be excluded via Exclude)

→ Google AI Overviews, AI Mode & AI features in Discover can no longer retrieve and display the content of the news section under www.example.com/news/ at all, neither in the text answers nor as link sources. Other areas of the website are not affected.

For the Generative Search AI Controls there is a complex inheritance logic at the parent-child property level, which can be used to control the effects more precisely. The CMA has obliged Google to deliver page-level settings by 3 March 2027.

3. Combining Google-Extended & Search Generative AI Controls

Both mechanisms, Google-Extended and Search Generative AI Controls, work on an opt-out basis. This means you have to actively exclude something. There are essentially four ways to combine these functions, shown here as a matrix:

Classic Search Only: blocks all AI tools (both block)	Search Only: blocks chatbots (Google-Extended blocks)
Blocked by Google-Extended Blocked by Search Generative AI Control Meaning: A complete barrier against Google’s AI services. Content is neither used for the long-term AI training of future models nor does it appear live in the Gemini apps or Vertex AI. At the same time, it is completely blocked for the AI search features (AI Overviews, AI Mode, AI features in Discover). In the classic search results (normal Google Search), however, the website remains fully indexed and visible.	Blocked by Google-Extended Meaning: Content is not used for future AI model training and is blocked in the Gemini apps. But: in Google Search (AI Overviews, AI Mode, AI features in Discover), the search AI may query the website live in the search index (real-time grounding) and display links prominently as clickable sources, which can still bring valuable AI search visibility.
Non-AI-Search: blocks only AI search, but no chatbots (AI Controls block)	Search & AI Search: blocks nothing (no block)
Blocked by Search Generative AI Control Meaning: The content is completely off-limits for the AI search results (AI Overviews, AI Mode, AI features in Discover). Google may not display any text or place any links for them. But: since Google-Extended is allowed, Google may continue to use the website texts in the background without restriction to train future Gemini language models and to ground answers in Gemini apps/Vertex AI.	Blocked by nothing Meaning: Google has full access to the website. The content flows into permanent model training and is used and linked live for real-time grounding in Gemini apps, Vertex AI and in all AI search results of Google Search.

4. Example strategy

The “walled garden” strategy maximises organic discoverability while consistently protecting premium content. Website operators use a permeable paywall (“flexible sampling”) that allows classic search engine crawlers to read the content and rank it in classic search, while human visitors are incentivised to sign up.

So that Google cannot generate free AI summaries from the content, website operators activate Google-Extended and the Search Generative AI Control in Search Console in parallel. This hybrid approach preserves visibility in classic search but prevents the content from being reused by AI answers. The classic Google Search Console performance report may change as a result, however, because AI feature metrics also flow into it.

Which setting fits your business model?

The two switches control different things, and most websites should leave both open. Which setting fits depends on the business model. Visibility in AI answers can also be shaped actively, not only blocked. With our GEO strategy, we work out together with you what is right for your business.

Book a free diagnosis call

Sources:

• https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
• https://support.google.com/webmasters/answer/16908024
• https://www.seroundtable.com/google-search-generative-ai-control-explained-41569.html
• https://www.seroundtable.com/google-search-generative-ai-controls-41653.html
• https://www.gov.uk/government/news/cma-secures-fairer-deal-for-publishers-and-improves-google-search-services-in-uk
• https://ppc.land/google-gives-site-owners-a-toggle-to-exit-ai-overviews-and-ai-mode/
• https://ppa.co.uk/cma-publishes-final-decision-on-google-search-publisher-controls
• https://developers.google.com/search/docs/appearance/ai-features?hl=de