Webflow & Websites
11 min read

AI SEO in 2026: What Google Allows, How to Show Up in AI Answers, and Where the Limits Are

Magnifying glass made of colorful toy bricks next to a brick staircase, where a robot arm places the top brick
Image: AI-generated
In short

You can use AI for SEO: Google objects not to the tool but to masses of pages without added value, which can fall under its spam policy on scaled content. According to Google, regular SEO gets you into AI Overviews and AI Mode, with no llms.txt or special markup needed. In ChatGPT, Perplexity, and Claude, your visibility depends on letting their search bots read your site, while you can still block their training bots.

Google has revised its documentation on AI and search several times in 2026. Its guide to optimizing for generative AI search was last updated on July 10, 2026, and its guidance on AI-generated content on October 1, 2026. Search Console now has a dedicated report for the AI features in Search, which Google says it rolled out worldwide as of August 31, 2026. And OpenAI, Perplexity, and Anthropic document their crawlers (programs that fetch websites automatically) in enough detail that you can control each one separately.

Can you use AI to write SEO content?

Yes, as long as the end result is a page that helps readers. Google explicitly calls generative AI useful for researching a topic and adding structure to original content. The line is drawn by the spam policy on scaled content abuse: many pages generated mainly to manipulate rankings rather than to help users. AI is one of several examples listed there, alongside scraping (automatically copying other people's content), stitching together content from other pages without adding value, and pages that make little sense to a reader but contain search keywords.

The consequences are serious. According to Google, sites that violate its spam policies may rank lower or not appear in the results at all. Google detects violations through automated systems and, as needed, human review. A manual action, meaning a penalty imposed by Google staff, shows up in the Manual actions report in Search Console.

The second catch is errors. Language models don't look up facts. They predict a likely sequence of words. That's why Google warns about inaccuracies, also known as hallucinations, and calls it critical to check everything manually before you publish it. That includes title elements, meta descriptions, structured data, and image alt text.

For online stores, Google adds rules from Merchant Center (Google's tool for product data): AI-generated images must contain the IPTC DigitalSourceType metadata with the value TrainedAlgorithmicMedia. IPTC is a standard for metadata stored directly in the image file. AI-generated product titles and descriptions must be specified separately and labeled as AI-generated.

Where does AI actually help in day-to-day SEO?

With tedious work that a person can check quickly. Our take: the more similar items there are and the clearer the criteria for checking them, the more time AI saves.

Task What AI handles well What you need to check
Collecting topics Grouping questions from customer emails, sales notes, and search terms Whether customers really ask it that way
Title tags and meta descriptions Drafts for hundreds of existing pages in a single pass Length, accuracy, no duplicates
Alt text Descriptions for product photos and graphics Whether the text describes what is actually shown
Structured data Drafting the machine-readable details for a product, price, or event Whether every detail is also visible on the page
Technical analysis Sorting crawl exports, redirect lists, and server logs Spot checks on the live page

What you shouldn't hand off is the substance. Google advises creating content based on what you know about the topic and considering what experience you can bring to it. A language model doesn't know your prices, your projects, or the questions your customers typically ask. That is exactly what sets your page apart from ten others answering the same question.

Recurring steps can be automated. For example, a workflow in Make or n8n (two automation platforms) can create a draft meta description and alt text for every new CMS item and send it to a person for approval instead of publishing it directly. Workflows like this are one of our core services.

What does Google say about GEO, llms.txt, and optimizing for AI answers?

To Google, it's all SEO. Its guide says, in essence, that optimizing for generative AI search means optimizing for the search experience, so it is still SEO. From Google's point of view, GEO (generative engine optimization, meaning optimization for AI answers) is not a separate discipline.

According to Google, there are no additional requirements for the AI features in Google Search: a page must be indexed and eligible to be shown with a snippet (the short text excerpt in a search result). Google describes two features. AI Overviews quickly sum up a topic and link out for further reading. AI Mode is meant for questions that call for more exploration, reasoning, or complex comparisons.

The guide clears up these assumptions:

  • llms.txt and other AI files: An llms.txt is a text file meant to summarize a website for language models. According to Google, you don't need files like this, custom markup, or Markdown, because Google Search doesn't use them, not even for its AI features.
  • Bite-sized chunks: You don't need to break your content into small pieces. Google's systems recognize multiple topics on one page and show the relevant part.
  • Writing for machines: A special writing style is unnecessary. The systems understand synonyms and meanings.
  • Special schema: Structured data isn't required for generative AI search, and there is no special schema.org markup for it. It remains useful for rich results (enhanced displays in search results).
  • Collecting mentions: According to Google, inauthentic "mentions" across the web are less helpful than they might seem.

Google also names what does count: publicly accessible, crawlable content, the technical requirements of Search, a clean JavaScript implementation, little duplicate content, relevant images and videos, plus Merchant Center feeds and a Google Business Profile, which can make your products and services visible in AI answers too.

One detail matters for how you structure a page: AI Overviews and AI Mode can use "query fan-out." The system runs several related searches across subtopics and sources on its own and builds the answer from them. Our take: this means a page can get cited even if it doesn't rank near the top for the main question, as long as it answers one sub-question especially well. Sections that each answer one question completely help with this, without you having to chop up your text.

How do you show up in ChatGPT, Perplexity, and Claude?

Mainly by letting their search bots read your site. The providers distinguish between training, search, and fetches that a user triggers in a chat. OpenAI and Anthropic have a separate bot for each job. Perplexity has two and says neither of them collects content for training AI foundation models. OpenAI also runs OAI-AdsBot, which, according to OpenAI, checks ad landing pages for safety.

Provider Bot Job according to the provider If you block it
OpenAI OAI-SearchBot Showing websites in ChatGPT search Not shown in ChatGPT search answers
OpenAI GPTBot Collecting content for model training No training; according to OpenAI, search is independent of it
OpenAI ChatGPT-User Fetches for user actions in ChatGPT and Custom GPTs According to OpenAI, robots.txt may not apply
Perplexity PerplexityBot Showing and linking websites in Perplexity search results Perplexity recommends allowing it
Perplexity Perplexity-User Fetching pages when users ask a question According to Perplexity, it generally ignores robots.txt
Anthropic ClaudeBot Collecting web content for its models New content missing from training data
Anthropic Claude-SearchBot Improving search results for users Less visibility in search results
Anthropic Claude-User Fetching pages when users ask Claude Less visibility for user queries
Google Google-Extended Training future Gemini models and grounding No training of future Gemini models, no grounding in the Gemini app and Vertex AI; inclusion and ranking in Google Search are unaffected

Grounding means the model receives content from the Google Search index while it answers. According to Google's overview of its crawlers, Google-Extended affects neither inclusion nor ranking in Google Search, which means it doesn't control AI Overviews or AI Mode.

If you want to block the training bots but stay in search answers, these lines in your robots.txt (the file at yourdomain.com/robots.txt that tells bots what they may read) are enough:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

OAI-SearchBot, PerplexityBot, and Claude-SearchBot can then keep reading. Google-Extended, however, is not a pure training bot: if you block it, the Gemini app also stops using your content for grounding. If you want Gemini to use your content in its answers, leave that block out. Watch out: a block for all bots (User-agent: *) with Disallow: / applies to every bot that doesn't have its own block, including the search bots.

Whether to block the training bots is a trade-off. Our take: for company websites whose content is marketing (services, case studies, guides), there is little reason to. If you sell content, such as studies, paywalled expert articles, or databases, you have good reasons.

Also check your server logs to see whether the search bots visit at all, and check your firewall or CDN (a service that caches and delivers your site) for bot blocks you never consciously set up. To tell whether a bot is genuine, compare it against the IP lists that OpenAI, Perplexity, and Anthropic publish.

How do you measure whether you appear in AI answers?

On Google, with a new report in Search Console. On ChatGPT, Perplexity, and Claude, with spot checks and your web analytics. Google does not report clicks from its AI features separately.

The Generative AI performance report shows only impressions for AI Overviews and AI Mode, meaning how many times links to your site were shown there. You can break the data down by page, country, date, and device, but not by query. The same data is also part of the regular Performance report under the Web search type, mixed in with the classic results. Despite the worldwide rollout, Google says that not every property (a website set up in Search Console) has access to the report yet.

Here is how to get something out of it anyway: each month, note the pages with the most AI impressions and put their clicks from the Web report next to them. That is a rough approximation, because those clicks also include the classic results. If impressions rise while clicks fall, the AI may already be answering the question itself. What helps then is something on the page that no summary can replace: a calculator, a template, a pricing example, a direct contact person.

For ChatGPT, Perplexity, and Claude, two simple methods help:

  1. Test questions: Write down 10 to 20 questions your customers really ask and check them every month. Note whether you are cited, who gets mentioned instead, and which page serves as the source. Ask each question twice, because AI answers are not always the same.
  2. Web analytics: Visits from these services show up as referrals, for example from chatgpt.com or perplexity.ai, as long as the service passes along where the visit came from.

Where are the limits of AI in SEO?

In volume, in first-hand experience, and in control over what AI systems do with your content.

  • Volume backfires: A generator that churns out 300 city pages with only the place name swapped would be a typical case for the scaled content policy, going by Google's description. Better to have 20 pages with real information.
  • Experience can't be generated: Your own numbers, photos, project examples, and well-founded opinions have to come from a person. Without them, you end up with interchangeable mass-produced content, while Google recommends content based on your own experience.
  • No separate switch for AI Overviews: For control, Google only names nosnippet, data-nosnippet, max-snippet, and noindex. These are the same controls as for regular results: with nosnippet, you lose the text excerpt there too, and with noindex, the entire page.
  • robots.txt is not copy protection: For fetches triggered by users, it may not apply, according to OpenAI, and Perplexity says Perplexity-User generally ignores it. Anthropic, on the other hand, says its bots respect robots.txt and Crawl-delay (a directive that tells a bot how long to pause between two requests).

What does this mean for you?

Our take: you don't need a separate GEO budget. You need solid SEO, clear bot rules, and AI in the right places. Things you can do this week:

  1. Check your robots.txt: Allow the search bots, make a deliberate decision about the training bots, and document that decision.
  2. Open Search Console: Note the ten pages with the most AI impressions and put their clicks next to them.
  3. Set up test questions: 10 to 20 real customer questions, checked monthly.
  4. Use AI for the tedious work: Titles, meta descriptions, alt text, and analysis, always with sign-off from a person.
  5. Add something of your own to every important page: Price ranges, how your process works, figures from experience, your own photos, a contact person.
  6. Label your store data: For Merchant Center, embed the IPTC metadata in your AI image files and mark AI-generated product texts.

We use Claude, ChatGPT, and Perplexity in our own toolkit. If you want to set up workflows like these in Make or n8n, or make your Webflow site technically ready for AI answers, take a look at our Webflow services or get in touch.

Frequently asked questions

How quickly does a robots.txt change take effect in ChatGPT and Perplexity?

OpenAI says its systems need about 24 hours to adjust after a robots.txt update, and Perplexity says it can take up to 24 hours. So check the result the next day at the earliest.

Do I have to label AI-generated text on my website?

For regular content, Google only says that information about how a piece of content was created could give readers more context. In Merchant Center, on the other hand, Google requires AI-generated product data to be labeled. Whether any legal obligations apply to you beyond that is something to clarify with your legal advisor.

Should I also block ChatGPT-User and Perplexity-User?

Not reliably through robots.txt, because users trigger these fetches. If you block them on your server, the service can't open your page when someone asks about it specifically. Our take: for company websites, that rarely makes sense.

Do I need a special GEO tool?

Not to get started. Google warns against tools that promise ranking success or advertise supposedly internal Google metrics, because no third-party tool has access to Google's ranking or AI systems. Search Console, web analytics, and a fixed list of test questions are enough to begin with.

Sources

  1. Google Search Central: Optimizing for generative AI search
  2. Google Search Central: Guidance on using generative AI content on your website
  3. Google Search Central: Spam policies for Google web search
  4. Search Console Help: Generative AI performance report (Search)
  5. OpenAI: Overview of OpenAI Crawlers
  6. Perplexity: Perplexity Crawlers
This article was created with AI.
Tobias from blocks&colors
Tobias
CEO, blocks&colors
Got a project in mind?

Tell me what you're planning. You'll get an honest assessment within 24 hours.

Got it! We will get back to you asap!
Whoopsie! Something went totally wrong here... Maybe just send an email to hi@blocksandcolors.de? Sorry!
Or email me directly: hi@blocksandcolors.de