AI Search

OpenAI’s web-search tooling can look up current information and return cited answers. Evidence for this claim OpenAI's web search tool can retrieve current web information and return responses with inline citations and source metadata. Scope: OpenAI API web search; ChatGPT product behavior can differ. Confidence: high · Verified: 2026-07-14 OpenAI: Web search guide Google’s AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. features may broaden retrieval by running related searches across subtopics. Evidence for this claim Google says its AI search features may use query fan-out to issue multiple related searches across subtopics and data sources. Scope: Google AI Overviews and AI Mode; not every response necessarily uses fan-out. Confidence: high · Verified: 2026-07-14 Google: AI features and your website

TL;DR — AI search uses chatbots and AI features (ChatGPT, Perplexity, Google’s AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index.) to write you an answer instead of handing you a list of blue links. It reads a bunch of web pages and synthesizes them into one response, usually with a few citations. It’s growing fast, but Google still sends far more traffic than every AI tool combined — so this is a new channel to understand, not a reason to panic.

What AI search is

For about 25 years, search worked one way: you type a query, the engine hands you a ranked list of links, and you click one. AI search changes the last step. Instead of a list of links, you get a written answer — generated by an AI that read several pages and stitched their information together.

You’re already seeing it in a few places:

• Google AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. — the AI-written summary box at the top of many Google results, with a few cited links off to the side.
• ChatGPT, Perplexity, Gemini, Grok, Copilot — chatbots that can search the web and answer your question conversationally, with sources.

The plumbing behind almost all of them is the same idea: the AI does a few searches, reads the top results, and writes an answer based on what it foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore.. The technical name for that is RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. (retrieval-augmented generation), but you don’t need the jargon — just picture an assistant who Googles for you, reads the pages, and gives you the gist.

How it’s different from regular search

That last row is the one SEOs worry about: if the AI answers the question on the page, the user may never click through to your site.

Why you should care (even though it’s small)

Here’s the balance, and it’s important to hold both halves:

• AI search is still small. Google has about 90% of the search market. As Ryan Law put it (and I agree): “Google still sends far more traffic. And it’s still your best marketing channel.”
• But it’s growing fast and it converts. AI-referred traffic grew roughly 9.7x in a year. In Ahrefs’ June 2025 internal study, AI search visitors converted at 23x the rate of traditional organic visitors — they were 0.5% of traffic but 12.1% of signups during the measured 30-day period.

So: don’t bet the company on AI search yet, but don’t ignore it either. The single most useful thing you can do is build the kind of brand mentions and citations across the web that AI tools draw from. That, more than any technical trick, is what gets you quoted.

Want the deeper version — how RAG works, every major platform, the licensing deals, and my data from the Ahrefs Evolve talk? Switch to the Advanced tab.

OpenAI documents web retrieval with inline citations and source metadata. Evidence for this claim OpenAI's web search tool can retrieve current web information and return responses with inline citations and source metadata. Scope: OpenAI API web search; ChatGPT product behavior can differ. Confidence: high · Verified: 2026-07-14 OpenAI: Web search guide Google documents query fan-outQuery fan-out is the technique where an AI search system breaks a single user question into multiple related sub-queries, runs those searches concurrently, and synthesizes the retrieved results into one answer. Google confirms AI Overviews and AI Mode 'may use a query fan-out technique' issuing multiple related searches across subtopics. as a possible retrieval technique, not a universal step in every AI response. Evidence for this claim Google says its AI search features may use query fan-out to issue multiple related searches across subtopics and data sources. Scope: Google AI Overviews and AI Mode; not every response necessarily uses fan-out. Confidence: high · Verified: 2026-07-14 Google: AI features and your website

TL;DR — AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. synthesizes answers with LLMsA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4). + RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. instead of ranking links. The major surfaces — Google AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index./AI Mode, ChatGPT Search, Perplexity, Bing Copilot, Grok, Gemini — mostly run the same retrieve → read → generate → cite loop over different indexes, fed increasingly by licensing deals (Reddit, AP, major publishers). It’s growing fast (~9.7x in a year) and converts unusually well (23x for Ahrefs), but Google still owns ~90% of search and sends ~190x more traffic than ChatGPT. In Ahrefs’ 75,000-brand correlation study, the strongest correlate wasn’t on-page SEO — it was branded web mentions (0.664 with AI Overview brand visibility vs. 0.326 for Domain Rating). And most of the new acronyms (GEOGenerative Engine Optimization (GEO) is the practice of optimizing content and brand presence so AI-powered search engines and assistants — Google AI Overviews, ChatGPT, Perplexity — cite, recommend, or mention you when generating answers. Google's position is that it's still SEO., AEOAnswer Engine Optimization (AEO) is the practice of structuring content so engines deliver it as a direct answer — featured snippets, voice assistants, and AI search — rather than just a ranked link. Coined for voice search in 2018 and revived for the LLM era. Google's position is that it's still SEO., LLMOAI search optimization is the practice of making your brand and content visible, citable, and accurately represented across AI-powered search — Google AI Overviews, ChatGPT, Perplexity, Copilot. It's built on traditional SEO plus a heavier emphasis on off-site brand mentions and content AI systems can cite.) describe SEO and PR work you already know.

This is the pillar hub for everything AI search. I gave a full talk on this at Ahrefs Evolve 2025 — “GEO? AEO? LLMO? What’s With All This AI Stuff?” — and most of the data here comes from that research.

What AI search actually is

AI search is any search system that uses an LLM to generate a synthesized answer from multiple sources, rather than retrieving and ranking documents for you to pick from. It takes three shapes:

1. AI summaries on top of normal results — Google AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index., Bing Copilot answers.
2. Standalone answer engines that replace the results page — Perplexity, ChatGPT Search.
3. Conversational assistants that search on demand — ChatGPT with browse, Gemini, Grok.

The defining difference from traditional search is the last step. Classic search retrieves and ranks, then sends you to a page. AI search retrieves, reads, synthesizes, and writes — potentially without sending you anywhere.

How AI search works under the hood

There are three ways an AI system gets the knowledge it answers from, and modern AI search blends all three:

• Training data — the giant pre-training corpus (web crawls, books, code). It’s static until the next training run, so it’s stale by design and not where current facts come from.
• RAG (retrieval-augmented generation) — live web retrieval at query time. The system searches, fetches the relevant chunks, and feeds them to the LLM so the answer reflects the current web. This is the engine behind Perplexity, ChatGPT Search, AI Overviews, and Copilot.
• Licensed curated content — direct deals with publishers and platforms (more below).

In my Evolve talk I boil the model down to training data + RAG + temperature: the training gives general competence, RAG supplies the specific current facts, and a temperature setting controls how “creative” (read: unpredictable) the wording is. The practical takeaway is that being retrievable and citable at query time is what matters — your page doesn’t need to be “in the model,” it needs to be findable when the model goes looking.

Two RAG mechanics worth knowing:

• Query fan-outQuery fan-out is the technique where an AI search system breaks a single user question into multiple related sub-queries, runs those searches concurrently, and synthesizes the retrieved results into one answer. Google confirms AI Overviews and AI Mode 'may use a query fan-out technique' issuing multiple related searches across subtopics.. Instead of one search, the system issues several related searches across subtopics and blends the results. Google does this for AI Overviews and AI Mode; Perplexity’s Deep Research runs an agentic loop across dozens of searches and hundreds of sources.
• Citation vs. generation. The cited links you see aren’t necessarily the sources the answer was generated from — and the answer is generated, so it can be wrong even when the citations are real.

The major AI search engines

Google AI Overviews & AI Mode. Rolled out broadly in May 2024 (formerly SGE); AI Mode followed in 2025 for queries needing reasoning or comparison. It runs RAG + query fan-out over Google’s own index — and explicitly does not just pull the top organic results. AI Overviews appear on roughly 9.46% of desktop keywords (about 16% in the US) and, by Google’s own framing in my talk, on close to 19% of queries; by search volume that’s about 12.8% of all Google searches. The most-cited domains are Reddit, Wikipedia, and Quora. GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. is the crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index..

ChatGPT Search (OpenAI). Two bots, two jobs: OAI-SearchBot crawls for real-time search and does not collect training data; GPTBot is for training. That separation matters — you can allow ChatGPT to surface your pages while blocking training use, or vice versa. ChatGPT rewrites your question into targeted queries sent to search partners, and leans on licensed content (Reddit, AP, news publishers). It’s the dominant AI assistant by a wide margin (see market share below).

Perplexity. The original “answer engine” — a fusion of search engine and LLM. It runs its own crawler (PerplexityBot, which treats XML sitemapsAn XML sitemap is a UTF-8 file listing the canonical URLs on your site (with optional lastmod) so search engines can discover and prioritize them. It's a discovery and diagnostic aid, not a guarantee of indexing — and Google ignores its priority and changefreq tags. as the primary discovery path) and its own curated index, plus live retrieval per query through a multi-stage RAG pipeline (intent parsing → hybrid retrieval → multi-tier rerankingReranking is the second stage of a retrieval pipeline: after a cheap, broad first pass pulls a candidate set of documents or passages, a slower, more precise model re-scores and reorders that shortlist by true relevance before the results are served or handed to an LLM. → cited synthesis). Its Deep Research mode (Feb 2025) is an agentic loop. Usage grew from ~22M to ~45M active users across 2025, with hundreds of millions of monthly queries.

Microsoft Bing Copilot. Draws primarily from Bing’s web index with a retrieve → generate → cite flow, so pages that rank well in Bing are disproportionately likely to be cited. Microsoft shipped an AI Performance reportThe Google Search Console report that shows how your site actually performed in Google Search, built from real impressions and clicks. It reports four metrics — clicks, impressions, average CTR, and average position — and keeps the most recent 16 months of data. in Bing Webmaster ToolsMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility. (public preview, Feb 2026) that surfaces total citations, average cited pages, groundingGrounding is anchoring an AI model's answer to source documents it retrieves at the moment you ask — not to the patterns frozen into its weights during training. Retrieval-Augmented Generation (RAG) is the most common way to do it. queries, and page-level citation activity — the first real first-party visibility into AI citationsAn AI citation is the visible source link an AI answer engine shows next to its generated text — the clickable reference that credits the web page it used. A citation's presence is a separate thing from whether the cited page actually supports the statement, and from being retrieved (read behind the scenes) or merely mentioned (named without a link); citation is driven more by brand mentions and being retrievable than by traditional ranking..

Grok (xAI). Real-time search with exclusive access to X (Twitter) data — a source no other engine has. That live-social angle is its differentiator more than raw market share.

Google Gemini. Google’s assistant on Google’s index and knowledge graphThe Knowledge Graph is Google's database of entities — people, places, organizations, and things — and the factual relationships between them. It's separate from any single website's structured data: your schema markup is one of many possible inputs to the graph, not the graph itself., boosted by the Google–Reddit deal and a Google–AP news feed (Jan 2025). Small in my Sept 2025 data but growing fast since.

Others worth knowing: Apple Intelligence (not a standalone engine — routes some queries through ChatGPT), Meta AI (massive distribution across Facebook/Instagram/ WhatsApp, with a Reuters news deal), Amazon Alexa+ (voice-first, shopping-heavy), and You.com (an early entrant, now overshadowed).

Market share — who’s actually winning

From my Evolve 2025 research (Sept 2025), the AI assistant market was extremely top-heavy:

Meanwhile traditional search is still Google’s world: ~90.4% Google, ~4.08% Bing, the rest a rounding error. The headline is that two races are running at once — Google dominates search; ChatGPT dominates AI assistants — and the assistant standings keep shifting: an independent tracker, First Page Sage (report dated July 2026), put ChatGPT down to roughly the low-50s% and showed Claude as the biggest gainer; within that tracker’s own series Gemini actually drifted slightly down across the first half of 2026 (about 14% in January to 13.3% in June) — a different ranking than my Sept 2025 snapshot, and not directly comparable to it since the two use different tracking methodologies. Treat any single percentage here as a snapshot, not a constant, and check a live tracker if the exact number matters to you.

How much traffic does AI search actually send?

This is where you keep perspective. AI traffic is growing fast but still small:

• AI-referred traffic grew roughly 9.7x in a year (study of ~82K sites).
• In Ahrefs’ analysis of 76,000 sites, Google sent about 190x more website traffic than ChatGPT.
• But it converts. For Ahrefs, AI search is ~0.5% of traffic yet ~12.1% of signups — a 23x higher conversion rate than organic. AI visitors arrive with unusually high intent.

And AI Overviews are quietly eroding clicks: they reduce position-#1 CTR by about 34.5% (forecast 0.040 vs. actual 0.026). I project clicks could keep falling toward ~25% of current levels on affected queries. The page types that win AI traffic for us are free tools, product pages, and the homepage; the ones that lose are commodity informational pages the AI can fully answer itself.

Data sources and licensing deals

AI search increasingly runs on licensed data, not just crawled data:

• Reddit — licensed to OpenAI ($70M/yr) and Google ($60M/yr); Reddit reported $203M in total AI licensing revenue in 2024.
• News publishers — OpenAI has deals with News Corp ($250M+/5yr), AP, FT, Dotdash Meredith, Condé Nast, Hearst, the Washington Post, and more.
• Google + AP (Jan 2025), Reuters + Meta (Oct 2025), and Perplexity’s publisher revenue-share program round out the landscape.

Why it matters: licensed, structured sources (especially Reddit and major news) get a privileged position in AI answers, which is part of why those domains dominate citation counts.

The strongest optimization lever isn’t on-page SEO

The single most actionable finding from my talk: for appearing in AI Overviews, branded web mentions correlate far more strongly than classic authority metrics.

The strategic shift I push in the talk: move from “optimize your site” to “optimize how the internet talks about you.” Build the citations, studies, and third-party mentions that AI systems retrieve and trust.

A few more honest realities:

• llms.txtllms.txt is a proposed (not adopted) Markdown file at /llms.txt that gives AI systems a curated map of a site's most important pages. Proposed by Jeremy Howard in 2024, it's read mostly by coding agents like Claude Code — not search crawlers — and Google ignores it. does nothing for training. Google explicitly ignores it; no major AI uses it as a training or ranking signal. Don’t build your strategy around it.
• You don’t need special markup or “AI writing.” Google says standard SEO applies and that you shouldn’t “write in a specific way just for generative AI search.”
• Don’t reflexively block AI crawlersAI crawlers are bots from AI companies that fetch web pages to train language models, build AI-search indexes, or answer live user questions. They come in three categories, each with its own user-agent tokens and its own robots.txt controls.. Blocking means you can’t be cited — you’re opting out of a growing referral channel. (See AI crawlers for the training vs. AI-search vs. user-fetcher distinction.)
• The acronyms are mostly relabels. GEOGenerative Engine Optimization — visibility inside AI answer engines. (Generative Engine Optimization), AEO (Answer Engine Optimization), and LLMO (LLM Optimization) largely map onto content marketing, digital PR, and brand-building you already do. New names, mostly old work.

Where to go next

This hub is the map; the optimization playbook is its own deep dive. For the how-to — earning citations, building brand mentions, technical considerations, and what not to waste time on — head to AI search optimizationAI search optimization is the practice of making your brand and content visible, citable, and accurately represented across AI-powered search — Google AI Overviews, ChatGPT, Perplexity, Copilot. It's built on traditional SEO plus a heavier emphasis on off-site brand mentions and content AI systems can cite..

Related reading elsewhere on the site: AI crawlersAI crawlers are bots from AI companies that fetch web pages to train language models, build AI-search indexes, or answer live user questions. They come in three categories, each with its own user-agent tokens and its own robots.txt controls. (how the bots behind these tools work and how to control them), crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. (the discovery layer everything sits on), and AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. for the Google-specific feature in depth.

AI summary

A condensed take on the Advanced version:

• AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. = LLMA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4). + RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights.. It synthesizes a written, cited answer from multiple sources instead of returning a ranked list of links. Three forms: AI summaries on results (AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index., Copilot), standalone answer enginesAnswer Engine Optimization (AEO) is the practice of structuring content so engines deliver it as a direct answer — featured snippets, voice assistants, and AI search — rather than just a ranked link. Coined for voice search in 2018 and revived for the LLM era. Google's position is that it's still SEO. (Perplexity, ChatGPT Search), and assistants that browse on demand (Gemini, Grok).
• Under the hood: training data (general knowledge) + RAG (live, current facts) + query fan-outQuery fan-out is the technique where an AI search system breaks a single user question into multiple related sub-queries, runs those searches concurrently, and synthesizes the retrieved results into one answer. Google confirms AI Overviews and AI Mode 'may use a query fan-out technique' issuing multiple related searches across subtopics. (several searches blended). Being retrievable and citable matters more than being “in the model.”
• Platforms: ChatGPT leads AI assistants (81.13%, Sept 2025), then Perplexity (10.82%), Copilot (4.05%), Gemini (2.82%). Traditional search is still ~90% Google.
• Traffic reality: AI traffic grew ~9.7x in a year but Google sends ~190x more than ChatGPT. It converts unusually well — 23x for Ahrefs (0.5% of traffic → 12.1% of signups). AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. cut position-#1 CTR ~34.5%.
• Biggest lever = brand mentions, not on-page SEO. Branded web mentions correlate 0.664 with AI Overview visibility vs. 0.326 for Domain Rating. Shift to “optimize how the internet talks about you.”
• Don’t waste effort on: llms.txtllms.txt is a proposed (not adopted) Markdown file at /llms.txt that gives AI systems a curated map of a site's most important pages. Proposed by Jeremy Howard in 2024, it's read mostly by coding agents like Claude Code — not search crawlers — and Google ignores it. (Google ignores it), special “AI markup,” rewriting for AI, or blocking AI crawlersAI crawlers are bots from AI companies that fetch web pages to train language models, build AI-search indexes, or answer live user questions. They come in three categories, each with its own user-agent tokens and its own robots.txt controls. (that just opts you out of citations).
• GEOGenerative Engine Optimization — visibility inside AI answer engines. / AEO / LLMO mostly relabel existing content marketing, PR, and brand work.
• Next: the optimization playbookAI search optimization is the practice of making your brand and content visible, citable, and accurately represented across AI-powered search — Google AI Overviews, ChatGPT, Perplexity, Copilot. It's built on traditional SEO plus a heavier emphasis on off-site brand mentions and content AI systems can cite..

Official documentation

Primary-source documentation on how the engines handle AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity..

Google

• AI features in Google Search — how AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. and AI Mode work, and what controls (nosnippet, noindex) apply.
• AI optimization guide — Google’s stance that standard SEO applies, that llms.txtllms.txt is a proposed (not adopted) Markdown file at /llms.txt that gives AI systems a curated map of a site's most important pages. Proposed by Jeremy Howard in 2024, it's read mostly by coding agents like Claude Code — not search crawlers — and Google ignores it. is ignored, and that you shouldn’t write specifically for generative AI.
• In-Depth Guide to How Google Search Works — the crawl → indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. → serve pipeline AI features are built on.

Microsoft / Bing

• Introducing AI Performance in Bing Webmaster Tools (Feb 2026 public preview) — citations, average cited pages, groundingGrounding is anchoring an AI model's answer to source documents it retrieves at the moment you ask — not to the patterns frozen into its weights during training. Retrieval-Augmented Generation (RAG) is the most common way to do it. queries, and page-level citation activity.
• Generative AI and public websites (Copilot Studio guidance) — Microsoft’s structure/freshness recommendations and the “retrieve → generate → cite” flow.

OpenAI

• OpenAI crawlers documentation — the OAI-SearchBot (search) vs. GPTBot (training) distinction and how to control each.
• Introducing ChatGPT Search — how ChatGPT rewrites queries and uses search partners plus licensed content.

Quotes from the source

On-the-record statements from the search engines on AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity.. Each link is a deep link that jumps to the quoted passage.

Google — standard SEO applies

• “The best practices for SEO continue to be relevant because our generative AI features on Google Search are rooted in our core Search ranking and quality systems.” — Google Search Central docs. Jump to quote
• On llms.txtllms.txt is a proposed (not adopted) Markdown file at /llms.txt that gives AI systems a curated map of a site's most important pages. Proposed by Jeremy Howard in 2024, it's read mostly by coding agents like Claude Code — not search crawlers — and Google ignores it. and similar files: “Google Search ignores them.” Jump to quote
• “You don’t need to write in a specific way just for generative AI search.” Jump to quote

Microsoft / Bing — how to get cited

• “Bing respects all content owner preferences expressed through robots.txtA plain-text file at the root of a host that tells crawlers which URLs they may and may not request. It controls crawling, not indexing — a blocked URL can still be indexed if it's linked from elsewhere. and other supported control mechanisms.” — Bing WebmasterMicrosoft's free portal for monitoring and improving how a site appears in Bing search — the peer to Google Search Console, plus IndexNow instant indexing, richer backlink data, and keyword volumes. Because Bing's index also feeds Microsoft Copilot, it doubles as a window into AI-search visibility. Blog. Jump to quote

AI search platforms — cheat sheet

The major AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. surfaces at a glance. Market share is AI-assistant share from my Sept 2025 Evolve data — treat it as a fast-moving snapshot.

Fast facts

• Two bots, two jobs (OpenAI): OAI-SearchBot = search/retrieval (no training); GPTBot = training. Control them independently.
• Bing-ranking pages get cited by Copilot disproportionately — Bing index = Copilot groundingGrounding is anchoring an AI model's answer to source documents it retrieves at the moment you ask — not to the patterns frozen into its weights during training. Retrieval-Augmented Generation (RAG) is the most common way to do it..
• llms.txtllms.txt is a proposed (not adopted) Markdown file at /llms.txt that gives AI systems a curated map of a site's most important pages. Proposed by Jeremy Howard in 2024, it's read mostly by coding agents like Claude Code — not search crawlers — and Google ignores it.: ignored by Google; not a training/ranking signal anywhere.
• Strongest AI Overview correlate: branded web mentions (0.664) ≫ Domain Rating (0.326).

Replacing the search strategy with an AI-only strategy

AI answer enginesAnswer Engine Optimization (AEO) is the practice of structuring content so engines deliver it as a direct answer — featured snippets, voice assistants, and AI search — rather than just a ranked link. Coined for voice search in 2018 and revived for the LLM era. Google's position is that it's still SEO. are an additional discovery surface, not evidence that ordinary search disappeared. Keep crawlabilityCrawlability is how well search engine crawlers can discover, access, and fetch a site's pages. A crawlability issue is any technical condition — blocked access, broken links, server failures, or bloated URL inventory — that stops pages from reaching the index., indexability, useful pages, and traditional performance reporting intact while you add citation and mention measurement.

Calling every chatbot response “search”

A model answering from training, a model retrieving live web pages, and a search engine showing a generated summary have different evidence and controls. Record the platform, mode, date, prompt, citations, and whether live retrieval was actually used.

Optimizing for a secret AI ranking checklist

There is no universal AI-search score. Platforms retrieve different sources and change their answers across runs. Improve clear, verifiable passages and real authority, then measure on a defined prompt set instead of chasing unsupported tricks.

Counting a mention, citation, and click as the same outcome

A brand can be named without a link, cited without receiving a visit, or visited from a surface that analytics labels inconsistently. Keep those stages separate so visibility is not reported as traffic or revenue.

The RACE framework for AI search

• R: Reachable. Important content is crawlable, indexable, and available in the response or rendered form the retrieval system can use.
• A: Answerable. Individual sections answer a specific question clearly enough to stand alone when retrieved as a passage.
• C: Credible. Claims have identifiable sources, consistent entity facts, and real third-party corroboration where the topic calls for it.
• E: Evaluated. A versioned prompt set tracks retrieval, mentions, citations, referral visits, and business outcomes separately by platform.

The framework prevents “AI optimization” from becoming a bag of formatting hacks. Reachability and answerability create eligibility; credibility and evaluation tell you whether that eligibility becomes useful visibility.

Resources worth your time

My talk and slides

• GEO? AEO? LLMO? What’s With All This AI Stuff? — Ahrefs Evolve 2025 (YouTube) — the full talk this hub is built on. (Standing disclaimer applies: this is my understanding of these systems, not gospel.)
• Slides on SpeakerDeck — every chart and data point from the talk.

My related writing (Ahrefs)

• Insights From 56 Million AI Overviews — my May 2025 study of 55.8M AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. across 590M searches.
• What 12 Months of AI Search Traffic Tells Us About Conversions — the 23x conversion finding.
• AI Search Traffic by Page Type — which pages actually win AI traffic.
• Meet the New Web Crawlers: AI Bots Are Closing in on Search Engine Bots — the botA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index.-traffic side of AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity..

From others

• How AI Search Works (LLM search optimization) — Louise Linehan’s deep dive on citation patterns and freshness.
• r/TechSEO — the community where AI-search debugging gets hashed out.

From around the industry

• AI features in Google Search — developer docs — primary-source documentation on how AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. and AI Mode work and what controls apply.
• Google AI optimization guide — Google’s official stance: standard SEO applies, llms.txtllms.txt is a proposed (not adopted) Markdown file at /llms.txt that gives AI systems a curated map of a site's most important pages. Proposed by Jeremy Howard in 2024, it's read mostly by coding agents like Claude Code — not search crawlers — and Google ignores it. is ignored, no special markup needed.
• Introducing AI Performance in Bing Webmaster Tools (Feb 2026) — Microsoft’s first-party citation analytics: total citations, groundingGrounding is anchoring an AI model's answer to source documents it retrieves at the moment you ask — not to the patterns frozen into its weights during training. Retrieval-Augmented Generation (RAG) is the most common way to do it. queries, and page-level citation activity.
• OpenAI crawlers documentation — official breakdown of OAI-SearchBot (search/retrieval) vs. GPTBot (training) and how to control each independently.
• 2024 Publisher–AI deals timeline (Digiday) — running log of Reddit, news-publisher, and platform licensing deals shaping AI search data sources.
• AI traffic research study (SE Ranking) — large-scale study of AI-referred traffic trends across tens of thousands of sites.
• Perplexity RAG pipeline deep dive (ZipTie) — technical walkthrough of Perplexity’s multi-stage retrieval and rerankingReranking is the second stage of a retrieval pipeline: after a cheap, broad first pass pulls a candidate set of documents or passages, a slower, more precise model re-scores and reorders that shortlist by true relevance before the results are served or handed to an LLM. architecture.

Stats worth citing

• AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. traffic grew ~9.7x in 12 months (study of ~82K sites). The growth rate is the story, not the absolute size. Source
• Google sends ~190x more traffic to websites than ChatGPT. Keep AI search in perspective: it’s additive, not a replacement. Source
• AI search converts at 23x the rate of organic for Ahrefs — 0.5% of traffic but 12.1% of signups. Source
• AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. cut position-#1 CTR by 34.5% (forecast 0.040 vs. actual 0.026). Source
• AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. appear on ~12.8% of Google searches by volume; ~9.46% of desktop keywords (~16% in the US). My 55.8M-AIO study covered 590M searches. Source
• AI assistant market share (Sept 2025): ChatGPT 81.13%, Perplexity 10.82%, Copilot 4.05%, Gemini 2.82%. Source
• Branded web mentions correlate 0.664 with AI Overview visibility — more than twice Domain Rating’s 0.326. The biggest lever is how the internet talks about you. Source
• Reddit reported $203M in AI licensing revenue (2024) — including ~$70M/yr from OpenAI and ~$60M/yr from Google. Source

AI Search

AI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity.

Related: Crawling, AI Crawlers, AI Overviews, Indexing

AI Search

AI search refers to search systems that use large language modelsA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4). (LLMs) and retrieval-augmented generationRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. (RAG) to read multiple sources and generate a synthesized answer, instead of just retrieving and ranking a list of documents. It shows up in three forms: AI summaries layered on top of normal results (Google AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index., Bing Copilot answers), standalone answer enginesAnswer Engine Optimization (AEO) is the practice of structuring content so engines deliver it as a direct answer — featured snippets, voice assistants, and AI search — rather than just a ranked link. Coined for voice search in 2018 and revived for the LLM era. Google's position is that it's still SEO. that replace the results page (Perplexity, ChatGPT Search), and conversational assistants that search the web on demand (ChatGPT with browse, Gemini, Grok).

The core difference from traditional search is the last step. Traditional search retrieves and ranks documents and sends you to a page. AI search retrieves, reads, synthesizes, and writes a new response — often with citations, and often without sending you to any website at all.

Most AI search products work the same way under the hood: a RAG loop that issues live searches against an indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed., fetches the relevant chunks, and feeds them to an LLM that composes a cited answer. The model’s training data supplies general knowledge; the retrieved pages supply the current, specific facts. That split is why being crawlable and citable still matters — and why brand mentions across the web turn out to be the strongest correlate of showing up.

For SEOs, AI search is additive, not a replacement: Google still holds ~90% of search and sends far more traffic than any AI tool. But AI traffic is growing fast and converts unusually well, so it’s worth understanding where it comes from and how to earn citations.

Related: Crawling, AI Crawlers, AI Overviews, Indexing

Changelog