Retrieval-Augmented Generation (RAG)

The original RAG architecture combined a language model with information retrieved from an external index during generation. Evidence for this claim The original RAG paper combined a pretrained sequence-to-sequence model with a non-parametric dense-vector index retrieved during generation. Scope: Lewis et al.'s 2020 RAG architecture and experiments, not every modern retrieval system. Confidence: high · Verified: 2026-07-14 Lewis et al.: Retrieval-Augmented Generation Modern platform documentation uses the same broad retrieve-then-generate idea. Evidence for this claim Google Cloud describes RAG as retrieving relevant information from external knowledge sources and providing it to a model to improve generated responses. Scope: General RAG architecture in Google Cloud documentation; quality depends on retrieval, source quality, and generation. Confidence: high · Verified: 2026-07-14 Google Cloud: RAG overview

TL;DR — RAG (Retrieval-Augmented Generation) is how AI search engines look things up before they answer. Instead of replying purely from memory, the system first retrieves relevant passages from a search index, then generates an answer based on what it found. That’s why Google AI Overviews, ChatGPT Search, and Perplexity can cite fresh web pages — and why being in the index still matters.

What RAG is

A large language model (an LLM, the thing behind ChatGPT and similar tools) learns from a huge pile of text during training. But that training has a cutoff date, and the model can’t possibly memorize everything — so on its own it either doesn’t know recent or niche facts, or it makes something up that sounds right.

RAG fixes that by letting the model look things up. When you ask a question, a RAG system does two things in order:

1. Retrieval — it searches an index (like Google’s or Bing’s) and pulls back the passages most relevant to your question.
2. Augmented generation — it hands those passages to the LLM, which writes an answer based on them and usually shows links to the sources.

The simplest way to picture it: instead of answering from memory alone, the AI does its homework first.

A quick example

Ask an AI search engine “what changed in the latest iPhone?” The model wasn’t trained on a product that launched last week. With RAG, it searches the web, retrieves a few recent articles, and writes its answer from those — with citations you can click. Without RAG, it would either say it doesn’t know or guess.

Why it matters to you

Here’s the part that surprises people: RAG doesn’t use a separate “AI index.” Google AI Overviews retrieve from Google’s normal search index. ChatGPT Search launched on Bing’s index and also runs its own crawler (OAI-SearchBot) — OpenAI hasn’t said exactly how the two are mixed today. Either way, the same basics that have always mattered — being crawlable, getting indexed, writing clearly — are exactly what decides whether your content can be retrieved and cited in an AI answer.

The other thing to know: RAG reduces wrong answers (hallucinations) but doesn’t eliminate them. The AI can still misread what it retrieved. So being the clearest, most direct source on a topic genuinely helps.

Want the real mechanics — embeddings, chunking, re-ranking, naive vs. agentic RAG, and the SEO playbook? Switch to the Advanced tab.

Lewis and colleagues’ 2020 system paired sequence generation with dense retrieval from a non-parametric index. Evidence for this claim The original RAG paper combined a pretrained sequence-to-sequence model with a non-parametric dense-vector index retrieved during generation. Scope: Lewis et al.'s 2020 RAG architecture and experiments, not every modern retrieval system. Confidence: high · Verified: 2026-07-14 Lewis et al.: Retrieval-Augmented Generation Google Cloud’s current overview defines RAG more broadly as supplying retrieved external knowledge to a model. Evidence for this claim Google Cloud describes RAG as retrieving relevant information from external knowledge sources and providing it to a model to improve generated responses. Scope: General RAG architecture in Google Cloud documentation; quality depends on retrieval, source quality, and generation. Confidence: high · Verified: 2026-07-14 Google Cloud: RAG overview

TL;DR — RAG is a two-phase, inference-time pattern: retrieval (find relevant passages in an external corpus) then augmented generation (feed those passages to an LLM to produce a grounded, cited answer). The weights never change — it combines the model’s parametric memory with non-parametric memory retrieved live. The retrieval phase chains chunking → embeddings → vector search → re-ranking → top-k. “Naive” RAG is retrieve-then-generate; advanced RAG adds query rewriting and re-ranking; agentic RAG adds iterative, multi-hop retrieval. Retrieval can ground answers but does not guarantee correctness; in one Gemma evaluation, insufficient context coincided with more incorrect answers. For SEO: there’s no separate AI index; crawlability, indexing, and passage-level clarity are the prerequisites for being retrieved.

The two phases (and why “inference time” is the whole point)

Break the acronym apart and you have the model: Retrieval plus Augmented Generation. A query comes in; the system retrieves the most relevant passages from an external corpus; it injects those passages into the LLM’s context window; the LLM generates an answer grounded in them.

The detail that everyone gets wrong: this happens at inference time, and the model’s weights are never touched. RAG is not training and it is not fine-tuning. The original 2020 paper from Patrick Lewis and colleagues at Facebook AI Research framed it as combining two kinds of memory — parametric memory (knowledge baked into the weights during training) and non-parametric memory (knowledge retrieved live from an index). RAG uses both at once. AWS puts the practical case plainly: retraining a foundation model for fresh or domain-specific knowledge is expensive, and “RAG is a more cost-effective approach to introducing new data to the LLM.”

(The naming, for what it’s worth, was an accident. Lewis later admitted: “We definitely would have put more thought into the name had we known our work would become so widespread… We always planned to have a nicer sounding name, but when it came time to write the paper, no one had a better idea.”)

Inside the retrieval phase

“Retrieve the relevant passages” is doing a lot of work in that sentence. In a real system it’s a pipeline:

1. Chunking. Documents get split into retrievable pieces. Chunk size is a real tradeoff — too small and a passage loses its context; too large and it floods the token budget with irrelevance. Strategies range from fixed token counts (100/256/512) to recursive/sliding windows to “Small2Big” (retrieve a small sentence, return its parent chunk for generation).
2. Embeddings. Each chunk is turned into a dense vector — a numeric representation of its meaning — so similarity is computed semantically, not by keyword match. This is why content about a topic gets retrieved even when it doesn’t use the exact query phrasing.
3. Vector search. The query is embedded too, and the system finds the chunks whose vectors sit closest to it. Most production stacks run hybrid search — dense vector retrieval plus BM25 keyword search — because each catches recall the other misses.
4. Re-ranking. A separate model re-scores the candidates by relevance to the query and reorders them, “effectively reducing the overall document pool.” Only the top survivors make it into the context.
5. Top-k into the prompt. The best passages are concatenated with the user’s query and handed to the generator.

Chunking is the fragile link. Anthropic identified that “traditional RAG solutions remove context when encoding information” — a chunk pulled out of its document loses the surrounding context that made it meaningful. Their Contextual Retrieval technique (prepending chunk-specific context before indexing) reduced failed retrievals by 49%, and by 67% combined with re-ranking. That’s a strong signal that the chunking problem is real — and that self-contained, context-rich passages are easier to retrieve correctly.

Naive, advanced, and agentic RAG

The survey literature (Gao et al., 2023) splits RAG into a useful taxonomy:

• Naive RAG — “a traditional process that includes indexing, retrieval, and generation.” Retrieve top-k once, generate once. It “struggles with precision and recall, leading to the selection of misaligned or irrelevant chunks.”
• Advanced RAG — adds “pre-retrieval and post-retrieval strategies.” Pre-retrieval: query rewriting and better indexing (including HyDE, where the model generates a hypothetical answer, embeds that, and retrieves documents that look like answers rather than questions). Post-retrieval: re-ranking and context compression.
• Modular / agentic RAG — the model retrieves, reasons about what’s still missing, and retrieves again, iterating across multiple hops. This is the current state of AI search. As Michael King put it: “The retrieve-once-then- generate pattern that defined the first wave is obsolete… Agentic RAG is now the default.”

This matters for SEO because content now has to survive multiple retrieval rounds and contradiction-checking — not just a single retrieval pass.

Does RAG eliminate hallucinations? No.

RAG can ground answers in retrieved sources, but the LLM can still misread or over-interpret what it pulled. Google Research (ICLR 2025) documented a counterintuitive result in one evaluation: Gemma produced incorrect answers on 10.2% of questions with no context and 66.1% with insufficient context. The researchers report that models can “excel with sufficient context but fail to recognize when context is insufficient.” Treat that as a model- and evaluation-specific warning, not proof that retrieval universally causes worse answers. The practical lesson is narrower: retrieval quality and context sufficiency need to be evaluated rather than assumed. Google operationalized the finding as an LLM re-ranker in its Vertex AI RAG Engine.

RAG vs. fine-tuning

These get conflated constantly, and they’re fundamentally different:

• RAG retrieves external information at query time. Weights unchanged. Best for fresh/changing information, citation requirements, and cost. The survey found “RAG consistently outperforms [unsupervised fine-tuning], for both existing knowledge encountered during training and entirely new knowledge.”
• Fine-tuning modifies the model’s weights in a separate training run. Best for changing style and behavior, or teaching stable domain knowledge that doesn’t change.

You’d reach for RAG to make a model know the latest facts; you’d reach for fine-tuning to change how it talks.

RAG in the wild: Google, ChatGPT, Perplexity

• Google AI Overviews. Google calls RAG “a technique (also known as grounding)… relying on our core Search ranking systems to retrieve relevant, up-to-date web pages from our Search index.” Two things follow. First, there is no separate AI index — “our generative AI features on Google Search are rooted in our core Search ranking and quality systems.” Second, Google runs query fan-out: “concurrent, related queries generated by the model to request more information.” A single question can spawn multiple sub-queries, each retrieving different content — so your content has to satisfy the implied sub-questions, not just the head query.
• ChatGPT Search. Launched (October 2024) with Bing as its data partner, and OpenAI’s own crawler documentation confirms OAI-SearchBot does independent fetching and indexing for search citations, separate from GPTBot’s training-crawl. OpenAI hasn’t published the current retrieval mix between Bing and its own index, and OpenAI has since positioned ChatGPT Search as a standalone competitor to Bing rather than a wrapper around it — so treat “it’s basically Bing” as a simplification. The documented, actionable lever is narrower and more durable: don’t block OAI-SearchBot in robots.txt, because that’s the crawler OpenAI itself names as the one that indexes content for search citations.
• Perplexity. Built on hybrid retrieval (Vespa.ai — BM25 + dense) with custom embedding models and a strict re-ranking threshold: by third-party analysis, only the top ~30% of 60-plus retrieved sources survive to the generation stage, and “citations are not retrofitted post-generation — they are structurally assigned during context assembly.” Deep Research runs the agentic loop across dozens of searches.

What RAG means for SEO

Strip away the jargon and the playbook is concrete:

• Being in the index is the prerequisite — full stop. No separate AI index means the crawl → index → retrieve chain has to be intact. If a page can’t be crawled and indexed, it can’t be retrieved into an AI answer. The same is true for the AI engines that build their own pools: AI crawlers like OAI-SearchBot and PerplexityBot have to be allowed to fetch you, or you’re invisible to those answers.
• Write self-contained passages. RAG retrieves fragments, not whole pages. As iPullRank’s Francine Monahan put it, AI systems examine “fragments of pages rather than the page as a whole” — so craft “stand-out passages and phrases” that answer a specific question on their own. This is exactly the H2/H3 structure and clear topic sentences good SEO already rewards. Google explicitly says not to chop your content into tiny pieces for AI — well-structured content chunks well on its own.
• Cover the sub-topics. Query fan-out means one question can trigger many retrievals. Depth across related sub-questions beats one page stuffed around a single keyword.
• Authority drives citation more than rank position. From an 8,000-citation analysis: “Strong organic search presence and broad web visibility leads to AI citations, not the other way around” — and “highly authoritative content from a lower-ranking page” sometimes gets cited over a less credible top-ranking one. My own data lines up (from my AI Overview citation research): mentions on heavily-linked pages are the strongest predictor of AI Overview inclusion (ρ ≈ 0.70), and branded web mentions correlated ~0.66 across 75,000 brands.
• Fresh content has an edge. AI citations skew meaningfully fresher than organic results, so currency matters.

If you want the one-sentence version: RAG didn’t replace SEO — it raised the stakes on the parts of SEO that were always about being findable and being clear.

AI summary

A condensed take on the Advanced version:

• RAG = Retrieval + Augmented Generation. Two phases at inference time: retrieve relevant passages from an external corpus, then feed them to an LLM to generate a grounded, cited answer. The model’s weights never change — it’s not training and not fine-tuning.
• It combines two memories: parametric (baked into weights) + non-parametric (retrieved live). That’s how AI answers cover information past the training cutoff.
• Retrieval is a pipeline: chunking → embeddings → vector search (often hybrid with BM25) → re-ranking → top-k passages into the prompt. Chunking is the fragile link; context-rich passages retrieve better (Anthropic cut failed retrievals 49%).
• Three flavors: naive (retrieve-once), advanced (query rewriting, HyDE, re-ranking), and agentic (iterative multi-hop) — agentic is now the AI-search default.
• It reduces, not eliminates, hallucinations. With insufficient context, one model’s hallucination rate jumped 10.2% → 66.1% — bad retrieval can beat no retrieval.
• RAG vs. fine-tuning: RAG for fresh/changing facts + citations + cost; fine-tuning for style/behavior and stable knowledge.
• Engines: Google AI Overviews retrieve from the core index (no separate AI index) with query fan-out; ChatGPT Search launched on Bing’s index and also runs its own crawler, OAI-SearchBot — the exact current mix isn’t published, so don’t block OAI-SearchBot; Perplexity via hybrid retrieval with a strict re-ranking threshold and citations assigned during context assembly.
• SEO: being crawlable + indexed is the prerequisite; write self-contained passages; cover sub-topics (fan-out); authority/E-E-A-T drives citation more than rank position; fresh content has an edge.

Official documentation

Primary-source documentation and definitions from the providers.

Google

• Google’s Guide to Optimizing for Generative AI Features — defines RAG as grounding over the core Search index; covers query fan-out.
• AI Overviews and AI Mode in Search — confirms no additional requirements beyond standard indexing and snippet eligibility.
• RAG and grounding on Vertex AI — Google Cloud’s retrieve-then-generate definition (Burak Gokturk).
• Deeper insights into RAG: the role of sufficient context — Google Research (ICLR 2025) on the insufficient-context failure mode.

Microsoft / Azure

• RAG and generative AI — Azure AI Search — RAG defined as grounding in proprietary content; query understanding, token constraints, and the move to agentic retrieval.

OpenAI

• Overview of OpenAI Crawlers — confirms OAI-SearchBot does independent fetching/indexing for ChatGPT Search citations, separate from GPTBot’s training crawl; doesn’t disclose the current mix with Bing’s index.

Anthropic

• Introducing Contextual Retrieval — the chunk-context-loss problem and a measured fix (49% / 67% fewer failed retrievals).

AWS

• What is Retrieval-Augmented Generation? — clean three-stage explainer and the RAG-vs-retraining cost argument.

Foundational papers

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., NeurIPS 2020 (the original RAG paper; parametric vs. non-parametric memory).
• Retrieval-Augmented Generation for LLMs: A Survey — Gao et al. (the naive / advanced / modular taxonomy, HyDE, re-ranking).

Quotes from the source

On-the-record statements from the providers and the original researchers. Deep links jump to the quoted passage where available.

Google — RAG is grounding, over the core index

• “A technique (also known as grounding) used to improve the quality, accuracy, and freshness of AI responses by relying on our core Search ranking systems to retrieve relevant, up-to-date web pages from our Search index.” — Google Search Central, AI optimization guide. Jump to quote
• “Our generative AI features on Google Search are rooted in our core Search ranking and quality systems.” — Google Search Central, AI optimization guide.

Google Cloud — the retrieve-then-generate definition

• “Retrieval Augmented Generation (RAG), a technique developed to mitigate these challenges, first ‘retrieves’ facts about a question, then provides those facts to the model before it ‘generates’ an answer – this is what we mean by grounding.” — Burak Gokturk, VP & GM, Cloud AI, Google Cloud (June 27, 2024). Jump to quote

The original RAG paper — parametric vs. non-parametric memory

• “retrieval-augmented generation (RAG) — models which combine pre-trained parametric and non-parametric memory for language generation.” — Lewis et al., NeurIPS 2020.

Patrick Lewis, lead author — on the name (via NVIDIA Blog, Rick Merritt)

• “We definitely would have put more thought into the name had we known our work would become so widespread.”
• “We always planned to have a nicer sounding name, but when it came time to write the paper, no one had a better idea.” Read the coverage

Microsoft — RAG as grounding in your content

• “Retrieval-augmented generation (RAG) is a pattern that extends LLM capabilities by grounding responses in your proprietary content.” — Microsoft, Azure AI Search documentation.

Anthropic — the chunking problem

• “traditional RAG solutions remove context when encoding information.” — Anthropic, Contextual Retrieval (Sept 19, 2024). Read the post

AWS — RAG vs. retraining

• “Retrieval-Augmented Generation (RAG) is the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response.” — AWS.
• “RAG is a more cost-effective approach to introducing new data to the LLM.” — AWS.

OpenAI — its own crawler for ChatGPT Search

• “OpenAI uses OAI-SearchBot and GPTBot robots.txt tags to enable webmasters to manage how their sites and content work with AI… a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training.” — OpenAI, Overview of OpenAI Crawlers. Read the docs

Michael King, iPullRank — the agentic shift (Search Engine Land)

• “The retrieve-once-then-generate pattern that defined the first wave is obsolete… Agentic RAG is now the default.” Read the coverage

RAG cheat sheet

The pipeline, end to end

query → [retrieval: chunk · embed · vector search (+BM25) · re-rank · top-k] → augment (passages into context) → generate (LLM writes grounded, cited answer)

RAG vs. fine-tuning

The three RAG generations

Engine retrieval pools at a glance

Fast facts

• RAG = Retrieval + Augmented Generation; coined in Lewis et al., 2020.
• It’s inference-time — weights never change.
• Hallucination isn’t solved: insufficient context took one model from 10.2% → 66.1%.
• Context-aware chunking cut failed retrievals by 49% (67% with re-ranking).
• Don’t pre-”chunk” your content for AI — clear H2/H3 structure chunks well on its own.

The mental models

1. Retrieve → Augment → Generate. Every RAG system is these three moves. When an AI answer is wrong, locate which stage failed: did it retrieve the right passages, did it pass enough context, or did the model misgenerate from good sources? Most AI-visibility problems are retrieval problems, not generation problems.

2. Parametric vs. non-parametric memory. The model has parametric knowledge (frozen in its weights, capped at its training cutoff) and non-parametric knowledge (retrieved live). Publishing content can’t touch the weights — but it can feed the live retrieval. That’s the entire reason SEO still applies to AI search.

3. RAG vs. fine-tuning is a knowledge-vs-behavior split. Need the model to know new or changing facts? RAG. Need to change how it behaves or writes? Fine-tuning. Don’t fine-tune to add facts that change weekly.

4. Retrieval quality is the bottleneck — and it cuts both ways. Better retrieval beats a bigger model. And insufficient retrieval can be worse than none. So the goal for your content isn’t just “get retrieved” — it’s “get retrieved as a sufficient, self-contained passage” that lets the model answer definitively.

5. The crawl → index → retrieve chain. There’s no separate AI index. If a page fails at crawl or index, it can never reach retrieval — for Google’s RAG or for AI engines building their own pools. Fix the chain first; optimize passages second.

Test yourself: Retrieval-augmented generation

Resources worth your time

My related writing & research

• What We Actually Know About Optimizing for LLM Search — Ahrefs’ write-up using my data: mentions on heavily-linked pages are the strongest predictor of AI Overview inclusion (ρ ≈ 0.70).
• Generative Engine Optimization — the SEO response to a RAG-powered search landscape.
• GEO? AEO? LLMO? What’s With All This AI SEO Stuff? — my Ahrefs Evolve 2025 talk on the AI search landscape and why the indexing prerequisite hasn’t changed.

The foundational papers

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020 (the origin).
• RAG for LLMs: A Survey — Gao et al. (the naive/advanced/modular taxonomy).

From others

• How AI Search Engines Work — Ryan Law (Ahrefs) on RAG as the grounding mechanism.
• Google AI Overviews: All You Need to Know — Ong & Law (Ahrefs) on RAG over the core index.
• What Is Retrieval-Augmented Generation? — NVIDIA (includes the Lewis naming anecdote).
• How Retrieval-Augmented Generation is Redefining SEO — Francine Monahan, iPullRank (passage-level optimization).
• Beyond RAG: why every AI search platform is now agentic — Michael King, Search Engine Land.
• How Perplexity AI Answers Work — Ishtiaque Ahmed, a technical breakdown of the retrieval/ranking/citation pipeline.
• How to get cited by AI: SEO insights from 8,000 AI citations — James Allen, Search Engine Land; authority and E-E-A-T drive AI citations more than rank position.
• How Perplexity uses Vespa.ai — Vespa.ai’s first-party account of Perplexity’s hybrid BM25 + dense retrieval architecture.
• Retrieval-augmented generation — Wikipedia — useful reference overview; covers RAG poisoning and the hallucination caveat.

Stats worth citing

• 10.2% → 66.1% hallucination jump — one model’s hallucination rate with insufficient retrieved context vs. no context at all; bad retrieval can beat no retrieval. Google Research, ICLR 2025. Source
• 49% fewer failed retrievals from context-aware chunking (Contextual Embeddings), rising to 67% when combined with re-ranking. Anthropic, 2024. Source
• ρ ≈ 0.70 — mentions on heavily-linked pages are the strongest predictor of Google AI Overview inclusion in my research; branded web mentions correlated ~0.66 across 75,000 brands. Source
• ~30% survival rate — by third-party analysis, only roughly the top 30% of 60+ retrieved sources clear Perplexity’s re-ranking threshold into the generation stage. Source
• RAG > unsupervised fine-tuning for knowledge tasks — “for both existing knowledge encountered during training and entirely new knowledge.” Source

Retrieval-Augmented Generation (RAG)

RAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights.

Related: Grounding, AI Search, Knowledge Cutoff

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is the framework that lets a large language model answer with information it never memorized during training. Instead of relying only on the patterns frozen into its weights, the system runs two phases: retrieval — find the most relevant passages from an external corpus (a search index, a knowledge base, the live web) — and augmented generation — pass those passages into the model’s context so it can write an answer grounded in real, citable sources. Coined in a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, RAG is now the mechanism powering Google AI Overviews, ChatGPT Search, and Perplexity.

The crucial thing to understand is that RAG is not training and not fine-tuning. It happens at inference time — the moment you ask a question — and the model’s weights never change. The original paper framed this as combining parametric memory (knowledge baked into the weights) with non-parametric memory (knowledge retrieved live). That’s why RAG is the cost-effective way to give a model fresh or proprietary information: you retrieve it per query instead of retraining the whole model.

Under the hood, the retrieval phase usually chains several steps: documents are split into chunks, each chunk is turned into an embedding, vector search finds the chunks closest to the query, a re-ranking pass reorders them by relevance, and the top-k survivors are handed to the model. Most production systems also blend in keyword (BM25) search for recall. “Naive RAG” is the simple retrieve-then-generate loop; advanced RAG adds query rewriting and re-ranking; agentic RAG — now the default in major AI search engines — retrieves, reasons about what’s missing, and retrieves again across multiple hops.

RAG reduces hallucinations but does not eliminate them. A model can still misread a retrieved passage, and Google research found that when the retrieved context is insufficient, hallucination rates can actually climb higher than with no retrieval at all. For SEO this is the whole point: there’s no separate AI index — being crawlable, indexed, and structured into clear, self-contained passages is the prerequisite for your content to be retrieved and cited.

Retrieval also isn’t guaranteed to happen at all. Production systems run a query classifier ahead of the retrieve step that decides, per query, whether to search — stable or well-known-to-the-model topics can get answered from parametric memory alone, with no retrieval and nothing to be cited. That’s a different measurement problem than “was my page retrieved”: see LLM visibility for why memory-driven answers don’t move the same way retrieval-driven ones do.

Related: Grounding, AI Search, Knowledge Cutoff

Changelog