Tokens and Context Windows

TL;DR — A token is the little piece of text an AI reads — usually a chunk of a word, roughly ¾ of an English word. A context window is how much text an AI can hold in its head at once (its short-term memory). Both matter for search because search systems do not routinely hand a model every page or an entire website. They usually grab only the relevant parts before they answer.

What a token is

Language models process text as tokens, which may be whole words or smaller character sequences depending on the tokenizer. Evidence for this claim OpenAI models process text as tokens, and token boundaries may be whole words or parts of words. Scope: OpenAI tokenization; token counts depend on the model-compatible encoding. Confidence: high · Verified: 2026-07-14 OpenAI: What are tokens? A context window is a model-specific limit on the tokens available to a request and response, and published limits can change by model version. Evidence for this claim Context-window limits are documented per model and can differ across model versions. Scope: OpenAI model metadata; published limits are product-specific and can change. Confidence: high · Verified: 2026-07-14 OpenAI: Models

Large language models — the tech behind ChatGPT, Gemini, and AI Overviews — don’t read text the way you do, word by word. They break it into tokens: small units that are often part of a word rather than a whole one. Google’s rule of thumb for its Gemini models is that one token is about 4 characters, and 100 tokens is roughly 60–80 English words. So a token is a bit less than a full word on average.

The process of chopping text into tokens is called tokenization, and the full set of tokens a model knows is its vocabulary. Common words might be a single token; longer or unusual words get split into several. And it’s not just text — Gemini tokenizes images, audio, and video too.

What a context window is

A context window is the most tokens a model can work with in one go. Think of it as short-term memory — that’s Google’s own analogy. Google describes that space as one shared pool covering the instructions the system gives the model, the back-and-forth of your conversation, any documents that got pulled in, your question, and the model’s answer — though the exact split isn’t identical everywhere; some providers cap how much output you can get back separately from the overall window, so check the specifics for whatever model you’re actually using.

The important part: once something falls outside the usable window, it’s gone — not slowly forgotten, just invisible, as if it was never there. The model can’t reach back for it unless the system deliberately re-feeds it, or the product instead rejects, truncates, or compacts your request before the model even sees it.

Why this matters for search

Here’s the connection people miss. Even when a page could fit inside a large context window, handing over every page or an entire site is expensive and usually irrelevant to the question. So when an AI search tool answers, it generally selects and chunks the most relevant passages first, then feeds only those to the model.

That’s the whole reason you hear advice like “put your answer up front” and “keep sections self-contained.” It’s not a style preference — it’s because the AI is working with a piece of your page, not the page. If a section only makes sense in the context of the rest of the page, it can lose its meaning the moment it’s pulled out on its own.

The thing people get wrong

A giant context window does not mean an AI reads your whole site. You’ll hear that Gemini can take in a million tokens, and think “great, it’ll read everything I publish.” It won’t. In a search or AI Overview scenario, the system still picks and chunks relevant passages before anything reaches the model — the huge window changes what’s possible in principle, not what a search engine actually feeds itself for your query.

And you don’t need to pre-cut your content into tiny token-sized blocks. Google says outright there’s no ideal page length and no requirement to break content into tiny pieces for AI. Write for people; the chunking happens on the machine’s side.

Want the real mechanics — how tokenization works, how big context windows have gotten and why there’s a ceiling, and what “Lost in the Middle” means for your content? Switch to the Advanced tab.

TL;DR — A token is the unit an LLM processes — a sub-word fragment, ~4 characters, with 100 tokens ≈ 60–80 English words (Google). Tokenization splits all input and output — text, images, audio, video — into tokens; the model’s known set is its vocabulary. A context window is the total token budget shared by input (system prompt + history + retrieved docs + your query) and output (the response) — Google’s analogy is short-term memory. Content outside the window is invisible, not gradually forgotten. Windows scaled from ~2K tokens to 1M+ (Gemini), with a hardware ceiling (“thermal limit” of TPUs). The reason this matters for search: systems manage cost and relevance by selecting and chunking before an LLM sees your content — and “Lost in the Middle” means even what is in the window isn’t used evenly. A bigger window does not mean AI reads your whole site.

What a token actually is

Tokenization maps text into integer token IDs using a model-compatible encoding; token boundaries are not the same as word boundaries. Evidence for this claim OpenAI models process text as tokens, and token boundaries may be whole words or parts of words. Scope: OpenAI tokenization; token counts depend on the model-compatible encoding. Confidence: high · Verified: 2026-07-14 OpenAI: What are tokens? Context-window sizes are product and model metadata, not a permanent property of all language models. Evidence for this claim Context-window limits are documented per model and can differ across model versions. Scope: OpenAI model metadata; published limits are product-specific and can change. Confidence: high · Verified: 2026-07-14 OpenAI: Models

Google is blunt about the granularity: “Gemini and other generative AI models process input and output at a granularity called a token.” A token is not a word and not a character — it’s a fragment. As Google’s docs put it, “Long words are broken up into several tokens. The set of all tokens used by the model is called the vocabulary, and the process of splitting text into tokens is called tokenization.”

The citable rule of thumb, straight from Google: “For Gemini models, a token is equivalent to about 4 characters. 100 tokens is equal to about 60-80 English words.” So roughly ¾ of a word per token, on average — but average is doing work there. Punctuation, non-English text, code, and unusual or long words tokenize less efficiently (more tokens per word), which is exactly why counting words is a poor way to estimate token usage.

Tokens also aren’t just text. Google: “All input to and output from the Gemini API is tokenized, including text, image files, and other non-text modalities.” Images, audio, and video frames all become tokens too. Google DeepMind’s engineers describe the context-window measure as “how many tokens — the smallest building blocks, like part of a word, image or video — that the model can process at once.”

What a context window is

A context window is the maximum number of tokens a model can hold and reason over in a single interaction. Google’s framing is deliberately simple: “An analogy for the context window is short term memory.”

The key mechanical point is that it’s one shared budget. Everything competes for the same space:

• the system prompt (the instructions the application gives the model),
• the conversation history so far,
• any retrieved or injected documents (the passages a search/RAG system pulled in),
• your input (the current query), and
• the model’s own output (the answer it generates).

Google’s Gemini docs describe input and output drawing from one shared pool. That exact accounting isn’t universal, though: other providers publish a separate maximum- output figure alongside the context window rather than treating it as one undivided budget — Anthropic’s model docs, for instance, list a “context window” size and a distinct “max output” cap side by side for each Claude model. Evidence for this claim Context-window limits are documented per model and can differ across model versions. Scope: OpenAI model metadata; published limits are product-specific and can change. Confidence: high · Verified: 2026-07-14 OpenAI: Models Check the documentation for the specific model and product you’re actually using rather than assuming one formula applies everywhere.

Whatever the exact accounting, once content falls outside the usable window, it isn’t gradually forgotten — it’s simply invisible, as if it never existed, unless a system explicitly summarizes it and re-injects a shorter version, or the product instead rejects, truncates, or compacts the request before the model ever sees it. That “it’s gone, not fading” behavior is the part that trips people up when they assume an AI “remembers” a long conversation the way a person would.

How big context windows are — and how fast that changed

The scale-up here has been dramatic. Early generative models handled only a couple thousand tokens; the progression ran through roughly 8K, then 32K, then 128K, before the jump Google flags in its docs: “Gemini is the first model capable of accepting 1 million tokens.”

There’s a physical ceiling, though. Google DeepMind pushed to experimental 10-million- token windows, and research scientist Nikolay Savinov — one of the research leads on the long-context project, who originally targeted 128,000 tokens before landing on 1 million — put the limit plainly: “10 million tokens at once is already close to the thermal limit of our Tensor Processing Units.” Bigger context isn’t free — it costs memory, compute, and literally heat. (Google DeepMind’s Denis Teplyashin has described the same cascade from 128K to 512K to 1M to 10M, each step opening new possibilities but running into harder engineering limits; research scientist Machel Reid has described what teams actually do with that space, like feeding an entire codebase or a 45-minute film into a single prompt.)

Two consequences worth internalizing:

1. The numbers you see are moving targets. “The largest context window” is a figure that keeps changing; don’t hard-code strategy to a specific token count.
2. Bigger ≠ automatically better. More capacity is not more comprehension — see “Lost in the Middle” below.

What a million tokens actually looks like

Abstract token counts are hard to feel, so Google offers concrete sizing for what 1M tokens can hold:

• “50,000 lines of code (with the standard 80 characters per line),”
• “All the text messages you have sent in the last 5 years,”
• “8 average length English novels,”
• “Transcripts of over 200 average length podcast episodes.”

That’s genuinely enormous — and it’s exactly why the “so the AI just reads my whole site” assumption feels reasonable and is still wrong. Which brings us to the search angle.

Why tokens and context windows matter for AI search

This is the core of it for anyone doing SEO or content. No full page — and certainly no full site — is reliably handed to a model whole in a search scenario. Even when a page fits technically, feeding all of it may be uneconomical or irrelevant to the query. That is why retrieval systems — AI Overviews, Copilot, RAG pipelines, chatbots with browsing — select, chunk, and pass only some of a page’s content to the model. (No search vendor publishes exactly what it retrieves per query, so this is inference from how retrieval architectures work generally and from public statements like the ones below — not a claim that every AI search product behaves identically.)

The token/context-window limit is the mechanical reason behind a whole stack of AI- search behavior you already know about on this site:

• It’s why chunking exists — content gets split into retrievable passages because the whole thing won’t fit, and Microsoft’s own Azure guidance says partitioning large documents into smaller chunks “can help you stay under the maximum token input limits of chat completion and embedding models.”
• It’s why retrieval (the “R” in RAG) has to pick a handful of passages before generation — the model can only be handed what fits in its budget.
• It’s why embedding models have their own token caps (many top out around a few thousand tokens per input), so long passages have to be split before they’re even turned into vectors.

The industry’s own framing of the problem is explicit. The llms.txt spec opens with: “Large language models increasingly rely on website information, but face a critical limitation: context windows are too small to handle most websites in their entirety.” That’s the whole motivation for the format — even if, as I’ll cover in the myths section, Google’s John Mueller has called llms.txt at best a token-saving “crutch” for coding tools, not a search-visibility mechanism.

”Lost in the Middle” — even what’s in the window isn’t used evenly

Here’s the subtler point. Getting your content into the window isn’t the finish line. The Stanford “Lost in the Middle” study (Liu et al., 2023) found that on the language models and tasks they tested, models used information at the start and end of their context far better than information buried in the middle. That’s a position effect measured on 2023-era models — not proof every current model always ignores the middle — but it’s held up as a general caution: stuffing more tokens in doesn’t guarantee better answers, and content that leads with its point survives both retrieval and in-context attention better than content that buries it.

The economics: why AI systems are selective on purpose

There’s a business-model reason retrieval systems chunk aggressively rather than feed whole pages. Google notes that “when billing is enabled, the cost of a call to the Gemini API is determined in part by the number of input and output tokens” — and output tokens typically cost several times more than input tokens. Tokens are literally metered. That gives AI systems a strong economic incentive to retrieve and pass only what’s necessary, which reinforces the same conclusion from a different direction: selective, passage-level retrieval isn’t a temporary limitation to wait out — it’s how these systems are designed to work.

Common myths, debunked

• “A 1-million-token context window means the AI reads my whole website at once.” No. In a search or RAG scenario, retrieval still selects and chunks content before it ever reaches the model’s window. A huge window changes what’s possible in principle, not what a search engine or AI Overview actually feeds itself per query. Google’s line is directly on point: “There’s no ideal page length, and in the end, make pages for your audience, not just for generative AI search.”

• “I need to write in exactly 200-word, token-sized chunks.” No. Google: “There’s no requirement to break your content into tiny pieces for AI to better understand it.” Chunking happens on the system side; your job is clear, well- structured content, not manual token accounting.

• “More tokens = a smarter model / better answers.” No. A larger window increases capacity, not comprehension — and “Lost in the Middle” shows models use the start and end of their context better than the middle. Bigger isn’t automatically better.

• “Tokens = words, so I can just count words to estimate usage.” Roughly, but not reliably. 100 tokens ≈ 60–80 English words is Google’s average, but punctuation, code, non-English text, and unusual words tokenize less efficiently — which matters when you’re estimating cost or budget.

• “llms.txt solves the context-window problem for my site.” The problem is real — it’s the spec’s own stated motivation — but Google doesn’t use llms.txt for search, and Mueller has framed it as a token-saving crutch for AI coding tools, not an SEO fix. The mitigations that actually matter are the same fundamentals: be crawlable and indexed, and write clear, retrieval-friendly content.

What this means for content and SEO

Strip the jargon and the takeaways are concrete — and they’re the same discipline that shows up across this cluster:

• Front-load the answer. Both retrieval and in-context attention favor the start (and end) of what a model sees. Lead with your point.
• Keep sections self-contained. Because a passage may be pulled out of your page and handed to a model on its own, it should stand on its own. This is the same logic behind good H2/H3 structure and clear topic sentences.
• Don’t obsess over manual chunking. No ideal page length, no requirement to fragment. Write for humans; the system chunks.
• “Bigger context window” ≠ “AI reads my whole site.” Optimize for being found and selected, not for being ingested whole.

Tokens and context windows are the low-level plumbing under most of this cluster: they’re the reason grounding and retrieval have to select passages, the reason chunking prepares text the way it does, the constraint embeddings and semantic and vector search work within, and the limit passage ranking exists to work around. And they sit right next to the model’s other hard boundary — its knowledge cutoff — as one of the core limitations of any LLM.

AI summary

A condensed take on the Advanced version:

• A token = the unit an LLM processes — a sub-word fragment, ~4 characters, with 100 tokens ≈ 60–80 English words (Google). Tokenization splits all input and output — text, images, audio, video — into tokens; the model’s known set is its vocabulary.
• Word counts estimate tokens poorly — punctuation, code, non-English text, and unusual words tokenize less efficiently (more tokens per word).
• A context window = the total token budget shared by input (system prompt + history + retrieved docs + query) and output. Google’s analogy: short-term memory.
• Content outside the window is invisible, not gradually forgotten — gone unless a system summarizes and re-injects it.
• Windows scaled fast: ~2K → 8K/32K/128K → 1M+ (Gemini), with experimental 10M near the “thermal limit” of Google’s TPUs (Nikolay Savinov). The numbers keep moving.
• Bigger ≠ better: more capacity isn’t more comprehension. “Lost in the Middle” — models use the start and end of their context better than the middle.
• Why it matters for search: no full page or site fits in one window, so retrieval systems (AI Overviews, Copilot, RAG) select and chunk before an LLM ever sees your content. It’s the mechanical reason chunking, retrieval, and embedding token caps exist.
• Economics reinforce it: tokens are metered (output usually costs more than input), so systems are designed to retrieve selectively, not feed whole pages.
• Myths busted: a 1M window doesn’t mean AI reads your whole site; no need for manual token-sized chunks (Google says no ideal length, no requirement to fragment); more tokens ≠ smarter; llms.txt doesn’t solve it for search.
• Content upshot: front-load the answer, keep sections self-contained, don’t obsess over manual chunking, optimize for being selected, not ingested whole.

Official documentation

Primary-source documentation on tokens and context windows.

Google

• Understand and count tokens (Gemini API) — what a token is, the ~4-characters / 100-tokens-≈-60-80-words rule of thumb, tokenization and vocabulary, and multimodal tokenization.
• Long context (Gemini API) — the short-term-memory analogy, the 1-million-token milestone, the historical progression, and the concrete “what a million tokens looks like” examples.
• What is a long context window? Google DeepMind engineers explain (Google Blog) — DeepMind engineers on tokens as the smallest building blocks and the hardware ceiling.
• Gemini Developer API pricing — token-based billing; input vs. output token costs.
• Optimizing your website for generative AI features (Google Search Central) — “no ideal page length” and “no requirement to break your content into tiny pieces.”

Microsoft / Azure

• Chunk large documents for vector search (Azure AI Search) — chunking to stay under the maximum token input limits of chat-completion and embedding models.
• What are AI Tokens? (Microsoft Copilot help) — Microsoft’s consumer-facing explainer of tokens and how token limits cap how much you can ask about at once.

Industry problem statement

• llms.txt spec (llmstxt.org) — frames the context-window limit as the origin problem: windows “too small to handle most websites in their entirety.”

Quotes from the source

On-the-record statements from Google and Microsoft, plus the industry framing of the context-window problem. Each link is a deep link to the quoted passage where the source page supports one.

Google — what a token is

• “Gemini and other generative AI models process input and output at a granularity called a token.” — Google AI for Developers, Understand and count tokens. Jump to quote
• “For Gemini models, a token is equivalent to about 4 characters. 100 tokens is equal to about 60-80 English words.” Jump to quote
• “Long words are broken up into several tokens. The set of all tokens used by the model is called the vocabulary, and the process of splitting text into tokens is called tokenization.” Jump to quote

Google — the context window

• “An analogy for the context window is short term memory.” — Google AI for Developers, Long context. Jump to quote
• “Gemini is the first model capable of accepting 1 million tokens.” Jump to quote

Google DeepMind engineers — tokens and the hardware ceiling (via Google Blog)

• “measures how many tokens — the smallest building blocks, like part of a word, image or video — that the model can process at once.” Jump to quote
• “10 million tokens at once is already close to the thermal limit of our Tensor Processing Units.” — Nikolay Savinov, Research Scientist, Google DeepMind. Read the coverage

Google — no ideal page length, no requirement to fragment

• “There’s no ideal page length, and in the end, make pages for your audience, not just for generative AI search.” — Google Search Central, Optimizing your website for generative AI features. Read the guide
• “There’s no requirement to break your content into tiny pieces for AI to better understand it.” — Google Search Central, same guide. Read the guide

Microsoft — chunking to stay under token limits

• “Partitioning large documents into smaller chunks can help you stay under the maximum token input limits of chat completion and embedding models.” — Microsoft, Azure AI Search documentation. Read the source

The industry problem statement — llms.txt spec

• “Large language models increasingly rely on website information, but face a critical limitation: context windows are too small to handle most websites in their entirety.” — llmstxt.org spec. Jump to quote

Tokens and context windows — cheat sheet

What each term is in one line

Token ↔ word rule of thumb (Google, Gemini)

(Caveat: averages only — punctuation, code, and non-English text use more tokens per word.)

How context windows grew

What 1 million tokens can hold (Google’s examples)

• ~50,000 lines of code (80 chars/line)
• 5 years of your text messages
• 8 average-length English novels
• Transcripts of 200+ podcast episodes

Fast facts

• Context window = one shared budget for system prompt + history + retrieved docs + query + output.
• Content outside the window is invisible — gone, not gradually forgotten.
• Bigger window ≠ better answers — “Lost in the Middle”: start/end beat the middle.
• In search, retrieval chunks and selects before the model sees content — a 1M window doesn’t mean AI reads your whole site.
• Google: no ideal page length, no requirement to fragment content for AI.
• Tokens are metered/billed (output usually costs more than input) — the economic reason systems retrieve selectively.

The mental models

1. Token = the model’s unit of reading, not the word. LLMs read in sub-word fragments (~4 characters, ~¾ of a word). When you estimate cost or budget, count tokens, not words — punctuation, code, and non-English text inflate the ratio.

2. The context window is short-term memory — a shared budget. System prompt, history, retrieved documents, your query, and the model’s output all draw from one pool. Anything outside it is invisible, not fading. If a model seems to “forget,” ask what fell out of the window.

3. Bigger window ≠ more understanding. Capacity and comprehension are different axes. “Lost in the Middle” means the start and end of the context get used better than the middle — so lead with your point, don’t just add more.

4. No page fits — so retrieval selects first. The whole reason AI search chunks and retrieves is that a page (let alone a site) doesn’t fit, or isn’t economical to feed, whole. The model never sees your page; it sees the passages a retrieval system chose. Optimize to be selected, not ingested whole.

5. The decision rule for content. Don’t manually cut content into token-sized blocks (Google says you don’t need to). Do front-load answers and write self-contained sections — because both retrieval and in-context attention reward content that stands on its own and leads with its point.

Context-window mistakes

Treating words and tokens as interchangeable

Tokenization varies by model, language, punctuation, and code. Use the tokenizer for the actual model when a hard limit matters; a word-count ratio is only planning guidance.

Filling the entire advertised context window

Capacity does not mean every token receives equal attention or that output space is free. Reserve room for instructions and the answer, remove duplicated material, and test the full task rather than celebrating a maximum input size.

Splitting content at a fixed character count

Blind cuts can separate a heading, definition, table, or qualifier from the passage it explains. Chunk on semantic boundaries and preserve enough local context for each unit to stand alone.

Measure page sections before chunking

Run this in the Chrome DevTools Console. It reports characters and whitespace-split words for each headed section without pretending those counts equal model tokens:

const headings = [...document.querySelectorAll('main h2, main h3')];
console.table(headings.map((h, i) => { let text = ''; for (let n = h.nextElementSibling; n && !/^H[23]$/.test(n.tagName); n = n.nextElementSibling) text += ` ${n.innerText || ''}`; return { heading: h.textContent.trim(), characters: text.trim().length, words: text.trim() ? text.trim().split(/\s+/).length : 0, nextHeading: headings[i + 1]?.textContent.trim() || '' }; }));

Use the output to find oversized or empty sections, then run the final text through the tokenizer for the model you will actually use.

Find likely semantic boundaries in plain text

This regular expression matches Markdown second- and third-level headings and keeps the heading text in capture group 1:

^#{2,3}\s+(.+)$

Use multiline mode. Headings are candidate boundaries, not a guarantee that every section is complete enough to retrieve alone.

Resources worth your time

My related writing

• What We Actually Know About Optimizing for LLM Search — my Ahrefs piece on how AI retrieval treats content, including Dan Petrovic’s Chrome DocumentChunker research on how pages are chunked into ~200-word passages before they’re processed — the token/context-window limit is exactly why that chunking happens.
• Generative Engine Optimization — my broader take on optimizing for a retrieval-and-generation search landscape shaped by these context limits.

My speaking

• GEO? AEO? LLMO? — my AI search webinar — where retrieval, chunking, and context limits fit into AI search. (My standing disclaimer applies: this is my understanding of these systems, not going to be 100% complete or accurate.)

Official

• Understand and count tokens (Gemini API) and Long context (Gemini API) — Google’s own definitions and the million-token examples.
• Optimizing your website for generative AI features (Google Search Central) — “no ideal page length,” “no requirement to break your content into tiny pieces.”

From around the industry

• What is a long context window? Google DeepMind engineers explain — Nikolay Savinov on the TPU thermal ceiling, plus Machel Reid and Denis Teplyashin on what long context unlocks.
• Chunk large documents for vector search (Microsoft Azure AI Search) — chunking to stay under maximum token input limits.
• The llms.txt spec — the “context windows are too small to handle most websites in their entirety” problem statement, and the format built around it.
• SEO Chunk Optimization is Overrated (Despina Gavoyannis, Ahrefs) — “You can’t control how Google, ChatGPT, or Perplexity chunk your content. Their pipelines change based on cost, model, and context.”
• Chunked, Retrieved, Synthesized — Not Ranked (Duane Forrester) — “If traditional SEO optimized for clicks, GenAI systems optimize for chunks… Structure still wins.”
• What is a context window? (IBM) — a neutral, thorough reference explainer.

Stats worth citing

• ~4 characters per token; 100 tokens ≈ 60–80 English words — Google’s own rule of thumb for Gemini models, the most citable token-to-word conversion. Source
• 1,000,000 tokens — Gemini is the first model capable of accepting 1 million tokens in a single window, up from ~2K in early generative models. Source
• ~10,000,000 tokens = near the TPU thermal limit — Google DeepMind’s Nikolay Savinov on the current physical ceiling for context-window size. Source
• 1 million tokens ≈ 8 novels / 200+ podcast transcripts / 50,000 lines of code — Google’s concrete sizing for what a million-token window can hold. Source
• Token-based billing (output usually costs more than input) — per Google, the cost of a Gemini API call is determined in part by input and output token counts, the economic reason AI systems retrieve selectively rather than feed whole pages. Source

Test yourself: Tokens and Context Windows

Five quick questions on how LLMs read text and how much they can hold at once. Pick an answer for each, then check.

Tokens and Context Windows

A token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory.

Related: Chunking, Large Language Model (LLM), Knowledge Cutoff

Tokens and Context Windows

A token is the granularity at which large language models read and write. It isn’t a word or a character — it’s a sub-word fragment. Google’s rule of thumb for Gemini is that a token is about 4 characters, and 100 tokens ≈ 60–80 English words. Splitting text into tokens is called tokenization, and the full set of tokens a model knows is its vocabulary. Tokenization isn’t limited to text: Gemini also tokenizes images, audio, and video.

A context window is the maximum number of tokens a model can “see” in a single interaction — the system prompt, the conversation history, any retrieved or injected documents, your input, and the model’s own output all share that one budget. Google’s own analogy is short-term memory. Once content falls outside the window, it’s simply gone — not gradually forgotten, just invisible, as if it were never there (unless a system explicitly summarizes and re-injects it).

Context windows have grown fast — from a couple thousand tokens in early models to 1M+ tokens in Gemini — but a bigger window doesn’t mean an AI reads your whole site. In most search and RAG scenarios, retrieval systems still select and chunk content before it ever reaches the model. That’s the mechanical reason chunking, passage-level retrieval, and “keep content clear and self-contained” advice exist at all.

Related: Chunking, Large Language Model (LLM), Knowledge Cutoff

Changelog