Embeddings

TL;DR — An embedding turns text into a list of numbers — coordinates in a giant space — where things that mean similar things end up close together. That’s how AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. and modern search engines match your content to a query by meaning instead of just matching keywords. You can’t “add” embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. to a page; they’re how the machine reads what’s already there.

What an embedding is

An embedding represents input as a numeric vector intended to preserve useful relationships for tasks such as semantic similarity. Evidence for this claim Embeddings represent inputs as numerical vectors that can be compared for relatedness and used for search, clustering, and classification. Scope: OpenAI embedding models and documented uses; vector dimensions and behavior vary by model. Confidence: high · Verified: 2026-07-14 OpenAI: Embeddings guide Geometry and dimensionality are model-specific rather than universal meanings attached to each coordinate. Evidence for this claim Learned vector representations can encode useful distributional relationships between words. Scope: Word2vec-era language representations; observed vector relationships are model- and training-data-specific, not ground truth. Confidence: high · Verified: 2026-07-14 Mikolov et al.: Efficient Estimation of Word Representations

Computers don’t understand words — they understand numbers. An embedding is the bridge: a model reads a piece of text and turns it into a list of numbers (called a vector) that captures its meaning. Think of it like a location on a map. Two pieces of text that mean similar things get placed close together; unrelated text ends up far apart.

So “laptop for gaming” and “high-performance laptop for games” land near each other, even though they barely share words. “Banana bread recipe” lands somewhere else entirely. The model learned this by reading enormous amounts of text and noticing which words and phrases show up in similar contexts.

Why this matters for search

Old-school search matched keywords: you typed a word, the engine foundA 302 (\"Found\") is a temporary redirect: it forwards users to a new URL while telling search engines the original URL should stay in the index. It's a weak canonicalization signal, not the zero-equity dead end of SEO folklore. pages containing that word. Embeddings let search match meaning. That’s why Google can answer a long, conversational question even when the best page doesn’t use your exact words — and it’s how AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. tools (the ones that read sources and write you an answer) decide which passages are relevant enough to pull in.

The short version of the pipeline behind AI answers:

1. Content gets broken into chunks (passages).
2. Each chunk gets turned into an embedding (a vector).
3. Your question gets turned into an embedding too.
4. The system finds the chunks whose vectors are closest to your question’s vector.
5. Those chunks become the source material the AI writes its answer from.

What this means for your content

Here’s the part people overstate, so let me be clear: there’s no “embedding optimization” you submit to Google. Google’s Danny Sullivan said exactly that about BERTSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it. (one of Google’s embedding-based systems): “There’s nothing to optimize for.”

What actually helps is the same advice as ever, just with a clearer reason behind it. Write content that genuinely and thoroughly covers a topic. Coherent, focused content produces clean, consistent embeddings that sit close to the questions it should answer. Keyword-stuffed, scattered content produces a muddier signal. You’re not feeding a formula — you’re writing something a model (and a human) can clearly understand.

Want the real mechanics — dimensions, cosine similarity, the word2vec-to-BERT history, and how Google actually uses embeddings in ranking? Switch to the Advanced tab.

TL;DR — An embedding is a dense vector of floating-point numbers (typically hundreds to a few thousand dimensions) that encodes meaning, produced by an encoder model — not a generative LLMA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4).. Similar meaning → nearby vectors, measured with cosine similarity. The field moved from static word embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. (word2vec, GloVe) to contextual ones (BERTSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it.) to sentence-level and modern API embeddings. Google uses embedding-based retrieval (Neural Matching / RankEmbed, RankEmbedBERT) alongside its keyword indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed. — hybrid, not a replacement. Embeddings are also the retrieval backbone of RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights.. There’s no embedding knob to turn; topical coherence is what makes content cluster near the right queries.

What an embedding actually is

Embeddings support similarity and retrieval, but they are not a direct measure of truth, quality, or ranking value. Evidence for this claim Embeddings represent inputs as numerical vectors that can be compared for relatedness and used for search, clustering, and classification. Scope: OpenAI embedding models and documented uses; vector dimensions and behavior vary by model. Confidence: high · Verified: 2026-07-14 OpenAI: Embeddings guide Research results depend on the trained model and evaluation setting. Evidence for this claim Learned vector representations can encode useful distributional relationships between words. Scope: Word2vec-era language representations; observed vector relationships are model- and training-data-specific, not ground truth. Confidence: high · Verified: 2026-07-14 Mikolov et al.: Efficient Estimation of Word Representations

An embedding is a dense numerical vector — a list of floating-point numbers — that represents text (or images, audio, video) as a point in a high-dimensional space. OpenAI’s documentation states it plainly: “An embedding is a vector (list) of floating point numbers.”

The defining property is geometric: semantically similar content has similar vectors. Text that means roughly the same thing points in roughly the same direction; unrelated text points elsewhere. That’s not a happy accident — the model is trained so that words and phrases used in similar contexts end up with similar vectors. Meaning becomes position.

A few things worth getting precise:

• Encoder, not generator. Embeddings come from encoder models whose job is to compress meaning into a fixed-size vector. That’s a different architecture and purpose from a generative LLM, which predicts the next tokenA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory.. (More on the internal-vs-API distinction below.)
• Dense, not sparse. Unlike one-hot or bag-of-words representations (mostly zeros, one slot per vocabulary word), embeddings pack meaning into every dimension. Google’s ML glossary frames embeddings as lower-dimensional, dense representations that fix what one-hot encoding can’t express — they let a model recognize that “hot dogs and shawarmas are more related than hot dogs and salads.”
• Higher dimensions ≠ always better. More dimensions can capture more nuance, but they cost more to store and compute, and the gain is task-dependent. It’s a trade-off, not a “bigger is better” dial.

Measuring similarity: cosine similarity

To compare two embeddings you measure the distance — or really the angle — between them. The standard metric is cosine similarity: it measures the angle between two vectors regardless of their length, scoring from −1 (opposite) through 0 (unrelated/orthogonal) to 1 (identical direction). Smaller distance = more related.

Many embedding APIs normalize vectors to unit length, which makes cosine similarity and dot product produce the same ranking — OpenAI notes cosine is the conventional, slightly cheaper choice. Voyage AI (the embedding provider Anthropic recommends) puts the intuition cleanly: “the cosine similarity between two embeddings captures the semantic relatedness of the corresponding original passages.” Doing that nearest-neighbor comparison at scale is its own problem — that’s the job of vector searchVector search finds content by comparing the meaning of a query against stored content as numerical vectors, retrieving the closest ones in a high-dimensional embedding space. At scale it uses approximate nearest neighbor (ANN) algorithms — not exact comparison — to search billions of vectors in milliseconds..

How we got here: the evolution

The story runs from single words to whole passages, and from fixed meanings to context-aware ones.

• word2vec (Google, 2013). Mikolov et al. introduced two architectures (CBOW and Skip-Gram) for learning dense word vectors from huge corpora. The famous result: the vector “King” − “Man” + “Woman” lands closest to “Queen” — evidence that vector arithmetic captures semantic relationships. (Caveat: that analogy is illustrative, not guaranteed every time; depending on the model it can land on “kings” or “monarch.”) These are static embeddings — one fixed vector per word, so “bank” gets the same vector in “river bank” and “bank account.”
• GloVe (Stanford, 2014). A count-based alternative built on global co-occurrence statistics rather than a predictive network — different objective, similarly useful embeddings. Also static.
• Universal Sentence Encoder (Google, 2018). Embeddings for whole sentences, not just words. The core idea: “Sentences are semantically similar if they have a similar distribution of responses” — “How old are you?” and “What is your age?” invite the same answers, so they embed close together.
• BERT (Google, 2018; deployed in Search Oct 2019). The big shift: contextual embeddings. The same word gets a different vector depending on the surrounding sentence, because BERT is bidirectional — it reads the words before and after a token to fix its meaning. So “bank” in “river bank” and “bank account” finally get different vectors.
• Sentence-BERT (2019). Solved BERT’s scaling problem for similarity search. Vanilla BERT needs both sentences fed in together, which is computationally brutal at scale; SBERT produces fixed-size sentence embeddings you can compare with cosine similarity, cutting the cost of finding the most similar pair in a large corpus from hours to seconds.
• Modern embedding APIs (2024–present). OpenAI’s text-embedding-3 family, Google’s Gemini embeddings, Voyage, and Cohere’s embed-v4.0 — multilingual, increasingly multimodal (text, image, audio, video in one space), and resizable via Matryoshka Representation Learning (truncate the vector to fewer dimensions without retraining, trading a little accuracy for storage and speed).

How Google uses embeddings in Search

Google’s ranking pipeline isn’t purely semantic or purely keyword — it’s hybrid, and the embedding pieces supplement the classic inverted index rather than replacing it. From Google’s own ranking-systems documentation and Pandu Nayak’s DOJ antitrust testimony, the named systems include:

• BERT — Google’s words-in-context system. At launch it helped Search “better understand one in 10 searches in the U.S. in English,” especially longer conversational queries where prepositions like “for” and “to” change the meaning.
• Neural Matching / RankEmbed — embedding-based retrieval that translates queries and documents into the same vector space to surface conceptually matching results even without shared keywords. Nayak described it as a supplement: “RankEmbed identifies a few more documents to add to those identified by the traditional retrieval.” Retrieval there is based on a dot product / distance measure in the embedding space.
• RankEmbedBERT — a later evolution combining RankEmbed’s retrieval with BERT’s language understanding, trained on quality-rater scores and search logs, and notably better on complex, long-tail queries.

The practical reading: keyword presence still matters because lexical retrieval (the inverted index, BM25-style) still does the first-pass narrowing. The embedding systems add conceptually-related candidates and re-rank. Both signals are in play — which is exactly why “BERT killed keywords” is wrong.

Embeddings in AI search (the RAG pipeline)

This is where embeddings touch AI OverviewsAI Overviews are the AI-generated summary box Google shows above or within its regular search results, written by Gemini models from pages retrieved out of Google's normal Search index. It's a Search feature, not a separate platform or index. and AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity. assistants most directly. Retrieval-Augmented Generation (RAG) uses embeddings as its retrieval layer:

1. Index: content is chunked into passages, each passage is embedded, and the vectors go into a vector database.
2. Retrieve: the user’s query is embedded, a nearest-neighbor search finds the closest chunks (via vector searchVector search finds content by comparing the meaning of a query against stored content as numerical vectors, retrieving the closest ones in a high-dimensional embedding space. At scale it uses approximate nearest neighbor (ANN) algorithms — not exact comparison — to search billions of vectors in milliseconds.), and the top-K chunks are handed to the LLM as context.
3. Generate: the LLM writes an answer grounded in those retrieved chunks.

Structure matters here because each chunk is retrieved in isolation. Dan Petrovic’s research (cited in Ahrefs’ What We Actually Know About Optimizing for LLM Search) found Chrome processes only the first ~30 passages of a page for embeddings and chunks them in roughly 200-word passages with overlap to preserve cross-chunk context. If a section can’t stand on its own once it’s pulled out of the page, it represents your content poorly.

Token embeddings vs. text embedding APIs

A distinction that trips people up: the embeddings inside an LLM and the embeddings you get from an API are not the same thing.

• Token embeddings are the model’s internal representations — each token gets a vector that’s transformed layer by layer during generation. They’re machinery for producing the next token.
• Text embedding APIs (OpenAI, Google, Voyage, Cohere) produce a single fixed-size vector for an entire input string, purpose-built for retrieval and similarity. Often a separate model with a different training objective.

When SEOs talk about “embedding a page” or running cosine similarity for internal linking or keyword clustering, they mean the API kind.

What this means for SEO

• Semantic coherence beats keyword density. Because models understand context, “laptop for gaming” and “high-performance laptop” already sit close. Stuffing doesn’t help — it produces topically scattered content with a muddier embedding.
• Structure for chunked retrieval. Passages get embedded and retrieved on their own. Put important content early, keep sections self-contained, use clear semantic HTML.
• Topical comprehensiveness. Content that genuinely covers a subject ends up near more of the related queries in vector space. That’s the mechanism behind “build topical authority.”
• There’s no embedding knob. Danny Sullivan, on BERT: “There’s nothing to optimize for… The fundamentals of us seeking to reward great content remain unchanged.” The cosine-similarity scores you get from a tool are analysis aids, not inputs you submit to Google.

Embeddings are the connective tissue under most of this cluster: they’re what semantic searchSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it. runs on, what vector search compares, what chunkingChunking is splitting a document into smaller passages so AI systems can embed, index, and retrieve the single most relevant piece — not the whole page — in response to a query. It's a foundational step in RAG pipelines and the conceptual cousin of Google's passage ranking. prepares text for, and the retrieval backbone of RAG. They’re also why crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. still matters first — content has to be fetched before any system can embed it.

AI summary

A condensed take on the Advanced version:

• An embedding = a dense vector of floating-point numbers that encodes meaning; similar meaning → nearby vectors. OpenAI: “An embedding is a vector (list) of floating point numbers.”
• Produced by encoder models, not generative LLMsA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4). — different architecture and purpose. Dense, not sparse.
• Similarity is measured with cosine similarity (the angle between vectors, −1 to 1). Normalized vectors make cosine and dot product equivalent for ranking.
• Evolution: word2vec (2013) and GloVe (2014) gave static word vectors → USE (2018) embedded whole sentences → BERTSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it. (2018) made embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. contextual (same word, different vector by context) → Sentence-BERT (2019) made similarity search fast → modern multimodal, resizable API models (2024–present).
• Google uses embeddings in a hybrid pipeline: BERT (words in context), Neural Matching / RankEmbed (embedding retrieval that supplements the keyword indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.), and RankEmbedBERT. Keywords still do first-pass retrieval — embeddings add and re-rank.
• RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. pipeline: chunk → embed → store in a vector DB → embed the query → retrieve nearest chunks → LLM generates a grounded answer. Each chunk is retrieved in isolation, so structure matters.
• SEO upshot: no “embedding optimization” exists — Danny Sullivan: “There’s nothing to optimize for” with BERT. Topically coherent, self-contained, thorough content clusters near the queries it should answer.

Official documentation

Primary-source documentation on embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. from Google and the major embedding providers.

Google

• Machine Learning Glossary — Embeddings — the dense-vs-sparse definition and why one-hot encoding can’t express relatedness.
• A guide to Google Search ranking systems — BERTSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it., Neural Matching, RankBrain, Passage RankingPassage ranking is a Google AI system that scores individual sections ('passages') of a page so a single page can earn multiple relevance scores for different queries. Google still indexes whole pages — only the ranking changed., and MUM described in Google’s own words.
• Understanding searches better than ever before (BERT) — the October 2019 announcement of BERT in Search.
• Advances in Semantic Textual Similarity — the Universal Sentence Encoder (2018) and the “similar distribution of responses” idea.
• Gemini API — Embeddings — Google’s current multimodal embedding model, dimensions, and use cases (RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights., semantic searchSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it., rerankingReranking is the second stage of a retrieval pipeline: after a cheap, broad first pass pulls a candidate set of documents or passages, a slower, more precise model re-scores and reorders that shortlist by true relevance before the results are served or handed to an LLM., clustering).

OpenAI

• Vector embeddings guide — “An embedding is a vector (list) of floating point numbers”; cosine similarity guidance.
• New embedding models and API updates — the text-embedding-3 family and Matryoshka dimension shortening (Jan 2024).

Other providers

• Cohere — Embeddings — multilingual embed-v4.0, separate query/document input types, compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. options.
• Voyage AI — Quickstart — Anthropic’s recommended provider; “the cosine similarity between two embeddings captures the semantic relatedness of the corresponding original passages.”

Foundational papers

• word2vec — Mikolov et al., 2013 — the original dense word vectors and the king−man+woman analogy.
• GloVe — Pennington, Socher, Manning, 2014 — global co-occurrence word vectors.
• BERT — Devlin et al., 2018 — bidirectional, contextual embeddings.
• Sentence-BERT — Reimers & Gurevych, 2019 — fast sentence embeddings for similarity search.

Quotes from the source

On-the-record statements from Google and the embedding providers. Each link is a deep link that jumps to the quoted passage on the source page.

What an embedding is

• “An embedding is a vector (list) of floating point numbers.” — OpenAI, Vector embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. guide. Jump to quote

Google — BERTSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it. and language understanding

• “At its core, Search is about understanding language.” — Pandu Nayak, VP of Search, Google (BERT announcement, October 2019). Jump to quote
• “BERT will help Search better understand one in 10 searches in the U.S. in English.” — Google Search Blog, October 2019. Jump to quote

Google — on optimizing for BERT

• “There’s nothing to optimize for with BERT… The fundamentals of us seeking to reward great content remain unchanged.” — Danny Sullivan, Google Search Liaison. Jump to quote

Google — semantic similarity (Universal Sentence Encoder)

• “Sentences are semantically similar if they have a similar distribution of responses.” — Google Research, “Advances in Semantic Textual Similarity” (May 2018). Read the source

Voyage AI (Anthropic’s recommended provider) — cosine similarity

• “The cosine similarity between two embeddings captures the semantic relatedness of the corresponding original passages.” — Voyage AI Quickstart. Read the source

Pandu Nayak (DOJ antitrust testimony) — RankEmbed

• “RankEmbed identifies a few more documents to add to those identified by the traditional retrieval.” — coverage of Pandu Nayak’s testimony, DOJ v. Google. Read the coverage

The mental models

1. Meaning becomes position. An embedding turns text into coordinates. Close together = similar meaning; far apart = unrelated. Everything else — semantic searchSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it., clustering, RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. retrieval — is just measuring distance in that space.

2. Static vs. contextual. word2vec and GloVe give each word one fixed vector (“bank” is always the same). BERT and its descendants give each occurrence a vector that depends on context (“river bank” ≠ “bank account”). Older keyword-similarity tools built on word2vec can’t tell those apart; BERT-based ones can.

3. TokenA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory. embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. ≠ text embedding APIs. Inside an LLMA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4)., token embeddings are working machinery for generating the next token. Text embedding APIs output one condensed vector per string, built for retrieval and similarity. Different models, different jobs. When SEOs “embed a page,” they mean the API kind.

4. Hybrid retrievalHybrid search runs keyword (lexical/BM25) search and vector (semantic) search together, then merges the two result lists into one ranking — commonly with Reciprocal Rank Fusion — so it catches both exact-term matches and meaning-based matches that either method alone would miss., not replacement. Google runs lexical retrieval (the keyword inverted indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.) and embedding-based retrieval (Neural Matching / RankEmbed). Keywords narrow first; embeddings add conceptually-related candidates and re-rank. Neither alone is the whole system.

5. The RAG loop. Chunk → embed → store → embed the query → retrieve nearest chunks → generate a grounded answer. Because each chunk is retrieved in isolation, write self-contained passages and front-load what matters.

6. The decision rule for content. There’s no embedding to optimize. Ask instead: does this passage clearly and thoroughly cover the thing it claims to? Coherent content clusters near the right queries; scattered, stuffed content doesn’t.

Embeddings — cheat sheet

What it is in one line A dense vector (list of floating-point numbers) encoding meaning; similar meaning → nearby vectors; compared with cosine similarity.

Cosine similarity scale

Normalized (unit-length) vectors → cosine similarity and dot product give the same ranking.

The evolution, at a glance

Google’s named embedding systems

The RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. retrieval loop

1. Chunk content into passages
2. Embed each chunk → vector
3. Store vectors in a vector databaseVector search finds content by comparing the meaning of a query against stored content as numerical vectors, retrieving the closest ones in a high-dimensional embedding space. At scale it uses approximate nearest neighbor (ANN) algorithms — not exact comparison — to search billions of vectors in milliseconds.
4. Embed the incoming query
5. Nearest-neighbor (vector) search → top-K chunks
6. LLMA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4). generates an answer grounded in those chunks

Fast facts

• Dimensions: typically hundreds to a few thousands — more nuance, more storage/compute; not always better.
• Produced by encoder models, not generative LLMs.
• Matryoshka lets you truncate to fewer dimensions without retraining.
• Switching embedding models means re-embedding everything — different models’ spaces aren’t compatible.
• No “embedding optimization” to submit to Google — Danny Sullivan: “There’s nothing to optimize for” (BERT).

Test yourself: Embeddings

Resources worth your time

My related writing & speaking

• What We Actually Know About Optimizing for LLM Search — the Ahrefs piece citing my research and Dan Petrovic’s findings on how Chrome chunks and embeds page passages (the ~30-passage limit, 200-word chunks).
• GEO? AEO? LLMO? — my AI search webinar — where retrieval pipelines and embeddingsEmbeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords. fit into AI searchAI search uses large language models and retrieval-augmented generation (RAG) to synthesize an answer from multiple sources rather than returning a ranked list of links. Examples include Google AI Overviews, ChatGPT Search, and Perplexity..

Foundational papers (the history)

• word2vec — Mikolov et al., 2013 — dense word vectors and vector arithmetic.
• GloVe — Pennington, Socher, Manning, 2014 — global co-occurrence word vectors.
• BERT — Devlin et al., 2018 — contextual, bidirectional embeddings.
• Sentence-BERT — Reimers & Gurevych, 2019 — fast sentence-level similarity.

From others

• The shift to semantic SEO: What vectors mean for your strategy — Ann Robison, Search Engine Land: “Vectors are to AI what structured dataStructured data is a standardized way of labeling page content (using the schema.org vocabulary in JSON-LD, Microdata, or RDFa) so search engines can understand its meaning. It's not a direct ranking factor — its value is rich results and entity understanding. is to search engines.”
• How to leverage cosine similarity for ecommerce SEO — Search Engine Land practitioner guide to applying cosine similarity in real SEO workflows.
• Introduction to Vector Databases and How to Use AI for SEO — Search Engine Journal overview of how vector databasesVector search finds content by comparing the meaning of a query against stored content as numerical vectors, retrieving the closest ones in a high-dimensional embedding space. At scale it uses approximate nearest neighbor (ANN) algorithms — not exact comparison — to search billions of vectors in milliseconds. connect to SEO.
• SEO Use Cases for Vectorizing the Web with Screaming Frog — iPullRank’s practitioner walkthrough of embeddings for SEO.
• Semantic Search Explained: Vector Models’ Impact on SEO Today — Lumar on dense vs. sparse, cosine similarity, and hybrid searchHybrid search runs keyword (lexical/BM25) search and vector (semantic) search together, then merges the two result lists into one ranking — commonly with Reciprocal Rank Fusion — so it catches both exact-term matches and meaning-based matches that either method alone would miss..
• Embeddings Explained: Unlocking the Future of SEO — Edd Dawson’s SEO-focused explainer covering practical use cases for keyword clustering and internal linkingAn internal link is a hyperlink from one page on a website to another page on the same website. Internal links help search engines discover your pages and pass ranking signals (PageRank and anchor-text context) between them..
• How Google Search Works (inverted index, RankBrain, RankEmbed, DeepRank) — the DOJ-testimony view of Google’s embedding systems.
• sbert.net — the Sentence Transformers library, if you want to generate embeddings yourself.

Embeddings

Embeddings are dense numerical vectors — lists of floating-point numbers — that represent the meaning of text in a high-dimensional space. Semantically similar content lands close together, so search and AI systems can match by meaning, not just keywords.

Related: Vector Search, Semantic Search, Retrieval-Augmented Generation (RAG)

Embeddings

An embedding is a dense vector of numbers — a list of floating-point values — that encodes the meaning of a word, sentence, or document. An embedding model maps text into a shared high-dimensional space where semantically similar content clusters together and unrelated content sits far apart. The distance between two vectors (usually measured with cosine similarity) is how search engines, LLMsA large language model (LLM) is a deep-learning model trained on massive text corpora to predict the next token and generate human-like text. LLMs use the transformer architecture and power AI search features like Google's AI Overviews (Gemini) and Bing Copilot (GPT-4)., and RAGRAG is the retrieve-then-generate pattern behind AI search: the system retrieves relevant passages from an external index at query time, injects them into the model's context, and generates an answer grounded in those sources — without changing the model's weights. systems answer “how related is this content to this query?” without relying on exact keyword overlap.

Put another way: embeddings turn language into coordinates, and the closer two texts’ coordinates are, the closer their meaning. In the OpenAI API reference, an embedding is represented as a vector containing a list of floating-point values.

Embeddings are produced by encoder models trained so that text used in similar contexts ends up with similar vectors. That training is what separates them from a keyword indexStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.: the position in vector space encodes meaning, not just presence. Modern models output anywhere from a few hundred to a few thousand dimensions — more dimensions can capture more nuance, but cost more to store and compute.

They power most of the modern retrieval stack: semantic searchSemantic search is meaning-based retrieval — matching what a user means, not just the words they typed. Search engines detect entities, expand synonyms, infer intent, and rank by conceptual relevance, which is why keyword stuffing lost its power and topical depth gained it. (find by meaning), clustering and deduplication, search rerankingReranking is the second stage of a retrieval pipeline: after a cheap, broad first pass pulls a candidate set of documents or passages, a slower, more precise model re-scores and reorders that shortlist by true relevance before the results are served or handed to an LLM., and the retrieval layer in RAG pipelines, where content is chunked, embedded, stored in a vector databaseVector search finds content by comparing the meaning of a query against stored content as numerical vectors, retrieving the closest ones in a high-dimensional embedding space. At scale it uses approximate nearest neighbor (ANN) algorithms — not exact comparison — to search billions of vectors in milliseconds., and matched against an embedded query. Google’s own search pipeline uses embedding-based retrieval (Neural Matching / RankEmbed) alongside its classic keyword index.

The SEO takeaway is not that there’s a special embedding knob to turn. Google introduced systems such as BERT to better understand the intent and context of queries, not to create a new markup requirement for publishers. Clear, coherent content remains the practical goal.

Related: Vector Search, Semantic Search, Retrieval-Augmented Generation (RAG)