Guide : Embeddings

TL;DR — An embedding turns text into a liste of numbers — coordinates in a giant space — où choses que mean similaire choses fin up fermer ensemble. That’s how AI search and modern moteur de recherches match votre content to a requête by meaning au lieu de simplement matching keywords. Vous pouvez’t “add” embeddings to une page; they’re how the machine reads what’s déjà là.

Ce que an embedding is

An embedding represents input as a numeric vector intended to preserve utile relationships pour tasks tel as semantic similarity. Evidence for this claim Embeddings represent inputs as numerical vectors that can be compared for relatedness and used for search, clustering, and classification. Scope: OpenAI embedding models and documented uses; vector dimensions and behavior vary by model. Confidence: high · Verified: 2026-07-14 OpenAI: Embeddings guide Geometry and dimensionality are model-specific plutôt que universal meanings attached to chaque coordinate. Evidence for this claim Learned vector representations can encode useful distributional relationships between words. Scope: Word2vec-era language representations; observed vector relationships are model- and training-data-specific, not ground truth. Confidence: high · Verified: 2026-07-14 Mikolov et al.: Efficient Estimation of Word Representations

Computers don’t comprendre words — ils comprendre numbers. An embedding is the bridge: a model reads a piece of text and turns it into a liste of numbers (appelé a vector) que captures its meaning. Think of it comme a emplacement on a map. Two pieces of text que mean similaire choses obtenir placed fermer ensemble; unrelated text ends up far apart.

So “laptop for gaming” and “high-performance laptop for games” land near chaque autre, même though ils barely share words. “Banana bread recipe” lands somewhere sinon entirely. The model learned ce by reading enormous amounts of text and noticing qui words and phrases montrer up in similaire contexts.

Pourquoi ce matters pour search

Old-school search matched keywords: vous typed a word, the engine trouvé pages containing que word. Embeddings let search match meaning. That’s pourquoi Google peut réponse a long, conversational question même quand the meilleur page doesn’t utiliser votre exact words — and it’s how AI search outils (the ones que lire sources and écrire vous an réponse) decide qui passages are relevant suffisant to pull in.

La version courte of the pipeline behind AI réponses:

1. Content obtient broken into chunks (passages).
2. Chaque chunk obtient turned into an embedding (a vector).
3. Votre question obtient turned into an embedding aussi.
4. The system trouve the chunks whose vectors are closest to votre question’s vector.
5. Ceux chunks become the source material the AI writes its réponse from.

Ce que cela signifie pour votre content

Here’s the partie personnes overstate, so let me be clair: there’s aucun “embedding optimization” vous submit to Google. Google’s Danny Sullivan said exactly que à propos de BERT (un of Google’s embedding-based systems): “There’s nothing to optimize pour.”

Ce que en réalité helps is the même advice as ever, simplement with a clearer raison behind it. Écrire content que genuinely and thoroughly covers a topic. Coherent, focused content produces clean, consistent embeddings que sit fermer to the questions it devrait réponse. Keyword-stuffed, scattered content produces a muddier signal. You’re pas feeding a formula — you’re writing something a model (and a human) peut clearly comprendre.

Vouloir the réel mechanics — dimensions, cosine similarity, the word2vec-to-BERT history, and how Google en réalité uses embeddings in ranking? Switch to the Avancé tab.

TL;DR — An embedding is a dense vector of floating-point numbers (typically hundreds to a few thousand dimensions) que encodes meaning, produced by an encoder model — pas a generative LLM. Similaire meaning → nearby vectors, mesuré with cosine similarity. The field déplacé from static word embeddings (word2vec, GloVe) to contextual ones (BERT) to sentence-level and modern API embeddings. Google uses embedding-based retrieval (Neural Matching / RankEmbed, RankEmbedBERT) alongside its keyword index — hybrid, pas a replacement. Embeddings are aussi the retrieval backbone of RAG. There’s aucun embedding knob to turn; topical coherence is ce que rend content cluster near the correct requêtes.

Ce que an embedding en réalité is

Embeddings prise en charge similarity and retrieval, but ils ne sont pas a direct mesurer of truth, quality, or ranking valeur. Evidence for this claim Embeddings represent inputs as numerical vectors that can be compared for relatedness and used for search, clustering, and classification. Scope: OpenAI embedding models and documented uses; vector dimensions and behavior vary by model. Confidence: high · Verified: 2026-07-14 OpenAI: Embeddings guide Rerésultats de recherche depend on the trained model and evaluation setting. Evidence for this claim Learned vector representations can encode useful distributional relationships between words. Scope: Word2vec-era language representations; observed vector relationships are model- and training-data-specific, not ground truth. Confidence: high · Verified: 2026-07-14 Mikolov et al.: Efficient Estimation of Word Representations

An embedding is a dense numerical vector — a liste of floating-point numbers — que represents text (or images, audio, video) as a point in a high-dimensional space. OpenAI’s documentation states it plainly: “An embedding is a vector (liste) of floating point numbers.”

The defining property is geometric: semantically similaire content has similaire vectors. Text que signifie roughly the même chose points in roughly the même direction; unrelated text points elsewhere. That’s pas a happy accident — the model is trained so que words and phrases utilisé in similaire contexts fin up with similaire vectors. Meaning becomes position.

A few choses worth getting precise:

• Encoder, pas generator. Embeddings come from encoder models whose job is to compress meaning into a fixed-size vector. That’s a différent architecture and objectif from a generative LLM, qui predicts the suivant token. (Plus on the internal-vs-API distinction ci-dessous.)
• Dense, pas sparse. Unlike one-hot or bag-of-words representations (mostly zeros, un slot per vocabulary word), embeddings pack meaning into every dimension. Google’s ML glossary frames embeddings as lower-dimensional, dense representations que fix ce que one-hot encoding can’t express — ils let a model recognize que “hot dogs and shawarmas are more related than hot dogs and salads.”
• Plus élevé dimensions ≠ toujours meilleur. Plus dimensions peut capture plus nuance, but ils cost plus to store and compute, and the gain is task-dependent. It’s a trade-off, pas a “bigger is better” dial.

Measuring similarity: cosine similarity

To comparer two embeddings vous mesurer the distance — or really the angle — entre les. The standard metric is cosine similarity: it measures the angle entre two vectors regardless of leur length, scoring from −1 (opposite) via 0 (unrelated/orthogonal) to 1 (identical direction). Plus petit distance = plus connexe.

Nombreux embedding APIs normalize vectors to unit length, qui rend cosine similarity and dot product produce the même ranking — OpenAI notes cosine is the conventional, slightly cheaper choice. Voyage AI (the embedding provider Anthropic recommends) puts the intuition cleanly: “the cosine similarity entre two embeddings captures the semantic relatedness of the corresponding original passages.” Doing que nearest-neighbor comparison at scale is its propre problem — that’s the job of vector search.

How we got ici: the evolution

The story runs from unique words to whole passages, and from fixed meanings to context-aware ones.

• word2vec (Google, 2013). Mikolov et al. introduced two architectures (CBOW and Skip-Gram) pour learning dense word vectors from huge corpora. The famous result: the vector “King” − “Man” + “Woman” lands closest to “Queen” — evidence que vector arithmetic captures semantic relationships. (Caveat: que analogy is illustrative, pas guaranteed every temps; selon the model it peut land on “kings” or “monarch.”) Ces are static embeddings — un fixed vector per word, so “bank” obtient the même vector in “river bank” and “bank account.”
• GloVe (Stanford, 2014). A count-based alternative construit on global co-occurrence statistics plutôt que a predictive network — différent objective, similarly utile embeddings. Aussi static.
• Universal Sentence Encoder (Google, 2018). Embeddings pour whole sentences, pas simplement words. The core idea: “Sentences are semantically similaire si ils have a similaire distribution of réponses” — “How old are vous?” and “Ce que is votre age?” invite the même réponses, so ils embed fermer ensemble.
• BERT (Google, 2018; deployed in Search Oct 2019). The big shift: contextual embeddings. The même word obtient a différent vector selon the surrounding sentence, parce que BERT is bidirectional — it reads the words avant and après a token to fix its meaning. So “bank” in “river bank” and “bank account” finalement obtenir différent vectors.
• Sentence-BERT (2019). Solved BERT’s scaling problem pour similarity search. Vanilla BERT nécessite les deux sentences fed in ensemble, qui is computationally brutal at scale; SBERT produces fixed-size sentence embeddings vous pouvez comparer with cosine similarity, cutting the cost of finding the la plupart similaire pair in a grand corpus from hours to seconds.
• Modern embedding APIs (2024–présent). OpenAI’s text-embedding-3 family, Google’s Gemini embeddings, Voyage, and Cohere’s embed-v4.0 — multilingual, increasingly multimodal (text, image, audio, video in un space), and resizable via Matryoshka Representation Learning (truncate the vector to fewer dimensions sans retraining, trading a little accuracy pour storage and speed).

How Google uses embeddings in Search

Google’s ranking pipeline isn’t purely semantic or purely keyword — it’s hybrid, and the embedding pieces supplement the classic inverted index plutôt que replacing it. From Google’s propre ranking-systems documentation and Pandu Nayak’s DOJ antitrust testimony, the named systems inclure:

• BERT — Google’s words-in-context system. At launch it helped Search “meilleur comprendre un in 10 searches in the U.S. in English,” surtout plus long conversational requêtes où prepositions comme “for” and “to” modifier the meaning.
• Neural Matching / RankEmbed — embedding-based retrieval que translates requêtes and documents into the même vector space to surface conceptually matching results même sans shared keywords. Nayak décrit it as a supplement: “RankEmbed identifies a few plus documents to ajouter to ceux identified by the traditional retrieval.” Retrieval là is fondé on a dot product / distance mesurer in the embedding space.
• RankEmbedBERT — a plus tard evolution combining RankEmbed’s retrieval with BERT’s language understanding, trained on quality-rater scores and search logs, and notably meilleur on complex, long-tail requêtes.

The practical reading: keyword presence encore matters parce que lexical retrieval (the inverted index, BM25-style) encore fait the first-pass narrowing. The embedding systems ajouter conceptually-related candidates and re-rank. Les deux signals are in play — qui is exactly pourquoi “BERT killed keywords” is incorrect.

Embeddings in AI search (the RAG pipeline)

Ce is où embeddings touch AI Overviews and AI search assistants la plupart directement. Retrieval-Augmented Generation (RAG) uses embeddings as its retrieval couche:

1. Index: content is chunked into passages, chaque passage is embedded, and the vectors go into a vector database.
2. Retrieve: the user’s requête is embedded, a nearest-neighbor search trouve the closest chunks (via vector search), and the top-K chunks are handed to the LLM as context.
3. Generate: the LLM writes an réponse grounded in ceux retrieved chunks.

Structure matters ici parce que chaque chunk is retrieved in isolation. Dan Petrovic’s research (cited in Ahrefs’ Ce que We En réalité Know À propos de Optimizing pour LLM Search) trouvé Chrome processes seulement the premier ~30 passages of une page pour embeddings and chunks les in roughly 200-word passages with overlap to preserve cross-chunk context. Si a section can’t stand on its propre une fois it’s pulled out of lune page, it represents votre content poorly.

Token embeddings vs. text embedding APIs

A distinction que trips personnes up: the embeddings à l’intérieur an LLM and the embeddings vous obtenir from an API ne sont pas the même chose.

• Token embeddings are the model’s internal representations — chaque token obtient a vector that’s transformed couche by couche during generation. They’re machinery pour producing the suivant token.
• Text embedding APIs (OpenAI, Google, Voyage, Cohere) produce a unique fixed-size vector pour an entier input string, purpose-built pour retrieval and similarity. Souvent a separate model with a différent training objective.

Quand SEOs talk à propos de “embedding a page” or running cosine similarity pour internal linking or keyword clustering, ils mean the API kind.

Ce que cela signifie pour le SEO

• Semantic coherence beats keyword density. Parce que models comprendre context, “laptop for gaming” and “high-performance laptop” déjà sit fermer. Stuffing doesn’t aider — it produces topically scattered content with a muddier embedding.
• Structure pour chunked retrieval. Passages obtenir embedded and retrieved on leur propre. Put important content early, garder sections self-contained, utiliser clair semantic HTML.
• Topical comprehensiveness. Content que genuinely covers a subject ends up near plus of the connexe requêtes in vector space. That’s the mechanism behind “construire topical authority.”
• There’s aucun embedding knob. Danny Sullivan, on BERT: “There’s nothing to optimize pour… The fundamentals of us seeking to reward great content remain unchanged.” The cosine-similarity scores vous obtenir from a outil are analysis aids, pas inputs vous submit to Google.

Embeddings are the connective tissue sous la plupart of ce cluster: they’re ce que semantic search runs on, ce que vector search compares, ce que chunking prepares text pour, and the retrieval backbone of RAG. They’re aussi pourquoi exploration encore matters premier — content has to be récupéré avant quelconque system peut embed it.

AI summary

A condensed prendre on the Avancé version:

• An embedding = a dense vector of floating-point numbers que encodes meaning; similaire meaning → nearby vectors. OpenAI: “An embedding is a vector (liste) of floating point numbers.”
• Produced by encoder models, pas generative LLMs — différent architecture and objectif. Dense, pas sparse.
• Similarity is mesuré with cosine similarity (the angle entre vectors, −1 to 1). Normalized vectors faire cosine and dot product equivalent pour ranking.
• Evolution: word2vec (2013) and GloVe (2014) gave static word vectors → UTILISER (2018) embedded whole sentences → BERT (2018) made embeddings contextual (même word, différent vector by context) → Sentence-BERT (2019) made similarity search fast → modern multimodal, resizable API models (2024–présent).
• Google uses embeddings in a hybrid pipeline: BERT (words in context), Neural Matching / RankEmbed (embedding retrieval que supplements the keyword index), and RankEmbedBERT. Keywords encore do first-pass retrieval — embeddings ajouter and re-rank.
• RAG pipeline: chunk → embed → store in a vector DB → embed the requête → retrieve nearest chunks → LLM generates a grounded réponse. Chaque chunk is retrieved in isolation, so structure matters.
• SEO upshot: aucun “embedding optimization” exists — Danny Sullivan: “There’s nothing to optimize pour” with BERT. Topically coherent, self-contained, thorough content clusters near the requêtes it devrait réponse.

Documentation officielle

Primary-source documentation on embeddings from Google and the major embedding providers.

Google

• Machine Learning Glossary — Embeddings — the dense-vs-sparse definition and pourquoi one-hot encoding can’t express relatedness.
• A guide to Recherche Google ranking systems — BERT, Neural Matching, RankBrain, Passage Ranking, and MUM décrit in Google’s propre words.
• Understanding searches meilleur que ever avant (BERT) — the October 2019 announcement of BERT in Search.
• Advances in Semantic Textual Similarity — the Universal Sentence Encoder (2018) and the “similar distribution of responses” idea.
• Gemini API — Embeddings — Google’s current multimodal embedding model, dimensions, and utiliser cas (RAG, semantic search, reranking, clustering).

OpenAI

• Vector embeddings guide — “An embedding is a vector (list) of floating point numbers”; cosine similarity guidance.
• Nouveau embedding models and API updates — the text-embedding-3 family and Matryoshka dimension shortening (Jan 2024).

Autre providers

• Cohere — Embeddings — multilingual embed-v4.0, separate requête/document input types, compression options.
• Voyage AI — Quickstart — Anthropic’s recommended provider; “the cosine similarity between two embeddings captures the semantic relatedness of the corresponding original passages.”

Foundational papers

• word2vec — Mikolov et al., 2013 — the original dense word vectors and the king−man+woman analogy.
• GloVe — Pennington, Socher, Manning, 2014 — global co-occurrence word vectors.
• BERT — Devlin et al., 2018 — bidirectional, contextual embeddings.
• Sentence-BERT — Reimers & Gurevych, 2019 — fast sentence embeddings pour similarity search.

Quotes from the source

On-the-record statements from Google and the embedding providers. Chaque lien is a deep lien que jumps to the quoted passage on the source page.

Ce que an embedding is

• “An embedding is a vector (list) of floating point numbers.” — OpenAI, Vector embeddings guide. Jump to quote

Google — BERT and language understanding

• “At its core, Search is about understanding language.” — Pandu Nayak, VP of Search, Google (BERT announcement, October 2019). Jump to quote
• “BERT va aider Search meilleur comprendre un in 10 searches in the U.S. in English.” — Recherche Google Blog, October 2019. Jump to quote

Google — on optimizing pour BERT

• “There’s nothing to optimize pour with BERT… The fundamentals of us seeking to reward great content remain unchanged.” — Danny Sullivan, Recherche Google Liaison. Jump to quote

Google — semantic similarity (Universal Sentence Encoder)

• “Sentences are semantically similaire si ils have a similaire distribution of réponses.” — Google Research, “Advances in Semantic Textual Similarity” (May 2018). Lire the source

Voyage AI (Anthropic’s recommended provider) — cosine similarity

• “The cosine similarity entre two embeddings captures the semantic relatedness of the corresponding original passages.” — Voyage AI Quickstart. Lire the source

Pandu Nayak (DOJ antitrust testimony) — RankEmbed

• “RankEmbed identifies a few plus documents to ajouter to ceux identified by the traditional retrieval.” — coverage of Pandu Nayak’s testimony, DOJ v. Google. Lire the coverage

The mental models

1. Meaning becomes position. An embedding turns text into coordinates. Fermer ensemble = similaire meaning; far apart = unrelated. Everything sinon — semantic search, clustering, RAG retrieval — is simplement measuring distance in que space.

2. Static vs. contextual. word2vec and GloVe give chaque word un fixed vector (“bank” is toujours the même). BERT and its descendants give chaque occurrence a vector que dépend on context (“river bank” ≠ “bank account”). Older keyword-similarity outils construit on word2vec can’t tell ceux apart; BERT-based ones peut.

3. Token embeddings ≠ text embedding APIs. À l’intérieur an LLM, token embeddings are working machinery pour generating the suivant token. Text embedding APIs output un condensed vector per string, construit pour retrieval and similarity. Différent models, différent jobs. Quand SEOs “embed a page,” ils mean the API kind.

4. Hybrid retrieval, pas replacement. Google runs lexical retrieval (the keyword inverted index) and embedding-based retrieval (Neural Matching / RankEmbed). Keywords narrow premier; embeddings ajouter conceptually-related candidates and re-rank. Neither alone is the whole system.

5. The RAG loop. Chunk → embed → store → embed the requête → retrieve nearest chunks → generate a grounded réponse. Parce que chaque chunk is retrieved in isolation, écrire self-contained passages and front-load ce que matters.

6. The decision rule pour content. There’s aucun embedding to optimize. Demander à la place: fait ce passage clearly and thoroughly cover the chose it claims to? Coherent content clusters near the correct requêtes; scattered, stuffed content doesn’t.

Embeddings — cheat sheet

Ce que c’est in un line A dense vector (liste of floating-point numbers) encoding meaning; similaire meaning → nearby vectors; comparé with cosine similarity.

Cosine similarity scale

Normalized (unit-length) vectors → cosine similarity and dot product give the même ranking.

The evolution, En un coup d’œil

Google’s named embedding systems

The RAG retrieval loop

1. Chunk content into passages
2. Embed chaque chunk → vector
3. Store vectors in a vector database
4. Embed the incoming requête
5. Nearest-neighbor (vector) search → top-K chunks
6. LLM generates an réponse grounded in ceux chunks

Fast facts

• Dimensions: typically hundreds to a few thousands — plus nuance, plus storage/compute; pas toujours meilleur.
• Produced by encoder models, pas generative LLMs.
• Matryoshka lets vous truncate to fewer dimensions sans retraining.
• Switching embedding models signifie re-embedding everything — différent models’ spaces aren’t compatible.
• Aucun “embedding optimization” to submit to Google — Danny Sullivan: “There’s nothing to optimize for” (BERT).

Testez vos connaissances: Embeddings

Ressources utiles

My connexe writing & speaking

• Ce que We En réalité Know À propos de Optimizing pour LLM Search — the Ahrefs piece citing my research and Dan Petrovic’s findings on how Chrome chunks and embeds page passages (the ~30-passage limite, 200-word chunks).
• GEO? AEO? LLMO? — my AI search webinar — où retrieval pipelines and embeddings fit into AI search.

Foundational papers (the history)

• word2vec — Mikolov et al., 2013 — dense word vectors and vector arithmetic.
• GloVe — Pennington, Socher, Manning, 2014 — global co-occurrence word vectors.
• BERT — Devlin et al., 2018 — contextual, bidirectional embeddings.
• Sentence-BERT — Reimers & Gurevych, 2019 — fast sentence-level similarity.

From others

• The shift to semantic SEO: Ce que vectors mean pour votre strategy — Ann Robison, Moteur de recherche Land: “Vectors are to AI what structured data is to search engines.”
• How to leverage cosine similarity pour ecommerce SEO — Moteur de recherche Land practitioner guide to applying cosine similarity in réel SEO workflows.
• Introduction to Vector Databases and Comment utiliser AI pour le SEO — Moteur de recherche Journal overview of how vector databases connecter to SEO.
• SEO Utiliser Cas pour Vectorizing the Web with Screaming Frog — iPullRank’s practitioner walkthrough of embeddings pour le SEO.
• Semantic Search Explained: Vector Models’ Impact on SEO Today — Lumar on dense vs. sparse, cosine similarity, and hybrid search.
• Embeddings Explained: Unlocking the Future of SEO — Edd Dawson’s SEO-focused explainer covering practical utiliser cas pour keyword clustering and maillage interne.
• How Recherche Google Fonctionne (inverted index, RankBrain, RankEmbed, DeepRank) — the DOJ-testimony view of Google’s embedding systems.
• sbert.net — the Sentence Transformers library, si vous vouloir to generate embeddings yourself.