Embeddings in AI are numerical representations of information, such as words or passages, that let a model compare items mathematically. A text embedding is a vector, meaning an ordered list of numbers. Those numbers can place related items near one another in a model's representation, helping applications find meaning beyond an exact keyword match. An embedding is a representation used by a system, rather than a written answer to a question.
| Term | Plain meaning |
|---|---|
| Vector | An ordered list of numbers |
| Dimension | One coordinate in that list |
| Embedding space | The mathematical space in which vectors are compared |
| Static word embedding | One representation for a word across its uses |
| Contextual embedding | A representation incorporating surrounding information |
How do embeddings work in AI?
Imagine a college website with separate pages for its library, examination office and hostel. A student searches for information about borrowing a book. An embedding-based search system can compare a representation of that request with representations of the pages. The aim is to identify related content even when a page uses a different phrase, such as lending rules. This is an illustration of the search task, not a claim about a particular college service.
The important step is turning information into comparable numbers. Google's text embedding documentation describes dense vectors as representations designed to capture meaning. It explains that this helps searches find passages aligned with a query's meaning, instead of depending only on direct word or syntax matches.
Numbers make comparison possible, but they do not make the representation easy to read. An embedding with many dimensions is not a spreadsheet whose columns have obvious labels such as topic, politeness or difficulty. Google's teaching material cautions that individual dimensions are rarely intuitive. The useful relationship is often between complete vectors, rather than an explanation attached to each coordinate.
What is the difference between encoding and embedding?
The Google Machine Learning Crash Course introduction illustrates the problem with a food recommendation dataset containing 5,000 meals. A simple representation can allocate one position to each meal, putting a one at that meal's position and zeroes elsewhere. This is called one-hot encoding. Each meal then needs a vector with 5,000 entries.
That method identifies an item but does not itself express useful similarity between meals. Two foods do not become close in meaning just because their identifying positions happen to be adjacent. Large, mostly empty vectors also increase the work needed in a neural network. Embeddings provide a way to represent the information in a smaller, dense space and capture relationships learned for the task.
Here is an original simplified analogy: suppose a recommendation system placed dishes on a map using sweetness and how liquid they are. A sweet drink could sit near other sweet drinks, while a dry savoury snack would be farther away. Real embedding coordinates are learned and usually harder to interpret. This imagined map explains similarity without suggesting that a production model literally uses those two labels.
What are static and contextual embeddings?
A static word embedding gives a word a single vector. Google's guide to obtaining embeddings uses word2vec to explain this approach and its limits. A word can have different meanings, but a single representation has to serve all of them. That can make it difficult to distinguish a meaning that depends on the surrounding sentence.
Consider the word bank in two original examples: someone sits on a river bank, while someone else visits a bank to submit a form. Contextual embeddings incorporate information around the word, allowing its representation to change with the sentence. This is why the distinction matters when readers see claims that an AI system understands context. The model uses information about the sequence to build a representation.
Context is not the only influence. The training data and task also shape the resulting space. A food model trained to distinguish vegetarian dishes can organise examples differently from a model trained to recommend breakfast. Relatedness is therefore a relationship within a particular learned representation, rather than a universal score that stays fixed across every application.
What are embeddings used for, and what are their limits?
Semantic search and recommendations are practical uses. In search, a system can rank stored passages against a query representation. In recommendations, it can compare items through the patterns learned from data. Google's documentation also describes storing embeddings in a vector database to support retrieval as the collection grows. That storage and retrieval layer is separate from the text a user eventually reads.
A close match should still be judged against the actual task. Returning a passage about college admissions does not establish that it contains the correct deadline for your course. Similarity helps select candidates; the selected document still needs to be read for the requested fact. This follows from what an embedding represents: a mathematical relationship, with no automatic certificate that a passage answers every part of a question.
For the neighbouring concepts, see our explainers on retrieval and RAG, context windows and fine tuning. Understanding embeddings first makes it easier to distinguish finding relevant information from generating a response or changing a model through training.
