Definition
Search "how do I get money back for a cancelled train" and a good help centre shows the article titled "Refund rules for IRCTC e-tickets", even though the two share almost no words. The search understood meaning, using embeddings.
In LLM applications, an embedding is a vector, often with several hundred to a few thousand numbers, produced by an embedding model to represent the meaning of a word, sentence or document. Texts with similar meanings get vectors that point in similar directions, so cosine similarity measures how related they are. Embeddings are stored in vector databases and power semantic search, retrieval-augmented generation (RAG), recommendation, clustering, duplicate detection and classification. Inside an LLM, every token is also turned into an embedding at the first layer.
In simple words
Imagine a huge map where every sentence has a location. Sentences about train refunds cluster in one neighbourhood, recipes in another, cricket in a third. To find what someone means, you look for the nearest neighbours of their question on the map.
Overview
- An embedding is a vector of numbers that represents the meaning of a word, sentence, image or document.
- Texts with similar meanings have embeddings that point in similar directions.
- Cosine similarity compares embeddings, from −1 to 1.
- Embedding models are usually smaller and cheaper than chat models.
- Embeddings are stored in vector databases for fast similarity search.
- They power semantic search, recommendations, clustering and retrieval-augmented generation.
Why it matters
Embeddings connect LLMs to your own data. Almost every serious LLM product, from document chatbots to customer-support search, uses them to find the right information before the model writes an answer.
Key terms
- Embedding model
- A model that turns text into a fixed-length vector representing meaning.
- Vector database
- A store that finds the nearest vectors to a query quickly, even among millions.
- Semantic search
- Search by meaning rather than exact keywords.
- Chunking
- Splitting long documents into passages before embedding them.
- Nearest neighbours
- The stored vectors most similar to a query vector.
- Hybrid search
- Combining keyword scores and embedding similarity.
Types
Embeddings come in several types, and they are used in several types of task.
1. Word and token embeddings
Vectors for individual words or tokens.
- How it works: Static methods (Word2Vec, GloVe) give one vector per word; inside LLMs, each token's embedding is updated by every layer using context.
- Example: "Delhi" and "Mumbai" lying close together as cities.
- When to use it: Use them for analysing vocabulary and as inputs to classic models; most applications now use sentence embeddings.
2. Sentence and document embeddings
One vector for a whole sentence, paragraph or document.
- How it works: Embedding models trained with contrastive learning map paraphrases close together and unrelated texts far apart.
- Example: "How do I reset my password?" and "I forgot my login" having similarity 0.8 or more.
- When to use it: Use them for semantic search, FAQ matching, duplicate detection and RAG.
3. Multilingual embeddings
Embedding spaces shared by many languages.
- How it works: Sentences with the same meaning in Hindi and English land close together, so search works across languages.
- Example: A Hindi question retrieving the right English policy document.
- When to use it: Use them for Indian-language and cross-language search and support. They are especially valuable in India, where users mix Hindi, English and regional languages in one question.
4. Image and multimodal embeddings
Vectors for images, audio or video, often in a space shared with text.
- How it works: Models such as CLIP embed images and captions together, so text can search pictures and vice versa.
- Example: Searching a photo gallery for "sunset at the beach" with no tags.
- When to use it: Use them for image search, deduplication and zero-shot classification.
5. Embedding tasks: search, clustering, classification and anomaly detection
The main ways embeddings are used once computed.
- How it works: Nearest-neighbour search finds similar items; k-means groups them; a small classifier on embeddings predicts labels; distance from typical embeddings flags unusual items.
- Example: Clustering 10,000 customer comments into themes without labels.
- When to use it: Use embeddings as cheap, strong features whenever meaning matters more than exact words.
Explained in detail
How semantic search works
First, documents are split into chunks, typically a few hundred tokens each, often with a small overlap so that sentences are not cut in half. Each chunk is sent through an embedding model and the resulting vector is stored, along with the original text and metadata such as the title and date, in a vector database. When a user asks a question, the same embedding model turns the question into a vector, the database returns the chunks with the highest cosine similarity, and these are shown to the user or passed to an LLM to write an answer. Because similar meanings map to nearby vectors, "paise wapas" can match "refund", and a Hindi question can match an English passage when the embedding model is multilingual.
Choices that affect quality
| Choice | Effect |
|---|---|
| Chunk size | Too small loses context; too large mixes topics and dilutes the match |
| Embedding model | Must support your languages and domain; test several on real queries |
| Metadata filters | Restrict search to the right state, year or product before ranking |
| Hybrid search | Keywords catch exact codes and names; embeddings catch paraphrases |
| Re-ranking | A second, more careful model re-orders the top results |
Limits and care
Embeddings capture similarity, not truth: an outdated circular and the current one may be equally similar to a question, so store dates and prefer recent versions. Exact identifiers such as PNR numbers, section numbers of an Act or product codes are often matched poorly by embeddings, which is why hybrid search is common. Embeddings also reflect biases in the text they were trained on. Finally, embedding text sends it to the embedding model; if you use a cloud service, check what it does with your data, and do not embed sensitive personal information without permission.
Embeddings have many uses beyond search. Grouping complaint embeddings with k-means reveals common themes without reading every message. Comparing a new question with past questions finds duplicates in a forum. Recommending articles similar to the one a student just read uses the same nearest-neighbour idea. And a simple classifier trained on embeddings, such as logistic regression, often works well with only a few hundred labelled examples, because the embedding model has already learned much of the meaning.
How it works
- Split documents into overlapping chunks with titles and dates as metadata.
- Embed every chunk with a model suited to your languages.
- Store vectors and text in a vector database.
- Embed each query, retrieve the nearest chunks and optionally re-rank.
- Measure retrieval quality: does the right chunk appear in the top few results?
In code
import numpy as np
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer
docs = [
"refund rules for cancelled train tickets booked online",
"how to get money back after cancelling a railway ticket",
"tatkal ticket booking timings and charges",
"steps to change your registered mobile number",
"update phone number linked to your account",
"best time to visit hill stations in summer",
]
# TF-IDF + SVD gives small dense vectors: a classic, simple form of embedding (latent semantic analysis)
tfidf = TfidfVectorizer().fit(docs)
svd = TruncatedSVD(n_components=3, random_state=0).fit(tfidf.transform(docs))
embed = lambda texts: svd.transform(tfidf.transform(texts))
D = embed(docs)
def search(query, k=2):
v = embed([query])[0]
sims = D @ v / (np.linalg.norm(D, axis=1) * np.linalg.norm(v) + 1e-9)
return [(round(float(sims[i]), 2), docs[i]) for i in np.argsort(-sims)[:k]]
for qtext in ["money back for cancelled train", "change my phone number"]:
print(qtext)
for score, d in search(qtext):
print(f" {score:.2f} {d}")
print("Vector length per document:", D.shape[1])Output
money back for cancelled train 0.92 refund rules for cancelled train tickets booked online 0.40 how to get money back after cancelling a railway ticket change my phone number 1.00 steps to change your registered mobile number 1.00 update phone number linked to your account Vector length per document: 3
Try it yourself
A college notice-board bot
A college has 900 PDF circulars and students can never find the right one.
Match the step to its purpose
Why is each step done?
- Chunking
- Embedding
- Vector database
- Metadata filter
- Keyword search alongside
Common mistakes
- Using an English-only embedding model for Hindi or Tamil questions.
- Ignoring dates, so outdated documents are retrieved.
- Never measuring whether the right passage is retrieved.
Check yourself
Summary
- Embeddings are meaning vectors; similar texts point in similar directions.
- Chunk, embed, store, then retrieve nearest neighbours for semantic search and RAG.
- Choose multilingual models, use metadata and hybrid search, and measure retrieval.
Finished reading?
Sources: Jurafsky and Martin, Speech and Language Processing (3rd ed. draft): Large Language Models · Hugging Face course: tokenizers and transformer models