Module 22 — Embeddings and Retrieval
Introduction
Large Language Models (LLMs) are powerful, but they are limited to the knowledge available during training. They cannot automatically access new documents, company data, PDFs, or databases.
Retrieval-Augmented Generation (RAG) solves this problem by retrieving relevant information from external knowledge sources before generating a response.
The first stage of every RAG system is retrieval, where documents are converted into embeddings, indexed in a vector database, and searched using semantic similarity.
Modern RAG systems use:
- Text Embeddings
- Sentence Transformers
- Vector Databases
- Dense Retrieval
- Sparse Retrieval
- Hybrid Search
- Reranking
Applications include:
- Chat with PDFs
- Enterprise Search
- AI Assistants
- Customer Support Bots
- Legal Search
- Medical Knowledge Bases
- Research Assistants
- Code Search
In this module, you'll learn:
- Embeddings
- Sentence Transformers
- FAISS
- ChromaDB
- Pinecone
- Milvus
- Hybrid Search
- Reranking
- Dense Retrieval
- Sparse Retrieval
- Build a Document Search Engine
RAG Retrieval Pipeline
1 Documents 2 │ 3 Text Chunking 4 │ 5 Embedding Model 6 │ 7 Vector Embeddings 8 │ 9 Vector Database 10 │ 11 Similarity Search 12 │ 13 Retrieved Documents 14 │ 15 LLM Prompt 16 │ 17 Generated Answer
1. Embeddings
What are Embeddings?
Embeddings are dense numerical vectors that represent the semantic meaning of text.
Instead of comparing words directly, embeddings place similar meanings close together in a high-dimensional vector space.
Example
1"Cat" → [0.12, -0.44, 0.91, ...] 2 3"Dog" → [0.10, -0.41, 0.88, ...] 4 5"Car" → [-0.81, 0.25, -0.73, ...]
Semantic space
1Animals 2 3Cat ● 4 5Dog ● 6 7 8Car ● 9 10Vehicles
Advantages
- Semantic similarity
- Multilingual search
- Robust retrieval
- Better than keyword matching for many tasks
Generate Embeddings
1from sentence_transformers import SentenceTransformer 2 3model = SentenceTransformer( 4 "all-MiniLM-L6-v2" 5) 6 7sentences = [ 8 "Transformers are neural networks.", 9 "Cats are animals." 10] 11 12embeddings = model.encode(sentences) 13 14print(embeddings.shape)
2. Sentence Transformers
What are Sentence Transformers?
Sentence Transformers are Transformer models optimized to generate sentence-level embeddings.
Unlike standard BERT, they produce fixed-length vectors suitable for similarity search.
Popular Models
- all-MiniLM-L6-v2
- all-mpnet-base-v2
- bge-large
- e5-large
- multilingual-e5
Architecture
1Sentence 2 3↓ 4 5Transformer Encoder 6 7↓ 8 9Pooling Layer 10 11↓ 12 13Sentence Embedding
Applications
- Semantic Search
- Document Retrieval
- Clustering
- Recommendation Systems
Similarity Example
1from sentence_transformers import ( 2 SentenceTransformer, 3 util 4) 5 6model = SentenceTransformer( 7 "all-MiniLM-L6-v2" 8) 9 10embeddings = model.encode( 11 [ 12 "Artificial Intelligence", 13 "Machine Learning" 14 ], 15 convert_to_tensor=True 16) 17 18score = util.cos_sim( 19 embeddings[0], 20 embeddings[1] 21) 22 23print(score)
3. FAISS
What is FAISS?
FAISS (Facebook AI Similarity Search) is a high-performance vector similarity search library developed by Meta.
Features
- Billion-scale search
- CPU and GPU support
- Fast nearest-neighbor search
- Efficient indexing
Architecture
1Embeddings 2 3↓ 4 5FAISS Index 6 7↓ 8 9Nearest Neighbor Search 10 11↓ 12 13Top-K Documents
Build a FAISS Index
1import faiss 2import numpy as np 3 4dimension = 384 5 6index = faiss.IndexFlatL2(dimension) 7 8vectors = np.random.rand( 9 100, 10 dimension 11).astype("float32") 12 13index.add(vectors) 14 15query = np.random.rand( 16 1, 17 dimension 18).astype("float32") 19 20distances, indices = index.search( 21 query, 22 k=5 23) 24 25print(indices)
4. ChromaDB
What is ChromaDB?
ChromaDB is an open-source vector database built specifically for LLM applications.
Features
- Persistent storage
- Metadata filtering
- Fast similarity search
- Python API
- Local deployment
Architecture
1Embeddings 2 3↓ 4 5Chroma Collection 6 7↓ 8 9Similarity Search
Example
1import chromadb 2 3client = chromadb.Client() 4 5collection = client.create_collection( 6 "documents" 7) 8 9collection.add( 10 ids=["1"], 11 documents=["Transformers are amazing."], 12 embeddings=[[0.1] * 384] 13)
5. Pinecone
What is Pinecone?
Pinecone is a managed cloud vector database for large-scale semantic search.
Features
- Fully managed
- Serverless options
- Metadata filtering
- Horizontal scaling
- High availability
Applications
- Enterprise RAG
- Recommendation systems
- AI search
Typical workflow
1Documents 2 3↓ 4 5Embeddings 6 7↓ 8 9Pinecone Index 10 11↓ 12 13Search
6. Milvus
What is Milvus?
Milvus is an open-source vector database designed for billion-scale similarity search.
Features
- Distributed architecture
- GPU acceleration
- Multiple indexing algorithms
- Metadata filtering
- Scalable deployments
Applications
- Enterprise AI
- Image search
- Video retrieval
- Large-scale RAG
Comparison of Vector Databases
| Database | Deployment | Best For |
|---|---|---|
| FAISS | Local library | Fast local search |
| ChromaDB | Local / Self-hosted | Small to medium RAG systems |
| Pinecone | Managed cloud | Production applications |
| Milvus | Distributed | Large-scale enterprise systems |
7. Dense Retrieval
What is Dense Retrieval?
Dense retrieval uses dense embeddings to retrieve semantically similar documents.
Pipeline
1Question 2 3↓ 4 5Embedding 6 7↓ 8 9Vector Search 10 11↓ 12 13Relevant Documents
Advantages
- Semantic understanding
- Synonym matching
- Better contextual retrieval
8. Sparse Retrieval
What is Sparse Retrieval?
Sparse retrieval uses keyword-based representations such as BM25 or TF-IDF.
Pipeline
1Query 2 3↓ 4 5BM25 6 7↓ 8 9Keyword Matching 10 11↓ 12 13Documents
Advantages
- Exact keyword matching
- Fast retrieval
- Strong lexical precision
Limitations
- Doesn't capture semantic similarity well
9. Hybrid Search
What is Hybrid Search?
Hybrid search combines dense retrieval and sparse retrieval to leverage the strengths of both approaches.
Architecture
1Query 2 3↓ 4 5Dense Search 6 7+ 8 9Sparse Search 10 11↓ 12 13Score Fusion 14 15↓ 16 17Final Ranking
Advantages
- Higher recall
- Better precision
- Improved enterprise search performance
10. Reranking
What is Reranking?
Initial retrieval often returns many relevant candidates. A reranker uses a more computationally expensive model to reorder those candidates by relevance.
Pipeline
1Query 2 3↓ 4 5Top 20 Documents 6 7↓ 8 9Cross Encoder 10 11↓ 12 13Top 5 Documents
Benefits
- Improved answer quality
- Better ranking accuracy
- More relevant context for the LLM
Cross-Encoder Reranking Example
1from sentence_transformers import CrossEncoder 2 3model = CrossEncoder( 4 "cross-encoder/ms-marco-MiniLM-L-6-v2" 5) 6 7pairs = [ 8 ( 9 "What is a Transformer?", 10 "A Transformer is a deep learning architecture." 11 ), 12 ( 13 "What is a Transformer?", 14 "The weather is sunny." 15 ) 16] 17 18scores = model.predict(pairs) 19 20print(scores)
Practice — Build a Document Search Engine
Step 1: Generate Embeddings
1from sentence_transformers import SentenceTransformer 2 3model = SentenceTransformer( 4 "all-MiniLM-L6-v2" 5) 6 7documents = [ 8 "Transformers revolutionized NLP.", 9 "PyTorch is a deep learning framework.", 10 "RAG improves LLM knowledge." 11] 12 13embeddings = model.encode(documents)
Step 2: Create a FAISS Index
1import faiss 2import numpy as np 3 4dimension = embeddings.shape[1] 5 6index = faiss.IndexFlatL2(dimension) 7 8index.add( 9 np.array(embeddings).astype("float32") 10)
Step 3: Search Documents
1query = "How does Retrieval-Augmented Generation work?" 2 3query_embedding = model.encode([query]) 4 5distances, ids = index.search( 6 np.array(query_embedding).astype("float32"), 7 k=2 8) 9 10for i in ids[0]: 11 print(documents[i])
What You'll Learn
- Generate semantic embeddings.
- Index documents using FAISS.
- Perform nearest-neighbor similarity search.
- Build the retrieval layer for a RAG application.
Choosing the Right Retrieval Solution
| Scenario | Recommended Solution |
|---|---|
| Local experimentation | FAISS |
| Personal RAG projects | ChromaDB |
| Production cloud deployment | Pinecone |
| Large distributed systems | Milvus |
| Semantic search | Dense Retrieval |
| Exact keyword matching | Sparse Retrieval (BM25) |
| Highest retrieval quality | Hybrid Search + Reranking |
Best Practices
| Recommendation | Benefit |
|---|---|
| Split large documents into meaningful chunks | Improves retrieval accuracy |
| Use high-quality embedding models | Better semantic representations |
| Store metadata with vectors | Enables filtering by source, author, or date |
| Combine dense and sparse retrieval | Higher recall and precision |
| Apply reranking before generation | Improves context relevance |
| Normalize and deduplicate documents | Reduces redundant retrieval |
| Tune the number of retrieved documents (Top-K) | Balances context quality and token usage |
Module Summary
After completing this module, you will be able to:
- Explain the role of embeddings in Retrieval-Augmented Generation (RAG).
- Generate semantic embeddings using Sentence Transformers.
- Build vector indexes with FAISS.
- Understand how ChromaDB, Pinecone, and Milvus store and retrieve embeddings.
- Compare dense retrieval, sparse retrieval, and hybrid search strategies.
- Apply reranking models to improve retrieval quality.
- Design and implement a document search engine as the retrieval component of a RAG system.
- Choose the appropriate vector database and retrieval strategy based on project requirements.
Next Module: Module 23 – Building End-to-End RAG Systems, where you'll learn document chunking, metadata filtering, retrieval pipelines, LangChain, LlamaIndex, context injection, conversational memory, evaluation, and build a complete PDF Chatbot and Enterprise RAG application.