RAG and Fine-Tuning: Giving Models Knowledge
Most problems with a model’s answers are either missing knowledge or inconsistent behaviour. Start with which fix applies to which, then go deep on building retrieval that works — from splitting documents to measuring the answers.
6 articles · start with the first and read in order, or jump to what you need
Choosing an approach
Prompting, retrieval or fine-tuning — and how RAG relates to tools and agents.
- 1 · IntermediatePrompting, RAG or Fine-Tuning: Which to UseThree ways to make a model better at your task, in the order to try them — because most teams reach for the most expensive one first.6 min read
- 2 · IntermediateRAG, MCP and Agents Solve Three Different ProblemsThey get compared as if you pick one. You do not — they answer separate questions: what the model knows, what it can reach, and who decides what happens next.4 min read
RAG in depth
How a retrieval system is built, step by step — and how to tell whether it works.
- 3 · BeginnerHow RAG Works, Step by StepRetrieval-augmented generation end to end: what happens when documents are indexed, what happens when a question arrives, and where each step goes wrong.6 min read
- 4 · IntermediateChunking and Embeddings for RAGHow to split documents so the right passage can be found, what embeddings are and how to choose a model, and why metadata matters as much as the text.6 min read
- 5 · AdvancedKeyword, Vector and Hybrid Search for RAGWhy keyword and vector search fail in different ways, how hybrid search and reranking combine them, and how query rewriting finds what the raw question misses.5 min read
- 6 · IntermediateHow to Evaluate a RAG SystemMeasure retrieval and answers separately, build a test set from real questions, use model-based judges carefully, and know which fix each failure points to.5 min read
Common questions: RAG & Fine-Tuning
How does RAG work?
Retrieval-augmented generation runs in two phases. Ahead of time, documents are split into chunks, each chunk is turned into an embedding, and the chunks are stored in a search index. When a question arrives, the most relevant chunks are retrieved, the best are reranked, and the model answers from them, citing its sources.
What is chunking in RAG?
Chunking is splitting documents into passages that can be retrieved individually. Chunks should be small enough to be precise and large enough to make sense on their own; splitting along the document’s headings and sections, and keeping metadata such as source, date and permissions with each chunk, usually works better than a fixed character count.
What is hybrid search in RAG?
Hybrid search runs keyword search, which matches exact terms such as names and codes, and vector search, which matches meaning, then merges the two result lists. A reranker then re-reads the query with each top candidate to keep only the most relevant passages.
How do you evaluate a RAG system?
Measure retrieval and answers separately on a test set of real questions. For retrieval, check whether the passage containing the answer was found and how highly it ranked. For answers, check faithfulness — whether every claim is supported by the retrieved passages — plus relevance, correctness, and whether the system declines when the documents do not contain the answer.
Should I use RAG or fine-tuning?
Use retrieval (RAG) when the model lacks facts, especially private or current information. Consider fine-tuning only when a consistent style, format or behaviour cannot be held with good prompts at your volume. Try improving the prompt before either.
What is the difference between RAG and MCP?
RAG is about what a model knows: relevant documents are retrieved and placed in front of it at request time. MCP, the Model Context Protocol, is about what it can reach: a standard way to connect models to tools and data. They solve different problems and are often used together.
Can fine-tuning teach a model new facts?
It is the wrong tool for that. Facts learned in training are hard to update and hard to trace back to a source. Retrieval keeps facts in documents you can update and cite.