Skip to main content

RAG and Fine-Tuning: Giving Models Knowledge

Most problems with a model’s answers are either missing knowledge or inconsistent behaviour. Start with which fix applies to which, then go deep on building retrieval that works — from splitting documents to measuring the answers.

6 articles · start with the first and read in order, or jump to what you need

Common questions: RAG & Fine-Tuning

How does RAG work?

Retrieval-augmented generation runs in two phases. Ahead of time, documents are split into chunks, each chunk is turned into an embedding, and the chunks are stored in a search index. When a question arrives, the most relevant chunks are retrieved, the best are reranked, and the model answers from them, citing its sources.

What is chunking in RAG?

Chunking is splitting documents into passages that can be retrieved individually. Chunks should be small enough to be precise and large enough to make sense on their own; splitting along the document’s headings and sections, and keeping metadata such as source, date and permissions with each chunk, usually works better than a fixed character count.

What is hybrid search in RAG?

Hybrid search runs keyword search, which matches exact terms such as names and codes, and vector search, which matches meaning, then merges the two result lists. A reranker then re-reads the query with each top candidate to keep only the most relevant passages.

How do you evaluate a RAG system?

Measure retrieval and answers separately on a test set of real questions. For retrieval, check whether the passage containing the answer was found and how highly it ranked. For answers, check faithfulness — whether every claim is supported by the retrieved passages — plus relevance, correctness, and whether the system declines when the documents do not contain the answer.

Should I use RAG or fine-tuning?

Use retrieval (RAG) when the model lacks facts, especially private or current information. Consider fine-tuning only when a consistent style, format or behaviour cannot be held with good prompts at your volume. Try improving the prompt before either.

What is the difference between RAG and MCP?

RAG is about what a model knows: relevant documents are retrieved and placed in front of it at request time. MCP, the Model Context Protocol, is about what it can reach: a standard way to connect models to tools and data. They solve different problems and are often used together.

Can fine-tuning teach a model new facts?

It is the wrong tool for that. Facts learned in training are hard to update and hard to trace back to a source. Retrieval keeps facts in documents you can update and cite.