4.1. Retrieval-Augmented Generation (RAG) Fundamentals

💡 First Principle: RAG is a pipeline with two halves that run at different times — an indexing half (offline: chunk → embed → store) and a query half (at request time: embed the question → retrieve relevant chunks → ground the model). Almost every RAG question is really asking which stage a problem lives in and which knob fixes it.

Why care: RAG is the standard answer to the grounding problem from Phase 1 — the model lacks your private or current facts, so you retrieve them and put them in the prompt. Getting the pipeline mental model right means you can diagnose a RAG failure (bad retrieval vs. bad generation vs. bad chunking) instead of blindly swapping models.

⚠️ Common Misconception: "RAG means the model reads the whole document each time." Retrieval fetches only the most relevant chunks via an index — full documents won't fit the context window and would dilute the model's attention. Retrieval exists precisely to narrow the context to what matters.