1.2.2. How Retrieval-Augmented Generation Works
💡 First Principle: RAG works because meaning can be measured. Text is converted into embeddings — vectors where similar meanings sit close together — so "How do I get my money back?" retrieves the refunds document even though it shares no keywords with it. Retrieval finds meaning matches, then the model reasons over only those matches.
The pipeline runs in two halves. At indexing time, source content is split into chunks, each chunk is embedded into a vector, and the vectors are stored in a searchable index alongside the text. At query time:
Each stage explains a behavior you will configure later. Chunking explains why giant unstructured documents ground poorly — a chunk must be self-contained enough to be useful alone. Ranking explains why scoping generative answers to the right sources beats attaching everything — irrelevant sources add noise that competes for the top ranks. And hybrid search (combining keyword and vector matching, optionally with semantic reranking) explains the configuration choices you will meet in Azure AI Search in Phase 2.
⚠️ Exam Trap: Retrieval quality bounds answer quality. If the right content never reaches the top-ranked chunks, no amount of prompt engineering fixes the answer — scoping, source quality, and index configuration are the levers that matter.
Reflection Question: Why does a semantically-similar-but-wrong chunk (say, last year's refund policy) produce a more dangerous answer than no chunk at all?
