4.1.1. The RAG Pipeline: Chunk, Embed, Index, Retrieve, Ground
💡 First Principle: Each pipeline stage has one job, and a failure at any stage propagates downstream. Bad chunking produces incoherent retrieval candidates; bad embeddings retrieve the wrong chunks; good retrieval but a weak prompt still grounds poorly. Diagnose by stage, not by symptom.
The flow: Chunk large documents into passages small enough to embed and focused enough to be independently meaningful. Embed each chunk into a vector that captures its meaning. Index the vectors (and original text) in a searchable store. At query time, embed the question the same way, retrieve the nearest chunks, and ground the model by inserting those chunks into the prompt with instructions to answer from them and cite sources.
⚠️ Exam Trap: When a RAG answer is wrong, the instinct is to blame the model. But if the retrieved chunks didn't contain the answer, no model swap will fix it — the failure is upstream in chunking, embedding, or retrieval. A scenario describing "the model answers from the wrong information" is usually a retrieval-quality problem, not a generation problem.
Reflection Question: A RAG assistant confidently answers using a chunk that's topically related but not actually relevant to the question. Which pipeline stage is the prime suspect, and why isn't switching to a bigger model the fix?
