2.3.3. Model Adaptation: RAG and Fine-Tuning
💡 First Principle: There are two different gaps between a foundation model and your business: a knowledge gap (it does not know your current, proprietary facts) and a behavior gap (it does not respond in your required style, format, or domain-specific way). Retrieval Augmented Generation closes the knowledge gap at the moment of each question; fine-tuning closes the behavior gap by changing the model itself. Diagnose the gap before choosing the technique.
Retrieval Augmented Generation (RAG) connects a model to a knowledge source. When a user asks a question, the system first retrieves the most relevant passages from company content (policies, product manuals, contracts), adds them to the prompt, and asks the model to answer using that material, often with citations. Updates take effect as soon as the source content changes; no retraining is needed. On AWS, Amazon Bedrock Knowledge Bases provides managed RAG.
Fine-tuning continues training a foundation model on a curated set of examples so that it adopts specific behavior: a consistent tone, a specialized output format, domain terminology, or better performance on a narrow task. It requires good labeled examples, costs money for training and for hosting or storing the customized model, and must be repeated when needs change. It does not reliably teach facts that change frequently, and it cannot cite sources.
| Technique | Closes which gap | Relative cost and effort | Data freshness | Typical use |
|---|---|---|---|---|
| Prompt engineering | Instructions, tone, format | Lowest | Whatever is in the prompt | Drafting, summarizing, formatting tasks |
| RAG | Knowledge (current, proprietary facts) | Low to moderate | Updates as soon as source content changes | Policy Q&A, product support, document search with citations |
| Fine-tuning | Behavior (style, format, domain task performance) | Moderate to high | Frozen at training time | Consistent brand voice, specialized classification or extraction |
| Continued pre-training / custom model | Deep domain knowledge or unique capability | Highest | Frozen at training time | Highly specialized domains where the model is the competitive asset |
These techniques combine. A bank might fine-tune a model to write in its regulated disclosure style and use RAG to supply current product rates. The strategist's job is to sequence investment: start with prompting, add retrieval for knowledge, and fund fine-tuning only when there is evidence that behavior, not knowledge, is the remaining gap.
RAG also supports governance goals covered in Phase 4: answers grounded in approved sources are easier to verify, citations let users check claims, and grounding reduces (but does not eliminate) hallucination.
⚠️ Exam Trap: When the requirement is to answer questions using company documents that change often, with sources users can verify, the answer is RAG (for example, Amazon Bedrock Knowledge Bases), not fine-tuning. Fine-tuning is the answer when the model must consistently adopt a specific style, format, or specialized task behavior that prompting cannot achieve.
Reflection Question: A legal team wants an assistant that answers questions about the company's 2,000 active contracts, which are amended weekly, and always responds in the firm's citation format. Which adaptation techniques address which part of this requirement, and in what order would you fund them?
