2.1.2. Pretrained vs. Fine-Tuned Models
💡 First Principle: Pretrained models are generalists—broad knowledge, ready immediately. Fine-tuned models are specialists—deep expertise in specific domains, but require investment to create. The choice depends on whether general knowledge or specialized accuracy matters more for your use case.
Think of pretrained models like a well-educated generalist consultant—they know something about everything and can start working immediately. Fine-tuned models are like domain experts—they know their specialty deeply but it took years to develop that expertise. You hire the generalist for versatility; you hire the specialist for accuracy in their domain.
| Aspect | Pretrained Models | Fine-Tuned Models |
|---|---|---|
| Knowledge | Broad, general | Deep, domain-specific |
| Setup time | Immediate | Requires training data and compute |
| Cost | Usage-based only | Training + usage |
| Accuracy | Good for general tasks | Better for specialized tasks |
| When to use | General productivity, first deployments | Industry-specific terminology, specialized workflows |
Important clarification: Fine-tuning changes how the model behaves (its style, terminology, specialized knowledge). Grounding/RAG (covered in 2.3) gives the model access to specific documents. These solve different problems:
- Fine-tuning: "I need the model to understand medical terminology and respond like a clinician"
- Grounding/RAG: "I need the model to answer questions about our company's HR policies"
⚠️ Exam Trap: Questions may present fine-tuning and grounding as interchangeable. They're not. Fine-tuning changes model behavior; grounding provides document access. Most business scenarios need grounding, not fine-tuning.
Reflection Question: A law firm wants their AI assistant to understand legal terminology and cite relevant case law from their document library. Do they need fine-tuning, grounding, or both?
