1.2.4. Choosing Between Traditional ML and Foundation Models
First Principle: A foundation model is a powerful generalist, but it is not automatically the right tool. The shape of the task, the data you have, and the rules you must follow decide whether a traditional ML model or a foundation model (FM) fits better.
Both are "AI", so the choice is easy to get wrong in either direction: forcing an LLM onto a tabular scoring problem, or building a labeled-data pipeline for a task a prompt could solve in a day.
When a traditional ML model is usually the better fit:
- The output is a prediction on structured data. A score, a label or a number from rows and columns (credit risk, churn, fraud, demand) is classic supervised learning. AWS guidance notes that LLMs often struggle to interpret tabular data accurately.
- Regulators or auditors need the reason for each decision. A traditional model's individual predictions can be explained with feature attributions such as SHAP values. An LLM's explanation of its own answer is generated text, not a faithful account of how the answer was produced.
- Results must be repeatable. A trained traditional model returns the same output for the same input. FM outputs are nondeterministic, so re-running an input can produce a different answer.
- Operational constraints are tight. Millisecond latency, very high request volume, or a very low cost per prediction favor a small, purpose-built model over per-token FM calls.
- Labeled history already exists. That is exactly what supervised learning needs.
When a foundation model is usually the better fit:
- The task is generative or language-heavy: summarizing, drafting, translating, answering questions, conversing.
- The data is unstructured or multimodal: free text, documents, images and audio, often in many languages.
- There is little or no labeled data, and you want a working prototype quickly by prompting rather than training.
| Question to ask | Leans traditional ML | Leans foundation model |
|---|---|---|
| What is the output? | A score, label or number | New text, images or other content |
| What is the input? | Structured, tabular fields | Unstructured text, documents, media |
| Must each decision be explained? | Yes, per decision (feature attributions) | Less critical, or handled with grounding and human review |
| Must results be repeatable? | Yes, same input gives the same output | Some variation is acceptable |
| Operating limits? | Millisecond latency, huge volume, tiny cost per call | Moderate volume, per-token cost acceptable |
| Labeled data? | Available | Scarce or absent |
The two also combine. AWS guidance describes FMs generating synthetic data to train traditional models where real data is scarce or too sensitive to use.
Scenario: A lender wants to (1) approve or decline loan applications from structured applicant data, with a reason regulators can review for every decision, and (2) send each applicant a friendly, personalized letter explaining the outcome.
Reflection Question: Why might the lender use a traditional classification model for the decision itself and a foundation model only for drafting the letter from that decision's result?
💡 Tip: On the exam, look for the constraint words. "Explain each decision to regulators", "same result every time", "millions of predictions in milliseconds" and "tabular data" point to traditional ML. "Summarize", "generate", "converse", "unstructured" and "no labeled data" point to a foundation model.
