2.1.1. Algorithms, Models, Training, and Inference
💡 First Principle: An algorithm is the learning method; a model is what the method produces after studying data; training is the expensive, occasional act of producing the model; inference is the cheap, constant act of using it. Because the model is a snapshot of patterns in historical data, it knows only what that history contained and stays fixed until someone deliberately trains it again.
Think of a new insurance adjuster. Their training program (the algorithm: a method for learning) exposes them to 10,000 past claims with known outcomes (historical training data). What they carry out of the program (the model) is not the claims themselves but a set of learned instincts: which patterns tend to signal fraud, which damage photos look typical. On the job, every time they review a new claim and give a judgment (inference), they produce a prediction: "likely legitimate, 92% confidence." They do not relearn the job with each claim. If fraud tactics change, their instincts become outdated until they attend another training program.
| Dimension | Training | Inference |
|---|---|---|
| What happens | The algorithm learns patterns from historical data to produce a model | The model applies those patterns to new input to produce a prediction or output |
| Frequency | Occasional (initial build, then periodic retraining) | Continuous (every request) |
| Cost profile | Large, lumpy cost: compute, data preparation, expert time | Small per-request cost that scales with usage volume |
| Data used | Historical, labeled or curated examples | Live business data from each request |
| Business question | "Is our historical data good and representative enough to learn from?" | "Is the per-use cost, speed, and accuracy acceptable at our volume?" |
| Who mainly pays (managed generative AI) | Usually the model provider (foundation models are pretrained) | The customer, per token or request |
Three implications matter for strategy:
- Historical data encodes historical behavior, including its flaws. If past loan approvals were biased, a model trained on them learns that bias. If last year's customers were mostly from one region, the model may serve other regions poorly. Training data is a decision, not a given.
- Models are snapshots. A model reflects the world at the time its training data was collected. When products, customers, or markets change, the model does not notice on its own. Retraining or updating is a recurring, budgeted activity, not a one-time event.
- Foundation models shift the economics. With services such as Amazon Bedrock, the provider has already paid for training enormous general-purpose models. Your costs concentrate on inference (usage) and on any adaptation you choose (covered in 2.3.3). With custom ML in Amazon SageMaker AI, you pay for both training and inference.
The model also does not "remember" the data you send at inference time in the sense of learning from it. That is a common point of confusion, and also a common privacy question; whether a provider may use your inference data to improve its models is a contractual and configuration question you should ask explicitly. (For Amazon Bedrock, AWS states that customer prompts and outputs are not used to train the base models.)
⚠️ Exam Trap: "The model will improve automatically as customers use it" is a red flag in most scenarios. A deployed model is static between training runs. Improvement requires a deliberate process: collecting feedback, retraining or updating, evaluating, and redeploying. If a scenario describes declining performance, the fix involves monitoring and retraining, not waiting.
Reflection Question: A demand-forecasting model was trained on three years of pre-pandemic sales data and deployed last year. Its accuracy has dropped sharply. Using the training/inference distinction, explain why, and what business decision is needed.
