3.4.1. Choosing Foundation Models on Amazon Bedrock
💡 First Principle: Model choice is a constrained optimization. Start from the hard requirements — modality, context length, languages, features the application depends on, data-residency — which eliminate most candidates outright. Only then compare the survivors on quality for your task, latency and cost. A benchmark score earned on someone else's task is a hint, not evidence.
Amazon Bedrock exposes models from many providers behind one API, so you can swap models without rewriting application code. The model card for each model ("Models at a glance" in the Bedrock documentation) is where the hard filters live.
| Selection criterion | What to check | Why it eliminates candidates |
|---|---|---|
| Modality | Text, image, video, audio input; text, image or embedding output | An image-understanding task rules out text-only models immediately |
| Context window | Maximum input tokens | Long documents or long conversations need room; truncation silently drops content |
| Languages | Supported languages | A multilingual support desk needs a model evaluated in those languages |
| Features | Tool use, streaming, prompt caching, customization support, service tiers | An agent needs tool use; a fine-tuning plan needs a model that supports customization |
| Region and inference options | On-demand, cross-Region inference profiles, Provisioned Throughput, batch | A data-residency rule narrows which inference profiles are usable |
| Task quality | Your own prompts, scored by an evaluation job | The only criterion that reflects your workload |
| Latency and cost | Tokens per second, time to first token, price per input and output token | A smaller model that passes the quality bar usually wins on both |
How to compare the survivors: run the same prompt dataset through each candidate using an Amazon Bedrock evaluation job (see 3.4.6) — programmatic metrics for tasks with reference answers, an LLM-as-a-judge job for open-ended quality, or a human evaluation job when judgement needs a person. The console playground's compare mode is useful for a first look, but a handful of hand-typed prompts is not an evaluation.
Trade-offs across the modeling spectrum (custom vs. managed vs. pre-trained vs. FM):
| Approach | Example on AWS | Choose when | Cost of choosing it |
|---|---|---|---|
| Managed AI service | Amazon Textract, Comprehend, Transcribe, Rekognition | The task is a solved, generic problem (OCR, sentiment, transcription) | Least control over the model; bounded to what the API offers |
| Foundation model via API | Amazon Bedrock with prompting and RAG | Open-ended language or multimodal generation, fast time to value | Per-token cost at scale; quality depends on prompt and retrieval design |
| Customized FM | Bedrock fine-tuning, distillation or reinforcement fine-tuning | Prompting and RAG plateau below the quality bar, or a smaller cheaper model must match a larger one | Training cost, dataset curation, a new model version to evaluate and govern |
| Pre-trained open-weight model you host | SageMaker AI JumpStart model on a SageMaker AI endpoint | Need control over hosting, instance choice, or an open-weight model | You operate the endpoint: scaling, patching, utilisation |
| Custom model | SageMaker AI built-in algorithm or your own training script | Structured/tabular prediction, strict interpretability, or no FM fits | Data science effort; you own the full lifecycle |
⚠️ Exam Trap: A scenario that says "predict churn from customer records" is a tabular problem — XGBoost on SageMaker AI, not a foundation model. A scenario that says "summarize support transcripts" is a generative task — Bedrock, not a built-in algorithm. The spectrum is decided by the task type first and by operational preference second.
⚠️ Exam Trap: "Largest model = best choice" is almost never the intended answer. When the stem gives a latency or cost constraint, the correct answer is usually the smallest model that passes an evaluation on the team's own prompts.
