4.4.1. Foundation Model Inference Options on Amazon Bedrock
💡 First Principle: Amazon Bedrock hosts the model for you, so the deployment decision becomes a purchasing decision: how do you pay for capacity, and how much latency variability can the workload tolerate? Five levers cover almost every scenario.
| Option | How you pay | Best for | Key constraint |
|---|---|---|---|
| On-demand (Standard tier) | Per input and output token | Default for interactive apps with variable traffic | Shares your account's on-demand quota |
| Priority / Flex service tiers | Token price premium (Priority) or discount (Flex) | Priority: customer-facing latency; Flex: evaluations, summarization, agent background work that can wait | Set per request with the service tier parameter; quota is shared with Standard |
| Reserved tier | Fixed price per 1K tokens-per-minute, 1 or 3 month term | Mission-critical apps that cannot tolerate throttling | Minimum 100,000 input TPM and 10,000 output TPM; overflow spills to Standard |
| Provisioned Throughput | Hourly per Model Unit, no commitment, 1 month or 6 month | Dedicated capacity for a base or custom model | Billed until you delete it; cross-Region inference profiles do not support it |
| Batch inference | Select models at 50% of on-demand price | Large offline jobs (JSONL in S3, results to S3) | Asynchronous; no tool calling or structured output; not for provisioned models |
Cross-Region inference is a routing choice layered on top of on-demand inference. A geographic inference profile (US, EU, APAC) keeps processing inside that geography, which is the answer when data residency rules apply. A global profile can route to any supported commercial Region and costs roughly 10% less. There is no extra routing charge, the price is set by the source Region, and CloudTrail records the Region that actually served the request in additionalEventData.inferenceRegion.
Custom models (fine-tuned or distilled inside Bedrock) have two serving paths: purchase Provisioned Throughput, or create a custom model deployment for on-demand, pay-per-use inference. On-demand custom model deployment is currently limited to specific base models (for example Amazon Nova Micro, Lite and Pro in US East (N. Virginia), Meta Llama 3.3 70B Instruct in US West (Oregon)) customized on or after 16 July 2025. Either way, you invoke the model through the ARN of the deployment or provisioned model, not the base model ID.
Amazon Bedrock Marketplace adds 100+ specialized models that are not in the serverless catalog. You subscribe and deploy the model to an endpoint managed by SageMaker AI (you pick instance type and count), then call it through the Bedrock InvokeModel or Converse APIs so it works with Bedrock tools such as Knowledge Bases and Guardrails.
⚠️ Exam Trap: Provisioned Throughput is not the default fix for throttling. If the scenario is bursty and cost-sensitive, a cross-Region inference profile raises effective throughput with no commitment. Provisioned Throughput (or the Reserved tier) is correct when the stem stresses guaranteed capacity for steady, high volume. Batch inference is correct only when nobody is waiting for the answer.
