5.4.4. FM Inference Cost: Pricing Choices and Usage Optimization
💡 First Principle: With FMs you pay per token, not per instance-hour, so cost follows behaviour: how long the prompts are, how much the model writes, how often identical context is resent, and how urgently each answer is needed. Optimising FM cost means matching each workload's urgency and repetition to the cheapest mode that still meets it.
Pricing and capacity modes (4.2.7).
| Mode | How you pay | Fits | Does not fit |
|---|---|---|---|
| On-demand, Standard tier | Per input and output token | Everyday interactive traffic; the default | — |
| Priority tier | Per token at a premium over Standard (75% on the pricing page) | Customer-facing calls that need the fastest response without a capacity reservation | Background work |
| Flex tier | Per token at a discount to Standard (50% on the pricing page) | Work that tolerates longer processing: evaluations, summarisation, some agent steps | User-facing latency targets |
| Batch inference | 50% lower than on-demand for supported models | Large asynchronous jobs: prompts in JSONL in S3, results written back to S3 | Real-time answers; tool calling; structured output; prompt caching |
| Provisioned Throughput | Hourly per model unit, whether used or not; no-commitment, 1-month or 6-month terms | Steady high volume; hosting a customized model | Spiky or low traffic |
| Reserved tier | Fixed price per 1K tokens-per-minute of reserved capacity, 1- or 3-month terms, overflow to Standard | Mission-critical apps needing reserved capacity (target 99.5% uptime) | Small workloads (minimums apply) |
You select a service tier per request with the service_tier parameter (serviceTier in the Converse API): priority, default (Standard), flex or reserved. Standard is used when it is omitted, and CloudWatch reports a ResolvedServiceTier dimension showing which tier actually served each request. On-demand quota is shared across Priority, Standard and Flex, while Reserved capacity is separate.
Usage optimisation (4.2.9). The cheapest token is the one you never send:
- Prompt caching — cache a long, static prompt prefix (system instructions, tool definitions, a shared document) so later requests read it from cache. Cache reads are billed at a reduced rate and, per the runtime metrics, do not count toward your tokens-per-minute quota; cache writes can cost more than normal input tokens on some models. Put static content first and variable content last, because any change to the prefix causes a miss. Many models use a 5-minute TTL that resets on each hit; several newer models also support a 1-hour TTL, so check the model card. Some models also cache eligible prefixes implicitly, with no checkpoints. Prompt caching is for on-demand inference, not batch.
- Intelligent prompt routing — one endpoint routes each request between two models in the same family, sending simple prompts to the smaller model. AWS cites up to 30% lower cost. It is optimised for English prompts.
- Model distillation — fine-tune a smaller student model on responses from a larger teacher for one use case, then serve the student.
- Right-size the model — use model evaluation (5.4.2) to find the smallest model that meets the quality bar; the price gap between model sizes is usually larger than any discount.
- Cap and trim — set
maxTokensto what the use case needs, ask for concise output formats, and trim conversation history and retrieved context instead of resending everything.
Attributing FM cost. Before optimising, find out who is spending. Three mechanisms do different jobs:
| Mechanism | Granularity | Appears in |
|---|---|---|
| Application inference profiles with cost allocation tags | Per application or team, aggregated per usage type per day | Cost Explorer and Cost and Usage Reports |
Per-request metadata (requestMetadata in Converse, X-Amzn-Bedrock-Request-Metadata header on InvokeModel) | Per call, up to 16 key-value pairs | Model invocation logs only (needs logging enabled) |
| IAM principal attribution | Per calling identity (optionally tagged by team or cost center) | Invocation logs (identity.arn); Cost Explorer and CUR 2.0, aggregated per usage type per day, once IAM principal cost allocation tags are activated |
Application inference profiles work with InvokeModel and Converse on the bedrock-runtime endpoint; for the Responses and Chat Completions APIs on the bedrock-mantle endpoint, Amazon Bedrock Projects play the same cost-allocation role. Request metadata never reaches the bill as a cost allocation tag: to turn it into money you multiply logged token counts by the published rates, or join the logs to the Cost and Usage Report. AWS Budgets then sets the quota part of 4.2.5 — a budget scoped to the profile's tag can alert or trigger an action before a team overruns.
⚠️ Exam Trap: Batch inference is 50% cheaper, but it is asynchronous and writes results to S3. It is the wrong answer for a chatbot and the right one for "summarise four million support tickets overnight". The Flex tier is the per-request alternative when a job must stay on the synchronous API but can tolerate slower responses.
⚠️ Exam Trap: Provisioned Throughput is billed every hour it exists. For bursty traffic that is idle most of the day, it costs more than on-demand, not less.
