4.4. Deploying Foundation Models, RAG and Agents
💡 First Principle: A foundation model (FM) changes what you deploy but not why deployment decisions matter. You still trade latency against cost, still version everything you ship, and still need a rollback path. What changes is the unit of deployment: instead of one model artifact behind one endpoint, a generative AI application is a stack of model, prompt, retrieval index and agent code. Each layer has its own hosting choice, its own versioning mechanism and its own refresh cycle.
Consider a support assistant built on Amazon Bedrock. The model is hosted by AWS, the prompt lives in Prompt Management, the product manuals sit in a knowledge base, and an agent calls an order-lookup API. Any of those four layers can break production on its own: a prompt edit that drops the citation instruction, a knowledge base that was never re-synced after the manuals changed, an agent release that points at the wrong tool. Section 4.3 taught CI/CD for model artifacts. This section extends the same discipline to the other three layers.
Think of the stack like a restaurant. The model is the chef (hired or trained in-house), the prompt is the recipe card, the knowledge base is the pantry, and the agent is the waiter who takes orders and fetches things. You can replace any one of them without rebuilding the kitchen, but only if each has a label, a version and a way to swap it back.
⚠️ Common Misconception: "Serverless" FM inference means you have nothing to configure. On Amazon Bedrock you still choose a throughput model (on-demand, service tier, Provisioned Throughput, batch), a routing scope (in-Region or a cross-Region inference profile) and, for imported models, you accept cold starts when an idle model is restored. The infrastructure is managed; the cost and latency decisions are still yours.
