4.4.2. Bringing Models Built Outside AWS
💡 First Principle: A model trained elsewhere is just weights plus a runtime contract. The deployment question is which contract you can satisfy with the least operational work: Bedrock's serverless import contract, or a SageMaker AI container you control.
Amazon Bedrock Custom Model Import takes Hugging Face format weights from Amazon S3 and serves them on demand through InvokeModel, InvokeModelWithResponseStream and, for most architectures, Converse. You create a model import job that points at the S3 prefix; Bedrock detects the architecture.
| Requirement | What Bedrock expects |
|---|---|
| Supported architectures | Mistral, Mixtral, Flan (T5), Llama 2 / 3 / 3.1 / 3.2 / 3.3 and Mllama, GPTBigCode, Qwen2 / 2.5 / 2-VL / 2.5-VL / Qwen3 (causal LM), GPT-OSS |
| Files | .safetensors weights, config.json, tokenizer_config.json, tokenizer.json, tokenizer.model |
| Size | Weights under 200 GB for text models, under 100 GB for multimodal models; context length under 128K |
| Not supported | Embedding models; batch inference and CloudFormation for imported models |
Billing and cold starts. Bedrock sizes each model copy in Custom Model Units (CMUs) at import time and scales the number of copies with demand. You pay per CMU per minute, billed in 5-minute windows from the first successful call. Bedrock removes models that are not active; the next invocation starts a restore and may return ModelNotReadyException, which the SDK retries with exponential backoff. A workload that cannot tolerate that first-call delay after idle periods needs either steady traffic or a hosting option you keep warm.
SageMaker AI is the answer when the model falls outside the import contract (unsupported architecture, custom pre/post-processing, a specific GPU type, or embeddings). Package the artifacts in S3 and pair them with a container: an AWS Deep Learning Container, a large model inference (LMI) container, or your own image in Amazon ECR. Large language models do not need to be tarred: set the model data source to an S3 prefix with CompressionType of None and SageMaker AI copies each object under the prefix into /opt/ml/model, which avoids a slow decompress step at startup.
⚠️ Exam Trap: Custom Model Import is not "upload any model". An embedding model, an unsupported architecture, or a need for batch inference points to SageMaker AI instead. Watch also for "no infrastructure to manage" plus a supported architecture: that combination points to Custom Model Import, not a SageMaker AI endpoint.
