2.2.2. Dynamic Model Selection and Provider Switching

💡 First Principle: Hard-coding a model ID in your application creates a brittle dependency — when a model is deprecated, a better/cheaper option releases, or a provider has an outage, every service that hard-coded the model ID requires a code change and redeployment. The solution is a provider abstraction layer that separates model selection from application logic.

The exam specifically tests Skill 1.2.2 — architecture patterns that enable dynamic model selection without code changes. The canonical AWS implementation uses a combination of configuration management and a routing layer:

Architecture: dynamic model routing with AWS AppConfig

Model configuration via AppConfig:

{
  "primary_model": "anthropic.claude-3-sonnet-20240229-v1:0",
  "fallback_model": "anthropic.claude-3-haiku-20240307-v1:0",
  "max_tokens": 1024,
  "temperature": 0.3,
  "routing_rules": {
    "high_complexity": "anthropic.claude-3-opus-20240229-v1:0",
    "low_latency": "anthropic.claude-3-haiku-20240307-v1:0"
  }
}

This pattern means a model upgrade or A/B test change is a configuration update (seconds) rather than a code deploy (minutes to hours), and it's instantly rollbackable.

⚠️ Exam Trap: AWS AppConfig has a configuration cache TTL — your application may continue using the old model ID for up to the cache duration after you update the config. For immediate switches (incident response, security patch), you need to invalidate the cache or use a shorter polling interval.

Reflection Question: Your company is migrating from Claude 3 Sonnet to Claude 3.5 Sonnet across 12 microservices. Without a provider abstraction layer, what is the minimum number of deployments required? With AppConfig-based dynamic routing, what is the answer?