3.4. Reflection Checkpoint

Key Takeaways

Before proceeding, ensure you can:

• Map business problems to the correct ML problem category (classification, regression, clustering, anomaly detection, recommendation, forecasting)
• Select the appropriate SageMaker built-in algorithm for a given data type and problem
• Distinguish between Bedrock (serverless foundation models) and JumpStart (SageMaker-managed pre-trained models)
• Identify when AWS AI services (Rekognition, Comprehend, Textract) are sufficient vs. when custom training is needed
• Explain the relationship between epochs, batch size, and learning rate and predict the effect of changing each
• Configure SageMaker AMT with the right search strategy (Bayesian, random, grid) based on budget and parameter space
• Diagnose overfitting vs. underfitting from training/validation metric patterns and apply the correct regularization
• Choose between data parallelism and model parallelism based on model size and dataset size
• Use Model Registry for versioning and approval workflows
• Select the correct evaluation metric (precision, recall, F1, RMSE, AUC) based on the business problem's cost structure
• Distinguish between Clarify's pre-training and post-training bias detection capabilities
• Use Debugger to diagnose training convergence failures

Connecting Forward

In the next phase, you'll take trained models and deploy them into production. You'll learn how to select the right endpoint type, provision infrastructure, set up auto scaling, build CI/CD pipelines for ML, and implement deployment strategies that minimize risk. The model quality decisions you've made in Phase 3 determine what you're deploying—Phase 4 determines how well you deploy it.

Self-Check Questions

1. A retail company needs to classify customer support tickets into 15 categories. They have 50,000 labeled examples. The categories are imbalanced—"billing" has 10,000 examples while "accessibility" has 200. Recommend an algorithm, an evaluation metric, and one data preparation step to address the imbalance.

2. A training job on ml.p3.2xlarge produces a model with 88% validation accuracy. The team runs AMT with Bayesian optimization across 5 hyperparameters, producing a model with 91% accuracy. The best model used a learning rate of 0.001 and dropout of 0.3. Explain why Bayesian optimization was a better choice than grid search for this scenario.

3. SageMaker Debugger fires a VanishingGradient alert during the training of a 50-layer neural network. Training accuracy is 52% after 20 epochs. Explain the root cause and two specific remedies.