The MLA-C01 exam retires on September 28, 2026
You can still take and pass the exam until then — plan your exam date accordingly.
3.3.2. Model Bias Detection with SageMaker Clarify
💡 First Principle: A model can amplify biases present in training data, producing predictions that systematically disadvantage certain groups. SageMaker Clarify detects these biases both before training (data-level) and after training (prediction-level), providing specific metrics and explanations. The exam tests both when to use Clarify and how to interpret its outputs.
Post-Training Bias Metrics (different from the pre-training metrics in 2.3.1):
| Metric | What It Measures | Concern If |
|---|---|---|
| Disparate Impact (DI) | Ratio of positive outcomes between groups | Far from 1.0 |
| Conditional Demographic Disparity (CDD) | Disparity conditioned on other attributes | Significant after controlling for legitimate factors |
| Counterfactual Fliptest | Whether changing a protected attribute changes the prediction | High flip rate |
SHAP Values (Shapley Additive Explanations): Clarify uses SHAP to explain individual predictions—which features contributed most and in which direction. This is critical for interpretability: you can tell a loan applicant not just that they were rejected, but which factors (income, credit history, employment length) drove the decision and by how much.
Partial Dependence Plots (PDPs): Show how a single feature's value affects predictions on average, across the entire dataset. Useful for understanding the learned relationship between a feature and the target.
⚠️ Exam Trap: Clarify's pre-training bias detection (data-level) and post-training bias detection (model-level) use different metrics. Pre-training uses CI and DPL (data distribution metrics). Post-training uses DI and CDD (prediction outcome metrics). A question about "bias in the training data" points to pre-training metrics. A question about "biased predictions" points to post-training metrics.
Reflection Question: A lending model has DPL of 0.0 (training data is perfectly balanced between demographic groups) but Disparate Impact of 0.6 (Group B’s approval rate is only 0.6 of Group A’s, so Group A is approved at about 1.67× Group B’s rate). How is this possible, and what does it mean?
