The MLA-C01 exam retires on September 28, 2026
You can still take and pass the exam until then — plan your exam date accordingly.
2.3.1. Detecting and Mitigating Bias in Training Data
💡 First Principle: Bias in ML starts with biased data, not biased algorithms. An algorithm trained on unbiased data produces fair results; the same algorithm trained on biased data produces biased results. The exam tests your ability to detect pre-training bias using specific metrics and mitigate it using specific techniques.
Pre-Training Bias Metrics (SageMaker Clarify):
| Metric | What It Measures | Example | Concern If |
|---|---|---|---|
| Class Imbalance (CI) | Normalised size gap between the two facet values: (n_a − n_d) / (n_a + n_d) | 80% male, 20% female in training data → CI = 0.6 | CI far from 0 |
| Difference in Proportions of Labels (DPL) | Whether positive labels are distributed equally across facets | 60% of males labeled "hire," 30% of females | DPL far from 0 |
| KL Divergence | How much one distribution diverges from another | Label (outcome) distributions differ significantly between facet groups | Large divergence values |
| Jensen-Shannon Divergence | Symmetric version of KL divergence | Comparing label distributions between groups | Large divergence values |
Reading a CI value back to group sizes. CI is a normalised gap, not a percentage-point difference — which is the single most common misreading of this metric. Because the two facet values together make up the whole dataset, a reported CI converts straight back:
- smaller facet value = (1 − CI) / 2 of the records
- larger facet value = (1 + CI) / 2 of the records
So a CI of 0.85 does not describe an 85/15 split. It means (1 − 0.85) / 2 = 7.5% of records fall in the smaller group and 92.5% in the larger — a ratio of roughly 12 to 1. The 80/20 split in the table above is CI = 0.6. CI = 0 is perfect parity, and CI = 1 means one facet value is absent from the data entirely.
⚠️ Common Misconception: "CI measures how imbalanced my target classes are." It does not. CI is computed over facet values — male vs female, one age band vs another — never over the target label’s classes. A dataset that is 99% non-fraud and 1% fraud has a class balance problem, addressed with resampling or class weights; that is a modelling concern and CI says nothing about it. CI answers a different question: who is represented in the data. The exam tests this distinction directly.
Mitigation Techniques:
| Technique | How It Works | When to Use | AWS Tool |
|---|---|---|---|
| Random oversampling | Duplicate minority class samples | Moderate imbalance, sufficient data | Data Wrangler built-in |
| Random undersampling | Remove majority class samples | Large dataset, extreme imbalance | Data Wrangler built-in |
| SMOTE | Generate synthetic minority samples | Moderate imbalance, tabular data | Data Wrangler SMOTE transform |
| Stratified sampling | Maintain class proportions during splitting | All cases (should be default) | SageMaker Processing |
| Reweighting | Assign higher weights to minority class | When resampling is impractical | Algorithm-level (class_weight parameter) |
⚠️ Exam Trap: SageMaker Clarify is used for both pre-training bias detection (data analysis) and post-training bias detection (model predictions). When the question asks about bias before training, the relevant metrics are data-level metrics like CI and DPL. When the question asks about bias after training, the relevant metrics are model-level like Disparate Impact and Conditional Demographic Disparity. Don't mix the two contexts.
Reflection Question: A loan approval dataset has 70% applications from urban areas and 30% from rural areas. The approval rate is 65% for urban and 40% for rural. Which bias metrics would flag this, and what mitigation would you recommend?
