2.3.1. Detecting and Mitigating Bias in Training Data
💡 First Principle: Bias in ML starts with biased data, not biased algorithms. An algorithm trained on unbiased data produces fair results; the same algorithm trained on biased data produces biased results. The exam tests your ability to detect pre-training bias using specific metrics and mitigate it using specific techniques.
📌 Availability note: On AWS these metrics were packaged by Amazon SageMaker Clarify, which is no longer open to new customers (existing customers can continue to use it, and no new features are planned). They are published, standardized formulas over label counts and group proportions, so AWS's recommended path for a new team is to compute them directly with pandas and scikit-learn in a SageMaker Pipelines processing step and log the results to a SageMaker AI MLflow App. The definitions below are identical either way, and the definitions are what the exam tests.
Pre-Training Bias Metrics (the standardized definitions Clarify reports):
| Metric | What It Measures | Example | Concern If |
|---|---|---|---|
| Class Imbalance (CI) | Normalised size gap between the two facet values: (n_a − n_d) / (n_a + n_d) | 80% male, 20% female in training data → CI = 0.6 | CI far from 0 |
| Difference in Proportions of Labels (DPL) | Whether positive labels are distributed equally across facets | 60% of males labeled "hire," 30% of females | DPL far from 0 |
| KL Divergence | How much one distribution diverges from another | Label (outcome) distributions differ significantly between facet groups | Large divergence values |
| Jensen-Shannon Divergence | Symmetric version of KL divergence | Comparing label distributions between groups | Large divergence values |
Reading a CI value back to group sizes. CI is a normalised gap, not a percentage-point difference — which is the single most common misreading of this metric. Because the two facet values together make up the whole dataset, a reported CI converts straight back:
- smaller facet value = (1 − CI) / 2 of the records
- larger facet value = (1 + CI) / 2 of the records
So a CI of 0.85 does not describe an 85/15 split. It means (1 − 0.85) / 2 = 7.5% of records fall in the smaller group and 92.5% in the larger — a ratio of roughly 12 to 1. The 80/20 split in the table above is CI = 0.6. CI = 0 is perfect parity, and CI = 1 means one facet value is absent from the data entirely.
⚠️ Common Misconception: "CI measures how imbalanced my target classes are." It does not. CI is computed over facet values — male vs female, one age band vs another — never over the target label’s classes. A dataset that is 99% non-fraud and 1% fraud has a class balance problem, addressed with resampling or class weights; that is a modelling concern and CI says nothing about it. CI answers a different question: who is represented in the data. The exam tests this distinction directly.
Mitigation Techniques:
| Technique | How It Works | When to Use | AWS Tool |
|---|---|---|---|
| Random oversampling | Duplicate minority class samples | Moderate imbalance, sufficient data | Data Wrangler built-in |
| Random undersampling | Remove majority class samples | Large dataset, extreme imbalance | Data Wrangler built-in |
| SMOTE | Generate synthetic minority samples | Moderate imbalance, tabular data | Data Wrangler SMOTE transform |
| Stratified sampling | Maintain class proportions during splitting | All cases (should be default) | SageMaker Processing |
| Reweighting | Assign higher weights to minority class | When resampling is impractical | Algorithm-level (class_weight parameter) |
⚠️ Exam Trap: Bias analysis has two phases: pre-training bias detection (data analysis) and post-training bias detection (model predictions). When the question asks about bias before training, the relevant metrics are data-level metrics like CI and DPL. When the question asks about bias after training, the relevant metrics are model-level like Disparate Impact and Conditional Demographic Disparity. Don't mix the two contexts.
Reflection Question: A loan approval dataset has 70% applications from urban areas and 30% from rural areas. The approval rate is 65% for urban and 40% for rural. Which bias metrics would flag this, and what mitigation would you recommend?