Black Friday sale: an extra 40% off every course until Monday, November 30.New Year sale: an extra 30% off every course until Friday, January 1.Memorial Day sale: an extra 25% off every course until Monday, May 31.SysAdmin Day sale: an extra 25% off every course until Sunday, August 1.Back to School sale: an extra 25% off every course until Sunday, August 22.Labor Day sale: an extra 25% off every course until Monday, September 6.Black Friday sale: an extra 40% off every course until Monday, November 29.New Year sale: an extra 30% off every course until Saturday, January 1.Choose your certification →

Copyright (c) 2026 MindMesh Academy. All rights reserved. This content is proprietary and may not be reproduced or distributed without permission.

2.3.1. Detecting and Mitigating Bias in Training Data

💡 First Principle: Bias in ML starts with biased data, not biased algorithms. An algorithm trained on unbiased data produces fair results; the same algorithm trained on biased data produces biased results. The exam tests your ability to detect pre-training bias using specific metrics and mitigate it using specific techniques.

📌 Availability note: On AWS these metrics were packaged by Amazon SageMaker Clarify, which is no longer open to new customers (existing customers can continue to use it, and no new features are planned). They are published, standardized formulas over label counts and group proportions, so AWS's recommended path for a new team is to compute them directly with pandas and scikit-learn in a SageMaker Pipelines processing step and log the results to a SageMaker AI MLflow App. The definitions below are identical either way, and the definitions are what the exam tests.

Pre-Training Bias Metrics (the standardized definitions Clarify reports):
MetricWhat It MeasuresExampleConcern If
Class Imbalance (CI)Normalised size gap between the two facet values: (n_a − n_d) / (n_a + n_d)80% male, 20% female in training data → CI = 0.6CI far from 0
Difference in Proportions of Labels (DPL)Whether positive labels are distributed equally across facets60% of males labeled "hire," 30% of femalesDPL far from 0
KL DivergenceHow much one distribution diverges from anotherLabel (outcome) distributions differ significantly between facet groupsLarge divergence values
Jensen-Shannon DivergenceSymmetric version of KL divergenceComparing label distributions between groupsLarge divergence values

Reading a CI value back to group sizes. CI is a normalised gap, not a percentage-point difference — which is the single most common misreading of this metric. Because the two facet values together make up the whole dataset, a reported CI converts straight back:

  • smaller facet value = (1 − CI) / 2 of the records
  • larger facet value = (1 + CI) / 2 of the records

So a CI of 0.85 does not describe an 85/15 split. It means (1 − 0.85) / 2 = 7.5% of records fall in the smaller group and 92.5% in the larger — a ratio of roughly 12 to 1. The 80/20 split in the table above is CI = 0.6. CI = 0 is perfect parity, and CI = 1 means one facet value is absent from the data entirely.

⚠️ Common Misconception: "CI measures how imbalanced my target classes are." It does not. CI is computed over facet values — male vs female, one age band vs another — never over the target label’s classes. A dataset that is 99% non-fraud and 1% fraud has a class balance problem, addressed with resampling or class weights; that is a modelling concern and CI says nothing about it. CI answers a different question: who is represented in the data. The exam tests this distinction directly.

Mitigation Techniques:
TechniqueHow It WorksWhen to UseAWS Tool
Random oversamplingDuplicate minority class samplesModerate imbalance, sufficient dataData Wrangler built-in
Random undersamplingRemove majority class samplesLarge dataset, extreme imbalanceData Wrangler built-in
SMOTEGenerate synthetic minority samplesModerate imbalance, tabular dataData Wrangler SMOTE transform
Stratified samplingMaintain class proportions during splittingAll cases (should be default)SageMaker Processing
ReweightingAssign higher weights to minority classWhen resampling is impracticalAlgorithm-level (class_weight parameter)

⚠️ Exam Trap: Bias analysis has two phases: pre-training bias detection (data analysis) and post-training bias detection (model predictions). When the question asks about bias before training, the relevant metrics are data-level metrics like CI and DPL. When the question asks about bias after training, the relevant metrics are model-level like Disparate Impact and Conditional Demographic Disparity. Don't mix the two contexts.

Reflection Question: A loan approval dataset has 70% applications from urban areas and 30% from rural areas. The approval rate is 65% for urban and 40% for rural. Which bias metrics would flag this, and what mitigation would you recommend?

See how it connects
Alvin Varughese
Written byAlvin Varughese
Founder•26 professional certifications earned