4.3.2. Bias Across the AI Lifecycle

💡 First Principle: Bias is not a single defect in a dataset; it is any systematic difference in how an AI system treats comparable people, and it can enter at every stage, from how the problem is framed to how the outputs are used. Because populations and behavior change, a system that was fair at launch can become unfair later. Bias management is therefore a lifecycle activity with ongoing monitoring, not a one-time cleanup.

Where bias enters:

• Problem framing: choosing a target that encodes unfairness, such as predicting "past successful hires" when past hiring was biased, or using healthcare spending as a proxy for health need when spending differs across groups.
• Data collection (historical and sampling bias): data reflects past discrimination or under-represents some groups.
• Labeling: human labelers apply inconsistent or prejudiced judgments.
• Feature choice: variables such as postal code can act as proxies for protected characteristics.
• Evaluation: overall accuracy hides poor performance for smaller groups.
• Deployment: a model is used on a different population or for a different purpose than it was built for.
• Feedback loops: a model's decisions influence the data it later learns from (for example, policing or lending patterns that reinforce themselves).

Bias drift is the change in a system's fairness over time as populations, behaviors, or data sources shift. A credit model that was balanced at launch may become skewed when the applicant mix changes or a new marketing channel brings in different customers.

Mitigation at a business level:

• Define fairness criteria with legal, compliance, and business owners at intake.
• Require representative data and evaluation across relevant groups before approval.
• Require the team to measure bias in the data and in model outputs with per-group fairness metrics, and to keep tracking those metrics on deployed models. (Amazon SageMaker Clarify, the managed tool for this, is closed to new customers; AWS now directs teams to standard bias metrics and SHAP explanations computed in their own SageMaker AI pipelines, and to Amazon Bedrock Evaluations for generative AI.)
• Monitor fairness metrics in production with thresholds and owners.
• Keep humans in the loop for high-impact decisions and provide recourse to affected people.
• Re-assess when the population, data sources, or use changes.

⚠️ Exam Trap: Cleaning or rebalancing training data once does not eliminate bias. When a scenario describes a model whose outcomes became uneven over time, the answer involves ongoing bias monitoring (bias drift) and re-evaluation, and recognizing that bias can enter at any lifecycle stage, including deployment and feedback loops.

Reflection Question: A lending model was tested and found fair at launch. Eighteen months later, approval rates for younger applicants have fallen sharply relative to others with similar credit profiles. Name two lifecycle stages where this bias might have entered after launch, and the controls that should have caught it.