Definition
AI bias is when a computer system makes systematically unfair decisions against certain groups, because it learned from data that reflected past prejudice or left those groups out.
At a glance
- AI does not invent fairness; it copies the patterns in its training data, including historical discrimination[1].
- Amazon scrapped a resume-screening tool in 2018 after it taught itself to penalize the word “women’s”[2].
- NIST found many facial recognition systems were 10 to 100 times more likely to falsely match Black or East Asian faces than white faces[3].
- It is a business risk, not just an ethics issue: lawsuits, fines, lost customers, and reputational damage.
How it works
An AI learns by studying past examples and copying what it finds. If those examples are unbalanced or carry old prejudice, the AI absorbs it and applies it at scale, often unnoticed. A tool can look objective and still quietly bake in discrimination.
Why it matters
Biased hiring tools invite discrimination lawsuits; biased facial recognition or credit decisions wrongly reject customers and make headlines. Regulators are moving too: the EU AI Act treats recruiting and HR AI as high-risk, requiring bias testing, human oversight, and records, with employer duties phasing in across 2026 to 2027[4].
What you can do
Ask vendors how their AI was tested for bias and get results in writing. Keep a human in the loop for consequential decisions. Check the data represents the people it affects, and monitor outcomes over time, since bias can surface after launch.
Bottom line
AI bias is a mirror, not a malfunction: treat it as a manageable business risk, demand testing, keep humans in the loop, and watch the outcomes.