Definition
When software makes a decision about a person, the business that runs it stays answerable for the result.
At a glance
- The software is a tool; the operator owns the outcome of any loan denial, hiring screen, or price it sets[1].
- Four tests: can you explain it, trace it, justify it, and fix harm if it goes wrong?
- Regulation is making it mandatory in the EU and proposed in the US.
- Audited, explainable systems cut legal, discrimination, and reputation risk.
Why it matters
Real tools have gone wrong: risk scores in lending and criminal justice showed bias, and one ride-hailing service’s wait times tracked neighborhood ethnicity and income[1]. The danger is the “black box,” where even operators can’t say why a decision was made[1].
The law
The EU AI Act classifies systems by risk and requires high-risk ones like credit scoring and hiring to be assessed before launch and monitored after[3]. The proposed US Algorithmic Accountability Act would have the FTC mandate impact assessments for systems making critical decisions in employment, housing, healthcare, and finance[4][5].
What to do
Treat it like a financial audit, but of the system’s data, design, and decisions[2]. Document how each system works, run bias checks before and after launch (models drift), give people a way to appeal, and get third-party audits — increasingly expected, not optional[2].
Bottom line
Automated systems pass blame straight to the business, so document how they decide, test for bias, and be ready to show your work.
References
- Algorithmic accountability. Wikipedia en.wikipedia.org
- AI Auditing in the EU AI Act: Compliance, Accountability, and the Future of Ethical AI. Sutra Academy www.sutraacademy.ai
- EU AI Act Update 2025: Code of Practice, Enforcement, Industry Reactions. TTMS ttms.com
- Bill SB2164 — Algorithmic Accountability Act of 2025: FTC-mandated impact assessments for AI systems. Codify Legal Publishing codifylegalpublishing.com
- S.2164 - 119th Congress (2025-2026): Algorithmic Accountability Act of 2025. Congress.gov, Library of Congress www.congress.gov
Comments
Questions, corrections, and links welcome. Be specific and civil.