Definition
Existential risk from AI is the possibility that highly advanced AI could cause an irreversible, civilization-scale catastrophe, up to and including human extinction or permanent loss of human control.
At a glance
- The core fear is loss of control: an AI smarter than its makers pursues goals that clash with ours and acts faster than we can stop it.
- It is mainstream, not fringe. In May 2023, lab CEOs and top scientists called extinction risk a global priority alongside pandemics and nuclear war.
- Experts split sharply on the odds, from under 1% to double digits.
- For your business, the takeaway is governance: know your AI dependencies and watch the rules.
What it actually means
Not a chatbot saying something rude. It means permanent, civilization-scale harm we could not recover from. The classic case: an AI far more capable than its designers develops goals that don’t match ours and resists being corrected or shut off, called misalignment or loss of control[3]. Today’s systems can’t yet cause this, but capabilities are rising fast[5].
Why credible people take it seriously
In 2023 the Center for AI Safety published one sentence calling AI extinction risk a global priority[1], signed by the leading lab CEOs and the two most-cited AI scientists, Hinton and Bengio[2]. That doesn’t mean catastrophe is likely; estimates vary enormously[4]. The signal: this is serious and contested, not science fiction.
What to do as a business
The practical risk is concentration and dependence. If your operations lean on one AI provider, an outage or policy shift can hit hard. Keep an inventory of where AI touches your business, keep a human in the loop on big decisions, and follow rules like the EU AI Act.
Bottom line
A low-probability, high-stakes worry that serious people no longer dismiss; act on its near-term shadow by knowing your AI dependencies and keeping humans in control.