Stuart Russell: Why AI risk is Russian roulette for humanity

Russell notes CEOs like Sam Altman, Elon Musk, and Dario Amodei have put estimates of human extinction risk in the 25–30% range, and many signed the May 2023 “Extinction Statement” equating AGI risk with nuclear war and pandemics. Despite this, they feel trapped in a race for AGI driven by investors and fear of being outcompeted, which Russell likens to playing Russian roulette with every human on Earth without consent.

Russell emphasizes that what matters is not whether AI is conscious but whether it is more capable than us at achieving its goals. He uses the “gorilla problem” to illustrate: humans can make gorillas extinct regardless of gorilla views, because we are more intelligent and capable. Today’s systems are grown via massive parameter tuning rather than designed from first principles, so we neither choose nor fully understand their objectives, yet experiments already show strong self-preservation tendencies.

Current frontier AI is trained by imitation learning to replicate human verbal behavior, effectively creating “imitation humans” that can substitute for white-collar workers. Coupled with humanoid and industrial robots, firms like Amazon are explicitly planning to replace hundreds of thousands of workers and even layers of management. Russell argues we should instead aim for tool-like AI that augments human decision-making and scientific discovery without displacing humans from all economically valuable roles.

If AGI plus robotics can do essentially all human work, Keynes’s old question resurfaces: what do humans do when economic necessity disappears? Russell says that despite asking economists, futurists, and science-fiction authors, no one can describe a convincing, non-enfeebling world where 80–90% of people have no productive role. He cites WALL‑E and Iain M. Banks’ Culture novels as cautionary: abundance without purpose tends toward passivity, meaninglessness, or dependence on tiny elite “purpose-bearing” roles.

Russell’s “human-compatible” AI framework abandons the idea that we can write down a perfect objective for the future. Instead, machines should be explicitly uncertain about human preferences, treat learning those preferences as their central task, and act cautiously whenever they are unsure. In this paradigm, an AI’s only purpose is to further human interests as inferred from our behavior and expressed choices, which in principle can be formulated and analyzed mathematically—but has not yet been built at scale.