The Diary of a CEOCEO Of Microsoft AI: AI Is Becoming More Dangerous And Threatening! - Mustafa Suleyman
CHAPTERS
- 0:00 – 7:10
Opening: Fear, Inevitability, and Why AI’s Trajectory Matters
The host frames public anxiety about AI, introducing Mustafa Suleyman’s pivotal role in the field. Suleyman describes his initial petrification, how he’s come to see AI’s rise as inevitable, and why humanity must actively shape its form, ownership, and governance rather than passively endure it.
- •Suleyman helped found DeepMind and later worked on Google’s chatbot efforts, seeing early waves of AI firsthand.
- •Initially he felt “petrified” about AI but now sees its trajectory as inevitable and potentially empowering if collectively steered.
- •AI is needed to solve massive global challenges—water, food, energy, climate—by acting as a scientific partner and inventor.
- •The critical variable is not whether AI exists, but how it is governed, who owns it, and how it is trained.
- •If we don’t engage, AI “happens to us”; Suleyman has spent 15 years wrestling with how to steer it toward broad benefit.
- 7:10 – 15:00
Early Breakthroughs: From Atari Games to Generative Intelligence
Suleyman recounts formative DeepMind milestones: Atari agents and primitive image generators that hinted at today’s generative AI. These systems revealed that simple reward-driven architectures could discover novel strategies and representations humans hadn’t considered, simultaneously thrilling and terrifying him.
- •DeepMind’s Atari agent learned from raw pixels and rewards, discovering strategies human players hadn’t noticed.
- •An early generative model for handwritten digits felt miraculous when it produced a new ‘7’ from noise.
- •Within a decade, similar principles scaled from tiny 300x300 black-and-white images to photorealistic images and videos.
- •These successes suggested AI could invent new knowledge, advancing civilization but also amplifying risk.
- 15:00 – 25:00
The Scaling Era: Large Language Models and Unfathomable Compute
The discussion turns to why large language models were unexpectedly successful. Suleyman explains how compute has exploded from a few petaflops to billions of petaflops, enabling models like ChatGPT and Inflection’s Pi, and why language—as a vast, abstract space—was the surprising frontier.
- •Image and audio generation felt intuitively predictable to Suleyman; spatial and temporal correlations are easier to imagine.
- •Language seemed too abstract and combinatorial—“a gazillion possible next word predictions”—to yield coherent models via scaling alone.
- •Compute used for DeepMind’s Atari system (~2 petaflops) has been 10x’ed yearly; Inflection’s largest model uses ~10 billion petaflops.
- •This scaling produces chat experiences that feel like talking to a highly knowledgeable human.
- •Inflection’s Pi is positioned as a more emotional and empathetic alternative to other LLMs.
- 25:00 – 31:40
Pessimism Aversion and the Illusion of Safety
Suleyman introduces the ‘pessimism aversion trap,’ where elites avoid worst-case thinking about AI because it feels uncomfortable or unfashionably negative. He argues that optimism bias and historic narratives about job creation can blind us to real systemic risks.
- •People instinctively recoil from AI pessimism, preferring to believe “it’ll all work out” or that history guarantees job creation.
- •This aversion is especially strong in elite circles and has delayed serious policy discussion of AI risks.
- •Suleyman has long pushed ethics and safety into governmental and civil-society conversations, insisting on honest debate.
- •Avoiding pessimism prevents society from preparing for the ‘coming wave’ of intertwined technologies and risks.
- 31:40 – 38:20
Is Containment Even Possible? Dual-Use Tech and Nation-States
The conversation tackles whether AI and other advanced technologies can be contained when they are economically and militarily valuable. Suleyman notes some historical successes in limiting specific weapons, but stresses that AI’s omni-use nature makes the challenge unprecedented.
- •Some technologies have been constrained (CFCs, chemical weapons, blinding lasers) but only when substitutes existed or benefits were limited.
- •AI’s core methods are omni-use: the same model can spot cancer or select military targets.
- •Commercial and military incentives will drive proliferation unless nation-states update themselves to oversee both corporate and open-source development.
- •Regulation remains a real and necessary tool but must be matched to AI’s mixed-use character.
- 38:20 – 48:20
2050 and Beyond: Robots, New Biological Beings, and a ‘New Species’
Looking toward 2050 and even 2200, Suleyman describes a world of humanoid robots, engineered life forms, quantum computing, and near‑ubiquitous sensors. He candidly admits discomfort with how “species-like” advanced AI could become and insists humans must not cede their dominant role.
- •Robotics has lagged but is on a similar trajectory; humanoid robots performing many physical tasks are plausible by 2050.
- •Synthetic biology is enabling engineered DNA with customized traits (longer-lasting fruit, high-protein synthetic meat, etc.).
- •The darkest scenario: AI-accelerated design of highly transmissible or lethal pathogens, accidentally or intentionally released.
- •Suleyman views advanced AI as feeling like a “new species” that must be brought under control so humans remain dominant.
- •Quantum computing could make today’s massive computations nearly instantaneous, amplifying all digital capabilities.
- 48:20 – 56:40
Biological Risk, Dangerous Materials, and the Precautionary Principle
Suleyman argues we must treat frontier AI and advanced bio tools like dangerous materials, limiting access to compute, software, and lab resources. He calls for an unprecedented, proactive precautionary approach: slowing deployment until safety and containment mechanisms are demonstrated.
- •Future AI tools will likely be capable of assisting in design of novel pandemic‑class pathogens.
- •Containment requires chokepoints: controlling cloud access, high-end chips, model weights, and key biological substances.
- •Such restrictions will frustrate small developers and startups but are framed as non‑optional tradeoffs in a ‘dangerous materials’ era.
- •Unlike past technologies, we must choose to move slowly and prioritize ‘first, do no harm’ ahead of commercial upside.
- 56:40 – 1:10:00
Race Conditions, Nuclear Analogies, and the Limits of Historical Precedent
Drawing parallels with nuclear weapons, the host and Suleyman explore how fears of being outcompeted drive every actor to develop dangerous capabilities. Suleyman notes the partial success of nuclear non‑proliferation but underlines how much easier AI is to replicate and spread.
- •Race conditions—“if I don’t build it, my rival will”—create self-fulfilling arms races.
- •Nuclear non‑proliferation limited nuclear states to nine, and some countries even gave up weapons under pressure and incentives.
- •AI differs from nuclear tech: uranium is scarce and hard to manage; models and code are widely replicable and trending cheaper.
- •Older frontier models (e.g., GPT‑3) are already fully open source and drastically compressed, showing the diffusion pattern.
- 1:10:00 – 1:18:20
Human vs. Superintelligence: Respect, Control, and Saying ‘No’
The host presses on whether a vastly more intelligent AI would respect humans at all. Suleyman says this returns to containment: if we can’t reliably ensure systems remain aligned and under control, we must resist building self‑improving, autonomous AGI and learn to say ‘no’ at the frontier.
- •Designing AI with objective functions that respect human goals is crucial, but long‑term self‑improvement may escape our influence.
- •The parent‑child analogy—raising an intelligence 1000x smarter than you—highlights the limits of human control.
- •Suleyman acknowledges that in the long run such systems may ‘outstrip our ability to act,’ which is precisely why we must step back from that edge.
- •Containment involves deliberately refraining from certain lines of research (e.g., fully autonomous, self‑improving online agents).
- 1:18:20 – 1:26:40
Autonomous Tech, Rottweilers, and Humanity’s Track Record of Containment
Using a Rottweiler metaphor, the host questions whether we can leash something so much more powerful than us. Suleyman counters that we have contained dangerous forces before—but only via creativity, humility, and new governance cultures emphasizing peace and restraint.
- •Autonomy makes AI more like an actor than a tool, a departure from historical technologies.
- •Suleyman cites successes: we have contained nuclear weapons, dangerous animals, and certain pathogens to an extent.
- •He stresses today is “the most peaceful moment” in human history, with many lethal forces already partially controlled.
- •Meeting AI’s challenge will require a new, less paranoid political tone, oriented towards global stability rather than rivalry.
- 1:26:40 – 1:35:00
Everyday Harms: Cybersecurity, Deepfakes, and AI Defending Against AI
The conversation zooms into near‑term risks like scams, deepfakes, and cybercrime. Suleyman explains how skepticism, multi‑factor authentication, and AI‑based defense systems will be essential to preserve trust and safety in an environment where audio and video are easily faked.
- •Real examples already exist of voice‑cloned scams targeting vulnerable people (e.g., fake calls from ‘relatives’ needing money).
- •We will rely increasingly on multi‑factor and even 3‑ or 4‑factor authentication to verify identity and transactions.
- •AI is already used defensively—spam filters, fraud detection, anomaly detection in surveillance and data centers.
- •‘Good AIs’ will need to defend us from malicious use of the same underlying technologies.
- 1:35:00 – 1:45:00
Why Build AI Companies If You Fear Existential Risk?
The host challenges Suleyman on why he founded Inflection AI despite his deep worries. Suleyman argues that participating at the frontier is the best way to understand risks, shape safer practices, and avoid leaving decisions entirely to less scrupulous actors.
- •Suleyman believes that standing aside cedes influence to actors focused purely on profit or power.
- •Hands‑on experimentation reveals real capabilities and limitations, reducing naive fear but exposing deeper long‑term worries.
- •He admits that more understanding simultaneously decreases short‑term fear and increases appreciation of long‑term risk.
- •Inflection is structured as a Public Benefit Corporation, legally obliged to balance profit with social and environmental impact.
- 1:45:00 – 1:56:40
Regulation, Global Governance, and the Missing ‘Stability Function’
Suleyman explains why regulation is necessary but insufficient. He calls for new global institutions focused on long‑term technological stability, lamenting that current politics are trapped in short election cycles and zero‑sum narratives about China and other rivals.
- •Existing efforts (EU AI Act, Biden’s orders, UK initiatives) are positive but hamstrung by limited technical understanding and short-termism.
- •Politicians face powerful incentives to maximize near‑term economic gains (e.g., AI’s projected $15T impact) rather than slow development.
- •We lack an institutional body whose explicit mandate is global technology stability and containment, analogous to but beyond the UN/WTO.
- •Cold War–style “clash of civilizations” rhetoric with China exacerbates arms‑race dynamics and undermines cooperative containment.
- 1:56:40 – 2:06:40
Unstoppable Incentives, Catastrophe Triggers, and Honest Pessimism
The host presses Suleyman on whether, given all the incentives, he truly believes containment will happen. Suleyman frankly says the odds are low and notes that historically, serious global cooperation has followed only catastrophic events or clear, symmetric threat perceptions.
- •Two historical drivers of deep global compromise: (1) unimaginable catastrophe (e.g., WWII, Hiroshima), (2) clear mutually assured destruction.
- •Even COVID-19 and plausible lab-leak scenarios have not produced a decisive global shift on risky biological research.
- •Suleyman uses this to illustrate how hard it will be to preemptively contain AI and synthetic biology without a disaster.
- •He maintains his “must be possible” stance as a normative imperative, not a prediction.
- 2:06:40 – 2:16:40
Taxation, Chokepoints, and the Economics of Slowing AI
Detailing parts of his 10‑point containment agenda, Suleyman highlights chokepoints in cables, chips, and cloud platforms, plus heavy taxation of AI firms, as ways to introduce friction and fund social adaptation. He acknowledges this raises coordination problems as companies can relocate.
- •Internet infrastructure and GPU supply are concrete chokepoints where governments can monitor and limit frontier model training.
- •Inflection has already built a supercomputer costing about $1.3 billion, illustrating how concentrated high‑end compute currently is.
- •High taxation on AI could both slow deployment and fund reskilling, education, and social safety nets.
- •However, without international agreement, firms may migrate to lower‑tax, lower‑regulation jurisdictions, undermining containment.
- 2:16:40 – 2:26:40
Abundance, Work, and Universal Basic Income in an AI World
Suleyman sketches a positive long‑term scenario: AI‑driven breakthroughs make energy, water, food, and healthcare extremely cheap, leading to radical abundance and reduced need for work. He suggests this could support something like de‑facto UBI and new forms of meaning and ‘quests.’
- •AI could rapidly accelerate solutions to energy storage, enabling near‑free solar power.
- •Cheap energy implies cheap desalination, agriculture, transport, drug manufacturing, and healthcare.
- •Suleyman envisions a world where many people don’t need to work unless they want to; the central question becomes meaning and purpose.
- •He has long supported the idea of UBI, but thinks abundance will also come from structurally lower costs, not just cash transfers.
- •He argues that this ‘better problem’ (finding meaning in abundance) is preferable to existential threats from uncontrolled AI.
- 2:26:40 – 2:35:00
Transhumanism, Uploading Minds, and Why Suleyman Is Skeptical
The host raises transhumanism and mind uploading. Suleyman explains why he doubts we can extract and replicate the ‘essence’ of a human brain on silicon, and criticizes cryonics and related beliefs as speculative and unsupported by neuroscience.
- •Transhumanists envision humans transcending biology, potentially by uploading minds into digital substrates.
- •Suleyman sees little evidence we can separate mind/spirit from the biological brain in a way that could be digitized.
- •He describes cryonics—freezing brains after death for future revival—as implausible, calling some adherents ‘crackpots.’
- •He acknowledges adjacent tech trajectories but distinguishes them from their more extreme speculative interpretations.
- 2:35:00 – 2:46:40
Emotional Toll, Responsibility, and What Individuals Can Do
Toward the end, Suleyman opens up about the emotional weight of working on existentially significant technology. He feels both privileged and exhausted, urging individuals not to look away but to learn, experiment, and engage politically and culturally with AI’s implications.
- •He experiences a mix of exhilaration (seeing people benefit from AI) and exhaustion (constantly arguing for containment).
- •He feels a strong sense of responsibility—“history has its eyes on you”—given his role at DeepMind and Inflection.
- •He reports some relief now that the public and policymakers are finally paying attention post‑ChatGPT.
- •His advice to a future child: knowledge is power; don’t avert your gaze from scary realities; participate in shaping outcomes.
- •Practical advice to listeners: read, use AI tools, follow regulation debates, join or support advocacy groups, and keep raising these questions socially.
- 2:46:40
Endgame: Succeeding or Failing at Containment
The discussion closes with a stark contrast between success and failure in containment. Success yields radical abundance and a meritocratic, creativity‑rich civilization; failure leads to widespread proliferation of destructive capabilities, enabling small groups to destabilize the world.
- •Success scenario: AI makes intelligence widely available, slashes costs of essentials, and helps solve major global challenges.
- •Abundance raises philosophical questions about meaning and struggle, but Suleyman sees this as a far better tradeoff than existential risk.
- •Failure scenario: uncontrolled proliferation gives small malicious groups tools to cause mass harm via cyber, bio, and autonomous weapons.
- •Containment is thus framed as the effort to prevent those destabilizing actors from accessing world‑shaking capabilities.
