Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

Dwarkesh PodcastDwarkesh Podcast

We could be 12 months away from AIs that can overthrow the government

Paul Christiano is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF.

Dwarkesh PatelhostPaul Christianoguest
Nov 16, 20261h 38mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Great conversations run long. Their best ideas don't have to.

uListen is free on iPhone, iPad, Android, Chrome, and Claude.

See how

Episode Details

EPISODE INFO

Released
November 16, 2026
Duration
1h 38m
Channel
Dwarkesh Podcast
Watch on YouTube
▶ Open ↗

EPISODE DESCRIPTION

Paul Christiano is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen). Paul thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario). He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails. He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it. And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion. 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒

𝐒𝐏𝐎𝐍𝐒𝐎𝐑𝐒

• Jane Street has finally come around on formal methods. Ron Minsky, who co-heads Jane Street’s technology group, told me that up until recently, formal methods just weren’t worth the cost. But coding agents have made them easier to apply AND increased the importance of code verification. So now Jane Street is hiring Formal Methods Engineers in both their London and New York offices. If you’re interested, or if you want to watch my full conversation with Ron, go to https://janestreet.com/dwarkesh

• Antithesis is NOT another LLM-based code-review tool. It doesn’t try to reason abstractly about how a PR might break your system. Instead, it just runs your code. You start by defining the rules that your system must never break. Then, whenever an agent proposes a change, Antithesis runs your code through thousands of adversarial worlds, testing whether or not it still adheres to your rules. If you want to try it, go to https://antithesis.com/dwarkesh and use code DWARKESH for 50% off your first month.

• WorkOS lets you give agents their own sign-up flow. This matters because agents tend to pick tools they can use right away, rather than ones that require humans to register. Fortunately, providing this kind of dedicated path is easy: if you already use WorkOS for login, you just toggle on “Agent Registration” in the WorkOS dashboard. Once it’s enabled, you can decide what agents are allowed to do on their own, and what needs to wait until the person they’re working for gets involved, all with the permissions you’ve already set up. Learn more at https://workos.com/authkit To sponsor a future episode, visit https://dwarkesh.com/advertise. 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 0:00:00 – We could be 12 months away from AIs that can overthrow the government 0:11:44 – Why an AI might want to takeover 0:20:49 – How an AI takeover could happen 0:35:35 – If labs keep racing without security, China will just steal the weights of superintelligence 0:53:15 – The case for pausing now instead of later 0:59:31 – Superintelligence will be good at learning from real-world experience 1:11:32 – Crossing the sim-to-real gap 1:22:23 – METR says AI progress doubled, so why is revenue growth slowing? 1:27:57 – The best plan we have is to build the plane as we fly it

SPEAKERS

  • Dwarkesh Patel

    host

    Host of the Dwarkesh Podcast, interviewing researchers about AI, policy, and long-term risks.

  • Paul Christiano

    guest

    AI safety researcher associated with ARC (Alignment Research Center) and known for work on alignment and AI governance concerns.

EPISODE SUMMARY

In this episode of Dwarkesh Podcast, featuring Dwarkesh Patel and Paul Christiano, We could be 12 months away from AIs that can overthrow the government explores Christiano warns near-term superintelligence could trigger takeover via misalignment and leaks Paul Christiano argues there is a plausible near-term path to superintelligence driven by automation of AI R&D, compressing years of progress into months and making disruption possible within 6–12 months.

RELATED EPISODES

What Robert Caro learned about power after studying it for 50 years

What Robert Caro learned about power after studying it for 50 years

What will computer chips look like in 2045?

What will computer chips look like in 2045?

How did a few hundred Spanish soldiers topple two empires? – Si Sheppard

How did a few hundred Spanish soldiers topple two empires? – Si Sheppard

OpenAI researcher on agent swarms & recursive self-improvement

OpenAI researcher on agent swarms & recursive self-improvement

AI researchers debate how close we are to recursive self-improvement

AI researchers debate how close we are to recursive self-improvement

Ajeya Cotra – "This might be the clearest warning shot we ever get"

Ajeya Cotra – "This might be the clearest warning shot we ever get"

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.