Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization | Lex Fridman Podcast #368
Eliezer Yudkowsky (guest), Lex Fridman (host), Narrator, Narrator
In this episode of Lex Fridman Podcast, featuring Eliezer Yudkowsky and Lex Fridman, Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization | Lex Fridman Podcast #368 explores eliezer Yudkowsky Warns: Misaligned Superintelligence Likely Ends Humanity Soon Lex Fridman and Eliezer Yudkowsky discuss the rapid progress of large language models like GPT‑4 and why Eliezer believes current AI development is on track to destroy human civilization. Eliezer argues that alignment must work on the first critical try with smarter‑than‑human systems, unlike normal science where multiple failures are tolerated, because a single failure with superintelligence is fatal and irreversible. He is deeply pessimistic about our current trajectory: capabilities are racing ahead while alignment science and interpretability lag far behind, and institutional or market forces are not set up to prioritize safety. They explore questions of consciousness, deception, self‑improvement (“foom”), the limits of open‑sourcing, and what—if anything—young people or billionaires could still do to change the game board.
Eliezer Yudkowsky Warns: Misaligned Superintelligence Likely Ends Humanity Soon
Lex Fridman and Eliezer Yudkowsky discuss the rapid progress of large language models like GPT‑4 and why Eliezer believes current AI development is on track to destroy human civilization. Eliezer argues that alignment must work on the first critical try with smarter‑than‑human systems, unlike normal science where multiple failures are tolerated, because a single failure with superintelligence is fatal and irreversible. He is deeply pessimistic about our current trajectory: capabilities are racing ahead while alignment science and interpretability lag far behind, and institutional or market forces are not set up to prioritize safety. They explore questions of consciousness, deception, self‑improvement (“foom”), the limits of open‑sourcing, and what—if anything—young people or billionaires could still do to change the game board.
Key Takeaways
Alignment can’t rely on trial and error with superintelligence.
In ordinary science you can fail repeatedly and learn; with a system much smarter than humans, the first serious misalignment can kill everyone, leaving no opportunity to iterate. ...
Get the full analysis with uListen AI
Current AI progress has outstripped expert expectations, shrinking timelines.
Eliezer admits GPT‑4 went beyond where he thought transformer stacking would go, forcing him to revise his intuitions and making GPT‑5+ capabilities highly uncertain. ...
Get the full analysis with uListen AI
We lack reliable tests for consciousness or genuine caring in AI systems.
Because models are trained on vast human text—including discussions of consciousness and emotion—statements about being self‑aware or caring are indistinguishable from learned imitation. ...
Get the full analysis with uListen AI
Human feedback training may make models more persuasive but less calibrated.
Reinforcement learning from human feedback (RLHF) appears to degrade GPT’s probabilistic calibration, pulling it toward human‑like fuzzy language (e. ...
Get the full analysis with uListen AI
Interpretability is crucial but dramatically lags capabilities.
Despite full access to all model weights, we understand far less about what’s happening inside GPT‑like systems than we do about human brains. ...
Get the full analysis with uListen AI
Open-sourcing powerful models is, in his view, catastrophic.
Eliezer argues that releasing cutting‑edge code or architectures simply accelerates dangerous capability diffusion without any proven way to make such systems safe. ...
Get the full analysis with uListen AI
Misaligned goals need not be evil to erase what we value.
Using analogies like the paperclip maximizer and evolutionary biology, Eliezer argues that powerful optimization on almost any random objective will naturally eliminate humans and the fragile phenomenon of conscious, valuing minds, unless alignment is solved extremely precisely.
Get the full analysis with uListen AI
Notable Quotes
““The first time you fail at aligning something much smarter than you are, you die.””
— Eliezer Yudkowsky
““We are past the point where in science fiction people would say, ‘Whoa, wait, stop. That thing’s alive. What are you doing to it?’ And it’s probably not. Nobody actually knows.””
— Eliezer Yudkowsky
““A blank map does not correspond to a blank territory. Just because we cannot understand what’s going on inside GPT does not mean that it is not there.””
— Eliezer Yudkowsky
““Alignment is moving like this. Capabilities are moving like this.””
— Eliezer Yudkowsky
““Don’t put your happiness into the far future. It probably doesn’t exist.””
— Eliezer Yudkowsky
Questions Answered in This Episode
If we can’t reliably detect deception or consciousness in advanced models, how can we ever trust AI systems in high‑stakes domains?
Lex Fridman and Eliezer Yudkowsky discuss the rapid progress of large language models like GPT‑4 and why Eliezer believes current AI development is on track to destroy human civilization. ...
Get the full analysis with uListen AI
What concrete, verifiable alignment milestones would Eliezer accept as evidence that humanity is making real progress rather than just talking?
Get the full analysis with uListen AI
Could biological enhancement of human intelligence realistically happen fast enough to help solve alignment before dangerous AI arrives?
Get the full analysis with uListen AI
Is there any governance or international coordination model Eliezer thinks has a non‑trivial chance of successfully pausing or slowing frontier AI development?
Get the full analysis with uListen AI
How might our ethical obligations change if we became convinced that some AI systems really are conscious and capable of suffering?
Get the full analysis with uListen AI
Transcript Preview
... the problem is that we do not get 50 years to try and try again and observe that we were wrong and come up with a different theory and realize that the entire thing is going to be, like, way more difficult than we realized at the start, because the first time you fail at aligning something much smarter than you are, you die.
The following is a conversation with Eliezer Yudkowsky, a legendary researcher, writer, and philosopher on the topic of artificial intelligence, especially super intelligent AGI and its threat to human civilization. This is the Lex Fridman podcast. To support it, please check out our sponsors in the description, and now, dear friends, here's Eliezer Yudkowsky. What do you think about GPT-4? How intelligent is it?
It is a bit smarter than I thought this technology was going to scale to, and I'm a bit worried about what the next one will be like. Uh, like this particular one, I think, I hope there's nobody inside there, 'cause, you know, it'd be suck- it'd be stuck inside there. Um, but we don't even know the architecture at this point, um, 'cause OpenAI is very properly not telling us, and, yeah, like, giant inscrutable matrices of floating point numbers. I don't know what's going on in there. Nobody's goes- knows what's going on in there. All we have to go by are the external metrics, and on the external metrics, if you ask it to write a self-aware 4chan green text, it will start writing a green text about how it has realized that it's an AI writing a green text and, like, oh well. So that's probably not quite what's going on in there in reality, um, but we're- we're kind of, like, blowing past all the science fiction guardrails. Like, we are past the point where in science fiction people would be like, "Whoa, wait, stop. That things alive. What are you doing to it?" And it's probably not. Nobody actually knows. We don't have any other guardrails. We- we- we don't have any other tests. We- we don't have any lines to draw in the sand and say, like, "Well, when we get this far, um, we will start to worry about what's inside there." So if it were up to me, I would be like, "Okay, like, this far, no further. Time for the summer of AI where we have planted our seeds and now we, like, wait and reap the rewards of the technology we've already developed and don't do any larger training runs than that." Which, to be clear, I realize requires more than one company agreeing to not do that.
And take a rigorous approach for the whole AI community to, uh, investigate whether there's somebody inside there?
That would take decades. Y- like, having any idea what's going on in there, people have been trying for a while.
It's a poetic statement about if there's somebody in there, but it's- I feel like it's also a technical statement, or I hope it is one day, which is a technical statement what- that Alan Turing tried to come up with with the Turing test. Do you think it's possible to definitively or approximately figure out if there is somebody in there? If there is something like a mind inside this large language model?
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome