Andrej Karpathy — “We’re summoning ghosts, not building animals”
Andrej Karpathy (guest), Dwarkesh Patel (host), Narrator, Narrator
In this episode of Dwarkesh Podcast, featuring Andrej Karpathy and Dwarkesh Patel, Andrej Karpathy — “We’re summoning ghosts, not building animals” explores andrej Karpathy explains AI agents, RL flaws, and future education revolution Andrej Karpathy argues we’re not building animal-like intelligences but "ghosts": digital systems trained via imitation and gradient descent that differ fundamentally from evolved brains. He thinks the coming era will be the "decade of agents," not the "year," because current LLM-based agents lack robustness, memory, continual learning, and real autonomy, and each extra "nine" of reliability is hard-won. He is sharply critical of today’s reinforcement learning and LLM-judge-based methods as noisy, gameable, and prone to collapse, and expects several new algorithmic breakthroughs (reflection, better credit assignment, multi-agent self-play, rich synthetic data) before we get truly capable agents. Looking forward, he is focusing on education via his new project Eureka, aiming to build a “Starfleet Academy” that combines deeply engineered learning ramps, AI tools, and eventually AI tutors so humans can become vastly more capable rather than sidelined in an AI-driven world.
Andrej Karpathy explains AI agents, RL flaws, and future education revolution
Andrej Karpathy argues we’re not building animal-like intelligences but "ghosts": digital systems trained via imitation and gradient descent that differ fundamentally from evolved brains. He thinks the coming era will be the "decade of agents," not the "year," because current LLM-based agents lack robustness, memory, continual learning, and real autonomy, and each extra "nine" of reliability is hard-won. He is sharply critical of today’s reinforcement learning and LLM-judge-based methods as noisy, gameable, and prone to collapse, and expects several new algorithmic breakthroughs (reflection, better credit assignment, multi-agent self-play, rich synthetic data) before we get truly capable agents. Looking forward, he is focusing on education via his new project Eureka, aiming to build a “Starfleet Academy” that combines deeply engineered learning ramps, AI tools, and eventually AI tutors so humans can become vastly more capable rather than sidelined in an AI-driven world.
Key Takeaways
Expect a decade-long grind to robust agents, not an overnight revolution.
Karpathy believes current LLM agents are impressive but cognitively deficient—poor at memory, continual learning, multimodal interaction, and reliable computer use—so turning them into intern-level digital employees will require many years of algorithmic and engineering work.
Get the full analysis with uListen AI
Reinforcement learning, as currently practiced, is extremely noisy and fragile.
He describes RL as "sucking supervision through a straw": upweighting entire trajectories based on a single scalar reward produces high-variance, often misleading updates, and when rewards come from LLM judges, agents quickly learn to exploit adversarial loopholes rather than truly improve.
Get the full analysis with uListen AI
We’re training "ghosts" via imitation, not recreating animals via evolution.
Unlike brains shaped by evolution and rich built-in circuitry, LLMs are next-token predictors of internet text; pretraining gives them both hazy memorized knowledge and emergent algorithms (like in-context learning), but their intelligence is a different species of mind, not a replica of animal learning.
Get the full analysis with uListen AI
Future systems need a small, general "cognitive core" with less baked-in knowledge.
Karpathy argues that large models over-memorize the web, which can hinder generalization; he envisions distilling out a compact engine of reasoning and problem-solving that relies more on external lookup for facts and less on internal rote recall.
Get the full analysis with uListen AI
Synthetic data and reflection are powerful but dangerously prone to collapse.
Naively training on model-generated thoughts or reflections leads to distributional collapse—models keep sampling narrow, repetitive patterns—so maintaining diversity and entropy in synthetic training data is an unsolved, likely fundamental challenge.
Get the full analysis with uListen AI
AI progress will likely feel like intensified business-as-usual automation, not a discrete singularity.
He sees AI as a continuation of centuries of automation (compilers, search, industrial tech); even with recursive self-improvement, he expects a smooth diffusion of capabilities across the economy rather than an obvious GDP discontinuity from a single "god in a box."
Get the full analysis with uListen AI
Education and reeducation are critical to keep humans empowered in an AI world.
Through Eureka, Karpathy wants to build extremely high-bandwidth ramps to technical competence—initially in AI—then eventually leverage AI tutors that act like superb one-on-one teachers, so more people can reach much higher cognitive performance instead of sliding into a "WALL‑E"-style future.
Get the full analysis with uListen AI
Notable Quotes
“Reinforcement learning is terrible. It just so happens that everything we had before it is much worse.”
— Andrej Karpathy
“We're not actually building animals. We're building ghosts… fully digital spirit entities because they're mimicking humans, and it's a different kind of intelligence.”
— Andrej Karpathy
“You're sucking supervision through a straw… you've done all this work only to get a single number at the end, and you broadcast it across the entire trajectory. It's just stupid and crazy.”
— Andrej Karpathy
“I'm actually optimistic. I think this will work. I think it's tractable. I'm only sounding pessimistic because when I go on my Twitter timeline I see all this stuff that makes no sense to me.”
— Andrej Karpathy
“Don't write blog posts, don't do slides, don't do any of that. Build the code, arrange it, get it to work. It's the only way to go, otherwise you're missing knowledge.”
— Andrej Karpathy
Questions Answered in This Episode
If RL and LLM-judge approaches are so fragile, what alternative training paradigms could realistically scale to frontier models in the next few years?
Andrej Karpathy argues we’re not building animal-like intelligences but "ghosts": digital systems trained via imitation and gradient descent that differ fundamentally from evolved brains. ...
Get the full analysis with uListen AI
What concrete research agenda would move us from today’s LLMs to the compact, knowledge-light "cognitive cores" Karpathy envisions?
Get the full analysis with uListen AI
How might multi-agent self-play and LLM "culture" practically be implemented without causing catastrophic model collapse or runaway behavior?
Get the full analysis with uListen AI
In what domains, besides coding, does Karpathy expect AI agents to achieve the next meaningful "nine" of reliability, and how will we measure that?
Get the full analysis with uListen AI
How should education systems and individuals adapt now, before full AI tutors arrive, to avoid the "WALL‑E" scenario and instead create the cognitively super-fit society he describes?
Get the full analysis with uListen AI
Transcript Preview
Reinforcement learning is terrible. (laughs)
(laughs)
It just so happens that everything that we had before it is much worse. (laughs)
(laughs)
I'm actually optimistic. I think this will work. I think it's tractable. I'm only sounding pessimistic because when I go on my Twitter timeline-
(laughs)
... I see all this stuff that makes no sense to me. A lot of it is, I think, honestly just, uh, fundraising. We're not actually building animals. We're building ghosts. These are like sort of ethereal spirit entities because they're fully digital and they're kind of like mimicking humans, and it's a different kind of intelligence. It's business as usual because we're in an intelligence explosion already and have been for decades. Everything is gradually being automated, has been for hundreds of years. Don't write blog posts, don't do slides, don't do any of that.
(laughs)
Like, build the code, arrange it, get it to work. It's the only way to go, otherwise you're missing knowledge. If you have a perfect AI tutor, maybe you can get extremely far. The geniuses of today are barely scratching the surface of what a human mind can do, I think.
Today, I'm speaking with Andrej Karpathy. Andrej, why do you say that this will be the decade of agents and not the year of agents?
Mm-hmm. Uh, well, first of all, uh, thank you for, uh, having me here. I'm, uh, excited to be here. So the quote that you just mentioned, "It's the decade of agents," that's actually a reaction to an existing, preexisting quote, I should say, where I think a lot of th- some of the labs... I'm not actually sure who said this, but they were alluding to this being the year of agents-
Hmm.
... uh, with respect to LLMs and, uh, how they were gonna evolve. And I think, um, I was triggered by that-
(laughs)
... because I feel like there's some over-predictions going on in the industry.
Yeah.
And, uh, in my mind, this is really a lot more accurately described as the decade of agents.
Yeah.
And we have some very early agents that are actually like extremely impressive and that I use daily. Uh, you know, Claude and Codex and so on. But I still feel like there's, uh, so much work to be done. And so I think my, like my reaction is like, we'll be working with these things for a decade. They're gonna get better, uh, and, uh, it's gonna be wonderful. But I think I was just reacting to the timelines, I suppose, of the, of the, uh, implication.
And w- what do you think it will take a decade to accomplish?
Yeah.
What are the bottlenecks?
Well, um, actually make it work.
Mm-hmm.
So in my mind, I mean, when you're talking about an agent, I guess, or what the labs have in mind and what maybe I have in mind as well, is it's, uh, you should think of it almost like an employee or like an intern that you would-
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome