Dwarkesh PodcastDario Amodei on Dwarkesh Patel: Why the Exponential Ends
Why the big blob of compute predicts log-linear gains through 2025: AIME-tested RL and pre-training confirm the curve; SWE task breadth is the remaining gap.
At a glance
WHAT IT’S REALLY ABOUT
Dario Amodei argues scaling continues, but exponential progress is ending soon
- Dario Amodei claims the core scaling thesis hasn’t changed since his 2017 “Big Blob of Compute” view: compute, data (quantity/quality), training time, scalable objectives, and stability tricks dominate progress more than clever new algorithms.
- He argues reinforcement learning now shows scaling behavior similar to pretraining, and that perceived “missing” abilities (like on-the-job learning) may be less necessary than people think—though he expects continual learning to likely arrive within 1–2 years anyway.
- Amodei predicts a “country of geniuses in a datacenter” could arrive in ~1–3 years (with high confidence within 10), but emphasizes economic diffusion and real-world bottlenecks (procurement, regulation, manufacturing, deployment) make impacts fast yet not instantaneous.
- They debate why labs don’t simply buy unlimited compute, how AI companies can be profitable amid huge capex and uncertain demand, and what governance/regulation is needed to reduce risks (bio, autonomy, offense-dominant security) without killing benefits; they also discuss export controls, authoritarianism, and Anthropic’s “constitution” approach to alignment.
IDEAS WORTH REMEMBERING
5 ideasScaling is still mostly about “inputs,” not cleverness.
Amodei reiterates that progress is dominated by compute, data quantity/quality/distribution, training duration, and scalable objectives, plus engineering for stable training; algorithmic novelty matters less than many assume.
RL is becoming another predictable scaling regime.
He argues RL training (including verifiable tasks like math/coding and broader tasks) shows log-linear improvement with more training—mirroring pretraining-style scaling—and will generalize more as task distributions widen.
LLM training may resemble evolution more than human lifetime learning.
Humans start with strong biological priors; models start near-blank. Pretraining/RL can be viewed as “compressed evolution,” while in-context learning provides a limited analog of short-term learning.
Continual learning might not be required for transformative automation.
Amodei claims broad pretraining + RL generalization and large context windows can already approximate much “on-the-job learning,” and could be sufficient for massive economic value even if true continual learning lags.
Computer-use reliability is a key gating factor for ‘agentic’ work.
He highlights progress on computer-use benchmarks (e.g., OSWorld rising from low teens to ~65–70%) and frames remaining issues as reliability and deployment engineering rather than conceptual research blockers.
WORDS WORTH SAVING
5 quotes“The most surprising thing has been the lack of public recognition of how close we are to the end of the exponential.”
— Dario Amodei
“All the cleverness… doesn’t matter very much. There are only a few things that matter.”
— Dario Amodei
“If you had the country of geniuses in a data center, we would know it.”
— Dario Amodei
“It will be faster than anything we’ve seen in the world, but it still has its limits.”
— Dario Amodei
“It is hard for me to see that there won’t be trillions of dollars in revenue before 2030.”
— Dario Amodei
High quality AI-generated summary created from speaker-labeled transcript.
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome