Dwarkesh PodcastTerence Tao on Dwarkesh Patel: How Erdős Problems Exposed AI
Tycho Brahe data let Kepler derive orbital laws by regression on six points; AI solved 50 Erdős problems fast then stalled on cumulative partial progress.
At a glance
WHAT IT’S REALLY ABOUT
Terence Tao on AI math: verification, depth, and new workflows
- Kepler’s laws illustrate how massive hypothesis generation only matters when paired with high-quality data and rigorous verification, and AI similarly drives the cost of idea generation toward zero while shifting the bottleneck to validation and filtering.
- Scientific and mathematical progress is not just about correctness but about adoption, narrative, and context over time, which makes it hard to algorithmically identify “unifying” ideas or score partial progress in the moment.
- Current AI math successes (e.g., Erdős problems) are real but skewed by selection bias: systematic sweeps show low per-problem success rates, and recent “one-shot” breakthroughs plateaued after low-hanging fruit.
- AI is already making mathematical work richer and broader (code, plots, literature search, formatting, experimental math), but it has not yet reliably improved the deepest step—creating new techniques that bridge the last resistant gaps.
- Tao expects human–AI hybrids to dominate for a long time, with major gains coming from reorganizing research around AI’s breadth and building better semi-formal languages and benchmarks for strategies, plausibility, and intermediate progress.
IDEAS WORTH REMEMBERING
5 ideasAI makes hypotheses abundant; verification becomes the scarce resource.
Tao argues AI collapses the cost of generating theories, similar to how the internet collapsed communication costs, but science must scale evaluation, validation, and prioritization or drown in low-signal output.
High-quality data is the enabling constraint for meaningful “Kepler-like” pattern-finding.
Kepler’s success depended on Brahe’s extra decimal of accuracy; without strong datasets, mass ideation degenerates into “slop,” whether from humans or models.
Small datasets can create seductive but false laws—AI will amplify this failure mode.
Tao’s Bode’s-law example shows how fitting patterns to a few points can look profound until new data breaks it; AI’s speed increases the risk of overfitting and premature hype.
“Unifying concepts” are often only recognizable after adoption and downstream use.
Tao notes that ideas like deep learning architectures or numeric conventions gain importance through social standardization and future applications, not from an objective, context-free score at birth.
AI math progress is real but systematically overstated by spotlighting wins.
Across broad problem sweeps, Tao cites ~1–2% success per problem; social media highlights the solved subset, masking the large base rate of failures and stalled attempts.
WORDS WORTH SAVING
5 quotesAI has basically driven the cost of idea generation down to almost zero.
— Terence Tao
It’s made the papers sort of richer and broader, but not necessarily deeper.
— Terence Tao
Whenever we do a systematic study, any given problem, an AI tool has a success rate of maybe one or 2%. It’s just that they can apply a scale, and if you just pick the winners, it looks great.
— Terence Tao
Often the ultimately correct theory initially is worse in many ways.
— Terence Tao
If there was some framework that sort of mimics how scientists talk to each other in a semi-formal way… we don’t know how to capture [the subjective aspect] in a way that we can insert AI into them in any useful way.
— Terence Tao
High quality AI-generated summary created from speaker-labeled transcript.
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome