OpenAIWhat happens now that AI is good at math? — the OpenAI Podcast Ep. 17
Andrew Mayne and Sébastien Bubeck on aI’s math leap enables research, verification, and automated discovery workflows.
In this episode of OpenAI, featuring Andrew Mayne and Sébastien Bubeck, What happens now that AI is good at math? — the OpenAI Podcast Ep. 17 explores aI’s math leap enables research, verification, and automated discovery workflows Researchers describe math as a uniquely clean benchmark for AI progress because problems are unambiguous and solutions are often verifiable, making capability jumps easy to measure.
At a glance
WHAT IT’S REALLY ABOUT
AI’s math leap enables research, verification, and automated discovery workflows
- Researchers describe math as a uniquely clean benchmark for AI progress because problems are unambiguous and solutions are often verifiable, making capability jumps easy to measure.
- Ernest Ryu recounts using ChatGPT—through iterative human verification and steering—to resolve a 42-year-old open optimization problem about divergence in Nesterov’s accelerated gradient method.
- Sébastien Bubeck argues the apparent “sudden” math improvement wasn’t just scaling, but a bundle of training and reasoning innovations that expanded models’ ability to sustain long, consistent chains of thought.
- The conversation frames strong math reasoning as central to AGI because it demands long-horizon, self-correcting reasoning that should transfer to other sciences and enable “automated researcher” systems.
- They emphasize both upside (faster discovery, deeper literature connections, proof checking) and risk (shallow understanding, over-trust, low-quality AI-generated proofs), concluding humans remain essential for direction, standards, and accountability.
IDEAS WORTH REMEMBERING
5 ideasMath capability gains reflect more than scaling—multiple innovations compound.
Bubeck rejects “scaling alone” as the right framing, noting OpenAI’s progress came from concurrent research advances; this helps explain why users perceived an abrupt jump in reliability on tasks like scheduling and ledger-splitting.
AI can already contribute to research when paired with expert human verification.
Ryu’s 12-hour, multi-day interaction shows the model didn’t magically one-shot a proof; the human played verifier, corrected mistakes, and guided approaches—turning AI into a high-speed collaborator rather than an oracle.
Long, consistent reasoning is the core skill math trains in both humans and models.
They argue math rewards correctness across entire chains: one small error can invalidate everything, so models must learn self-correction and coherence over extended reasoning—properties expected to generalize to other scientific domains.
“AGI time” is a useful lens for progress: seconds → minutes → hours → days → weeks.
Bubeck frames capability not just as IQ-like performance, but duration of sustained competent work; the automated researcher goal is to push this horizon to weeks/months to enable deeper breakthroughs and experimental loops.
Erdős-problem wins illustrate two distinct superpowers: cross-literature connection and original discovery.
Early successes were sometimes “deep literature search” (finding answers in distant fields and translating them), which sparked controversy when framed as solving “open problems”; later they claim models produced genuinely new, publishable combinatorics results.
WORDS WORTH SAVING
5 quotesToday, two years later, the models are able to help Fields Medalists in their day-to-day work.
— Sébastien Bubeck
And that's how this, uh, 42-year-old open problem got resolved.
— Ernest Ryu
If at some point in your chain of reasoning there is a mistake, this will kill the entire argument.
— Sébastien Bubeck
So you can have AGI seconds, minutes, hours, days, and so on.
— Sébastien Bubeck
I'm worried about potentially having a shallower understanding of things because we rely too much on the tool.
— Sébastien Bubeck
QUESTIONS ANSWERED IN THIS EPISODE
5 questionsIn Ryu’s Nesterov accelerated gradient example, what were the key “human verifier” interventions (the exact kinds of mistakes corrected and steering decisions) that made the final proof succeed?
Researchers describe math as a uniquely clean benchmark for AI progress because problems are unambiguous and solutions are often verifiable, making capability jumps easy to measure.
For the Erdős-problem cases, how do you operationally distinguish “deep literature search + translation” from genuinely new mathematics, and what evidence would convince skeptics?
Ernest Ryu recounts using ChatGPT—through iterative human verification and steering—to resolve a 42-year-old open optimization problem about divergence in Nesterov’s accelerated gradient method.
What specific training or evaluation changes most improved sustained multi-step mathematical consistency (as opposed to just final-answer accuracy)?
Sébastien Bubeck argues the apparent “sudden” math improvement wasn’t just scaling, but a bundle of training and reasoning innovations that expanded models’ ability to sustain long, consistent chains of thought.
How would an “automated researcher” manage long projects practically—notes, intermediate lemmas, experiment logs, and revisiting ideas—without being bottlenecked by a single context window?
The conversation frames strong math reasoning as central to AGI because it demands long-horizon, self-correcting reasoning that should transfer to other sciences and enable “automated researcher” systems.
If models are already finding errors in papers internally, what should journals and arXiv-style platforms change (workflows, disclosures, automated checks) to handle both faster verification and more AI-generated submissions?
They emphasize both upside (faster discovery, deeper literature connections, proof checking) and risk (shallow understanding, over-trust, low-quality AI-generated proofs), concluding humans remain essential for direction, standards, and accountability.
Chapter Breakdown
From “laughable at math” to Fields Medal assistance
Andrew Mayne opens with Sébastien Bubeck and Ernest Ryu on how quickly AI math capabilities have advanced—from pre-reasoning models to systems that can meaningfully assist working mathematicians. They frame mathematics as an unexpectedly revealing domain for measuring reasoning progress and why the pace has surprised even experts.
A 42-year-old open problem solved with ChatGPT (and a human verifier)
Ernest recounts using ChatGPT to tackle a genuinely open optimization-theory question about Nesterov’s accelerated gradient method. Over about 12 hours across three evenings, he iterated with the model, corrected mistakes, guided approaches, and verified the final argument—resulting in a correct resolution of a decades-old question.
Calibrating progress: from everyday arithmetic failures to IMO gold
The group contrasts early model limitations (splitting expenses, time-zone scheduling) with sudden improvements that culminated in top human-level International Math Olympiad performance. They distinguish competition math (short, “canned” solutions) from research math, while noting the practical threshold: for most STEM users, today’s models cover nearly all needed mathematics—with caution and checks.
What changed: beyond scaling, toward reasoning systems
Sébastien argues the “scaling alone” framing misses the point: multiple innovations progressed together, not one silver bullet. He situates the progress historically (e.g., Minerva era) to show how quickly expectations have shifted, and emphasizes that modern models can often solve problems directly (not just via calculator tools).
Why math matters for AGI: long, consistent chains of thought
Math is presented as more than “cool”—it demands long-horizon, error-intolerant reasoning where a single mistake collapses the whole argument. That property makes it an ideal training/measurement ground for reasoning that should transfer to other domains, mirroring why humans learn math for disciplined thinking.
Erdős problems, literature search breakthroughs, and a communication trap
Sébastien explains Paul Erdős, the culture around his questions, and the online catalog of open problems. They describe early successes where models didn’t invent new proofs but performed deep “literature search + translation,” connecting results across fields—followed by controversy when those results were misunderstood as brand-new solutions.
From rediscovery to genuinely new combinatorics results
They describe rapid acceleration from “finding answers that already exist” to producing novel, publishable solutions. This raises deeper questions about what scientific creativity is—mere recombination plus reasoning, or rare sparks of genius—and whether AI can continuously extend human knowledge without bound.
The automated researcher and ‘AGI time’ (seconds → weeks → months)
The conversation shifts to building systems that can work autonomously over long horizons, not just within a single chat session. Sébastien introduces “AGI time” as the duration an AI can sustain human-like research thinking, arguing that the key frontier is extending this from days to weeks and beyond—an open research problem tied to the “automated researcher” vision.
Context limits, persistent workspaces, and the Codex analogy for math
Ernest highlights that typical chat context is roughly the size of a ~50-page paper, which is insufficient for deep breakthroughs that require far more thinking than the final write-up. He points to tools like Codex that operate over large codebases with persistent artifacts as a model for how math research agents could maintain long-running notes, summaries, and evolving work products.
Science acceleration in practice: lowering friction and expanding who can do what
Andrew shares a hands-on example of generating a benchmark dataset mid-workflow in minutes—something that would otherwise derail progress. Sébastien ties this to ‘science acceleration’ and emphasizes a two-way effect: mathematicians gain easy access to coding/experiments, and scientists in other fields gain access to advanced math.
Humans’ role as models surpass researchers: direction, meaning, and priorities
Sébastien predicts continued progress: systems that think for weeks, then years, plus agents that find mistakes in papers and even propose valuable new questions. He argues the human role becomes more about setting goals and ensuring science serves human needs (health, control over environment), since AI has no intrinsic stake in those outcomes.
Verification, shallow understanding risk, and how to learn math with ChatGPT
They discuss the dual-edged nature of AI: it can accelerate proof checking and improve trust, but overreliance can erode deep expertise and produce confident nonsense—especially from non-experts attempting grand proofs. They close with practical learning advice: use ChatGPT as an adaptive tutor, ask it for problems at your level, iterate socially, but still do the hard work of understanding and verifying.
EVERY SPOKEN WORD
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome