Skip to content
a16za16z

Inside OpenAI’s Breakthroughs in Mathematical Reasoning

a16z Infra Partner Lisha Li sits down with OpenAI mathematicians Mehtaab Sawhney and Mark Sellke to discuss how quickly AI’s mathematical capabilities are advancing, what recent results reveal about model reasoning, and what happens when AI begins making progress on problems mathematicians have struggled with for decades. Mehtaab and Mark unpack several recent results from OpenAI’s models, including advances in sphere packing and the construction of a non-sofic group. They explain why the surprising part isn’t simply that models can search more possibilities or work longer than humans: in many cases, the reasoning traces look remarkably similar to the work of an expert mathematician, including choosing promising approaches, backtracking when they fail, and combining ideas from across the literature. They also explore what this means for mathematics itself: how the role of human taste and judgment may change, whether AI could produce far more mathematics than humans can absorb, and why models that accelerate discovery may also make sophisticated results easier to understand. Timestamps: 00:00 - Intro 00:50 - From Practicing Mathematician to OpenAI: Meet Mark & Mehtaab 02:43 - Why GPT-5 Was the Conversion Moment 04:21 - Beyond Search & Connections: How Recent Progress Goes Deeper 09:51 - Reasoning Traces: Is It Lucky Sampling or Actual Backtracking? 11:44 - Why Math Papers Are a Bad Training Set for Real Mathematics 16:20 - The Astra 10-Problem Set: Favorites & Deep Dives 36:17 - The Harness vs the Model: What Actually Matters? 40:01 - What Even Is "Taste" in a Model? 57:32 - How Should the Math Community Adopt AI? 01:00:01 - Empirical vs Theoretical Math & the Positive Vision Resources: Follow Lisha Li on X: https://x.com/lishali88 Follow Mehtaab Sawhney on X: https://x.com/mehtaab_sawhney Follow Mark Sellke on X: https://x.com/MarkSellke Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.

Lisha LihostMehtaab Sawhneyguest
Sep 8, 20261h 5mWatch on YouTube ↗

CHAPTERS

  1. 0:00 – 0:50

    Cold open: Why AI changes the math grind (and what might still be out of reach)

    A rapid montage sets the theme: mathematicians often abandon promising ideas due to time and uncertainty, while AI can persist and explore. The speakers tease questions about whether models are “guessing,” how far reasoning can scale, and how the math community should respond.

    • Human math work often ends in giving up due to time/risk constraints
    • AI’s persistence can turn “reachable” results into a renaissance
    • Skepticism: is the model guessing or truly reasoning?
    • Open question: even with AI, some problems (e.g., P vs NP) may remain unsolved
    • How the community should adopt and credit AI-assisted work
  2. 0:50 – 2:43

    From practicing mathematicians to OpenAI: who Mark and Mehtaab are

    Lisha Li introduces Mark Sellke and Mehtaab Sawhney, highlighting their shared mathematical background and prior collaboration. They outline what drew them toward working on AI-driven mathematical reasoning at OpenAI.

    • Introductions and connection via academic lineage and prior coauthorship
    • Both became excited as math capabilities accelerated in recent models
    • Motivation: understand what enabled the apparent breakthrough-level performance
    • Framing: discuss both OpenAI’s approach and where AI-for-math is going
  3. 2:43 – 4:04

    The GPT-5 “conversion moment”: solving an Erdős problem via literature + direction

    Mark describes the moment he realized something had changed: GPT-5 quickly found a reference and path forward on an “open” problem whose status was hard to verify. What started as better search quickly felt like genuine mathematical usefulness.

    • Erdős problem lists are difficult to keep current; literature search is nontrivial
    • GPT-5 rapidly identified relevant references and provided a workable approach
    • Human time/effort can be wasted if a problem is already solved or “in reach”
    • This success repeated across multiple similar cases, increasing confidence
  4. 4:04 – 9:51

    Beyond search: execution, detail management, and making ideas actually work

    They discuss why recent progress is deeper than connecting references: models are unusually strong at carrying an idea through finicky technical details. This reduces the practical barrier that often stops humans from converting a plausible approach into a complete proof.

    • AI strength: reliably handling delicate “epsilon vs delta” style bookkeeping
    • Humans often abandon approaches when details become too risky or time-consuming
    • Models can doggedly execute without fatigue, improving the odds of completion
    • The “risk/reward” calculus changes when the marginal cost of trying is low
  5. 9:51 – 11:45

    Reasoning traces: backtracking, pruning, and avoiding “polluted context”

    The conversation turns to whether success is lucky sampling or real reasoning. They describe evidence of mistakes, backtracking, and systematic updating—plus the practical advantage of restarting fresh sessions to avoid cognitive inertia humans face.

    • Models do make mistakes, then revisit and correct (backtracking behavior)
    • They appear better than humans at updating which branches are promising
    • Parallelism: you can run many attempts or restart cleanly with no sunk-cost bias
    • Summarized chains-of-thought help show the process resembles expert notes
  6. 11:45 – 16:21

    Why math papers are a bad training set—and why reasoning can still emerge

    Lisha argues that papers and textbooks omit the struggle and motivation behind definitions, making them poor “how to think” datasets. Mark frames OpenAI’s work as training general-purpose reasoning behaviors (like backtracking and long-horizon planning) that then transfer into math.

    • Math writing often hides the real exploratory process and motivation
    • Question: how do “mathematician-like” traces arise from imperfect artifacts?
    • OpenAI trains general reasoning skills, not just domain-specific proof patterns
    • Releasing summarized reasoning traces helps dispel the “insane guessing” hypothesis
  7. 16:21 – 30:24

    Astra’s 10-problem set: deep dive on sphere packing and the LP bound breakthrough

    They unpack a favorite Astra result: understanding the linear programming (LP) upper bound for high-dimensional sphere packing, matching a conjectured asymptotic form. The key advance is both constructing the function achieving the bound and proving optimality within the LP framework.

    • Sphere packing is solved exactly only in a few special dimensions (1,2,3,8,24)
    • LP bounds reduce packing limits to constructing functions with Fourier-analytic constraints
    • Viazovska’s miracles in 8 and 24 motivate the LP method’s power and mystery
    • Astra derives a clean high-dimensional asymptotic for the LP bound and shows it’s tight
    • Result reframes and improves older bounds by identifying the best possible LP outcome
  8. 30:24 – 36:17

    Spherical and binary codes: symmetry, representation theory, and a linked discovery path

    Mehtaab explains error-correcting codes as geometry on hypercubes and spheres, closely related to packing. The model improves bounds using heavy symmetry and representation theory, and this line of work unexpectedly connects back to the sphere-packing asymptotics.

    • Binary codes as sets of separated vertices in a hypercube (Hamming distance)
    • Spherical codes as packings constrained to a sphere’s surface
    • Improved bounds use representation theory to exploit symmetry of sphere/cube
    • Interactive prompting mattered here: pushing the model further unlocked stronger theory
    • Taking limits connects code bounds back to sphere-packing constants and behavior
  9. 36:17 – 44:34

    Harness vs model: prompting, task-orientation, and what ‘taste’ means in practice

    They discuss how much results depend on the “harness” (prompting, interaction) versus raw model capability. “Taste” is treated pragmatically: better judgments that speed progress, plus the possibility of multi-agent setups where one model supervises another’s long work.

    • Sometimes the model stops after meeting the stated task; follow-up prompts can extend results
    • This can be harness/procedure rather than capability limitations
    • Taste as utilitarian judgment: picking promising directions that reduce time-to-solution
    • Multi-agent analogy: a ‘supervisor’ that steps back can unstick a ‘worker’ model
    • Collaboration (human-human or model-model) naturally improves search and perspective
  10. 44:34 – 54:45

    Non-sofic groups: approximating infinite groups by finite ones—and Astra’s counterexample

    Mehtaab introduces groups and the notion of soficity: being approximable by finite groups. Astra proves a landmark negative result—there exists a non-sofic group—contrasting with broader approximation conjectures and highlighting a proof that stays within group theory rather than heavy external machinery.

    • Sofic groups: countable groups approximable by finite groups
    • Motivation: finite-to-infinite approximation transfers desirable properties (e.g., surjunctivity)
    • Connection to Aldous–Lyons style approximation ideas via Cayley graphs
    • Context: a stronger conjecture was disproved using massive machinery; this proof is shorter
    • Key challenge: making ‘not approximable by any finite group’ concrete enough to prove
  11. 54:45 – 57:32

    Why the proof is impressive: concrete combinatorial obstructions and elegant short arguments

    They reflect on what makes the non-sofic result hard: the definition is slippery and global. Astra identifies the right obstruction and stitches together the needed literature and delicate combinatorics to rule out “conspiracies” that prior work couldn’t exclude.

    • Core difficulty: soficity is hard to “get your hands on” directly
    • Astra isolates a specific obstruction that blocks all finite approximations
    • Builds on existing results (e.g., by Kun and Tom) and adds a crucial final step
    • Observation: AI-generated proofs so far skew short/elegant, not sprawling 200-page verifications
    • Postmortems can generate further math; follow-up work by human authors emerged quickly
  12. 57:32 – 1:01:26

    How the math community should adopt AI: understanding, verification, and attribution pressure

    They discuss practical uptake: models can both generate more results and accelerate human understanding by summarizing papers and proof strategies. The community may face new bottlenecks in organizing, verifying, and communicating knowledge, alongside cultural questions about credit and workflow norms.

    • Models help absorb sophisticated math faster (e.g., summarize arXiv papers)
    • AI increases output volume, shifting bottlenecks to understanding and curation
    • Verification and attribution norms become more salient as AI assistance grows
    • Ideal outcome: mathematicians build on AI results rather than treating them as isolated curiosities
  13. 1:01:26 – 1:05:15

    Empirical vs theoretical math: a positive vision of faster applied math and broader participation

    They close with a forward-looking view: as routine parts become cheaper, human value may shift toward framing, explanation, and tackling deeper mysteries. Faster applied mathematics could benefit the world, and AI tools may let more non-specialists meaningfully engage with advanced math.

    • Greater value placed on explanation, shared frameworks, and communal understanding
    • Math may concentrate more on big, hard mysteries as smaller ones become routine
    • Even with rapid AI progress, some difficulty ceilings may remain (e.g., P vs NP)
    • Lower barriers: non-experts can learn and apply advanced math without scarce human gatekeepers
    • Upside: applied math and adjacent fields (e.g., physics) may accelerate substantially

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.