Terence Tao on Dwarkesh Patel: How Erdős Problems Exposed AI

1. 0:00 – 11:44

   Kepler was a high temperature LLM

   1. DPDwarkesh Patel0:00

      Okay. Today, I'm chatting with Terence Tao, who needs no introduction. Terence, I wanna begin by having you retell the story of how Kepler discovered the laws of planetary motion because I think this will be a great jumping off point to talk about AI for math.

   2. TTTerence Tao0:15

      Okay. Yeah. So I've always had an amateur interest in astronomy, and so I've, I've, I've loved stories of how the early astronomers worked out, um, the nature of the universe. Um, so, uh, Kepler was building on the work of Copernicus, um, who was himself building on the work of Aristarchus. Uh, so, uh, Copernicus very famously proposed the heliocentric model that, um, uh, instead of the planets and the Sun going around the Earth, that the Sun was at the center of the solar system and the other planets were, were going around, uh, the Sun. And Copernicus proposed that the orbits of the planets were perfect circles. And his theory kind of fit, uh, the observations that, um, the, the Greeks and the Arabs and the Indians had worked out over, over centuries. Um, I think, uh, Kepler got interested... Uh, like he learned about these, these theories, um, in his, in his studies and he made this observation that the ratios of the, uh, size of the orbits that Copernicus predicted seemed to have some geometric meaning. Um, I think, uh, uh, yeah, he, he started proposing that, uh, you know, if you, if you take, um, say the orbit of, of, um, say the Earth and you enclose it in I think maybe a cube, um, the, uh, the outer sphere of that, that encloses the cube almost matched perfectly the orbit of Mars and so forth. Um, and there were six planets known at the time, five gaps between them, and there were five perfect Platonic solids, uh, the cube, the tetrahedron, icosahedron, octahedron, and dodecahedron. And so he had this, this theory which he thought was absolutely beautiful, that he, he could inscribe these Platonic solids between the spheres of the planets and it seemed to fit and it, it, it seemed to be to him like, you know, God's design of the planets was, was matching this mathematical perfection of the Platonic solids. So he needed data to, um, confirm this theory. And at the time, there was only one really high-quality data set, um, [laughs] almost in existence, okay, which was the... So, uh, Tycho Brahe, this Danish astronomer, um, very wealthy, eccentric astronomer, had managed to convince the Danish government to fund this extremely expensive observatory, this, in fact, an entire island, um, where he had taken decades of observations of all the planets, Mars, Jupiter, um, every night, or at least every night for which the weather was clear, with the naked eye actually. This is, uh, he was the last of the, of the naked eye astronomers. And so he had all this data which Kepler could use to confirm his theory. And so Kepler started working with, with Tycho, but Tycho was very jealous of the data. He only gave him little bit, bits of it at a time. Um, and I think, uh, Ke-Kepler eventually just stole the data actually. He co- he copied it and, and, uh, had to have a fight with, with, uh, uh, Brahe's descendants. Um, but he did work out-- Uh, he did get the data, um, and then he worked out to kind of his disappointment that, um, his beautiful theory didn't quite work. Like the data was sort of off from his, um, Platonic solid theory by, you know, about 10% or something. And he tried all kinds of fudges, moving the circles around and things. And it, it, it didn't quite work. But he worked on this problem for, for, for years and years and eventually he figured out how to use the data to, to work out the actual orbits of, um, um, of the planets. Um, and that was an incredibly clever genius amount of data analysis, like actually. And, um, yeah, and then he eventually worked out that the, the, uh, uh, they also are actually ellipses, not circles, which was shocking to him. Uh, and then he worked out, um, uh, so he worked out the two laws of planetary, two laws of planetary motion, the ellipses also equal areas sweep out, uh, equal times. Um, and then 10 years later, yeah, after collecting a lot of data, the, the, the f- the, the furthest planets like, um, like Saturn and Jupiter were the hardest for him to, to work out, but then he, he finally worked out this third law also that, uh, um, uh, that the, uh, the orbits, the, the, the time it takes for a planet to complete its orbit was proportional to some power of, of the distance to the Sun. And these are the th- three famous Kepler's laws, laws of motion, um, and he had no explanation for them. It, it, uh, it, uh, it was just all driven by, by experiment and it took Newton a century late-later to give a theory that explained all three laws at once.

   3. DPDwarkesh Patel4:09

      The take I wanna try on you-

   4. TTTerence Tao4:11

      Mm-hmm

   5. DPDwarkesh Patel4:12

      ... is that Kepler was a high temperature LLM. [laughs]

   6. TTTerence Tao4:15

      [laughs]

   7. DPDwarkesh Patel4:16

      Where, uh, Newton comes up with this, uh, explanation of why the three laws of planetary motion must be true. And of course the way that Kepler discovers the laws of planetary motion or figures out the relative orbits of the different planets is, as you say, a work of genius. But then, you know, he's, through his career he's just trying random relationships and in fact that the, in the book in which he writes down the third law of planetary motion, it's sort of an aside on the harmonics of the world, which is his book about, you know, all these different planets have these different harmonies and the reason there's so much famine and misery on Earth is because the Earth is mi-fa-mi, that's the note of Earth.

   8. TTTerence Tao4:50

      Mm-hmm.

   9. DPDwarkesh Patel4:50

      And so all this random astrology, but in, in there is the cube square law which tells you what relationship the, uh, the period has to a planet's distance from the Sun, which is, as you were detailing, uh, if you add that to Newton's F equals ma and then the equation for centripetal acceleration, you get the inverse square law.

   10. TTTerence Tao5:11

       Mm-hmm.

   11. DPDwarkesh Patel5:11

       And so Newton works that out, but the reason I, um, I think this is an interesting story is I feel like LLMs can do the kind of thing of like 20 years, let's try random relationships, some of which make no sense as long as there's a verifiable data bank like Brahe's data set-

   12. TTTerence Tao5:25

       Mm-hmm

   13. DPDwarkesh Patel5:26

       ... where, okay, I'm gonna try out random things about like musical notes. I'm gonna try out random things about Platonic objects. I'm gonna... All these different geometries. I have this bias that there's some important thing about the geometry of these orbits and then one thing works and as long as you can verify it, it can then dri- these empirical regularities can d-then drive actual deep scientific progress.

   14. TTTerence Tao5:44

       Traditionally when we talk about the history of science, um, idea generation has always been kind of the prestige part of science. Um, so I mean, a scientific problem comes with, uh, there's many steps. You know, you have to identify a problem, uh, and then you have to identify a good problem to work on, a fruitful problem.And then you need to, to collect data, um, you need to figure out a strategy to analyze the data, to make a hypothesis, um, and at this, at this point, you need to propose a good hypothesis, and then you need to validate. Yeah, so this, and then you need to write things up and explain. There's this, there's that, a dozen different components. Um, but yeah, the, the ones we celebrate are these sort of eureka genius moments of, of, uh, idea generation. Um, and, um, yeah, so, so Kepler certainly had to, to, as you say, cycle through many ideas and, and, and several which didn't work and, and, and I, I bet many that he didn't even, um, publish at all, um, because, yeah, they just, they just didn't fit. And that's an important part of the process, um, trying all kinds of, of, of random things and seeing if they worked. Um, but as you say, the, um, you know, the, uh, it, it had to be matched by an equal amount of verification, otherwise it's, it's slop. Like, you know, I mean, um, we, we celebrate Kepler, but we should also celebrate Brahe for, for his, his, his assiduous data collection with which was ten times more precise than, than any previous observation. And it, it wa- um, that extra decimal point of accuracy was actually essential for, for Kepler to get, uh, his, um, um, his, his results. Um, and, you know, and he was using, you know, Euclidean geometry and, and, and like, like the most advanced mathematics he could, uh, use at, at the time to, to match his, his models with the data. So, like, all aspects had to be in play. Uh, you know, the, the, the data and the theory and the, uh, the, the hypothesis generation. I'm, I'm not sure nowadays that hypothesis generation is the bottleneck anymore.

   15. DPDwarkesh Patel7:32

       Hmm.

   16. TTTerence Tao7:32

       Um, science has, has changed in, in, in the century since. Um, so, um, classically sort of the, the two big paradigms for, for, for science were theory and experiment. Um, then in the, uh, twentieth century, uh, numerical simulation came along, and so you can also do, do computer simulations of, of, of, of, uh, to test theories. Um, but then finally in the late twentieth century, we had big data. Now we, we had the, the, the era of data analysis. Um, and so a lot of new progress is actually driven now by analyzing massive data sets first, collecting large data sets, and then drawing the patterns from them to, to deduce thoughts, which is a little bit different from how science used to work, where you, you make a few observations or you just have one out of the blue idea, and then you collect data to test your idea. That's the classic scientific method. Um, now it's almost reverse. You collect big data first, and then you, you, you try to, to get hypotheses from it. Um, I mean, Kepler was maybe one of the first early data scientists, but, but even, even he didn't start with Ty- uh, um, Tycho's data set and, and analyze it. He, he had, he had some preconceived theories first. But it, it seems like this is less and less the way we make progress in, in, in, um, uh, just because, uh, you know, the data is, is just so much more massive. It's just so much more useful. Um.

   17. DPDwarkesh Patel8:48

       Oh, interesting. I, I, I actually feel like the more the twentieth century science that you're describing is actually very well describes what happened with Kepler, where he did have these ideas, um, fifteen ninety-five and ninety-six is where he comes up with first polygons and then, uh, platonic objects theory, but they were wrong. And then a few years later, he gets Brahe's data, and it's only after twenty years of just trying random things that he gets this empirical regularity. And so it actually feels a bit closer to Brahe's data is analogous to some massive data bank of simulations, and then we, he, he now, he, now that you've got the data, you can keep trying random things. But if it wasn't, Kepler would be out there just writing books about harmonics and platonic objects, and there would be nothing to actually verify against.

   18. TTTerence Tao9:35

       Yeah. Yeah. Yeah. So the, uh, the, the, the data was extremely important, um, but the distinction I was trying to make was that sort of traditionally, you make a hypothesis, and then you test it against data.

   19. DPDwarkesh Patel9:45

       Yeah.

   20. TTTerence Tao9:46

       Um, but, um, now with, um, machine learning and data analysis and statistics and so on-

   21. DPDwarkesh Patel9:51

       Mm

   22. TTTerence Tao9:51

       ... you can, you can start with data and, um, through, say, statistics, work out, um, um, laws that, um, were not present before. So, and so Kepler, so Kepler's third law is a little bit like this, except that, uh, for the third law, instead of having the thousand data points that Brahe had, Kepler had like six data points. Um, like every planet, uh, you knew the length of the orbit, uh, and the, the distance to the sun, and there was like five or six data points, and he did, uh, what we would now call regression. You know-

   23. DPDwarkesh Patel10:18

       Yeah

   24. TTTerence Tao10:19

       ... he, he could fit a curve to these six data points, and he got a square cube law, which was amazing. Uh, but actually, he was quite lucky, I mean, that these six data points gave him the right conclusion. Um, you know, it's, uh, that's not enough data to be really reliable. Um, there was a later astronomer, uh, Johannes Bode, um, who took the same, the same data actually, um, the, the distances to, to the planets, uh, and inspired by Kepler, I think, he had a prediction that the, the, the, the distances to the planets formed basically a shifted geometric progression. Uh, he also fit a curve.

   25. DPDwarkesh Patel10:49

       Mm.

   26. TTTerence Tao10:50

       Um, except there was, there was one, there was one point missing. Uh, so there was a big gap between Mars and Jupiter. Uh, his law predicted that there was a missing planet. So, uh, it was a kind of a, a crank theory, except, um, when Uranus was discovered by Herschel, the, the distance to Uranus fit exactly this-

   27. DPDwarkesh Patel11:04

       Hmm

   28. TTTerence Tao11:05

       ... this pattern. Um, and then Ceres was discovered, uh, this asteroid between, um, um, I think in, in the asteroid belt, and it also fit the pattern. And so people got really excited that, that, that, that Bode had discovered this, this amazing new law, um, of, of, of nature. Um, but then Neptune was discovered, and it was, it was completely like way off.

   29. DPDwarkesh Patel11:22

       [chuckles]

   30. TTTerence Tao11:22

       Um, and, um, you know, and, and basically, it was just a numerical fluke. You know, there was-
2. 11:44 – 26:10

   How would we know if there’s a new unifying concept within heaps of AI slop?

   1. DPDwarkesh Patel11:44

      But maybe to ask the question about the analogy more explicitly, does this analogy make sense to if we have, you know, in the future, we'll have smarter and smarter AIs, and we'll have millions of them, and then they can go out and hunt for all these empirical regularities. It sounds like you don't think the bottleneck in science isFinding more things that are for each given field, their equivalent of the third law of planetary motion, so that then later on somebody can say, "Oh, we need a way to explain this. Let's work out the math. Here, here's the inverse, uh, square law of gravity."

   2. TTTerence Tao12:17

      Right. So I think AI has basically driven the cost of idea generation down to almost zero.

   3. DPDwarkesh Patel12:22

      Yeah.

   4. TTTerence Tao12:23

      Um, in a very similar way to how the internet d- uh, drove the cost of communication down to almost zero.

   5. DPDwarkesh Patel12:27

      Yeah.

   6. TTTerence Tao12:27

      Um, which is an amazing thing, but it, you know, it, it, it doesn't make-- it doesn't create abundance by itself. Um, yeah, so now the bottleneck is, is, is different. Um, so we're now in a situation where suddenly people can generate thousands of theories, uh, for a, a, a, a, a given scientific problem, and now we have to, to verify them, evaluate them. Um, and this is something which we, we have to, to change our structures of science to actually sort this out. So, you know, in fact, traditionally, we, we build walls, you know. So in, in the past, you know, before we had, uh, AI slop, you know, we, we had sort of amateur scientists, you know, create-

   7. DPDwarkesh Patel13:00

      Mm-hmm

   8. TTTerence Tao13:00

      ... you know, have their own theories of the universe, uh, many of which, uh, were basically of very little value.

   9. DPDwarkesh Patel13:06

      Yeah.

   10. TTTerence Tao13:06

       Um, and so we built these, like, you know, peer review publication systems and things to kind of filter out, um, and try to, to isolate the high signal, um, ideas to, uh, to test. Um, but, uh, but now that we can generate these, these, these, uh, these possible explanations at massive scale, um, and some of them are good and a lot are terrible, um, I mean, human reviewers, we just, it's just, um, they're already being overwhelmed, actually. I mean, many, many journals are reporting AI-generated submissions are just, are just, are just flooding-

   11. DPDwarkesh Patel13:36

       Mm

   12. TTTerence Tao13:36

       ... their, their submissions. So it's great that we can generate all kinds of things now with AI, but it, it means that we have to, the rest of, the rest of the aspects of science have to catch up. Uh, yeah, so verification, validation, um, and, and assessing, uh, what ideas actually move the subject forward and, and what, which ones are dead ends or, or, or, or red herrings. Um, and that's, that's not something we're, we've, we know how to do at scale. Um, you know, for each individual paper, we can discuss it with, you know, have a debate among scientists and get a consensus in a few years. But when we're generating, you know, a thousand of these every day, it, you know, this doesn't work. Yeah.

   13. DPDwarkesh Patel14:13

       Uh, so I think there is this incredibly interesting question of if you have billions of AI scientists-

   14. TTTerence Tao14:17

       Mm-hmm

   15. DPDwarkesh Patel14:18

       ... not only how do you gauge which ones are real progress, but how do you-- I mean, this is actually a question that human scientists had to face, and we've solved somehow, and I'm w- I actually am not sure how we solved this. But in any given field, let's say in the 1940s, and there, there's, if you're at Bell Labs or if you're just generally trying to, there's these new technologies coming out, uh, uh, pulse code modulation, basically how do you transfer signals? How do you digitize signals? How do you transfer them over analog wires? And then, but there's, like, all these papers about the engineering constraints there and the details, and then there's one which is like, comes up with the idea of the bit-

   16. TTTerence Tao14:49

       Mm-hmm

   17. DPDwarkesh Patel14:50

       ... which has implications across many different fields. And you need some system which can then look at that and say, "Okay, we need to apply this to probability. We need to apply this to computer science," et cetera. Um, and, uh, for, in the future, the AIs are coming up with, you know, the next version of this kind of unifying concept, and how would you identify it among millions of papers which might actually constitute progress, but which have much less general-

   18. TTTerence Tao15:12

       Right

   19. DPDwarkesh Patel15:13

       ... unifying ideas?

   20. TTTerence Tao15:13

       So a lot of it's the test of time. Um, so, so many great ideas didn't actually get a great reception at the time that, uh, they were first proposed. It was only after some other scientists realized that, that, uh, they could take it further and apply them to their own, um... You know, deep learning itself, uh, was like a, a niche area of AI for a long time, that this, the idea of, of getting answers entirely through training on data and, and not through first principles, you know, reasoning was, was, was very controversial, and it, it took a long time before it actually started, uh, bearing fruit. You know, you mentioned the bit. You know, I mean, there are, there were other proposals for computer architectures than the zero one that is universal today. I think there were, there were trits, you know, zero one, uh, uh, three-valued logic and, you know, in an alternate universe, maybe a different paradigm would have, would have showed up. Um, people have argued that, you know, the transformer, for example, is, is the foundation of all modern large language models. And it was the first, um, deep learning architecture that really was, was sophisticated enough to capture language. But it didn't have to be that way. There, there could have been some other architecture that, um, uh, was the first to do it. And once that was adopted, uh, it would become the standard. Um, so I think a, um, one reason why, uh, it's hard to assess whether a, a given idea is gonna be fruitful is that it, it, it depends on the future. It, it depends on, and it, it just, it depends on, on, also on the culture and society, like, like which ones get ado-adopted, which ones don't. Um, you know, um, the base ten, um, uh, numeral system in, in mathematics, extremely useful, uh, much better than the Roman numeral system, for instance. Um, but again, there's nothing special about ten. Uh, it, it's, it's a system that we, it's useful for us because everyone else uses it. Um, and we've standardized it, and we've built all our, our computers and our, and our, our number, uh, representation systems around it. Um, and so we're stuck with it now, actually. Um, you know, people are, some people occasionally push for other systems than decimal, but, um, it's, there's, there's just no, there's just no, uh, there's too much inertia. Um, so you, you can't look at any given scientific achievement purely in isolation and give it an objective grade, uh, uh, without being aware of the context both in the, the past and the future. And so it, it, it may never be something that you can just reinforcement learn the same way that, uh, that, that you can for much sort of more localized, um, problems. Um...

   21. DPDwarkesh Patel17:31

       Mm. Yeah. It seems often in the history of science when what, when a new theory comes up that in retrospect we realize is correct, it seems to make implications that just either make no sense because they're wrong-

   22. TTTerence Tao17:45

       Mm

   23. DPDwarkesh Patel17:45

       ... and we realize later on why they're wrong, or they're correct, but seem, seem wildly implausible at the time. So y- as you talked about, Aristarchus, uh, had heliocentrism in the third century, uh, BC, and then, um, the ancient Athenians were like, "This can't be because it would, if the Earth is going around the sun, we should see the relative position of the stars change as we're going around the sun. And the only way that wouldn't be the case is if they're so far away that, um, that you don't notice any parallax," which is actually the correct implication. But there's times when actually the implication is incorrect, and we just need to graduate to a better level of understanding. So Leibniz wouldyou know, chide Newton and disagree with Newton's theory of gravity on the basis that it implied action at a distance.

   24. TTTerence Tao18:26

       Mm-hmm.

   25. DPDwarkesh Patel18:27

       Um, and then there's we don't know the mechanism.

   26. TTTerence Tao18:29

       Mm-hmm.

   27. DPDwarkesh Patel18:30

       And, um, and Newton himself was sort of stunned that inertial mass and gravitational mass were the same quantity. So all these things w- they were eventually resolved by Einstein.

   28. TTTerence Tao18:36

       Yes. Yes.

   29. DPDwarkesh Patel18:37

       But it was still progress. And so the question for an system of peer review for AI would be, even if you can falsify a theory-

   30. TTTerence Tao18:45

       Mm.
3. 26:10 – 30:31

   The deductive overhang

   1. DPDwarkesh Patel26:10

      One takeaway I had from, uh, reading and watching your stuff on The Cosmic Distance Ladder, by the way, I, I highly, highly, highly recommend people watch your series with Thru the Wormrun on The Cosmic Distance Ladder. But, um, one takeaway was that the deductive overhang in many fields could be so much bigger than people realize, where if, if you just had the right insight about how to study a problem, you might be surprised at how much more you could learn about the world. And I wonder if you think that's sort of a product of astronomy at the particular times in history that you're studying, or is this that based on the data that is incident on the Earth right now, we could actually divine a lot more than we happen to know?

   2. TTTerence Tao26:52

      Right. So astronomy was one of the first sciences to really embrace data analysis and, and, and squeezing every last possible drop of information out of the information they had because, because data was the bottleneck. Um, I mean, it still is the bottleneck. I mean, it's, it's really hard to, to collect astronomical data. So astronomers are the best or, uh, you know, um, almost, uh, uh, world-class in, in extracting, you know, almost like Sherlock, you know. So they like extracting all kinds of conclusions from little traces of data. Um, I hear that, uh, that, uh, a lot of quant, uh, hedge funds, uh, they, their, their preferred hire is an astronomy PhD.

   3. DPDwarkesh Patel27:28

      Oh, interesting.

   4. TTTerence Tao27:28

      That, uh, they, they also are very interested for other reasons in extracting signals-

   5. DPDwarkesh Patel27:32

      Yeah

   6. TTTerence Tao27:32

      ... from, from, from various random bits of data.

   7. DPDwarkesh Patel27:35

      Okay. Speaking of clever ideas, one of my listeners, Shawn, solved the puzzle that Jane Street made for my audience and posted a great walkthrough on X. For context, Jane Street trained a ResNet and then shuffled all 96 layers and then challenged people to put them back in the right order using only the model's outputs and training data. You can't brute force this. There's more possible orderings than atoms in the universe. So Shawn broke the problem into two different parts. First, pair the layers into 48 different blocks, and second, put those blocks in the right order. For pairing, Shawn realized that in a well-trained ResNet, the product of two weight matrices in a residual block should have a distinctive negative diagonal pattern. And this arises as a way for the model to keep the residual stream from growing out of control. From this insight, he was able to recover the right pairings. For ordering, Shawn noticed that the model seemed to improve if he sorted the blocks by the size of their residual contributions. Starting with that rough approximation, he combined a clever ranking heuristic with local swaps to recover the exact right order. His full walkthrough is linked in the description. Don't worry if you didn't get to this puzzle in time, though. There's still one up about backdoor LLMs that even Jane Street doesn't know how to solve. You can find it at janestreet.com/dwarkesh. All right, back to Terence.

   8. TTTerence Tao28:51

      We, we, we do underexplore sort of, um, how to extract extra information from, from various signals. Um, like, um, um, I, I just to pick, to, to pick one random study. I, I remember reading once that, that, uh, people had discovered or trying to, to measure how often, um, scientists actually read the citations, um, that the papers that they cite. Uh, so how, how do you measure this? Okay, you, you, um, you, you could try to survey, um, um, uh, different scientists, but they, they had some clever, um, uh, trick. So, so, um, so many citations, uh, have little typos, like, like, uh, like, you know, um, a number is, is wrong or, or punctuation was wrong. And they, they measured how often a, a, a typo got copied from one reference to, to the next, and, and they could infer whether a-an author was actually just copying it, cutting and pasting a reference without actually checking it. Um, and so from that, they, they were able to infer some, some measure of, of sort of, uh, how much attention people were paying.

   9. DPDwarkesh Patel29:46

      Right.

   10. TTTerence Tao29:46

       So there are also clever tricks to extract, um, you know, so these questions you, you posed earlier of, you know, how can we assess whether, um, a, a, a, a scientific development is fruitful or, uh, or interesting or, or represents real progress. You know, maybe there are, um, really useful metrics and, or, or, um, footprints of this, of this, of this, um, of, of this phenomenon in, in a data, data set. You know, we can, we can examine citations and, um, and like how often something is mentioned in a conference or something. And, and maybe there, there's, there's, there's a lot of, uh, social, sociology of science research to be, to be done and, and that could actually, um, detect these things. Um, yeah, maybe we usually get some astronomers on the case actually.

   11. DPDwarkesh Patel30:28

       [laughs] Um,

4. 30:31 – 46:43

   Selection bias in reported AI discoveries

   1. DPDwarkesh Patel30:31

      okay. Uh, uh, so I, I think this brings us, uh, nicely toThe progress that from the outside it seems like AI for math is making.

   2. TTTerence Tao30:39

      Mm-hmm.

   3. DPDwarkesh Patel30:39

      And I think you had a post recently where you pointed out that over the last few months, AI programs have solved fifty out of the eleven hundred odd Erdős problems. But then I think, I don't know if it's still correct, but as of a month ago, you said that there had been a pause because the low-hanging fruit had been picked. First of all, I'm, I'm curious if actually that is still the case, that we have picked the low-hanging fruit, and now we're at, now we're at this plateau currently.

   4. TTTerence Tao31:02

      It, it does seem so. I mean, there's still activity at the Erdős... Yeah, so, so fifty-odd problems have been solved with AI assistance, which is great, but there's like six hundred to go.

   5. DPDwarkesh Patel31:10

      Right.

   6. TTTerence Tao31:10

      Um, and people are still chipping away at, at one or two of these right now. Um, um, we're seeing a lot fewer sort of pure AI solutions now, where, um, the, the AI just one-shots the problem. Um, so, so there, there was a month where that happened and, and that has stopped. Um, not for lack of trying. I know of three separate, uh, uh, attempts to get frontier model AIs to just at- at- attack every single one of the problems simultaneously.

   7. DPDwarkesh Patel31:35

      Right.

   8. TTTerence Tao31:35

      Um, and they pick out some minor observations or, or maybe they, they, they found that some problem was al- already solved in the literature, but there hasn't been any further AI purely powered solution yet. Um, people are using AI a lot, um, uh, currently. Yeah, so someone might use AI to generate a, a possible, um, proof strategy, and then another, uh, person will use an, a separate AI tool to critique it, um, uh, or rewrite it, uh, or generate some numerical data for it, or do a literature survey. Um, and, and some problems have been solved by a, a ongoing conversation between lots of humans and lots of AI tools. Um, but, uh, it, it, it does seem like it, it, it was this, this one-off thing. Um, so maybe one analogy to, to, for, for these problems is like, um, imagine like, um, there's, there's, there's all these-- that you're in some sort of mountain range with all kinds of, of cliffs and walls. And, and, uh, maybe there's a, there's a, there's a little, um, uh, wall which is maybe like three feet high and one that's, uh, six feet high and then there's fifteen feet high, and then there's, there's, uh, there's some mile high cliffs. Um, and you're trying to climb as many of these cliffs as possible, but it's in the dark. Uh, we don't know which ones, uh, are tall, which, which ones are short. And, um, so, you know, we try to light some candles and make some maps and, and slowly we, we kind of figure out, uh, some of them are, are, are climbable. Some of them we can identify some, some partial, um, um, track in the wall that you can reach first. Um, and then these, these AI tools, they're kind of like these jumping machines that can kind of jump, you know, two meters in the air, you know, higher than any human. And sometimes they jump in the wrong direction, and sometimes they, they crash, but sometimes they, they, they can reach, um, um, the tops of, of, of, uh, the lowest, um, you know, um, uh, walls that we, we couldn't reach before. And so we've just basically set them loose in this mountain range, hopping around and, you know, and then there was this exciting period where they, they could actually find all the-

   9. DPDwarkesh Patel33:26

      [laughs]

   10. TTTerence Tao33:26

       ... um, all the low ones, um, and they, they could reach them. Um, but then, uh, there's been no, uh, you know, I mean, maybe if the next time there's a big advance in the models, then they will try it again, and maybe a, a few more will be, will be, uh, will be breached. Um, but it, it, it's a different style of doing ma- mathematics than, um, sort of the... You know, so normally we would hill climb and, you know, we would, uh, we would make little m- markers and, and, and try to identify partial things and, um, you know. The- these tools, they either succeed or they fail. Um, and they, they've been really bad at creating sort of partial progress or identifying intermediate, um, um-

   11. DPDwarkesh Patel34:05

       Hmm

   12. TTTerence Tao34:06

       ... stages that you should, you should focus on first. Um, a- a- again, going back to, to this, this previous discussion, you know, we don't have a way of evaluating partial progress.

   13. DPDwarkesh Patel34:13

       Yeah.

   14. TTTerence Tao34:13

       Um, the sa- the same way, way you could, you can evaluate a one-shot succ- success or failure of solving a problem.

   15. DPDwarkesh Patel34:19

       So there's two different ways to, um, think through what you've just said, and one of them is m- more bearish on AI progress, and one of them is more bullish. And the bearish one being, oh, they're only getting to a certain height of wall, which is not as high as humans are reaching. Um, and the second is that, well, they have this powerful property that o- once they achieve a certain waterline, they can fill every single problem that is available at that waterline, which we simply can't do with humans, where we can't make a million copies of you and, uh, give each of them a million dollars of inference compute and have you do a hundred years of subjective time research on, um, a hundred different problems at the same time or a million different problems at the same time. But once AIs reach Terence Tao level, they could do that.

   16. TTTerence Tao35:01

       [laughs]

   17. DPDwarkesh Patel35:01

       Um, and then once they reach intermediate levels, they could do, they could do the intermediate version of that. So the same reason that we should be bearish now is the reason we should be especially bullish, not even when they achieve superhuman intelligence, but just when they achieve human level intelligence, because their human level intelligence is qualitatively wider and more powerful than-

   18. TTTerence Tao35:20

       Yeah

   19. DPDwarkesh Patel35:20

       ... our human level intelligence.

   20. TTTerence Tao35:21

       I, I, I agree. Yeah, so they excel at breadth, uh, and humans excel at depth, um, like human experts at least. Yeah, so, um, I think they're very complementary. Um, but our current, uh, way of doing math and science is focused on depth because that, that's where the human, uh, expertise is, because humans can't do breadth. Um, but, uh, yeah, so we, we have to redesign, uh, the way we do science to take full advantage of, um, of this breadth capability that we now have. Um, so as I said, we do, uh, we should have a lot more effort in creating very broad classes of problems to work on rather than, than one or two, um, really, um-

   21. DPDwarkesh Patel35:57

       Mm

   22. TTTerence Tao35:58

       ... deep important problems. I mean, we should still have the deep important problems, um, and humans should still be working on them. Um, but, but n- now, now we, we have this other way of, of, of, of doing, um, of doing science. You know, I mean, uh, we can explore entire new fields of science by, by first getting the, these broad, um, moderately competent AIs to sort of map it out and clear out all the, the easy, make all the easy observations, okay, and then identify certain islands of difficulty, uh, which, you know, then human experts can come and, and, and, and work on. So I, I s- I see very much, uh, a future of very complementary, um, um, science. Eventually, you would hope to get both breadth and depth, you know, and, and, umSomehow get the best of both worlds.

   23. DPDwarkesh Patel36:41

       Mm.

   24. TTTerence Tao36:41

       Um, but I think we, we need practice with the breadth side. It gets so-- It's too new. Uh, we don't even have the paradigms really to, to, um, to make full advantage of it. But we will, um, and then science will be unrecognizable after that, I think.

   25. DPDwarkesh Patel36:56

       Mm. To, to this point about complementarity, the, uh, programmers have noticed that they're way more productive-

   26. TTTerence Tao37:02

       Mm-hmm

   27. DPDwarkesh Patel37:02

       ... as a result of these AI tools. And, um, I don't know if you as a mathematician feel the same way, but it does seem like one big difference between vibe coding and vibe researching is that with software, the whole point of the thing is to have some effect on the world through your work, and if it leads to you better understanding a problem or you coming up with some clean abstraction to embody in your code, that is instrumental to the end goal. Whereas maybe with research, you-- the reason we care about solving the millennium price problems is presumably that in the process of solving them, our, our... we discover new mathematical objects or better, be- new techniques and those who understand our civilization's understanding of mathematics. And so the proof is sort of instrumental to the int-intermediate, uh, work. I don't know if you agree with that dichotomy or if that in any way will explain the relative uplift we'll see in software versus research.

   28. TTTerence Tao37:59

       Right. Um, yeah, so, so certainly in, in math, the process is, is often more important than the problem itself. Um, the problem is kind of a proxy for, for measuring your progress. And I think even in software, there's, there's different types of software tasks. I mean, the, you know, like if you're just trying to create a webpage that does the same thing that a thousand other web pages do, um, there's, there's sort of no skill to be learned. Well, um, um, there's, there's still some skill maybe that the individual programmer could pick up. Um, but, you know, for, for kind of boilerplate type code, definitely, um, um, you know, it, it's, it's, it's something that you should definitely offload, offload to AI. Um, um, but, you know, sometimes once you make the code, you know, you still have to maintain it and, and, and, and there's issues with upgrading it and making it compatible with other things and, and that, um, I think, um, I've, I've heard that programmers are reporting, you know, that even if, if, if an AI can create the first prototype of, of a, um, of a tool, making it mesh with everything else and, and making it interact with the real world in the way they want, I mean, it's, that's an ongoing process and if you didn't have the, um, the skills of, uh, that you pick up from, from, um, um, from writing the code, um, that, that may, that may, may impact, uh, your ability to ma-maintain it down the road. Um, so yeah, certainly, um, mathematicians, we, you know, we've, we've used problems to build intuition and, and to, to train people to, to have a, a good idea as, as what's true, what, what to expect, what is, what is provable, what is, what is difficult. Um, and so yeah, just getting the answers right away may actually, yeah, inhibit that process. Um, I mean, so as, um, I made this distinction between theory and experiment before. Um, so, um, in most sciences, there's an equal division between there's a theoretical side and experimental side. Um, but in math has been almost unique is that it's almost entirely theoretical. Uh, we, we, we, um, we place a premium on sort of trying to, to, to have coherent, clean theories of, of, of why things are true and, and false. And we haven't done much experiments as to like, you know, maybe we have two different ways to solve a problem. W-which one is, is more effective? Um, we have, we have some intuition, but we haven't done large scale studies where we take a thousand problems and we, and we, we just test them. Um, but we can do that now. So I think AI type tools, we really will, will actually revolutionize the ex- the experimental side of math-

   29. DPDwarkesh Patel40:19

       Mm

   30. TTTerence Tao40:19

       ... where, where, um, you don't care so much about, uh, individual problems and, and the process of solving them. But yeah, you, you, you wanna gather just large scale data about, about what things work and what things don't. Um, you know, same way that if, if you want to, to, if, if you're a software company and, and you want to, to roll out a thousand pieces of software, you know, you don't really wanna handcraft each one and learn lessons from each. You just wanna find what are the workflows that let you scale. Um, so we, we don't yet... We, we, we... The, the idea of doing mathematics at scale is at its infancy, um, but that's where AI is really gonna revolutionize the subject.
5. 46:43 – 53:00

   AI makes papers richer and broader, but not deeper

   1. TTTerence Tao46:43

      Yeah.

   2. DPDwarkesh Patel46:43

      Okay. So speaking of 2026 AI, you made a prediction in 2023-

   3. TTTerence Tao46:47

      Mm-hmm

   4. DPDwarkesh Patel46:47

      ... that I think by 2026, what was it? That it would, it would be like, like a colleague in mathematics or?

   5. TTTerence Tao46:52

      Yeah. A, a trustworthy coauthor if used correctly. Um-

   6. DPDwarkesh Patel46:55

      Got it.

   7. TTTerence Tao46:55

      Yeah.

   8. DPDwarkesh Patel46:55

      Which is looking pretty good in retrospect.

   9. TTTerence Tao46:57

      Yeah. I'm, I'm, I'm pretty pleased. Yeah.

   10. DPDwarkesh Patel46:58

       Um, so, you know, let, let, let's see if you can continue this streak. Um, you personally are 2X more productive as a result of AI. What year would you say that?

   11. TTTerence Tao47:10

       Um, yeah, so productivity I think is not quite a one-dimensional, um, quantity. Um, like I'm definitely noticing that the style in which I do mathematics is changing quite a bit-

   12. DPDwarkesh Patel47:21

       Mm

   13. TTTerence Tao47:21

       ... and the type of things I do. So for example, my, my papers now have a lot more code, a lot more pictures, um, um, uh, I, I, because it's so easy to, to generate these things now. So some plot which would have taken me hours to do now I can, I can do in minutes. But in the past, I just wouldn't have put the plot in-

   14. DPDwarkesh Patel47:38

       Mm

   15. TTTerence Tao47:38

       ... my paper in the first place. I would, I would just talk about it in words. Um, so it's h- hard to measure, to measure what 2X means. Um, so yeah, on the one hand, you know, the, I, I think the type of papers that I would write today, if I had to do them without AI assistance, they would definitely take five times longer, but-

   16. DPDwarkesh Patel47:55

       Interesting

   17. TTTerence Tao47:56

       ... but I would not write my papers that way.

   18. DPDwarkesh Patel47:58

       5X? So-

   19. TTTerence Tao48:00

       Yeah

   20. DPDwarkesh Patel48:00

       ... that, that's-

   21. TTTerence Tao48:01

       But, but it's, it, it's because, but the, the, these are sort of, uh, auxiliary ti- I mean, uh, you know, the, um, you know, so, so things, yeah, things like, like, um, like doing a much deeper re- literature search, um, uh, s- supplying a lot more numerics.

   22. DPDwarkesh Patel48:13

       Yeah.

   23. TTTerence Tao48:13

       Um, I mean, they, they, they, they, they enrich the paper. Um, so yeah, the, the, the, um, the core of what I do, like actually solving, um, the most difficult part of a, of a math problem, that hasn't changed too much. I still use pen and paper for that. But, um, you know, um, yeah, there's lots, there's lots of silly things. I, I, I use, um, an, an, an AI agent now to, to reformat, like s- sometimes, um, all my parentheses are not quite the right size. You know, I used to manually change them by hand, and now I can get an AI agent to sort of do all that quite nicely now in the background. Um-So yeah, they, they've, they really sped up lots of secondary tasks. Uh, they haven't yet sort of, uh, um, sped up the, the core thing that I do. But it, it's allowed me to sort of add more things to, to, to my papers. Um, yeah. But, um, by the same token, like if I were to write a paper I wrote in twenty twenty again and not add all these extra features but just have something of the same sort of level of functionality, yeah, then it actually doesn't have, hasn't saved that, that much, uh, to be honest. Uh, yeah, so it's, it's made, made the papers sort of richer and broader, but not necessarily deeper.

   24. DPDwarkesh Patel49:19

       Hmm. You made this distinction between artificial cleverness and artificial intelligence.

   25. TTTerence Tao49:25

       Mm-hmm.

   26. DPDwarkesh Patel49:26

       And I would like to better understand those concepts. W- uh, what is an example of, um, uh, i- intelligence that is not just cleverness?

   27. TTTerence Tao49:36

       Yeah. So, um, it's-- Intelligence is famously hard to define. It's one of these things that you s- you kind of know it when you see it. Um, but when I, when I, when I talk to someone, um, and we're trying to s-collaboratively solve a math problem together, um, there's this conversation where, you know, we-- neither of us knows how to solve the problem, um, initially, but, um, one of us has some idea and, and it looks promising. And, and so then, then we have some sort of prototype strategy, and then we test it, and then it doesn't work, but then we, we, we modify it, and there's some ada-adaptivity and, and, um, and, and, uh, continual improvement of, of, of the idea over time. And eventually, um, you know, w-we sort of-- we've, we've sort of like mapped out what doesn't work, what does work, and, and, and we can kind of see a path forward, but it's evolving with our discussion. Um, and this isn't not quite what the AIs do. The AIs can kind of mimic this a little bit. So to go back to this analogy of, of these jumping robots, uh, you know, so, um, you know, they can jump and fail and jump and fail and, and jump and fail, but, but what they can't do is they kind of, uh, they jump a little bit and they, they, they reach some handhold with... and they, um, but then they sort of stay there, and then they pull other people up, and then, uh, they try to just jump from there.

   28. DPDwarkesh Patel50:55

       Right.

   29. TTTerence Tao50:55

       Um, there, there isn't this cumulative process which is, uh, sort of built up interactively. Um, it, it, it seems to be a lot more trial and error and just repetition-

   30. DPDwarkesh Patel51:06

       Mm
6. 53:00 – 59:20

   If AI solves a problem, can humans get understanding out of it?

   1. DPDwarkesh Patel53:00

      One big question I have is how plausible is it that if we just keep training AIs, they get better and better at, you know, solving problems in lean, that they will continue to solve more and more impressive problems, and then we will in retrospect be surprised at how little insight we got from some lean solution to proving the Riemann hypothesis or something? Or do you think it is a necessary condition of solving the Riemann hypothesis, even by an AI that is like totally doing it in lean, that the constructions which are made, the definitions which are created, even in the, the lean program, have to advance our understanding of mathematics? Or do you think it could just be assembly code gobbledygook?

   2. TTTerence Tao53:38

      Uh, yeah, we don't know. I mean, some problems have been basically solved by pure brute force. The four color theorem is, is a famous example.

   3. DPDwarkesh Patel53:46

      Yeah.

   4. TTTerence Tao53:46

      Um, we have still not found a conceptually elegant proof of this theorem. Um, it, it basically... And, and maybe we never will. I mean, some, some problems may only be solvable by just splitting into some enormous number of cases and, and doing brute force uninsightful computer analysis on, on each case. I mean, pa-part of the reason we, we prize problems like the Riemann hy-hypothesis is that we're pretty sure that, that something amazing has to, uh, a new type of mathematics has to be created or a new connection between two previously unconnected areas of mathematics has to be discovered to, to make this work. We, we don't even know what the shape of the solution is, but it doesn't feel like a problem that will be solved just by exhaustively checking cases or something. Um, I mean, it could be false actually. So we, we could actually, uh, okay, there is an unlikely scenario that, that, that the hypothesis is false, and there's just this, this, this... You can just compute, "Oh, here's a zero off, off the line," and a massive computer cal-cal-calculation verifies it. That would be very disappointing. Um-I don't know. I, I, I, I do feel that, you know, fully autonomous one-shot approaches are not the right approach for these problems. I mean, I, I think you, you will get a lot more mileage out of the interplay between, between humans collaborating with these tools. Um, and, uh, I can see one of these problems being solved by, by some, uh, smart humans as- assisted by some extremely powerful AI tools. But the exact dynamic may be very different from what we envisioned right now. I mean, it, it, it could be a collabora- collaboration of a type that we just-- doesn't exist yet. Um, yeah, I mean, we-- There may be a way to, to generate, you know, a million variants of the human's data function and do some data analysis, AI-assisted data analysis, and we, we, we discover some pattern between connecting them which, which we didn't know about before, and w- and this lets you transform the problem into, into a different area of mathematics. I mean, there could be all kinds of, of scenarios.

   5. DPDwarkesh Patel55:50

      So suppose the AI figures it out, and latent in the Lean-

   6. TTTerence Tao55:56

      Mm.

   7. DPDwarkesh Patel55:56

      -is some brand-new construction which, you know, if you realize the significance, would-

   8. TTTerence Tao56:02

      Mm.

   9. DPDwarkesh Patel56:02

      -we would be able to apply it in all of these different situations. How, how would we even recognize it, right? Like, if, um, if you just... A-again, a v-very naive question, but you-- i-if you, if you come up with the equivalent of, like, Descartes r-comes with this idea, oh, you can have this coordinate system where you can unify algebra and geometry.

   10. TTTerence Tao56:17

       Mm.

   11. DPDwarkesh Patel56:17

       But in Lean code, it would just look like R to R, and it-

   12. TTTerence Tao56:20

       Right

   13. DPDwarkesh Patel56:20

       ... wouldn't look that significant or something. Or similarly, I'm sure there's other constructions which have th-this kind of property.

   14. TTTerence Tao56:25

       Well, the, the beauty of formalizing a proof in something like Lean is that s- you can take any piece of it and study it a-atomically. Um, so, um, you know, so when I read a paper with my humans, uh, with, uh, which solves some, some difficult problem, you know, there's often some big sequence of lemmas and theorems and things. Um, and so ideally, the author will talk us, talk their way through, you know, what's important, what's not. But, but sometimes th-they don't reveal what, what, um, what steps were the important ones and which ones are just kind of boilerplate, um, standard, um, steps. But you can study each lemma in isolation, and some of them I can say, "Oh, th-this looks fairly standard. This, this, this resembles something I'm, I'm familiar with. I'm pretty sure, um, there's nothing interesting going on here. But this lemma, oh, that's, that's something I haven't seen before, and I could see why if you could, if you had this result, that would really help prove the main result." Like, you could, you know, you can assess whether something's, uh, uh, um, are really sort of key to your, um, uh, to, to your argument or not. And Lean really facilitates that. You know, you can, you can, you can-

   15. DPDwarkesh Patel57:25

       Mm

   16. TTTerence Tao57:25

       ... you know, the, the individual steps are, are identified really precisely. Um, I think in the future, there'll be, um, you know, there'll, there'll be entire professions of, of mathematicians who might take a giant, um, Lean-generated proof and maybe, you know, do some ablation on it or something, try to remove steps or p-parts of it-

   17. DPDwarkesh Patel57:41

       Mm

   18. TTTerence Tao57:41

       ... and, um, and try to find, find more elegant ways, you know, um, you know, maybe get some other AIs to sort of do some reinforcement learning. H-how can you make the proof more elegant? And, and, and, uh, um, maybe other AIs will grade whether this, this proof looks better or not. Um, um, one thing that will change quite a bit, uh, in, in the near future is, is that un-until recently, writing papers was, uh, the most time-consuming and expensive part, um, of, uh, of the job. And so you did, you did it very rarely. You know, you, you, you, you only wrote up your results once everything was, all the other parts of your argument were, um, were checked out and, and things, 'cause y- just rewriting it again, refactoring was just a total pain. But that's one thing that's become a lot easier now with modern AI tools. So, you know, you don't have to have just one version of, of your paper. You, you know, you can, once you have one, you know, people can generate hundreds more. Um, so yeah, one giant messy Lean proof may not be very, uh, uh, meaningful or, um, understandable on its own, but, but p- other people can, can, can refactor it and do all kinds of, of, of, of things with it. Um, we have seen if, with the Erdős problem website, you know, that people will, will... An AI will, will generate a proof, and then here's 3,000 lines of code that, that verify the proof. But then we-- people got other AIs to ex-ex-summarize the proof and, and, and people write their own proofs. Um, there's actually, um, um, post-processing. Once you actually have one proof, um, we, we actually have a lot of tools now to, to deconstruct it and, and interpret it. It's a very nascent area of, of, of, of science or, or mathematics, but, um, I'm not as worried about, um... You know, so, so, so some people are concerned, "What if the Riemann hypothesis proof was a completely incomprehensible proof?" I, I think once you have the artifact of a proof, we can do a lot of, of, of, of, of analysis on it.

   19. DPDwarkesh Patel59:19

       Mm.

7. 59:20 – 1:09:48

   We need a semi-formal language for the way that scientists actually talk to each other

   1. DPDwarkesh Patel59:20

      You posted recently that it would be helpful to have a formal or semi-formal language for mathematical strategies as opposed to just mathematical proofs, which is what Lean specializes in. I would love to learn more about what that would involve or look like.

   2. TTTerence Tao59:34

      Um, we don't really know. Um, I mean, uh, we've been very lucky in mathematics that, uh, that we have worked out the laws of, of logic and mathematics, but this is actually a fairly recent, um, accomplishment. I mean, it was started by Euclid, um, you know, a millennia ago, but, but only in like the early 20th century did we finally list, "Okay, here are the, the axioms of, of mathematics," or well, the standard axioms of what we call ZFC and the axioms of first order logic, and this is what a proof is and, and, and, and this we've managed to automate and, and, and have a formal language for. Um, but there could be some way to assess plausibility of certain, um... You know, so if you, you have a conjecture that something is true, um, you t- you, you test a few examples, uh, and it works out. Like, how does this increase your, your, your confidence that the conjecture is true? We have a few sort of mathematical ways to, to, to model this, uh, uh, Bayesian probability, for example. Um, but they're not, uh, but they're, they're-- You often have to, they're often, uh, you have to set certain base assumptions and, and, and people, and it's, it's, it's, there's a lot of subjectivity still in, in, uh, in, um, in these tasks. So it, it is, it's, it's not clear, um, I, I mean, it's, this is more of a wish than, um, than a, than, than a plan to, to, uh, to build these languages. But-Just seeing how successful having f- a formal framework in place like Lean has made deductive proofs so much, um, uh, easier to automate and, and, and train AI on. If there was some similar framework... Yeah, so the, the bottleneck for using AI to, to, to create strategies and, and, and make conjectures is we have to rely on human experts to, um, uh, and the test of time to, to validate whether something's plausible or not. If there was some semi-formal framework where this could be done semi-automatically in a way that, that, uh, um, isn't sort of easily hackable, um, to, you know... So of course, yeah, the, the, well, uh, it's really important with these formal proof assistants that, that, that there are just no, um, uh, there, there's no backdoors or exploits that, um, uh, that you can do to somehow get your, your certified proof without actually proving it, because reinforcement learning is just so, so good at finding these, these, the, the, these backdoors. Um, but, um, yeah, if, if it's some framework that sort of mimics how, um, scientists talk to each other in a semi-formal way, you know, using data and, and argument, but, but also, um, you know, constructing narratives and, and, and, and th-th-there's some sub-sub- there's some subjective as-aspect of science that we don't know how to capture in a way that, that, that, uh, we can insert AI into them in any useful way. Um-

   3. DPDwarkesh Patel1:02:16

      Interesting.

   4. TTTerence Tao1:02:17

      So yeah, this is a, this is a future problem. Um, I mean, there are research efforts to, you know, uh, to try to create automated conjectures and, and, and, and, uh, and, and maybe there are ways to benchmark these and, and get some, some way to simulate this, but this is, it's, it's all very, very new science.

   5. DPDwarkesh Patel1:02:35

      Can, can you help me get some intuition for... I have two sub-questions. One, it would be very helpful to have a tangible sense of-- It would be helpful to have a specific example of w- what, uh, s- uh, something like this would look like, that the way scientists communicate that we can't formalize yet. And two, it seems almost definitionally paradoxical to say h- building up some narrative or building up some natural language explanation, and then also having something which you could have formalized. And I'm sure there's some intuition behind where that overlap is, and I'd love to understand that better.

   6. TTTerence Tao1:03:19

      All right, so, so an example of, of a conjecture. So, um, Gauss, um, was interested in the prime numbers. Uh, and, uh, he computed, he c- he created one of the first mathematical data sets. He just computed the first hundred thousand prime numbers or so, um, hoping to find patterns. Um, and, uh, he did find a pattern, but maybe not, not the pattern he was expecting. He, he found a statistical pattern in the primes that, that if you count how many primes there are up to one hundred, one thousand, um, uh, um, one million and so forth, they get sparser and sparser, but the, the, the, uh, the, the, the drop-off in, in, in the density was inversely proportional to the natural logarithm of, of, of, of, of, of, of the range of numbers. So he conjectured what we now call the prime number theorem. Um, the number of primes up to X is like X divided by the natural log of X. Um, and he had no way to prove this. Um, it was, it was data-driven. Um, so this, this was a, a conjecture. Um, it was revolutionary for its time because, um, it was maybe the first really important conjecture of, of math that was st-st-statistical in nature, you know? So normally, you, you would talk about a pattern like maybe the spacing between the primes has a certain regularity or something. But, um, yeah, but this was really something which it, it didn't tell you exactly how many primes there were in any given range. It just gave you an approximate approximation that got better and better as you, um, uh, went further and further out. But it, um, it, it helped. So it, it, it started the field of what we call an- analytic number theory. Um, but it was the first in many conjectures like this, m-many of which got proved, which sort of started, um, um, consolidating the idea that the prime numbers actually didn't really have a pattern, that they behaved like random, um, uh, random sets of, of numbers with a certain density. Um, I mean, they had some patterns, like, like they, they're almost all odd, okay? So there, there was some, there's... And, and they're not actually random. They're what's called pseudo-random. I mean, the, um, there's no random number generation involved in creating the prime numbers. But, um, over time, it became more and more productive to think of the primes as, as if they were just generated by some, some, some god rolling dice all the time and just creating this, this random set. Um, and this allowed us to make all these other predictions. Um, so there's a still open conjecture in, in, in number theory called the, the twin prime conjecture, that there should be infinitely many pairs of primes that are twins just two apart, like 11 and 13. We can't prove that, and there's actually good reasons why we can't prove it. But, um, uh, but because of this statistical random model of the primes, we are absolutely convinced it's true. Um, we, we know that if, if the primes were sort of generated by flipping coins or something, that we would just by random chance, just like infinite monkeys at a typewriter, we would see, um, twin primes appear over and over again. Um, and we have over time developed this very accurate conceptual model of what the primes should behave like based on statistics and probability, um, but it's all mostly heuristic and non-rigorous-

   7. DPDwarkesh Patel1:06:07

      Mm-hmm.

   8. TTTerence Tao1:06:07

      Um, but extremely accurate. Um, so the few times when we actually can prove things about the primes, it has matched up with the predictions of this, uh, what we call the random model of, of, of the primes. So we, we, we have this conjectural concept framework for understanding the primes that we, everyone believes in. And, you know, it's the same reason why we, we believe the Riemann hy-hypothesis is true, why we believe that cryptography based on the primes is basically, um, is mathematically secure, things like that. It's, it, it's all part of this, this, this, this belief. Um, in fact, one reason why we care about the Riemann hypothesis is that if the Riemann hypothesis failed, um, we, we knew it was false, it means it would-It would be a serious blow to this model that, that this-- it would mean there's a secret pattern to the primes that we were not aware of. Um, and, uh, I think we would very rapidly abandon any cryptography based on the primes, because if there was one pattern that we didn't know about, there's probably more, and these patterns can lead to exploits in, in crypto, and yeah, it's, it's gonna be a, it'd be a big, big shock. Um, so we really want to make sure that doesn't happen. Um, so yeah, it's, it's, um, so we've been convinced of, of things like the Riemann hypothesis and things over time, but, uh, some of it is experimental evidence, some is, uh, the few times we've been able to make theoretical results, they've always aligned. Um, you know, it is possible that the, the consensus is wrong and, and we've all just missed something very basic. Um, you know, there have been paradigm shifts in the past in scientific history. Um, yeah, but we, we don't really have a way of measuring this, um, I think partly 'cause we don't have enough data on, on, on how math or science develops. We, we have one timeline of history, and, you know, we, we have like, you know, 100 stories of turning points in history. If, if, if we had access to a million alien civilizations and each with a, the different development of, of history and, and of science in different orders, then maybe we'd actually have a, have a, have a decent shot, shot at, at, at a, at an understanding of how do we measure what is, uh, um, progress and, and, and what is a, a good strategy. And we could maybe start formalizing it and, and actually having a, a framework. Um, maybe if, um, what we need to do is actually start cr- creating lots of mini universes or simulations of, of AI solving very basic problems, you know, in arithmetic or whatever. But, but, um, uh, but coming up with their own strategies for doing these things and, and, and having these little laboratories to test. I mean, there are people who, who, who, who investigate, like, trying to, what's the smallest, uh, you know, neural network that can do ten-digit multiplication and things like that. I think, I think we could actually learn a lot just from, from evolving, um, uh, small AIs on, on, on simple problems. We could learn a lot. I was super excited when Mercury reached out about sponsoring the podcast because I've been banking with them for years. I think I opened my first account with them in 2023. Something I've come to appreciate over the last few years is that Mercury is constantly updating things and adding new features. Take their newest feature, Insights. Insights summarizes your money in and out, showing you your biggest transactions and calling out anything that deserves extra attention. Like maybe your revenue from a particular partner has gone down, or you've got a big uncategorized purchase that needs to be investigated. It's a super low-friction way for me to keep tabs on my business and make quick decisions. For example, I try to invest any cash that I don't need on hand to keep running the business. With Insights, with just a couple of clicks, I was able to see exactly how much money I spent in each month of 2025. And that lets me know exactly how much cash I'll need for the next year or so of operations, and then I can go invest the rest. Mercury just keeps adding new features like this. Go to mercury.com to check it out. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column NA, members FDIC.

8. 1:09:48 – 1:17:05

   How Terry uses his time

   1. TTTerence Tao1:09:48

      You have to, uh, learn about new fields not only very rapidly, but deeply enough to contribute to the frontier. So in some sense, you're also one of the world's greatest autodidacts. What, how does-- What is your process of learning about a new subfield in math? What does that look like? Yeah. So, um, I certainly, I identify with kind of the, yes, we talked about depth and breadth before. Yeah. And it's, it's not purely human-AI distinction. I mean, humans also, um, split, uh, so it's, I think it was, uh, Irving who split them into hedgehogs and foxes. Mm-hmm. And they said a hedgehog knows one thing very, very well, and a fox knows a l- a, a little bit about everything. Uh, so I definitely, I didn't, you know, I, I, I think of myself as a fox. Um, you know, I mean, I, I, I work with hedgehogs a lot, and sometimes I can be a hedgehog if need be. But, um, yeah, so, um, I've, I've always had a little bit of an, uh, obsessive streak. If, if there's something which I read about, which I feel like I should understand, I, I, I have the capability to understand this, but I don't understand why it works. There's, there's some magic in it that, um, you know, so someone was able to use a, a type of mathematics I'm not, I'm not familiar with and get a result which I would like to prove, and I can't do it by myself, but they could do it by, by their method, then I want to find out what was their trick. Um, it bugs me that they, someone else can, can do something which I think I, I can do, but, but I can't. Um, so I've always had that kind of obsessive completionist type, type streak. Um, I've had to wean myself off computer games because, um- [laughs] ...uh, I w- I w- I, I, I start a game, I wanna play it to completion, so all the levels. And, um, so, um, that's one, one way in which I, I learn new fields. Um, I collaborate with a lot of people, uh, who have taught me, um, other types of mathematics. Um, I just make friends with other mathematicians who's working on another area of mathematics, and I find their problems interesting, but I need to-- but they have to teach me, um, some of the basic tricks and, and what's known, what's not known, and, and I learn a lot from that. Um, I found that, that writing about my ex- uh, what I've learned, you know, I have a blog where I sometimes, um, um, uh, record things that I've learned. 'Cause in the past when I was younger, I would learn something and do this cool trick, and I say, "Okay, I'm gonna remember this." And then six months later, I'd, I'd, I'd forgotten. I, I, I remember remembering it, but I don't, uh, but I can't reconstruct my arguments. And it-- The first few times it was so frustrating to have understood something and then lost it, um, that I sort of resolved I should always write down anything cool that I've learned. Um, and that's, this is pa-part of why, how this blog came about. Um- How long does it take you to write a blog post? Um, it's something I often do when I don't want to do other work. You know, like, like there's some referee report or something. There's, there's, there's something that, that feels slightly unpleasant for me to do at the time. And so, uh, writing a blog, it feels creative and fun, like it's something that I, I do for myself. Um, so maybe, de-depending on, on, on the topic, it could be a, a quick, you know, half an hour or several hours. But I, um, it doesn't f- Because it's something that I do sort of voluntarily, it doesn't feel like it, it, it, it doesn't feel, uh-Time flies when I, when I write these things down. As opposed to sort of doing something which I have to do for administrative reasons, but it's just that it's, it's, it's drudgery. Okay. Those, those are tasks where the AI is really helping with nowadays actually. [chuckles]

   2. DPDwarkesh Patel1:13:06

      Is it, um, if, if like civilization could, could from first principles decide how to use Terry Tao's time, [laughs] you know, it's like a limited resource. Uh, uh, how, how, how, what is the biggest diff between-- [laughs] if the, if the veil of ignorance got to decide how to use Terry Tao's time versus what it does now?

   3. TTTerence Tao1:13:25

      Um, okay. So-

   4. DPDwarkesh Patel1:13:26

      This podcast wouldn't be happening. [laughs]

   5. TTTerence Tao1:13:28

      Yeah. So I could-- The, uh... As much as I complain about certain tasks that I don't want to do, but I have to do. So yeah, as, as you get more senior in, in academia, you get more and more responsibilities, and I get some more committees and, and, and whatever. Um, but I have also found that, um, a lot of events that I kind of reluctantly went to because I was obliged to, for one reason or another, um, because it's outside my comfort zone, I often find interactions with people who I wouldn't normally talk to, uh, like you, for instance. Um, and I would, I would learn interesting things and have interesting experiences. Um, and I, I would have opportunities to, to, to, to then network with other people that I would never have, have done before. Um, so I do believe a lot in serendipity. Um, I mean, I, I do optimize my time and, and, and, um, when I, um, um... So there's some portions of, of my, of my day where I do schedule very carefully. Um, but I, I have been willing to sort of leave some, some portions just, okay, I'm gonna do something which is, which is not my usual thing, and then maybe it'll be a waste of my time, but maybe I'll, I will learn something. And, uh, and more often than not, it, it's, um, I've, I've, um, I feel like I've, I've gotten a, a positive experience, which is not something I would have planned for. Um, and yeah, so I believe a lot in serendipity. Um, and maybe there's a danger actually that, uh, you know, in the mo-modern societies, it's not just AI, but we've become really good at optimizing everything. Um, and, and, and maybe we are optimizing, we're not optimizing other optimization. Um, that, uh, um, you know, with, with, with COVID, for example, um, we, we switched, um, like we, we switched a lot to remote meetings, um, and so everything was scheduled now. And so, uh, we kept busy, at least, uh, in, in academia. You know, we s- we met almost the same number of people that we met, um, uh, in person, but everything had to be planned. Um, and you had to schedule things in advance. Uh, and what we lost out on was sort of the, the casual, like, you know, knocking on a hallway, just meeting someone, uh, for, you know, while getting a coffee. Um, and the, the, the, this, the, this, um, um, yeah, serendipitous interactions that, uh, um, you may think are not optimal, but actually are really important. You know, when I was a grad student, um, I would go down to the library, um, to look, if I had to look for a, a journal article. Yeah, I had to physically go down to the library, check out the, the journal and read your article. And sometimes the next article, you know, you can just browse through and, and the next article is also interesting. Um, uh, sometimes it wasn't, but, but you could accidentally find interesting things, um, which is something which has basically been lost now because you can just type in, you know, if you, if you, if you want to access an article now, you just type it into, to a search engine or even an AI, and you can get instantly what you want, but you don't get sort of the accidental things that, uh, that you, you, you might have, have, um, gotten if you'd done it more inefficiently. Um, so, um, yeah. There've been times when, I mean, um, um, I s- I spent a year once at the Institute for Advanced Study, which is, uh, a great place to, uh, you know, there's no distractions. You, you, you're there to just do research. And, like, the first few weeks you're there, like, it's great. You're getting all these papers written up that, that you've been wanting to do for a long time. You've been thinking about problems for blocks of hours at a time. Um, but I find if I stay there for more than, um, um, uh, several months, like I'd, I, I run out of, of inspiration somehow. Like I get bored.

   6. DPDwarkesh Patel1:16:44

      Interesting.

   7. TTTerence Tao1:16:44

      I actually, you know, surf the internet a lot more.

   8. DPDwarkesh Patel1:16:46

      Hmm.

   9. TTTerence Tao1:16:46

      Um, you actually do need a certain level of distraction in your life. It, it somehow, uh, adds enough randomness, um, and, and that, uh, and temperature, high temperature if you need. Yeah. Um, so yeah. I don't know the, the optimal, uh, way to schedule my life. Uh, it just seems to work.

9. 1:17:05 – 1:23:43

   Human-AI hybrids will dominate math for a lot longer

   1. DPDwarkesh Patel1:17:05

      I'm very curious when you expect AIs that can, like, actually do frontier math better than the, at least as good, well as the best human mathematicians.

   2. TTTerence Tao1:17:15

      I mean, in, in some ways, they're al- they're already doing frontier math that is super intelligent that, that humans can't do, but it's a different frontier from what we're used to.

   3. DPDwarkesh Patel1:17:22

      Right.

   4. TTTerence Tao1:17:23

      Um, I mean, you could argue that calculators were doing frontier math-

   5. DPDwarkesh Patel1:17:26

      Right

   6. TTTerence Tao1:17:26

      ... uh, that, that humans, um, uh, could not accomplish. But it was, uh, but it wasn't, you know, number crunching and, um-

   7. DPDwarkesh Patel1:17:32

      Right

   8. TTTerence Tao1:17:32

      ... but, um-

   9. DPDwarkesh Patel1:17:34

      But, but replacing Terry Tao completely.

   10. TTTerence Tao1:17:38

       I mean, uh, the question, what do you want me for? Uh-

   11. DPDwarkesh Patel1:17:41

       [laughs]

   12. TTTerence Tao1:17:41

       I, um, uh-

   13. DPDwarkesh Patel1:17:43

       You'll just go on all the podcasts after. [laughs]

   14. TTTerence Tao1:17:46

       [laughs] I'm not sure we've-- It might not be the right question to ask. Um, I think with- within a decade, a lot of things that mathematicians currently do, um, what, what we spend a lot of the bulk of our time doing and a lot of stuff we put in our papers today can be done by AI. Um, but we will find that that actually wasn't the most important part of what we do. Um, you know, um, a hundred years ago, um, a lot of mathematicians were just solving differential equations, um, like, uh, people needed, uh, uh, physicists needed some exact solution to, to, to, to, to, to some system and, and they were just, they hired a mathematician at Labelbox to go through the calculus and, and work out the solution to this fluid equation, whatever. Um, a lot of what, um, uh, a 19th century ma-mathematician would do, um, you could make a call to, uh, Mathematica or, or Wolfram Alpha or a computer algebra package, or more, now more recently, an AI, and it would just solve the problem, you know, in a few minutes. Um, but we, we moved on. We d- we, we, we worked on different types of problems after that. Um-You know, once computers came along, you know, the, uh, uh, computers used to be human, right? People used to laboriously create log tables and, and, and, and, and work out primes as Gauss did, and that has all been outsourced to computers. Um, but, but, but we, we moved on. Um, in genetics, you know, um, to, uh, to, to sequence a-at, at the, at, at the genome of a single organism, that was an entire PhD of a geneticist. You know, it's, uh, so carefully, you know, separating all the chromosomes and, and whatever. Um, and now you can just spend $1,000 and send it to a sequencer and, and, and get it done. But genetics is not dead as a subject. Uh, you, you move to a different scale. You know, maybe you study whole ecosystems rather than individuals.

   15. DPDwarkesh Patel1:19:34

       I, I, I take your point, but on, on the question of, well, when is most mathematical progress or almost all mathematical progress happening by AI, so that if you find out, oh, this year a Millen- Millennium Prize problem has been solved, you would put, you know, a 95% odds that an AI did it autonomously. Surely there will be such a year.

   16. TTTerence Tao1:19:51

       Um, I guess. I mean, I, I, I, I do believe that, that hybrid, um, human plus AIs will, will dominate mathematics for a lot longer. It, it's, it will depend-- It will require some additional breakthroughs be- uh, beyond what we already have. Um, so it's, it's gonna be stochastic. Um, you know, I think, you know, AIs currently are very good at certain things but, but really terrible at others. Um, and, and while you can sort of add more and more frameworks on top to kind of reduce the error rates and, and, and, and make them, uh, uh, work with each other a bit more and so forth, um, I, I, um, it feels like we are-- we don't have all the, uh, the ingredients to, like, really have a s- truly satisfactory sort of, uh, replacement for all intellectual tasks. Um, it's, it is complementary currently.

   17. DPDwarkesh Patel1:20:39

       Yeah.

   18. TTTerence Tao1:20:39

       Um, it's not, not, uh, um, it, it is, it's not a replacement. Um, but maybe, uh, I mean, because current level AIs will accelerate science in so many ways, uh, hopefully, I mean, it, new discoveries, new breakthroughs will happen, um, more, uh, more quickly. I mean, um, it's possible that also by somewhat destroying serendipity, we, we actually inhibit certain types of progress. Um, anything is possible really at this point. I think, uh, this, uh, [chuckles] the world is very, very unpredictable-

   19. DPDwarkesh Patel1:21:11

       Yeah

   20. TTTerence Tao1:21:11

       ... at this, this point in time.

   21. DPDwarkesh Patel1:21:13

       What is your advice to somebody who would consider a career in math or is early in a career in math, especially in light of AI progress? How should they be thinking about their career differently, if at all, as a result of AI progress?

   22. TTTerence Tao1:21:28

       Yeah. So, uh, we live in a time of change. Um, it is, as I said, uh, it, it, we live in a particularly unpredictable era. Um, and, uh, I think in t- like things that we've taken for granted for c- centuries may not hold anymore. Um, so, um, yeah, the way we, uh, do everything, and not just mathematics, um, will change. And, um, you know, so I, I think, wh-which is, you know, I mean, in many ways I would prefer the much more boring, quiet era where things are much the same as they were ten years ago, 20 years ago. But, um, so I think one just has to embrace this, that this, it, we're, there's gonna be a lot of change, um, and that, um, you know, the things that you study, some of them m-may become obsolete or revolutionized, but s- but some things will be retained. Um, and, um, so y-y-you somehow always have to keep an eye on, uh, like, um, there'll be a lot of opportunities for, for things that you, you wouldn't be able to do before. Um, so I mean, in, in math, you know, you previously had to basically go through years and years of education, maybe a math PhD, before you could contribute to the frontier of, of math research. Um, but now it's quite possible at the high school level or, or whatever that, that you could get involved in a math project and actually make a real contribution because of all these AI tools and, and, and Lean and everything else. Um, so there'll be a lot of non-traditional opportunities to, to learn. Um, so you need a very adaptable, um, mindset. Um, you know, there'll be, there'll be worth pursuing things just for curiosity, you know, for playing, playing around and, uh, I mean, you still need to get your credentials for, I mean, for, uh, thank gosh, for a while it will still be important to, to sort of still go through traditional education and, and, uh, and, and learn math and science and so forth the old-fashioned way for a while. But, um, yeah, um, but you should also be open to, to very, very different ways of, of, of doing science, some of which don't exist yet. Um, yeah, so it's, it's, it's a scary time, but also very exciting. Yeah.

   23. DPDwarkesh Patel1:23:40

       Awesome. That's a great note to close on.

   24. TTTerence Tao1:23:41

       Okay.

   25. DPDwarkesh Patel1:23:41

       Terence, thanks so much.

   26. TTTerence Tao1:23:42

       Yeah. Pleasure.

Episode duration: 1:23:44