AGI is still 30 years away — Ege Erdil & Tamay Besiroglu

1. 0:00 – 23:01

   AGI will take another 3 decades

   1. EEEge Erdil0:00

      Just think about the sheer scale of knowledge that these models have. It is actually quite remarkable that there's no, like, innovation that comes out of that, has a reasoning model, ever come up with a math concept that even seems, like, slightly interesting to a human mathematician. I- I've never seen that.

   2. TBTamay Besiroglu0:17

      Intelligence isn't the bottleneck. Making contact with the real world and getting a lot of data from experiments and from deployment just has this drastic impact.

   3. EEEge Erdil0:25

      There's just, like, this enormous amount of richness and detail in the real world that you just can't, like, reason about it.

   4. DPDwarkesh Patel0:32

      Right.

   5. EEEge Erdil0:32

      Like, you- you- you need to see it.

   6. DPDwarkesh Patel0:34

      Today I'm chatting with Tamer Besiroglu and Ege Erdil. They were previously running Epoch AI and are now, uh, launching Mechanize, which is a company dedicated to automating all work. One of the interesting points you made recently, Tamer, is that the whole idea of the intelligence explosion is mistaken or misleading. W- why don't you explain what you were talking about there?

   7. TBTamay Besiroglu0:57

      Yeah. I think it's not a very useful concept.

   8. DPDwarkesh Patel0:58

      Mm-hmm.

   9. TBTamay Besiroglu0:59

      Um, it's kind of like calling the Industrial Revolution a horsepower explosion. Like, sure, during the Industrial Revolution we saw this drastic acceleration in raw physical power, but there are many other things that were maybe equally important in explaining the acceleration of growth and technological change that we saw during the Industrial Revolution.

   10. DPDwarkesh Patel1:18

       Uh, w- what is a way to characterize the broader set of things that the horsepower perspective would miss about the Industrial Revolution?

   11. TBTamay Besiroglu1:25

       So I- I think in the case of the Industrial Revolution, it was a bunch of these complementary changes to many different sectors in the economy. So you had agriculture, you had transportation, you had law and finance, you had urbanization and moving from rural areas into- into cities. Um, there were just many different innovations that-

   12. DPDwarkesh Patel1:45

       Mm.

   13. TBTamay Besiroglu1:45

       ... kind of, you know, happened simultaneously that gave rise to this, um, change in the- the way of economically organizing our society. It wasn't just that we had, uh, more horsepower. That, I mean that was part of it, but that's not the, kind of central thing to focus on when thinking about the Industrial Revolution. And I think similarly for the development of AI, sure, we'll get, like, a lot of very smart AI systems, but that will be one part among very many-

   14. DPDwarkesh Patel2:13

       Hm.

   15. TBTamay Besiroglu2:14

       ... different moving parts that explain, you know, why we expect to get this transition and this acceleration in growth and technological change.

   16. DPDwarkesh Patel2:21

       Yeah. I- I wanna better understand how you think about that broader transformation. Um, before we do, the other really interesting part of your world view is that you have longer timelines to get to AGI than most of the people in San Francisco who think about AI. Um, when do you expect a drop-in remote worker replacement?

   17. EEEge Erdil2:40

       Yeah. Maybe for me that would be around, like, 2045 or ...

   18. DPDwarkesh Patel2:44

       Wow. Wait, and you?

   19. TBTamay Besiroglu2:46

       I'm a little bit more bullish. I mean, it depends what you mean by drop-in remote worker and whether it's able to do, like, literally everything that can be done remotely or do most things.

   20. DPDwarkesh Patel2:55

       Yeah.

   21. EEEge Erdil2:55

       I'm saying literally everything.

   22. TBTamay Besiroglu2:56

       For literally everything, yeah. I- I- it just shaped, I guess, predictions by five years or, like, by 20% or something.

   23. DPDwarkesh Patel3:02

       Why? 'Cause we've seen so much progress over even the last few years.

   24. EEEge Erdil3:06

       Mm-hmm. Yeah.

   25. DPDwarkesh Patel3:07

       We've gone from ChatGPT, like, two years ago to now we have models that can literally do reasoning, c- are better coders than me, um, and I- I studied software engineering in college. I mean, I did become a podcaster. I'm not saying I'm, like, the best coder in the world.

   26. EEEge Erdil3:21

       (laughs)

   27. TBTamay Besiroglu3:22

       (laughs)

   28. DPDwarkesh Patel3:22

       (laughs) But, um, if you made this much progress in the last two years, why would it take another 30 to get to full automation of, um, human brains?

   29. EEEge Erdil3:33

       Right. So-

   30. DPDwarkesh Patel3:33

       Wait, I- I said that wrong. You know what I'm saying. F- full automation of remote work.
2. 23:01 – 45:38

   Even reasoning models lack animal intelligence

   1. EEEge Erdil23:01

      different claim, right?

   2. DPDwarkesh Patel23:02

      Look, a ton of B2B software companies start off by building self-serve, consumer-grade products, and that's fine at first. Eventually though, you have to go after enterprise. The most successful and durable software companies of the last decade have all made this transition. But getting enterprise ready is hard. Single sign-on, role-based access controls, and comprehensive audit logs are all actually quite complex and tedious to build, and they're ripe for bugs and annoying edge cases. These features take a ton of engineering time and capital, which you should be spending on the core product. For example, one of Slack's PMs said that they spent $30 million building these features, and they were only half done. That's where WorkOS comes in. WorkOS has helped Vercel, Plaid, Vanta, OpenAI, and hundreds of others become enterprise ready with APIs to integrate all of these common features. If you want to learn more, go to workos.com and tell them that I sent you. Okay, so then this brings us to the intelligence explosion.

   3. EEEge Erdil24:04

      Mm-hmm.

   4. DPDwarkesh Patel24:05

      Because what people will tel- say is, "We don't need to automate literally everything that is needed for, uh, automating remote work, um, let alone all human labor in general. We just need to automate the things which are necessary to fully close the R&D cycle needed to make smarter intelligences." And if you do this, you get a very rapid intelligence explosion a-... and the end product of that explosion is not only an AGI, but something that is superhuman, potentially. These things are, like, extremely good at coding, and that- th- they're good at the kinds of things that you would think, and reasoning, and like, it seems like the kinds of things that would be necessary to a- automate, um, R&D at AI labs.

   5. EEEge Erdil24:47

      Mm-hmm.

   6. DPDwarkesh Patel24:47

      What do you make of that logic?

   7. EEEge Erdil24:49

      I mean, I think if you look at their capability profile, it is like, like ... If you compare it to, like, a random job in the economy, I agree they are better at doing sort of coding tasks that will be involved in R&D compared to a, like a random job in the economy. But I, in absolute terms, I don't think they are, like, that good. Uh, I think they are good at things that maybe impress us about human coders. Like, if you were wanted to see, like, oh, like what is, what makes a person a really impressive coder? You might look at their competitive programming performance. Uh, I mean, in fact, companies often hire people based on, es- if they are relatively junior, based on their performance on these kinds of problems. But that is just impressive in the human distribution. So if you look in absolute terms at what are the skills you need to actually automate the process of being a researcher, then what fraction of those skills do the AI systems actually have? Even in coding. Like, a lot of coding is you have a very large code base you have to work with. The instructions are very kind of vague. There isn't, uh, a ... For e- for example, you mentioned a meter eval, uh, in which because they needed to make it an eval, all the tasks have to be kind of, uh, compact and closed and have clear evaluation metrics. Like, here's a model. Like, get its loss on this, uh, you know, uh, data set as low as possible, uh, or whatever. Or like, here's a- another model and its lay- like, its, uh, embedding matrix has been scrambled. Just, uh, fix it to recover, like, most of its original performance, et cetera. Those are not problems that you actually work on in AI R&D. They're like very artificial problems. Uh, now if a human was good at doing those problems, you would infer, I think logically, that that human is likely to actually be a good researcher. But if an AI is able to do them, like the AI lacks so many other, uh, competences that a human would have. Not just a researcher, just an ordinary human, that we don't think about in the process of research. So our view would be automating research is, um, first of all, more difficult than people g- get credit for. I think you need more skills to do it, uh, and definitely more than models are displaying right now. And on top of that, even if you did automate the process of research, we think a lot of the software progress has been driven not by, uh, cognitive effort, though that has played a part, but it has been driven by compute scaling. We just have more GPUs that can do more experiments to figure out more things. Your experiments can be done at larger scales. And, um, that is just a very important driver. Like, if you just, uh, if you were 10 years ago, 15 years ago, you were trying to figure out what software innovations are gonna be important in 10 or 15 years, you would have had a very difficult time. In- in fact, you probably wouldn't even conceive of the right kind of innovations to be looking at.

   8. DPDwarkesh Patel27:35

      Mm-hmm.

   9. EEEge Erdil27:35

      Because you would be so far removed from the context of that time with much more abundant computes and all the things that people would have learned by that point. Um, so there, these are two v- two, uh, components of our view.

   10. DPDwarkesh Patel27:48

       Yeah.

   11. EEEge Erdil27:48

       Research is harder than people think and depends a lot on compute scale.

   12. DPDwarkesh Patel27:51

       S- s- well, uh, y- can you put a finer point on what exa- what i- what is the kind of thing ... What is an example of the kind of task which is very dissimilar from train a classifier or debug a classifier that is relevant to AI R&D?

   13. TBTamay Besiroglu28:05

       I- I- I think it's, like, you know, examples might be introducing novel, um, uh, having novel innovations that are very useful for unlocking innovations in the future. So that might be, i- i- you know, introducing some novel way of thinking about a problem.

   14. DPDwarkesh Patel28:22

       Yeah.

   15. TBTamay Besiroglu28:22

       Or introducing ... So, so maybe, um, a good example might be in, in mathematics where we have these reasoning models that are extremely good at solving math problems.

   16. EEEge Erdil28:31

       I mean, very short horizon math problems.

   17. TBTamay Besiroglu28:34

       Yeah, yeah, sure. Uh, maybe not extremely good, but certainly better than I can.

   18. EEEge Erdil28:38

       Yeah.

   19. TBTamay Besiroglu28:38

       And better than maybe most, most undergrads can. Um, and, and so, you know, they can do that very well, but they're not very good at coming up with novel conceptual schemes that are useful for making progress in mathematics. Um, so, you know, it's able to solve these problems that you can kind of neatly excise out of some very messy context and it's able to make a lot of progress there. But within some much messier context, it's kind of not very good at figuring out what directions are especially useful for, you know, you to build things or kind of tr- make incremental progress on that enables you to have a big kind of innovation later down the line. Um, so thinking about both this, like, larger context, as well as maybe much longer horizon, kind of much fuzzier things that you're optimizing for, I think it's much worse at those types of things.

   20. EEEge Erdil29:29

       Right. So, uh, I think one interesting thing is if you just look at these reasoning models, they know so much, especially the large ones, because I mean, they know in literal terms more than any human does in some sense.

   21. DPDwarkesh Patel29:40

       Yeah.

   22. EEEge Erdil29:40

       And while we have unlocked these reasoning capabilities on top of that knowledge, and I think that is actually what is enabling them to solve a lot of these problems. But, uh, if you actually look at the way they approach problems, they, um ... Like, they ... The reason what they do looks impressive to us is because we have so much less knowledge, and the model is ap- approaching the problems in a fundamentally different way compared to a human would. A human would have much more limited knowledge, and they would usually have to be much more creative in solving problems because they have this lack of knowledge while the model knows so much. Like, you would ask it some obscure math question where you need, like, some specific theorem from 1850 or something and then it would just, like, know that, if it's like a large model. So that makes the difficulty profile very different, and if you look at the way they approach problems, the reasoning models, they are usually not ...... creative. Uh, they are very effectively able to leverage the knowledge they have, uh, which is extremely vast, and that makes them very effective in a bunch of ways. But you might ask the question, has a reasoning model ever come up with a math concept that even-

   23. DPDwarkesh Patel30:49

       Yeah.

   24. EEEge Erdil30:49

       ... seems, like, slightly interesting to a human mathematician? And I've never seen that.

   25. DPDwarkesh Patel30:54

       I mean, they've been around for all of six months (laughs) .

   26. EEEge Erdil30:56

       But that's a long time, like-

   27. TBTamay Besiroglu30:57

       Well, lots of people have been... I mean, that- that's- that's a long time.

   28. EEEge Erdil30:59

       But, but like, just think about the-

   29. TBTamay Besiroglu31:00

       One, one mathematician might have been able to, like, do a bunch of work over that time, and-

   30. DPDwarkesh Patel31:04

       Right.
3. 45:38 – 1:01:31

   Intelligence explosion

   1. TBTamay Besiroglu45:38

      that gives us this acceleration.

   2. DPDwarkesh Patel45:39

      Hmm. All right, so this brings us back to, um, the intelligence explosion.

   3. TBTamay Besiroglu45:44

      Mm-hmm.

   4. DPDwarkesh Patel45:46

      Um, here is the argument for the intelligence explosion. Look, uh, you're right that certain kinds of things might take longer to come about, but this core loop of software R&D, uh, that's required, um, if you just look at, like, what kinds of progress is needed to make a more general intelligence, you may be right that it needs more experimental compute, but, like, we're just getting, uh, as you guys have documented, we're just getting, like, a shit ton more compute every single year for the next few years.

   5. TBTamay Besiroglu46:15

      Yep.

   6. DPDwarkesh Patel46:16

      So you can imagine a intelligence explosion for the next few years where in 2027 there'll be, like, 10X more compute than there is now, um, for AI and you'll have this effect where the AIs that are doing software R&D are finding ways to make running copies of them more efficient, which has two effects. One, you're increasing the population of AIs who are doing this research, so more of them in parallel can find these different optimizations. Um, and a subtle point that they'd often make here is software R&D in AI is not just ELI type coming up with new transformer-like architectures. To your point, it actually is a lot, uh, ver- uh, like, you gotta, like... I, I mean, I, I'm not a AI researcher, but, uh, I assume there's a, like, from the lowest level libraries to the kernels to making RL environments to finding the best optimizer to... Uh, there's just, like, so much to do and in, like, parallel you can be doing all these things or finding optimizations across them. And so you have, you have two effects, going back to this. One is you've... You know, if, if you look at the original GPT-4 compared to th- the current GPT-4.0, I think it's, like, w- what... It's, like, t- h- how much cheaper is it to run? It's, like, w- what... where... you, it's like-

   7. TBTamay Besiroglu47:33

      Maybe 100-

   8. DPDwarkesh Patel47:34

      Yeah, yeah. So you have this, like-

   9. TBTamay Besiroglu47:35

      ... times for the same capability or something.

   10. DPDwarkesh Patel47:37

       Right. So you, you're, they're finding ways in which to run more copies of them at, like, l- you know, 100X cheaper, uh, or something, which means that the population of them is increasing and the higher population is then helping you find more efficiencies. And does, not only does that mean you have more researchers, but to the extent that what's the complementary input is experimental compute, it's not the compute itself, it's the experiments. And the more efficient it is to run a copy or to develop a copy, the more parallel experiments you can run because now you can do a GPT-4 scale training run for much cheaper than you could do it in 2024 or 2023. Um, and so, uh, f- for that reason also, this, like, software-only singularity sees more researcher copies who can run experiments for cheaper, dot, dot, dot. They initially are maybe handicapped in certain, um, ways that you mentioned, but through this process they are rapidly becoming much more capable. What is wrong with this logic?

   11. TBTamay Besiroglu48:31

       So I, I think the logic, like, the logic seems fine.

   12. DPDwarkesh Patel48:35

       Yeah.

   13. TBTamay Besiroglu48:35

       Um, I, I think this is, like, a decent way to think about this problem, but I, I, I think the, it's useful to draw on a bunch of work that, say, economists have done for studying kind of the returns to R&D and what happens if you 10X your inputs, so the number of researchers, what happens to innovation or the rate of innovation. And, um, and there, you know, they, they, they point out these kind of two effects where, you know, as you do more innovation then you get to kind of stand on top of the shoulders of giants and you get the benefit from past discoveries and it makes you as a scientist more productive. But then there's also kind of diminishing returns that the low-hanging fruit has been picked and it becomes harder to make progress. And overall, you can summarize...... those estimates as thinking about the kind of returns to research effort. And, uh, you know, we've looked into the returns to research effort in software specifically, and we look at a bunch of domains in traditional software or, you know, um, like linear integer solvers or, or SAT solvers, uh, but also in AI, like computer vision and RL and, um, and language modeling. And there, um, like if this model is true that all you need is just cognitive effort, it seems like the estimates are a bit ambiguous about whether this results in this acceleration or whether it results in just merely exponential growth. And then you might also think about, well, it isn't just your research effort that you have to scale up to make these innovations, because you might have complementary input. So as you mentioned, experiments are, are, are the thing that might kind of bottleneck you, and I think there's a lot of evidence that, in fact, these experiments and scaling up hardware is just very important for, for getting progress in, in, in the algorithms and the architecture and, and so on. So in AI, um, th- this is true for software in general, where if you look at progress in software, it, it often matches very closely the rate of progress we see in hardware. Um, so for traditional software, we see about a 30%, roughly, increase per year, which kind of basically matches Moore's law. And in AI we've seen the same until you get to the deep learning era, and then you get this acceleration, which in fact coincides with the acceleration we see in compute scaling, which gives you a hint that actually the compute scaling might have been very important. Other pieces of evidence, you know, besides this condens- co- coincidental rate of progress, other kind of pieces of evidence are, um, you know, the, the fact that innovation in algorithms and architectures are often concentrated in GPU-rich labs and not in the GPU-poor parts, uh, uh, you know, uh, of the world, like academia or maybe smaller research institutes. That also suggests that having a lot of hardware is very important. If you look at specific innovations that seem very important, uh, the big innovations over the past five years, many of them have some kind of scaling or hardware-related motivation. So, you know, you might look at, uh, the transformer itself was about how to harness more parallel compute. Um, things like flash attention was literally about how to implement, uh, the attention mechanism more efficiently, or things like the chinchilla scaling law. And so many of these big innovations were just about how to harness your compute more effectively. That also tells you that actually the scaling of compute might be very important, and I think there's just, like, many pieces of evidence that points towards this complementarity picture. So, I would say that not only, like even if you assume that experiments are not particularly important, the evidence we have both from estimates of, of AI and other software, although the data is a bit... is not great, suggests that, you know, um, maybe you don't get this kind of hyperbolic, uh, faster than exponential, you know, super growth in the, um, in the overall algorithmic efficiency of systems.

   14. DPDwarkesh Patel52:31

       I- I- I'm not sure I buy the argument that because these two things, compute and AI progress, have risen so concomitantly that this is a sort of causal relationship. So broadly, the industry as a whole has been ge- getting more compute and as a result making more progress. Um, but within... if you look at the top players, there's been multiple examples of a company with much less compute but a more coherent vision, uh, more concentrated research effort being able to beat a incumbent who has much more compute. So OpenAI initially beating Google DeepMind, and if you remember, there were these emails that were released between Elon and Sam and so forth where they were like, "We gotta start this company because they've got this bottleneck on the compute, and, like, look how much more compute Google DeepMind has." Um, and then OpenAI made a lot of progress similarly now with OpenAI versus Anthropic and so forth. Um, and then I think just generally y- your argument is just like too outside view when we just do know a lot about, like, what is... like, you're just like, this very macroeconomic argument that I'm like, "Well, why don't we just ask the AI researchers?"

   15. TBTamay Besiroglu53:35

       I mean, AI researchers will often kind of overstate the extent to which this cognitive effort and doing research is important for driving these innovations, because that's often kind of convenient or useful. They will say the insight was, you know, w- w- was derived from some kind of nice idea about-

   16. DPDwarkesh Patel53:57

       Yeah.

   17. TBTamay Besiroglu53:57

       ... statistical mechanics or some nice equation in physics that says that we should do it this way and then, and then... but often that's kind of a ad hoc story that they tell to make it a bit more compelling to, uh-

   18. DPDwarkesh Patel54:10

       Or, or, or-

   19. TBTamay Besiroglu54:10

       ... to, to, to the kind of reviewers -

   20. DPDwarkesh Patel54:12

       ... so Dan- ... basically. Daniel mentioned this, like, um, survey he did where he... Daniel Gogotalo.

   21. TBTamay Besiroglu54:16

       Right.

   22. DPDwarkesh Patel54:17

       Um, he, he, he asked a bunch of AI researchers, "If you had one-thirtieth the amount of compute," and he did one-thirtieth because AIs will be... as opposed to, I think, 30 times faster. "If you had one-thirtieth the amount of compute, how much, how much would your progress slow down?" And they say, "I make a third of the amount of progress I normally do." So that's just a pretty good, like, substitution effect of you, you get one-tenth the compute, your progress only goes down one-third. Mm-hmm. Um, I was talking to an AI, uh, researcher the other day who's, like, just like one of these, like, cracked people, gets paid millions and... tens of millions of dollars a year probably. Um, and we asked him, "How much do these AI models help you in domains you already are f- you know, how much does these AI models help you in AI research?" And he said, "In domains that I- I'm already quite familiar with, where I just... closer to autocomplete, it's, like, saves me four to eight hours a week." Mm-hmm. Um, and then he said, "But in domains where I'm actually less familiar, where it's like I need to try new connections, I need to understand how these different parts relate to each other and so forth, um, it saves me close to 24 to 36 hours a week." Right? So that... and then that's, like, current models, and I'm just like-... he didn't get more compute, but it still saved him, like, a shit ton more time. Like, it just, like, draw that forward, it's like, that's a crazy implication or crazy trend, right?

   23. EEEge Erdil55:33

       I mean, I guess have we seen, uh, like, I'm skeptical of the claims that we have actually seen that much of an acceleration in, uh, the process of R&D. Like, these claims seem to me like they're not borne out by the actual data I'm seeing. Uh, so I'm not sure how much to trust them.

   24. DPDwarkesh Patel55:52

       I mean, the mo- more, uh, on the general intuition that cognitive effort alone can give you a lot of AI progress-

   25. EEEge Erdil55:56

       Right.

   26. DPDwarkesh Patel55:57

       ... we've had a- a- a- it seems like a big p- important thing the labs do is this, like, science of deep learning. Like, scaling laws is just a, you, I mean, a, a, oh, like, it ultimately netted out an experiment, but the experiment was motivated by cognitive effort.

   27. EEEge Erdil56:10

       So for what is worth, when you say that A and B are complementary, you're not saying, like, just as you can't get a lot of progress, like, just can, just as A can bottleneck you, B can also bottleneck you.

   28. DPDwarkesh Patel56:19

       Yeah.

   29. EEEge Erdil56:19

       So when you say you need, like, compute, uh, and experiments and data, but you also need cognitive effort, like, that doesn't mean the lab who has the most compute is gonna win, right? Like, that's a very simple point. Like, e- e- either one can be the bottleneck.

   30. DPDwarkesh Patel56:32

       Yeah, yeah.
4. 1:01:31 – 1:06:58

   Ege & Tamay's story

   1. EEEge Erdil1:01:31

      we probably shouldn't.

   2. DPDwarkesh Patel1:01:32

      Okay, sure. (laughs)

   3. EEEge Erdil1:01:33

      Do that.

   4. DPDwarkesh Patel1:01:33

      Uh, by the way, so for... The, uh, the audience should know, my most popular guest by far is Sarah Payne.

   5. EEEge Erdil1:01:37

      Mm-hmm.

   6. DPDwarkesh Patel1:01:39

      Um, not only is she my most popular guest, she's my most popular four guest, because like (laughs) all four of those episodes that I've done with her-

   7. EEEge Erdil1:01:46

      (laughs)

   8. DPDwarkesh Patel1:01:46

      ... are... Like from a viewer minute adjusted basis, I host the Sarah Payne podcast where I occasionally talk about AI.

   9. EEEge Erdil1:01:52

      (laughs)

   10. DPDwarkesh Patel1:01:53

       Um.

   11. EEEge Erdil1:01:53

       (laughs)

   12. DPDwarkesh Patel1:01:54

       And, uh, anyways, we did this three-part lecture series, um, where we're talking about like one of them was about India-Pakistan wars through history. One of them was about... Was it the S- the, uh, Ja-

   13. EEEge Erdil1:02:07

       War-

   14. DPDwarkesh Patel1:02:07

       ... Japan b- Like Ge-

   15. EEEge Erdil1:02:08

       Yep.

   16. DPDwarkesh Patel1:02:08

       ... Japanese culture before World War II. The third one was about the Chinese civil war.

   17. EEEge Erdil1:02:11

       Yep.

   18. DPDwarkesh Patel1:02:12

       And for all of them, my tutor, my history tutor was Ege. Um, and it just like, why does he know so much about-

   19. EEEge Erdil1:02:21

       (laughs)

   20. DPDwarkesh Patel1:02:21

       ... like fucking random like 20th century conflicts? Um, (laughs) uh, but he did, and he suggested a bunch of the good questions I asked her. We'll get into that in a s- Like, actually, wha- wha- what's going on there?

   21. EEEge Erdil1:02:31

       I- I- I don't know. I mean, I don't really have a good question. I think it's interesting. I mean, I read a bunch of stuff, but it's like kind of a boring answer. Like, I don't know. Like, imagine you ask like a, like a top AI researcher, like, "What's going on?" Like, "How are you so good?" And then they will probably give you like a boring answer.

   22. DPDwarkesh Patel1:02:46

       Mm-hmm.

   23. EEEge Erdil1:02:46

       Like, I don't know, like, "I did this (inaudible) (laughs)

   24. DPDwarkesh Patel1:02:48

       I mean, th- that i- that itself is interesting that often these kinds of questions elicit boring answers.

   25. EEEge Erdil1:02:52

       Yeah.

   26. DPDwarkesh Patel1:02:52

       Like, it tells you the nature, about like the nature of the skill.

   27. EEEge Erdil1:02:55

       Right.

   28. DPDwarkesh Patel1:02:55

       Um, how'd you find him?

   29. TBTamay Besiroglu1:02:56

       We, uh, we connected on like some, uh... On a Discord for Metaculus, which is this-

   30. DPDwarkesh Patel1:03:02

       Forecasting platform.
5. 1:06:58 – 1:33:34

   Explosive economic growth

   1. DPDwarkesh Patel1:06:58

      Ege and Tamek. Okay, so let me ask you about this. I can poke you from the b- I, I... So just- just to set the scene for the audience, um, we're gonna talk about, um, the possibility of this explosive economic growth and like greater than 30% economic growth rates. So, I wanna poke you both from the perspective of maybe suggesting that...... this isn't aggressive enough in the right kind of way because it's, maybe it's too broad. And then I'll poke you in the, from the perspective of, like, the more normal perspective that, hey, this is fucking crazy.

   2. EEEge Erdil1:07:29

      I imagine it would be difficult for you to do the second thing. (laughs)

   3. TBTamay Besiroglu1:07:34

      (laughs)

   4. DPDwarkesh Patel1:07:34

      (laughs) No, I mean, like, I think it might be fucking crazy. Let's see. Um, the big question I have about this broad automation, like I get what you're saying about the industrial revolution, but in this case, we can just make this, like, argument that you get, uh, you, you get this intelligence and then what, what you do next is you go, like, to the desert and you build this, like, Shenzhen of robot factories, which are building more robot factories, which are building... If you need to do experiments and you build bio labs and you build chemistry labs and whatever, they're labs-

   5. EEEge Erdil1:08:04

      Well, if you build Shenzhen in a, the desert, I agree, that looks much more plausible than a software-only singularity. Uh...

   6. DPDwarkesh Patel1:08:10

      But, but why... But you, the way you're framing it, it sounds like McDonald's and Home Depot and fucking whatever are growing at 30% a year as well, and not just, like-

   7. TBTamay Besiroglu1:08:22

      (laughs)

   8. DPDwarkesh Patel1:08:22

      ... is it the, you know, the aliens-level view of the economy, is it that, like, there's a robot economy in the desert that's growing at 10,000% a year and everything else is the same old same old? Or is it like-

   9. EEEge Erdil1:08:32

      No.

   10. DPDwarkesh Patel1:08:32

       You know what I mean?

   11. EEEge Erdil1:08:33

       I, I mean, there is a question about what would be, uh, possible or physically possible and what would be the thing that would actually be efficient, right?

   12. DPDwarkesh Patel1:08:42

       Yeah.

   13. EEEge Erdil1:08:42

       So it might be, it might be the case, and again, once you're scaling up the hardware part of the equation as well as the software part, then I think the case, uh, for this feedback loop gets a lot stronger. Uh, if you scale up data collection as well, I think it gets even stronger, like real world data collection by deployment and so on. Uh, but building Shenzhen in a desert, that's a pretty, like, like if you look, if you think about the, um, pipeline, so, so far we have relied, uh, first of all we're relying on the entire semiconductor supply chain. That industry depends on tons of inputs and materials and whatever it gets from probably tons of random place in the world. And creating that infrastructure, like doubling or tripling, whatever, that infrastructure, it, like the entire thing, that's very hard work, right? So probably you couldn't even do it even if you just have Shenzhen in a desert. Like that will be even more expensive than that. On top of that, so far, we have been drawing heavily on the fact that we have built up this huge stock of data, uh, over the past 30 years or something on the internet. Um, like imagine you were trying to train a state-of-the-art model, but you only have like 100 billion tokens, right, to train on. That, though that would be very difficult. So in a certain sense, we, our entire economy, uh, has produced this huge amount of data on the internet that we are now using to train the models. Uh, it's plausible that in the future when you need to get new competencies added to these systems, uh, the most efficient way to do that will be to try to leverage similar kind of modalities of data, which will also require this, like, like you would want to deploy the systems broadly because that's going to give you more data. And maybe you can do the, maybe you can get where you want to be without that, but it would just be less efficient if you're starting from scratch compared to if you're collecting a lot of data. Like I think this is actually a motivation for why labs want their, uh, LLMs to be deployed widely, because, like, sometimes when you talk to ChatGPT it's going to give you two responses and it's going to say, "Well, which one was good?" Or like it's going to give you one response and it's going to ask you, "It was as good or not?" Well, why are they doing that, right? That's a way in which they are getting, uh, user data, uh, through this extremely broad deployment. So I think you should just imagine that thing to be, continue to be efficient and continue to increase in the future because it just makes sense. And then there's a separate question of, well, suppose you didn't do any of that. Like suppose you just tried to imagine the most rudimentary, the most narrowest possible kind of infrastructure build out and deployment that would be sufficient to get, uh, this positive feedback loop that leads to, like, much more efficient AIs. I agree that loop could, in principle, be much smaller than the entire world. Uh, I think it couldn't, probably couldn't be as small as Shenzhen in a desert, but it could be much smaller than the entire world. But then there's a separate question of would you actually do that? Would that be efficient? I think some people have the intuition that there are just these, like, extremely strong, uh, constraints, maybe regulatory constraints, maybe sociopolitical constraints-

   14. DPDwarkesh Patel1:11:47

       Yeah.

   15. EEEge Erdil1:11:47

       ... to doing, like, doing this broad deployment.

   16. DPDwarkesh Patel1:11:50

       Yeah.

   17. EEEge Erdil1:11:50

       They just think it's going to be very hard. So I think that's part of the reason why they imagine these, like, more narrow scenarios-

   18. DPDwarkesh Patel1:11:56

       Yeah.

   19. EEEge Erdil1:11:56

       ... where they think it's going to be easier. But I think that's, like, I think that's overstated. Uh, I think people's intuitions for, like, how hard this kind of deployment is comes from cases where the deployments of the technology wouldn't be, like, that valuable. Uh, so may- it might come from housing, like, we have a lot of regulations in housing, maybe this comes from nuclear power, maybe this comes from supersonic flights. Uh, I mean, those are all technologies that would be useful if they were, like, maybe less regulated, but it, they wouldn't, like, double, uh, economic output.

   20. TBTamay Besiroglu1:12:28

       I, I think the, the core point here is just that the value of AI automation and deployment is just extremely large.

   21. EEEge Erdil1:12:34

       Yeah.

   22. TBTamay Besiroglu1:12:34

       Um, even just for workers, uh, at least the ones that, um, you know, at least after, you know, finding, you know, there might be some, uh, kind of displacement and there might be some tran- transition that you need to do in order to find a job that, that works for you, but, but otherwise the wages could still be very high for, for, for a while at least. And, and on top of that, the gains from owning capital might be very enormous. And in fact, a large share of the US population would benefit from this thing-

   23. EEEge Erdil1:13:04

       Yeah, they own a lot of housing, for example.

   24. TBTamay Besiroglu1:13:05

       ... you know, they benefit, they own housing, they own, they have 401 (k) s, those who do, you know, enormously better when you have this process of broad automation and AI deployment, and so I, I think there's, um, there, there could just be a very deep support for some of this, even when it's, like, totally changing the nature of labor markets and, you know, the, the skills and, and, and occupations that are in demand.

   25. EEEge Erdil1:13:30

       Yeah. So I would just say it's, like, complicated. I, I think the, like, what the political reaction to it will be when this starts actually happening.... I think, like, the easy thing to say is that, "Yeah, like, this will become, like, a big issue, and then the issue will be maybe controversial or something." But, like, what is the actual nature of the reaction in different countries? I think that's kind of hard to forecast. Like, I think the default view is like, "Well, people are gonna become unemployed, so it will just be very unpopular." I think that's, like, very far from obvious.

   26. DPDwarkesh Patel1:13:57

       Yep.

   27. EEEge Erdil1:13:58

       Uh, and I just expect heterogeneity in how different countries respond, and some of them are gonna, like, be more liberal about this and gonna allow broader deployment, and those countries probably end up doing better. So just like during the Industrial Revolution, some countries were just ahead of others. I mean, eventually, almost the entire world adopted the sort of norms and culture and values of the Industrial Revolution in various ways, but-

   28. DPDwarkesh Patel1:14:18

       And actually, uh, you, you say they might be more liberal about it, but they might actually, like, be less liberable, l- they might be less liberal in many ways, and-

   29. EEEge Erdil1:14:27

       Right.

   30. DPDwarkesh Patel1:14:27

       ... in fact, that might be, like, more functional in this world in which you have broad AI deployment.

Episode duration: 3:09:03