Dwarkesh Patel and Noah Smith on AGI and the Economy

Erik Torenberg 0:00 Are you dubious of the trope that, you know, labor provides meaning, and if people don't have a clear, uh, sense for labor, then it will be very difficult for them to obtain alternative sources of meaning?

Speaker 0:09 Humans have just adapted to so much.

Dwarkesh Patel 0:11 Agricultural revolution, Industrial Revolution, the growth of states. Once in a while, like a communist or fascist or good regime will come around or something. Like, the idea that being free and having millions of dollars is the thing that finally gets us, I'm just suspicious of.

Erik Torenberg 0:32 Dwarkesh,

Erik Torenberg 0:33 Noah, welcome. Our first podcast ever together as a trio.

Speaker 0:37 Yes. Excited.

Dwarkesh Patel 0:38 I'm very excited.

Erik Torenberg 0:39 So Dwarkesh, it's almost... You, you came out of the Scaling Era.

Dwarkesh Patel 0:42 Uh-huh.

Erik Torenberg 0:42 It's almost like you're a future historian. You're sort of telling the history as it's, as it's being, as it's being written, and so it's, it's only appropriate to ask you, what is your definition of, of AGI?

Dwarkesh Patel 0:53 Mm.

Erik Torenberg 0:53 And, and how has that evolved over time?

Dwarkesh Patel 0:55 Um-

Erik Torenberg 0:56 Except for, like, superintelligence, you know. Brings it down for us.

Dwarkesh Patel 0:58 I feel like I'm like five decades too young to be a historian.

Erik Torenberg 1:00 [laughs]

Dwarkesh Patel 1:00 You gotta be, like, in your 80s or something-

Erik Torenberg 1:02 Yeah. [laughs]

Dwarkesh Patel 1:02 ... before you could, um-

Erik Torenberg 1:03 Exactly

Dwarkesh Patel 1:03 ... uh, um-

Erik Torenberg 1:05 But we're living in history right now.

Dwarkesh Patel 1:06 So the ultimate definition is can do almost any job, say, like, 98% of jobs at least, as well, fast, uh, cheaply as a human. I think the more, um... The definition that's often useful for near-term debates is can automate 95% of white-collar work because there's a clear path to get to that, whereas robotics, you know, there's, like, a, a long tail of things you gotta do in the physical world, and robotics is slower. So automate fi- uh, white-collar work.

Speaker 1:32 Hmm. So it's, that's interesting 'cause it's an economic definition. It's not sort of a definition about, like, how it thinks, how it reasons-

Dwarkesh Patel 1:38 Mm-hmm

Speaker 1:38 ... et cetera. It's about, like, what it can do.

Dwarkesh Patel 1:40 Yeah. I mean, we've, we've been surprised what capabilities have come first in AI. Like, they can reason already. Um, and h- why they seem to lack the economic value we would've assumed would correspond to that level of capability. Um, this thing can reason, but it's making OpenAI $10 billion a year, and McDonald's and Kohl's make more than $10 billion a year, right? Um, so clearly there's m- m- more things relevant to automating entire jobs than we previously assumed. So then it's just useful to, like... W- who knows what all those things are, but once they can automate it, then it's AGI.

Erik Torenberg 2:15 And so when Ilya or Meta is using the word superintelligence, what, what, what do they mean? Do they mean the same, the same thing or something totally different?

Dwarkesh Patel 2:22 Um, I'm not sure what they mean. Like, there's a spectrum between God and just something that thinks like a human but much faster. Um, yeah, m- uh, uh, do you have some sense what the, you think they mean?

Speaker 2:36 God.

Dwarkesh Patel 2:36 [laughs]

Speaker 2:36 I think probably they, they

Speaker 2:38 mean something they would worship-

Dwarkesh Patel 2:39 Yeah

Speaker 2:39 ... as a god.

Dwarkesh Patel 2:39 Yeah.

Erik Torenberg 2:40 And so when Tyler says we've achieved AGI and, and you differ from him, wh- wh- where is the tangible difference there?

Dwarkesh Patel 2:46 Um, I'm just noticing that if there, there was a human who was working for me, they could do things for me that these models cannot do, right? And I'm not talking about something super advanced. I'm just saying, "I'm, I have transcripts from my podcast. I want you to rewrite them the way a human would, and then I'll give you feedback about what you messed up, and I want you to integrate that feedback as you get better over time, you learn my preferences, you learn my content." And they actually don't, they can't, like, learn over the course of six months how to become a better editor for me or how to become a better transcriber for me. And since they c- a human I, I hire would be able to do this. They can't, so therefore it's not AGI.

Speaker 3:17 Noah, I have a question. I am a natural general intelligence. You are a natural general-

Dwarkesh Patel 3:21 Mm-hmm

Speaker 3:21 ... intelligence, but we cannot easily do each other's jobs even though our jobs are fairly similar.

Dwarkesh Patel 3:25 Right.

Speaker 3:26 Um, put me in the Dwarkesh Podcast and I could not interview people nearly so well. If you had to write Substack, you know, articles, like, several times a week-

Dwarkesh Patel 3:34 Yes

Speaker 3:34 ... on economics, you might not do as well. So then, um, but we are general intelligences, and we're not exactly substitutable, so why should we use substitutability as the criterion for AGI?

Dwarkesh Patel 3:44 I mean, what, what else is it that we want them to do? I think w- with humans, we have more of a sense of, like, there's some other human who theoretically could do what you would do. A model is... An individual copy of a model might be, say, fine-tuned to do a particular job, and it would be fair to say then why expect this particular fine-tune to be able to do any job in the economy? But then there's a question of, well, there's many different models in the world, and each model might have many different fine-tunes or many different, um, instances. Any one of them should be able to do a particular white-collar job for it to count as AGI. I'm not... It's not that, like, every, any AGI should be able to do every single job, that, like, some artificial intelligence should be able to do this job for, like, this model to count as AGI.

Speaker 4:23 I see. Okay. But, so let's take another similar example. Let's take Star Trek.

Dwarkesh Patel 4:26 Yeah.

Speaker 4:26 Okay, you got Spock.

Dwarkesh Patel 4:27 Very Spock. [laughs]

Speaker 4:28 He's very logical. He can do stuff that the, that, you know-

Dwarkesh Patel 4:30 Yeah

Speaker 4:30 ... Kirk and whoever can't do. But then those guys can do stuff that Spock can't do.

Dwarkesh Patel 4:34 Right.

Speaker 4:34 Like get in touch with their emotions, intuition, stuff like that. They're both general intelligences, but they're alien to each other. So, you know, AI feels alien to me.

Dwarkesh Patel 4:42 Mm-hmm.

Speaker 4:42 It, sometimes it, it talks just like us. It, it was built off of our thoughts obviously. But then, um, you know, sometimes it talks just like us, and sometimes it's just, like, very alien. And so, but, but should we ever expect that to change such that it's n- it's no longer an alien intelligence?

Dwarkesh Patel 4:59 I think it'll continue to be alien, but I think eventually we will gain capabilities which are necessary to unlock, um, the trillions of dollars of economic value that are implied by automating human labor, which these models are clearly not generating right now. Um, so you could say, well, like, if I just put, if, if we substituted jobs right now, immediately there'd be a huge productivity dip. But over time, we would learn to do s- you know, start doing them better. I mean, maybe a better example is just that, like, you hire people to do things for you. I, I don't know if you actually hire people, but I, I assume-

Speaker 5:31 Occasionally, yeah.

Dwarkesh Patel 5:32 Okay. Like, why are, why are you still having to do that rather than hiring an AI? And I have, like, many rules where it's like an AI might be generating hundreds of dollars of value for me a month, but, like, humans are generating thousands of dollars or tens of thousands of dollars of value for me a month. Um, like, why is that the case? And I think it's just, like, the capab- AI's are lacking these capabilities. Humans have these capabilities.

Erik Torenberg 5:50 And is the main thing missing in your view sort of continual learning?

Dwarkesh Patel 5:54 The reason humans are so valuable is not just their raw intellect. It's not mainly their raw intellect, although that's important. It's their ability to build up context. It's to interrogate their own failures and pick up small efficiencies and improvements as they practice a task. Um, whereas with an AI model, its understanding of your problem, your business, will be expunged by the end of a session.Um, and then you're just, you're starting off at the baseline of the model. Like, with a human, you gotta train them over many months to make them useful employees.

Speaker 6:22 What will need to happen for that to be ch- ch- change-- Like, what, what needs to change in order for us to develop, for AI to develop that capability?

Dwarkesh Patel 6:29 Hmm. I mean, um, I, I, I probably wouldn't be a podcaster if I knew the answer-

Speaker 6:34 [laughs]

Dwarkesh Patel 6:34 ... to that question. [laughs] Um, uh, I, I, it just seems to me that, like, a lot of the modalities that we have today to teach LLM stuff do not constitute this kind of continual learning. For example, making the system prompt better is not the kind of continual learning that, or on-the-job training that my human em- employees experience, or RL fine-tuning is not this. But I, like, what the solution to this looks like, it's precisely because I don't have an obvious-

Speaker 6:58 Yeah

Dwarkesh Patel 6:58 ... solution that I think we're many years

Dwarkesh Patel 7:00 away.

Speaker 7:00 Okay, so here's my question about replacing jobs. Uh, you know, it seems to me that it's partly about demand.

Dwarkesh Patel 7:06 Mm-hmm.

Speaker 7:06 So for example, suppose that AI has already replaced my job-

Dwarkesh Patel 7:09 Yeah

Speaker 7:10 ... or, or can replace my job, so that suppose that anyone who goes onto, you know, fires up ChatGPT or whatever they want-

Dwarkesh Patel 7:16 Yeah

Speaker 7:16 ... or whatever model and says, "Search the web, find the most interesting topics that people are talking about, economics, and write me an insightful post telling me some cool new thing I should think about that."

Dwarkesh Patel 7:25 Yeah.

Speaker 7:25 And they just do that every day, and then they get a better blog than No Opinion.

Dwarkesh Patel 7:29 Yeah.

Speaker 7:29 I don't know if that's happened yet. I, I mean, I've tried that and I don't like it as much. But, but, but suppose that most people would like it as much, and so my job's been automated and people just don't realize it, or people have this sort of, like, idea in their mind of like, "Well, is it really a human?" and blah, blah, blah, and then j- uh, as generational turnover happens, young people won't care about reading a human. They'll care about-

Dwarkesh Patel 7:45 Right

Speaker 7:45 ... reading an AI. But in terms of functional capabilities, it's already there. But in terms of, of demand, it's not there. Uh, how much of that could there be?

Dwarkesh Patel 7:53 I expect there would be much less of that than people assume. If you just look at the example of Waymo versus Uber, I think previously you could have had this thing about people will hesitate to take automated rides. And in fact, in the cities where it's been deployed, people, like, love this product despite the fact that you gotta wait 20 minutes because the demand is so high. Um, and it's still, like, it got some glitches to iron out. But just, like, the seamlessness of using machines to do things for you, the fact that it can be, like, personalized to you, it can happen immediately, uh, I, I, I mean, it was like, okay, if it's like one thing people would be like, "Okay, well, doctors and lawyers will set up guilds, and so you won't be able to consult." Um, I think there might be guilds in who can call themselves a doctor or a lawyer, but I just think if, like, if genuinely ChatGPT can give me as good medical advice as a real doctor, the experience of just talking to a chatbot rather than spending three hours in a waiting room is so much better that I, I think a lot of sectors of the economy look like this, where we're like, we're assuming people will care about having a human, but in fact they will not. If, if you assume that they will genuinely have the capabilities that the human brings to bear.

Speaker 8:51 So it's interesting, you know, AI is, is better for diagnosis on a lot of things than, than humans.

Dwarkesh Patel 8:58 Mm-hmm.

Speaker 8:58 Right? But then, um, something about having humans to follow up with makes me also want to check with a human-

Dwarkesh Patel 9:05 Yeah

Speaker 9:05 ... after I've gotten diagnoses from an AI on something. And so it might, that might vary by job. Like, cars may be one thing, and then, um, but, but maybe it is about capabilities. I, I can't say. I'm just saying, like, I, I'm, I'm, I'm saying everybody seems to think that the, that AI is a perfect substitute for humans, and that's what it should be, and that's what it will be, and everyone seems to think of it in that case. However, every other tool that's ever been made, every other technological tool, was a complement to humans. It could do some things human could, humans could do. Maybe even it, it could do anything humans could do, but at different relative costs, different relative prices.

Dwarkesh Patel 9:39 Mm-hmm.

Speaker 9:39 So that you'd have humans do something and the tool do other things, and you'd have this complementarity between the two. And yet when people talk about AI and think about AI, they essentially never seem to think in these terms. They always seem to think in terms of perfect substitutability. And so I'm trying to get to the bottom of, like, why people insist on always thinking in terms of perfect substitutability when every other tool has been, uh, you know, complementary in the end.

Dwarkesh Patel 10:01 Well, human labor is also, um, complementary to other human labor, right? There's increasing returns to scale. Um, but that doesn't mean that there's, like, uh, you know, Microsoft has to hire some number of software engineers and, like, it will care about the cost of what the software engineers cost. Like, it will go to markets where they're, they can get the highest performance for the relative value those software engineers are bringing in. I think it'll be a similar story with AI labor and human labor. And the AI labor just has the benefit of having extremely low subsistence wages. Like, the marginal cost of keeping an H100 running is much lower than the cost of keeping a human alive for a year.

Speaker 10:37 Noah, would you say you're AGI-pilled to, in, in the sense that Dwarkesh described the term? And how do you u- understand, you know, we've talked a little bit about AI's effect on labor. Why don't you share your, wh- why you're perhaps a little bullish that there'll be a, you know, plenty for, for humans to do and that'll be more complementary?

Dwarkesh Patel 10:53 What is AGI-pilled?

Speaker 10:55 Well, just believe in d- d- Dwarkesh's, that it'll automate a huge swath of the economy. I mean-

Dwarkesh Patel 11:01 Or, or labor

Speaker 11:01 ... I'm, I am very unwilling to say, like, "Here's something technology will never be able to do."

Dwarkesh Patel 11:06 [laughs]

Speaker 11:06 I mean, that always seems like a bad bet. Here's two things people have been saying since the beginning of the Industrial Revolution, neither of which has ever remotely come close to being true, even in specific subdomains. Um, the first one is, "Here's a thing technology will never be able to do." And the second one is, "Human labor will be made obsolete." Those, people have been saying those two things, and you can just go, you can read it, you can even, you know, ask AI to go-

Dwarkesh Patel 11:32 [laughs]

Speaker 11:32 ... search and find, I have done this, and, and, and, and find you examples of people saying those two things. People have been saying those two things over and over and over and over and over, and it's never been true. That doesn't mean it could never be true. Sometimes something happens that never happened before, such as the Industrial Revolution itself. You know, you have this hockey stick where suddenly, like, like, "Oh, we'll never get rich. We'll never get rich. Oh, we're rich."

Dwarkesh Patel 11:50 [laughs]

Speaker 11:50 And so, you know, sometimes-

Dwarkesh Patel 11:51 Right

Speaker 11:52 ... things like that happens. Un- the unprecedented can happen. However, I'm always wary because I've seen it said so many times. And so, you know, within just the last 10 years or, or, or whatever, I've seen a couple predictions just spectacularly fail. So for example, in, in 2015, 10 years ago, I was sitting in the Bloomberg office in New York, and my colleague, um, was, w- who I, I won't name. But he was, he was physically yelling at me that truck drivers were in trouble and, you know, that truck drivers were all gonna be put out of a job by self-driving trucksAnd he just, he said this is gonna just devastate a sector of the economy. It's gonna devastate the working class, it's gonna devastate blue collar labor, blah, blah, blah. And I, at the same time I was reading, like, I, I always read the sci-fi, you know, top stories of the year, whatever. And so there were two stories in the same year about truckers being mass unemployed by, you know, self-driving trucks. And then 10 years later there's a trucker shortage and the number of truckers we hire is higher than ever.

Dwarkesh Patel 12:42 Mm-hmm.

Speaker 12:42 I'm not saying truckers will never be automated. They may. But however, I'm saying that was a spectacularly wrong prediction, and you also got Geoffrey Hinton's prediction that radiologists would be unemployed within a certain timeframe, and by that time radiologists' wages were higher than ever and employment was higher than ever.

Dwarkesh Patel 12:54 Yeah.

Speaker 12:55 I'm not saying this can't happen. I'm not smugly sitting here and saying there's a law of the universe that says you'll never see this kind of mass unemployment, blah, blah, blah. There, there... I mean, there were encyclopedia salespeople were mass unemployed by the internet. We've seen it happen in r- like, in real life, but these predictions keep coming wrong and keep coming wrong, and I'm trying to figure out why, why is that true? Why do they keep coming wrong? Is it simply that people overestimate progress in technical capabilities, or are there complementarities that people can't imagine from sort of, like, the O*NET division of tasks or the standard mental division of tasks?

Dwarkesh Patel 13:24 Mm. I, I think the b- b- the problem has been that people underestimate how many things are truly needed to automate human labor, and so they think, like, we've got reasoning, and now that we've got reasoning we've, like... Th- this is what it takes to take over a job, when I think in fact there's much more to a job than is assumed. Um, that's why I, you know, wrote this blog post where I'm like, "Look, it's not a couple years away. It might be longer than that." Then there's another question of, like, by 2100 will there be, um, jobs that humans are doing? Um, like, if you just, like, zoom out long enough, will we ever be able to make machines that can think and do physical labor, um, at least as cheaply and as well as humans can? And fundamentally the, the big advantage they have is, like, we can keep building more of them, right? So we make as many of those machines as the cost of producing them... Or sorry, the value they generate equals the cost of producing them. Um-

Speaker 14:17 And the cost will continue to go to, go to ga- go down.

Dwarkesh Patel 14:19 Right. Yeah. And it will be lower than the cost of, like, keeping a human alive. So if, like, even if a human could do the exact same labor, a human needs, like, a lot of stuff to stay alive, let alone, you know, to grow a human, everything. Um, an H100 costs $40,000 today. The u- the yearly cost of running it is, like, thousands of dollars. We can just buy more H100s, right? Again, i- i- if currently we had the algorithm for AGI, we could run it on an H100, um, and yeah.

Speaker 14:45 So however big the demand is, l- the latent demand-

Dwarkesh Patel 14:48 Right

Speaker 14:48 ... uh, that, that's unlocked by, by the more sup- we just increase the supply, basically-

Dwarkesh Patel 14:51 Right

Speaker 14:51 ... to meet, to meet the demand.

Dwarkesh Patel 14:52 Yeah.

Speaker 14:52 So first, um, what, why don't you sort of... When, when AGI is here, what does the, what does the world look like? Because,

Speaker 15:00 you know, Sam Altman was reflecting on his podcast with, uh, with Jack Altman the other week. He was saying, "You know, if you told me 10 years ago that we would have, um, you know, PhD level, um, you know, A- AI, I would think the world looks, looks a lot different."

Dwarkesh Patel 15:13 Right.

Speaker 15:13 But in fact it, it, it doesn't look that different.

Dwarkesh Patel 15:15 Yeah.

Speaker 15:15 And so is there, is there a potential where we, uh, have, you know, much, much more increased capabilities but actually the world does... It's like the, you know, Peter Thiel called the 1973 test or something. It's like we have these phones, but the world just looks-

Dwarkesh Patel 15:26 Right

Speaker 15:26 ... looks the same. We just have phones in our pockets.

Dwarkesh Patel 15:29 Yeah. I, I think if we have, like, chatbots that can answer hard math questions, I don't expect the world to look that different because the fraction of economic value that is generated by math is, like, extremely small. Um, I... But there's, like, other jobs that are much more mundane than quote-unquote PhD intelligence, which these... A chatbot just cannot do, right? A chatbot cannot edit videos for me. Um, and once those are automated, I actually expect a pretty crazy world because, uh, the big bottleneck to growth has been that human population can only increase at this slow clip. And in fact s- you know, one of the reasons that growth has slowed since the '70s is that in developing countries the population has, uh, plateaued. With AI, the popula- like, the capital and the labor are functionally equivalent, right? You can just, like, build more data centers or build more robot factories, and they can do real work or they can build more robot factories, and so you're gonna have this explosive dynamic. And once we get, like, that loop closed, I think it would just be, like, 20% growth plus.

Speaker 16:29 Do you see that feasible, possible, 20% growth? And T- Tyler I believe said 5%, right? He, he-

Dwarkesh Patel 16:34 0.5% more than the steady state.

Speaker 16:36 0.5%.

Dwarkesh Patel 16:36 Oh, yeah.

Speaker 16:37 0.5%.

Dwarkesh Patel 16:37 More.

Speaker 16:38 And, and, and, and what's the argument? Just that, um... Yeah, what, what is the argument for that?

Dwarkesh Patel 16:42 For Tyler's argument? Uh, bottlenecks. I think the problem with that argument is that, like, there's always bottlenecks, right? So you could have said before the Industrial Revolution, well, we will never 10X the rate of growth because there will be bottlenecks, and that doesn't tell you what... Like, you empirically have to just, like, look at the fraction of the economy that'll be bottlenecked and, like, what is the fraction that's not and then, like, actually derive the rate of growth. Um, the, the, the, the, the fact that there's bottlenecks doesn't tell you... Like, yeah, okay, there will be, like-

Speaker 17:06 Is he mostly referring to, uh, regulation? Or, or-

Dwarkesh Patel 17:09 Yeah, and just that, like, we live in a fallen world and people will have to use the AIs and there... Yeah, things like that.

Speaker 17:16 Who'll be buying

Speaker 17:17 all the stuff? So, so background, in economics GDP is what people are willing to pay for.

Dwarkesh Patel 17:23 Right.

Speaker 17:23 Who will be buying the stuff in a world where we get 20% growth?

Dwarkesh Patel 17:26 Uh, first of all, I don't know. You could have said, um, in 10,000 BC, like, the economy's gonna be a billion times bigger in, um, in 10,000 years. Like, what does it mean to produce a billion times more stuff than we're producing right now? Who is buying all this, like, stuff? And it just, like, you, you can't, like, predict that in advance.

Speaker 17:41 In 1700s I could tell you exactly who was buying stuff. It was everybody. You know, peasants. Like, I could tell you, like... You know, we, we, there... In fact, people wrote these, these things in the, about, around 1900 about what the world would look like in 100 years. You know-

Dwarkesh Patel 17:54 Yeah

Speaker 17:54 ... what, what, what we'll have. They didn't get exactly the right things-

Dwarkesh Patel 17:57 Right

Speaker 17:58 ... right that we'll have, but they correctly identified that it would be regular consumers who would be buying all these things, regular people. And so that would, that came true. It was obvious. But here's my point. Here's my point. Suppose that 99% of people do not have a job and are not getting paid an income-

Dwarkesh Patel 18:12 Yeah

Speaker 18:12 ... and all the money is going to sort of-Sam Altman, Elon Musk, and like five other guys.

Dwarkesh Patel 18:16 Right.

Speaker 18:17 Okay? And, and, and their captive AIs that they own because for some reason our property rights system still exists.

Dwarkesh Patel 18:22 Right.

Speaker 18:22 But okay, suppose that that's, that's the future we're contemplating, right? And so 99% of people or more don't have any job, they don't have any income, they're out on the street, and yet you're saying 20% growth a year. That growth is defined by people, consumers paying for things and saying, "Here is-

Dwarkesh Patel 18:39 And I wouldn't define it just-

Speaker 18:41 ... here's my money. Take my money."

Dwarkesh Patel 18:41 ... just as people.

Speaker 18:41 Okay, so then-

Dwarkesh Patel 18:42 I would just define it as like the, the ra- I mean, I-

Speaker 18:44 We will have AI producing agents

Dwarkesh Patel 18:46 ... assume the AIs are creating each other. Yeah, and, and that was like-

Speaker 18:47 No, no, that doesn't, that doesn't count in GDP. Only final good- only final consumer spending

Dwarkesh Patel 18:49 Okay, so we're like launching the Dyson spheres. We're not allowed to count that because like the AIs are doing it. I mean, like I wanna, I wanna know what the solar system will look like. I don't care like what like the semantics of that are, and I think the better way to capture what is physically happening is just like including the AIs when calculating numbers.

Speaker 19:02 Why will they do that? Why will they do that? Why will they do any of that?

Dwarkesh Patel 19:05 One argument is simply that if there's any agent, AI or human, who cares about colonizing the galaxy, um, even if like 99% of agents don't care about that, if like one agent cares, they can go do it. Colonizing the galaxy is a lot of growth 'cause the galaxy's really big, right? So it's very easy for me to imagine if like Sam Altman decides to launch the probes, how like, you know, breaking down Mars and sending out the, the, the virus probes like is like generates 20% growth.

Speaker 19:28 I think what you're getting at here is that AI will have to have property rights. AI, AI agents will have to be able to have-

Dwarkesh Patel 19:35 But even if a human wants to use AI-

Speaker 19:36 ... autonomous control of resources

Dwarkesh Patel 19:37 ... even if, even I, I guess it w- de- it depends on what you mean by autonomous. Today we already have-

Speaker 19:41 I mean, without getting into say that-

Dwarkesh Patel 19:42 ... computer programs that have autonomous use of resources, right?

Speaker 19:44 Okay, but the program goes off and colonizes the solar system.

Dwarkesh Patel 19:46 Right.

Speaker 19:47 It, it's not like a, a dude telling it, "Colonize the solar system now," and, and doing all this stuff. It's like the AI has made the decision to do it, and Sam Altman's sitting back there saying, "Oh, well, you know, you can do what you want."

Dwarkesh Patel 19:56 I'm just saying this is not a crux.

Speaker 19:58 You know?

Dwarkesh Patel 19:58 Like if Sam Altman could say it or the AI could say it, like if, if some agent cares about this-

Speaker 20:03 And they're not stopped from doing it

Dwarkesh Patel 20:05 ... this doesn't stop from doing it. Like this is just like physically you can easily see where the 20% growth is coming from.

Speaker 20:08 Let me make this a little more concrete. Suppose that AI is gonna produce a bounty of the things that humans desire, and that's gonna be what growth is. How will it get to the humans if the humans don't have a job? And if the humans don't have a job, why will AI be... So in other words, if there's no consumers to buy my cars, why am I building cars? You, you might be assuming there's some UBI or some sort of-

Dwarkesh Patel 20:30 No, no, I don't need to assume that. Although, I mean, uh-

Speaker 20:32 Let's assume there's not that.

Dwarkesh Patel 20:33 Yes. I, I don't need to assume that. It seems like you're saying, look, if like 99% of consumers are no longer consumers, where is this economic activity coming from?

Speaker 20:40 Yeah.

Dwarkesh Patel 20:41 And I'm just saying, okay, if like one person cares about colonizing the galaxy-

Speaker 20:46 Right

Dwarkesh Patel 20:46 ... that's generating a lot of demand. It takes a lot of stuff to colonize a galaxy.

Speaker 20:50 Okay.

Dwarkesh Patel 20:50 So like the, this world where like every, even if there's like not in the galaxy or in a world where everybody's like roughly contributing equivalent amounts of demand, um, the potential for one person alone to generate this demand is h- so high enough that like-

Speaker 21:01 So Sam Altman tells-

Dwarkesh Patel 21:02 I'm not saying this is-

Speaker 21:02 Sam Altman tells his infinite army of robots to go out and colonize the galaxy. We count that as consumption. We put a value on it, and that's GDP.

Dwarkesh Patel 21:08 Yeah, or like you could... It, it might be investment. Maybe he's like gonna defer his consumption-

Speaker 21:11 Investment

Dwarkesh Patel 21:11 ... to once he's like after colonizing the galaxy.

Speaker 21:13 Investment. Gotcha.

Dwarkesh Patel 21:13 Yeah.

Speaker 21:14 No, investment. That's it.

Dwarkesh Patel 21:14 And I'm not saying this is the world I want. I'm just saying like, just like think about it physically. If you're colonizing the galaxy, which we can do p- potentially after AGI, I mean, I'm not saying like it'll happen tomorrow after AGI, right? But like this is a thing that's physically possible. Is that growth? Like, like something's happening that's like explosive.

Speaker 21:29 Maybe. The, the thing is that, um, it's a very weird world. It doesn't look like the kind of economy we've ever had.

Dwarkesh Patel 21:36 Right.

Speaker 21:36 And it's not, you know, we created the notion of GDP to represent, you know, people p- exchanging money for goods and services, people like basically exchanging their labor for goods and-

Dwarkesh Patel 21:46 Right

Speaker 21:46 ... exchanging the value of their labor for goods and services. That's at a fundamental level, that's what GDP is. We're envisioning a radical shift of what GDP means to a sort of internal pricing that a few overlords set for the things that their AI agents want to do, and that's incredibly different than what we've called GDP in the past.

Dwarkesh Patel 22:02 Um, I mean, I, I think the economy will be, yeah, incredibly different from what, what was, was in the past. I, I wanna say that I'm not saying this is like the, the modal world. There's a couple of reasons why this might not end up happening. One is even if your labor is not worth that much, um, the property you own is p- potentially worth a lot, right? Like if you own the S&P 500 and there's been explosive growth, you're like a multi-multimillionaire or the land you have is like worth a lot if like the-

Speaker 22:29 Hmm

Dwarkesh Patel 22:30 ... AI can make such good use of that land to build the space probes, um, a- assuming the, our system of property rights continues into this regime. Um, and second, so w- I mean, in many cases it's hard to ascribe how much economic growth there has been over very long periods of time. For example, um, over 500 year... Like if you're comparing the basket of goods that we can produce as an economy today versus like 500 years ago, um, it's like not clear how you compare what you w- there's like we have antibiotics today. I wouldn't wanna go back 500 years for any amount of money because they don't have antibiotics, and I might die, and it'll, it'll just suck. So there's actually like no amount of money, um, in like 1,500 bucks to live in 1500 that we would rather have than live today. And so if we have those quality of goods for normal people, just like, you know, you can live forever, you have like euphoria drugs, whatever. These are things we can imagine now. Hopefully it'll be even more compelling than that. Then it's easy to imagine like, okay, it makes sense why this stuff is worth way more than the stuff that the world economy can produce for even for normal people today.

Speaker 23:29 Right. Right. Yeah. And so I, I guess I'm just thinking about, um, th- this, this is a thing that economists really struggled with in the early 20th century. It's this idea that, uh, you know, we had this capacity to expand production, expand production, expand production, and then the thing is that companies competed their profits to zero, and the profits crashed, and nobody wanted to expand production anymore 'cause they weren't making any profit. We're seeing this happen again in China right now-

Dwarkesh Patel 23:51 Mm-hmm

Speaker 23:52 ... with overproduction. We're seeing BYD having to take loans from its suppliers just to stay financially afloat even though it's the best car company in the world because the Chinese government has paid a million other car companies-

Dwarkesh Patel 24:03 Right

Speaker 24:03 ... to compete with BYD. And so you compete, you know, you overproduce, and so you have this overproduction. The question is, uh, so, so the, the solution was to expand consumption. This is the solution people are recommending for China now-

Dwarkesh Patel 24:13 Yeah

Speaker 24:13 ... to expand consumption so that you can refloat the profit margins of all these companies and have them continuous companies. And so the idea is if AI's producing all this stuff but it's overproducing services, it's overproducing whatever AI can produce, you know, and then-That will-- And, and the profits from this go, go negative, that makes the GDP contribution go to zero, and basically, OpenAI and Anthropic and, and xAI and whatever will just be sitting there saying, "Why am I, why am I doing this again? Why am I-- No one's buying this shit," you know? And so, like, um, at that point, we'll-- it seems like there will be corporate pressure on the government to do something to redistribute purchasing power so that they don't compete their profits to negative, and, uh, so they have some reason to, uh, create more economic activity so they can take a slice of it, which is essentially what happened in the early 20th century.

Dwarkesh Patel 25:00 Yeah, I disagree with this. I, I, I think, uh-

Speaker 25:02 Oh, I'm not saying this will happen. I'm saying, like, the, that would be the analogous thing.

Dwarkesh Patel 25:05 Um, I, I, I think there's just, like, I disagree. I, I would prefer it to be the case that the-- even as a libertarian, I would prefer for significant amounts of redistribution in this world because, like, the libertarian argument doesn't make sense if there's, like, no way you could physically pick yourself up by the bootstraps. Like, your labor is not worth anything, um, or your labor is worth less than subsistence, uh, subsistence, you know, calories or whatever, which is the more relevant thing. Um, uh, but, like, I don't think this is analogous to the situation in China. I think, like, what's happening in China is more due to the fact that you have this system of financial repression which redistributes money and also, um, currency manipulation, which basically redistributes ordinary people's money to these, uh, to basically producing one EV maker in every single province. Um, so it is the market distortion that the government is creating that causes this overproduction. Uh, we can go into what, like, the analogous thing in the AI case looks like, but I think if there isn't some market distortion, I just think, like, people will use AI where it has the highest rate of return. Like, if it's not space colonization, it'll be, like, you know, longevity drugs or whatever, um-

Speaker 26:07 I'm stressing, like, why would I, why would I in, why would I invest all this money into AI-producing stuff? Uh, why would I just invest the, the massive hundreds and billions of trillions or whatever of dollars into producing stuff for people who are all gonna be out of a job and won't be able to buy the stuff?

Dwarkesh Patel 26:21 But again, I don't think you'll be producing it for them.

Speaker 26:23 Okay.

Dwarkesh Patel 26:23 I think you'll be producing it for whoever does have-- Like so- there's stuff in the world.

Speaker 26:26 Okay.

Dwarkesh Patel 26:26 So somebody will have stuff. Maybe it's the AIs, maybe it's Sam Altman.

Speaker 26:29 Right.

Dwarkesh Patel 26:30 Um, you're producing it for whoever has the capability to buy your stuff. Um, and will they want AI? And I'm just saying, like, AI can do so many things, least of which is, like, colonizing the galaxy. People will, people are willing to pay a lot of stuff-

Speaker 26:42 Right. I'm just trying to-

Dwarkesh Patel 26:43 ... to colonize the galaxy.

Speaker 26:43 Right. I'm just trying to get this straight in my head of what this economy looks like, and I'm, I, what I'm, I'm seeing a picture of the trillions of dollars needed to build out all these data centers will be done not for profit, not to make money for, you know, from a consumer economy for the creators of the AI, but to satisfy the whims of a few robot lords to colonize the galaxy and-

Dwarkesh Patel 27:01 I, I think you're making two different points, and, uh, they're getting, um-

Speaker 27:05 Conflated

Dwarkesh Patel 27:05 ... they're getting wrapped into one. Um, I'm saying-

Speaker 27:09 Conflated.

Dwarkesh Patel 27:09 Yes. [laughs] Important word. Um, [laughs] I-- So there's one about, like, do you expect it to be the case that, uh, the robot overlord world happens? And I'm saying no, actually, like, even without redistribute-- First of all, I expect redistribution to happen. I hope it happens. But even if it doesn't, uh, and I don't think it'll happen because people, like-

Speaker 27:27 Mm

Dwarkesh Patel 27:27 ... corporations want the redistribution to happen. I think it's just like it'll be good to happen for independent reasons. But I don't buy this argument that, like, the corporations will be like, "We need somebody to buy our AI, therefore, we need to give the money to the ordinary consumers."

Speaker 27:37 You believe broad-based asset ownership will create a lot, whole lot of broad-based consumer demand even in the absence of labor income?

Dwarkesh Patel 27:44 I, I, I'm honestly, I don't have, like, a super strong opinion, but I think that's, like, plausible. Um, uh, but independent of that, I'm like, okay, even if that demand doesn't exist, just, like, the things you can do, uh, with a new frontier of technology, as long as one person wants it, there's, like, so much room to do things, you know? Like, uh, space colonization is an obvious example that like-

Speaker 28:04 But costs a lot of money.

Dwarkesh Patel 28:05 Right. That like, there's, like, obvious demand for the things that AI will be able to produce, right? Like, one of the things AI can produce is colonize the galaxy.

Speaker 28:11 Right. Exactly. So but the question is, like, who, you know, I can see a paperclip maximizing A- you know, autonomous intelligence-

Dwarkesh Patel 28:19 Right

Speaker 28:19 ... is colonizing the galaxy. But in terms of-

Dwarkesh Patel 28:20 That's a lot of growth.

Speaker 28:22 That is.

Dwarkesh Patel 28:22 [laughs]

Speaker 28:22 But i-in terms of, of, uh... And, and so by the way, I would like to say that I am a paperclip maximizer. I am the real paperclip maximizer. I wanna maximize rabbits-

Dwarkesh Patel 28:31 Uh-huh

Speaker 28:31 ... in the, in the galaxy.

Dwarkesh Patel 28:33 Uh-huh.

Speaker 28:33 I wanna turn the entire galaxy into floofy rabbits.

Dwarkesh Patel 28:35 Right.

Speaker 28:35 That's my goal.

Dwarkesh Patel 28:36 Yes.

Speaker 28:36 And so my goal with AGI is to enlist the AGIs to, to help me in this goal.

Dwarkesh Patel 28:40 [laughs]

Speaker 28:40 But then to align them towards rabbits.

Dwarkesh Patel 28:42 Yes.

Speaker 28:42 But anyway, um-

Dwarkesh Patel 28:43 Get this guy in front of the OpenAI board of directors.

Speaker 28:45 Yeah, exactly. I know.

Dwarkesh Patel 28:46 [laughs]

Speaker 28:46 I, I mean, like, objective, the social welfare function is floofiness.

Dwarkesh Patel 28:49 [laughs]

Speaker 28:50 [laughs] Um, but I guess my, my, my point here is, um, as long as it's, as long as AI still, uh, doesn't have property rights and it's, it's humans making all the economic decisions, be it Sam Altman and Elon Musk or some, you know, you and me, uh, then at that point, like, that really matters for what gets done because, uh, then if we're talking about the money needed to build all these massive data centers, which currently it's a lot of money. It's a ton of money required to build these data centers, and that money need will not go away. We can't just, like, say, "Oh, cost goes to zero," because we can say unit cost goes to zero, but total cost doesn't go to zero, nor has it. It has, it has increased. Uh, the total spend on data centers has increased, and I think everyone expects it to increase for the foreseeable future.

Dwarkesh Patel 29:34 Mm-hmm.

Speaker 29:34 The question is, is that money being spent because AI companies expect to reap benefits from consumers like you and me? Or, or to what extent is it that? And to what extent is it Sam Altman feels like doing some crazy stuff and Sam Altman's just godlike richer than everybody else, and so Sam Altman is actually consuming when he builds those data centers? He is, he is building those data centers so that he can indulge his godlike whims.

Dwarkesh Patel 30:01 Mm.

Speaker 30:02 And so that, that's-

Dwarkesh Patel 30:02 I mean, I think the more, more plausible than either a single godlike person is able to direct the whole economy a- or, um, like there's this broad-based consumer, every, every person is buying-

Speaker 30:13 It's not an example. These, these are extremes.

Dwarkesh Patel 30:13 Yes.

Speaker 30:14 They're thought examples.

Dwarkesh Patel 30:14 I think more plausible is, like, AIs will be integrated through all the firms in the economy. Um, a firm can have property. Um, firms will be, like, largely run by AIs even though there's nominally a human board of directors, and it might not even be nominal, right? Like, maybe the AI's are aligned and, like, genuinely give the board of directors an accurate summary of what's happening. But, like, day-to-day, they're being run by AIs, and firms can have property rights. Firms can demand things.

Speaker 30:39 So say all you have is a board of directors and AI.

Dwarkesh Patel 30:41 Yeah.I mean, in the ideal world.

Speaker 30:44 Okay. So then, then what we're basically looking at is capital i- is the, the labor share of income goes to zero-

Dwarkesh Patel 30:49 Mm-hmm

Speaker 30:50 ... or something approaching that.

Dwarkesh Patel 30:51 Yeah.

Speaker 30:51 Capital share of income-

Dwarkesh Patel 30:51 Depends on how you define-

Speaker 30:52 Right

Dwarkesh Patel 30:52 ... the AI labor.

Speaker 30:53 And, and capital share of income, and capital income is not, i- is distributed highly unevenly. It's more distributed much more unevenly than labor income.

Dwarkesh Patel 31:00 Mm-hmm.

Speaker 31:00 But it's still distributed reasonably broadly. Like, I have capital income, you have capital income. So like, um, uh, so at that point, we have just an extremely unequal society where owners get everything and then, you know, workers get nothing.

Dwarkesh Patel 31:12 Yeah.

Speaker 31:12 And then so we have to figure out what

Speaker 31:14 to do about that.

Dwarkesh Patel 31:14 Yeah, 100%. The hopeful situa- case here is the way our society currently treats retirees and old people who are not generating any economic value anymore, and if you just look at, like, the percent of your paycheck that's going, basically being transferred to old people, it's like, I don't know, 25% or something. Um, and, uh, you're willing to do this because they have a lot of political power. They've used that political power in order to, uh, lock in these advantages. They're not, like, so overwhelming you're like, "I'm gonna go to, like, Costa Rica instead." You're like, "Okay, I had to pay this money, I had to pay this concession, I'll do it." And hopefully humans can occupy that sort of like, um, can, can have, be in a similar position to this massive AI economy-

Speaker 31:54 Mm-hmm

Dwarkesh Patel 31:55 ... that old people today have in, um, in today's economy.

Speaker 31:58 All right.

Erik Torenberg 31:59 What do humans do? Uh, so, you know, let's say they get some, some money. They, like, they have enough to live. Uh, h- how do they spend their time? Is it art, religion, poetry-

Dwarkesh Patel 32:07 Podcasting

Erik Torenberg 32:07 ... drugs?

Dwarkesh Patel 32:08 Podcasting.

Speaker 32:08 [laughs]

Dwarkesh Patel 32:08 It's the final job. [laughs]

Erik Torenberg 32:10 Yeah, we're, we're out of the, out of the curve here.

Dwarkesh Patel 32:12 [laughs] Or it's a-

Speaker 32:14 We-

Dwarkesh Patel 32:14 We're the m- last man of history.

Erik Torenberg 32:16 [laughs]

Speaker 32:17 Wait, so here's an idea. How about sovereign wealth fund?

Dwarkesh Patel 32:19 Uh-huh.

Speaker 32:19 Okay, so sovereign wealth fund, we, uh, we tax Sam Altman and Elon Musk.

Dwarkesh Patel 32:24 Yeah.

Speaker 32:24 We tax them.

Erik Torenberg 32:25 We're using Sam as a metaphor here. He's a friend of the firm, you know. [laughs]

Speaker 32:27 Yeah, yeah, yeah. We tax him. We tax Mark and, and Ben. And so then we-

Dwarkesh Patel 32:31 [laughs]

Speaker 32:32 ... we use, we use their money to-

Dwarkesh Patel 32:33 Only the friends of the show will be taxed.

Speaker 32:35 Right.

Dwarkesh Patel 32:35 [laughs]

Speaker 32:36 We, we, we use that money to buy, we use that money to buy, like, um, shares in the things that those people have. So they get their money back [laughs] 'cause we're buying the shares back from them. Okay, so it's okay.

Dwarkesh Patel 32:46 Yeah.

Speaker 32:47 And then, and then we hire them.

Dwarkesh Patel 32:48 Yeah.

Speaker 32:48 Because then what we do is we hire a number of firms, including a16z, and pay them two and 20 or whatever-

Dwarkesh Patel 32:54 [laughs]

Speaker 32:54 ... to manage the investment of AI stuff on behalf of the, of the humans, but then the humans become broad-based sort of index fund shareholders or, or shareholders in whatever you guys choose to invest in, then you take a cut, and this could be the, the future economy. This is what my PhD advisor Miles Kimble has suggested. This is what the socialist Matt Bruenig has suggested, and this is what, um, Alaska actually does with oil.

Erik Torenberg 33:17 Capitalists like it. Socialists like it. Alaska likes it.

Dwarkesh Patel 33:21 I, I, I, I think sovereign wealth funds generally have a bad track record. There's some exceptions that have, like, managed-

Speaker 33:26 Mm

Dwarkesh Patel 33:26 ... to use their wealth well, like Norway or Alaska. But, um, there's just, like, these political economy problems that come up-

Speaker 33:31 Mm

Dwarkesh Patel 33:31 ... when, uh, there's this tight connection between the investment, which should theoretically be just highest rate of return and-

Speaker 33:39 Right

Dwarkesh Patel 33:39 ... um, uh, politicians. So I, uh, I, I, I don't have, like, have a strong alternative. Ideally, you just let the market decide how the investment should happen, and then you can just ta- take a tax, um, but then exactly where does that tax happen, I, I haven't thought it through. But I, I wouldn't want the government influencing where that investment happens, but I want the government taking a significant share of the returns of that investment.

Erik Torenberg 34:01 Yeah. Are you dubious of the trope that, you know, labor provides meaning, and if people don't have a clear, uh, sense for labor, then it will be very difficult for them to obtain alternative sources of meaning, or is that kind of a, you know, a capitalist, uh, [laughs] sort of trope that, that isn't necessarily true-

Dwarkesh Patel 34:16 Um

Erik Torenberg 34:16 ... from your perspective?

Dwarkesh Patel 34:17 I, I, m- my suspicion is that humans have just adapted to so much, like agricultural revolution, industrial revolution, the growth of states, the, um, you know, like once in a while, like a communist or fascist regime will come around or something. Like, the, the idea that, uh, being free and having millions of dollars is the thing that finally gets us-

Speaker 34:39 Yeah

Dwarkesh Patel 34:40 ... um, I'm just suspicious of.

Erik Torenberg 34:41 [laughs] Um-

Dwarkesh Patel 34:43 By the way, do we not disagree about the thing? I'm saying, like, once we get AGI, um, humans will not have high-paying jobs. Do we disagree about this?

Speaker 34:56 Uh, I think humans may have high-paying jobs-

Dwarkesh Patel 34:58 Okay

Speaker 34:59 ... because of, um, comparative advantage.

Dwarkesh Patel 35:01 Yes.

Speaker 35:01 You know, uh, if there's some... The, the key here is if there's some AI-specific resource constraint that doesn't apply to humans, um, then, then comparative advantage law takes over, and then humans get high-paying jobs even though AI would be better at any specific thing than a human-

Dwarkesh Patel 35:15 Mm-hmm

Speaker 35:15 ... because there's some sort of aggregate constraint. And the example I always use, of course, is Mark Andreessen, who, uh, is the fastest typist I have ever seen in my life and yet does not do his own typing. And so, you know, because there's a, there's a Mark Andreessen-specific aggregate constraint on Mark Andreessens. There is only one of him. Um, phew. And, [laughs] so then, um, so he hasn't taken all the, the secretaries' typist jobs, but, um, uh, because he has better things to do. And so if, if there's some sort of AI-specific resource constraint that hits, then humans could have... Now, I'm not saying there will be.

Dwarkesh Patel 35:47 Yeah.

Speaker 35:47 And I'm not saying there won't be.

Dwarkesh Patel 35:48 Yeah.

Speaker 35:48 I'm saying I don't know if there is.

Dwarkesh Patel 35:50 Yeah. The reason I find that implausible is that I, I think that is, that will be true in the short term because, um, right now there's 10 million H100 equivalents in the world. You know, in a couple years there might be 100 million. Human pop- and but, like, H100 is, has the same amount of FLOPS as a human brain, so theoretically they're, like, as good as a brain if you had the right algorithm. Um, uh, so there, there's like a lower population of AIs even if you had AGI right now than humans. But the key difference is that in the long run, you can just keep increasing the supply of compute or of robots. And so if it is the case, so if an H100 costs, um, a couple thousand dollars a year to run-Um, and you know, but this, the value of, like, an extra year of intellectual work is still, like, $100,000. So you're like, "Look, we've, we've saturated all the H100s and we still have to pay a human-- We're gonna pay a human $100,000 because, like, there's still so much intellectual work to do." In that world, the return on buying another H100, like an H100 costs for- $40,000. Just like in a year, that H100 will pay you over 200% return, right? So you'll just keep expanding that supply of compute until the, until basically, you know, the H100 plus depreciation plus running cost is the same as an extra year of labor. Um, and in that world, that's, like, much lower than human subsistence. So comparative advantage is totally consistent-

Speaker 37:10 Okay

Dwarkesh Patel 37:10 ... with, like, uh, with human wages being below subsistence.

Speaker 37:14 It, it is, but that comes from the common resource consumption. So you know, if, um, if basically all of the human, all of the land and energy that could be used to feed and clothe and shelter humans gets appropriated by H100s, then that is the case. However, if you pass a law that says, "This land is reserved for growing human food," and if you pass a law that says, "This land..." You know, that we have a minimum-- If, if we, if we actually were to just pass a simple law saying that you have to use these resources, these resources are reserved for humans-

Dwarkesh Patel 37:50 But, but at that point, the comparative advantage-

Speaker 37:51 Then, then comparative advantage breaks down

Dwarkesh Patel 37:53 ... but at that ar- at that point, like, human labor has nothing to do with this. The only reason the system works is that you are basically transferring resour- You've come up with a sort of, like, intricate way to transfer resources to humans. You're just like, "This resource is for you. You have this land and therefore you can survive." And this is just, like, an inefficient way to allocate resources to humans.

Speaker 38:12 It's true that it is an inefficient way to allocate resources.

Dwarkesh Patel 38:14 Um, so but that, that, like, that has nothing to do with, like-- I think people g- you know, people will, like, hear this argument of comparative advantage and be like, "Oh, there's some intrinsic reason that humans will-"

Speaker 38:21 We take UBI instead.

Dwarkesh Patel 38:22 Yeah.

Speaker 38:22 Okay.

Dwarkesh Patel 38:23 Um, um, yeah.

Speaker 38:26 Yeah. I mean, sure. It, it-- But then again, we typically do not see the first, best, most efficient political solution implemented for things like redistribution. In the real world, redistribution happens via things like the minimum wage or, you know, like letting the AMA decide how many doctors there's gonna be.

Dwarkesh Patel 38:45 Right.

Speaker 38:46 So, so redistribution in the real world is not always the most efficient thing. So I'm just saying that, like, comparative advantage, uh, if you're talking about will humans actually continue to get high-paid work, yes or no, it depends on political decisions that may-

Dwarkesh Patel 39:00 But what I'm saying, like-

Speaker 39:00 ... and it depends on physical constraints that will happen if we don't know.

Dwarkesh Patel 39:02 But the high-paid jobs are literally because, like-

Speaker 39:05 I know

Dwarkesh Patel 39:05 ... you have said that, like, there must be high-paying jobs politically. Like, yeah.

Speaker 39:08 I understand. You, in this case, you've said it in an indirect way, but you still said it.

Dwarkesh Patel 39:11 Right. Yeah. But like-

Speaker 39:12 That's right. You're absolutely right.

Dwarkesh Patel 39:13 Yeah. Yeah.

Speaker 39:14 You're not wrong.

Dwarkesh Patel 39:14 Yeah. Or I, I, I guess it's, like, incredibly different from, um, what somebody might assume.

Speaker 39:20 Right.

Dwarkesh Patel 39:20 Like, it just, it has almost nothing to do with the comparative advantage argument.

Speaker 39:23 Okay. Okay, sure, but that's true of a lot of jobs that exist now. Like, a lot of jobs that exist now, you know, like, I'm not sure what, like, university professors [chuckles] like, there's a lot of those jobs. Or, like, um, credit rating agencies or, you know, there, there's a lot of things where, you know, where probably we could wring out some significant TFP growth more or less-

Dwarkesh Patel 39:44 Mm-hmm

Speaker 39:44 ... by eliminating those things. But we don't because our, our politics is a clue-jocracy. I think this is one of Tyler's points in the book.

Dwarkesh Patel 39:49 Yeah. I, I, I mean, I do think it's important to, um, uh, like, point out in advance, like, basically it would be better if we just bit the bullet about AGI so that instead of doing redistribution by expanding Medicaid and then, you know, Medicaid can't procure all the amazing services that AI will create, um, it'd be better if we just, like, said, "Look, this is coming," and I'm not saying we should do a UBI today, but, like, the lo- if all human w- wages go to zero or go below subsistence, then the only way to deal with that is through some kind of UBI rather than, you know, if you happen to sue OpenAI, you get a trillion-dollar settlement, otherwise you're kinda screwed, right?

Erik Torenberg 40:27 Some people said the bear case for UBI was something around, like, COVID as an example. You, you gave people a bunch of money and what do they g- go do? Go riot in the streets. I'm, I'm teasing. [chuckles] But, like, are people g- gonna use that money in, in a, in a, an effective way?

Speaker 40:40 [chuckles] I mean, that was literally what happened. [laughing]

Erik Torenberg 40:43 Yeah. So, um, y- is UBI the form that you would think that-- Like, what is the mo- most effective method?

Dwarkesh Patel 40:49 I, the reason I favor UBI is, like, this thing where in a future world with explosive growth, we're going to see so many new kinds of goods and services that will be possible that are not available today. And so distributing just, like, a basket of goods is just inferior to saying, "Oh, if, like, we solve aging, here's, like, here's some fraction of GDP. Go, like, go, go spend your tens of millions on, p- partly on buying this aging cure," whatever this new thing that AI enables, rather than "Here's, like, here's a food stamps equivalent of the AGI world that you can

Dwarkesh Patel 41:21 have access to."

Speaker 41:22 Of course, I mean, this discussion may be academic because I believe that, you know, you said that we got phones and the world looked the same. I mean, no, it doesn't. Phones have destroyed the human race.

Dwarkesh Patel 41:32 [chuckles]

Speaker 41:33 Like, the fertility crash that's happening all around the world-

Dwarkesh Patel 41:36 Right

Speaker 41:36 ... nobody has replacement level fertility. Fertility is going far below replacement everywhere, uh, because of technology. And, uh-

Erik Torenberg 41:43 Is that the phone or the pill or-

Speaker 41:45 It, it, oh, well, no, it's the phone. I mean, well, no, the, the, the pill a- and other things like-

Erik Torenberg 41:51 Education

Speaker 41:51 ... women's education, whatever-

Erik Torenberg 41:52 [chuckles]

Speaker 41:52 ... like, lowered fertility, like, quite a bit, but some countries were still at replacement level, some were still around replacement level. But the crash we've seen since everybody got phones is epic and is just unbounded. Like, uh, you know, the human race does not have a desire, a collective desire to perpetuate itself. Um, we can, you know, yes, we're gonna get lonely, but we'll have company through AI and through the internet, uh, social media, you know, until there's just a few of us and we dwindle and dwindle. Um, yeah, I mean, like, technology has already destroyed the human race, and basically UBI is just, like, keeping us around on life support for a little while, while we, while that plays out.

Dwarkesh Patel 42:29 So I, I, I have a take about, like-I do think so far there's been a lot of negative effects from, you know, widespread TikTok use or whatever that, like, w-we're still pr- you know, like, learning about. Um, I am somewhat optimistic that in the long run there's some optimistic vision here that could work. Um, just because right now the ratio of, um... Like, it's impossible for Steven Spielberg to make every single TikTok, uh, and direct it in a sort of really compelling way that's, like, genuine content and not just video games at the bottom and some, like, you know, music video at the top. Um, in the future, it might genuinely be possible to give every single person their own dedicated Steven Spielberg and create, like, incredibly compelling but long narrative arcs that include other people they know, et cetera.

Speaker 43:16 Oh, yeah.

Dwarkesh Patel 43:17 So in the long run, I'm like, maybe this-

Speaker 43:19 Let's make that happen.

Dwarkesh Patel 43:20 I don't think TikTok is, like, the best possible medium.

Speaker 43:22 No.

Dwarkesh Patel 43:23 Um-

Speaker 43:23 But I don't think, I also don't think TikTok is unique in destroying the human race. I think that, um, interacting online instead of interacting in person, that's, that's the great-

Dwarkesh Patel 43:31 How do you make your money, Noah?

Speaker 43:32 That's the, that's the great filter. [laughs]

Dwarkesh Patel 43:34 [laughs]

Speaker 43:34 How do you make your money? Go ahead.

Dwarkesh Patel 43:35 I agree. [laughs]

Speaker 43:36 We're all making, we're all part of the problem-

Dwarkesh Patel 43:38 We're making money destroying our species

Speaker 43:39 ... but that's, but, but, but- You don't think we can isolate it- No, it doesn't matter ... to dating apps and sort of... No, I'm saying, like, I'm saying, like, as long as you can get your... You know, why did humans perpetuate the human species? It was not because they wanted to see the human species perpetuated. It was because it's like, "Oop, I had sex and there came a baby." And that's, that's done. We've severed that. Right. And that's, that's- But- That is the end. We did not evolve to want our species to continue. Right, but y-you're saying the, the reasons why we're not having babies is because we're, we can make friends on the internet, but is it that dating apps have created just a much more efficient market and, and thus there isn't the same- Maybe ... pair, pair bond? I don't know. I mean, like, you know, people are having less sex. Uh, you know, if, if Elon gets his way, everybody will just sit there gooning to some sort of, um- [laughs] ... Grok companion thing. The goonpocalypse, uh, seem- Yeah. [laughs] Seems upon us. But, but, but, like-

Dwarkesh Patel 44:24 Is this available right now?

Speaker 44:26 [laughs]

Dwarkesh Patel 44:26 What's the website?

Speaker 44:26 Oh, no. [laughs] Um, um, anyway. [laughs]

Dwarkesh Patel 44:34 [laughs]

Speaker 44:34 This podcast got silly, but anyway, um, I guess the, the point is that, you know, the idea of a humanity that just keeps increasing in numbers and spreading out to the galaxy, I don't see a lot of evidence that that is in our future and that we have to go to great lengths to make sure that that future is compatible with AGI, because I don't think it's happening in any case, AGI or not. By the way, [laughs] not to cope, uh, too hard, but- [laughs] ... in, in a world, [laughs] in a world where AGI happens- No ... um, how important is population? Like, uh, how important is sort of in-increasing population?

Dwarkesh Patel 45:06 I mean, po-population has so far been the decisive, uh, factor in terms of which countries are powerful and, um... Like, the reason China, if the US was not involved, the reason China could take over Taiwan, uh, is just that there's 1.4 billion Chinese people and there's 20 million Taiwanese people. Um, now, if in future your population is, uh, your effective labor supply is, like, largely AIs, then you just, uh, like, this dynamic just means that, like, your inference capacity is literally your geopolitical power, right?

Speaker 45:40 Right. I wanna shift to, uh, short term a bit.

Speaker 45:43 You've, you've had some people on the podcast, you had the AI 2027 folks who believe that AGI is perhaps two, two years away, may- I think they updated to three years away, and then you've also had some folks on who, who said it's not for 30-something years. Maybe you could steelman both, both arguments and then, uh, share how, wh-where you net it, net it out.

Dwarkesh Patel 45:59 Yeah. So two years, if I'm steelmanning them, is that, look, if you just look at the progress over the last few years, it's reasoning. This is, like, Aristotle's, like, the thing that makes humans humans is reasoning, and we just, like, it was not that hard, right? Like, train on math and code problems, um, and have it, like, think for a second, and you get reasoning. Like, that's crazy. So what is a secret thing that we won't get?

Speaker 46:21 Right.

Dwarkesh Patel 46:21 Um-

Speaker 46:21 Can I ask a stupid question?

Dwarkesh Patel 46:22 Yeah.

Speaker 46:23 Why was, uh, stuff like o3-type models, why are those called reasoning models but, like, GPT-4o is not called reasoning? What, what, what are they doing different that's reasoning?

Dwarkesh Patel 46:35 I, I, one, I think it's, um, like, GPT-3 can technically do a lot of things GPT-4 can, but it just does it way more- GPT-4 just does it way more reliably, and I think this is even more true of reasoning models relative to GPT-4o, where, like, 4o can solve math problems, and in, in fact, like, modern-day 4o has been probably trained a lot on math and code. But the original GPT-4 just wasn't trained that much on math and code problems, so, like, it didn't have whatever meta, uh, meta-circuits there exist for, like, how do you backtrack? How do you be like, "Wait, but I'm on the wrong track, I gotta go back, I gotta, like-

Speaker 47:08 Right

Dwarkesh Patel 47:08 ... I gotta pursue the solution this way"?

Speaker 47:10 Algorithmicically, I have a, you know, okay idea of-

Dwarkesh Patel 47:14 Right

Speaker 47:14 ... what a reasoning model does that the non-reasoning models-

Dwarkesh Patel 47:16 Right

Speaker 47:16 ... don't. But wh- in terms of how does that map to a thing that we call reasoning, what is the, what is the definition of what it means to reason that these people are using, the operational definition here? Like, 'cause I don't understand that myself.

Dwarkesh Patel 47:29 I mean, 4o can't get a gold in, uh, IMO.

Speaker 47:33 Okay, but, but-

Dwarkesh Patel 47:34 Uh, or on whatever

Speaker 47:34 ... I, I, I can reason and I can't get a gold in IMO, but I can reason.

Dwarkesh Patel 47:37 Yeah, but, like, I don't think-

Speaker 47:38 As can the clerk at the checkout

Dwarkesh Patel 47:39 ... I can't get a gold either, but I don't think I can reason as well as a, a math Olympiad, at least in the relevant domain. I agree that reasoning is not just about mathematics, but, um, this is true of any word you come up with. Like, the zebra, like, what is, you know, what, what about the thing that, like, is a mixture of a zebra and a giraffe and they have a baby? Is that a zebra still? Like, I agree there's edge cases to everything, but, like, there's a general conceptual category of zebra, um, and I think there's, like, a general conceptual category of reasoning.

Speaker 48:02 Okay, I'm just wondering what it is. Like, what, what... Like, when you have a, um, no, I'm saying, like, when you have a checkout clerk, right?

Dwarkesh Patel 48:09 Right.

Speaker 48:09 That checkout clerk wouldn't, would look at an IMO problem and be like, "What?"

Dwarkesh Patel 48:12 [laughs]

Speaker 48:12 But then, like, you have a checkout clerk, and the checkout clerk, you're like, you know, okay, so you put the thing in the, you know, on this shelf, and therefore someone has, he looked for it and didn't find it, so something else must have happened.

Dwarkesh Patel 48:24 Sure, but I think a reasoning model-

Speaker 48:25 That's reasoning

Dwarkesh Patel 48:26 ... I think a reasoning model will be more reliable and be better at solving that kind of problem than 4o.

Speaker 48:30 Right. Okay, so you're, you're steelmanning the AI 2027.

Dwarkesh Patel 48:34 Yes. So b- basically, like, look, a lot of things we previously thought were hard have just been incredibly easy. So whatever additional bottlenecks you are anticipating, whether it's this continual learning, uh, on-the-job training thing, um, whether it's computer use, uh, th-this is just gonna be the kinda thing where in advance it's just like how would we solve this? And then deep learning just works so well that we, like, I don't know, try to, like, train it to do that, and then it'll work. Um, the long timeline people will say, uh, w- like, I don't know. There's, there's a sort of longer argument. I don't know how much to bore you with this. But basically, uh, the things we think of as very difficult and requiring intelligence have been some of the things that machines have gotten first. So just adding numbers together we got in the '40s and '50s. Um, reasoning might be another one of those things where we think of it as the apogee of, like, human abilities, but in fact it's only been recently optimized by evolution over the last few million years. Whereas things like just moving about in the world and having common sense and so forth, and having this long-term memory has, you know, evolution spent hundreds of millions if not billions of years optimizing those kinds of things. So th- those might be much harder to b- build into these AI models.

Speaker 49:41 I mean, the, the reasoning models still go off on these crazy hallucinations that they'll never, like, admit were, were wrong and will just, like, gaslight you infinitely on some crap it made up. [laughs] Like, they'll still-- Like, just knowing truth from falsehood.

Dwarkesh Patel 49:53 Yeah.

Speaker 49:54 You know? I've, I've met a couple humans who don't seem to be able to know truth from falsehood. They're weird.

Dwarkesh Patel 49:58 Yeah.

Speaker 49:58 [laughs] And you know, so but, o- o3 sometimes does this.

Dwarkesh Patel 50:02 I mean, to me, I, I think it's an interesting question. Do they hallucinate more than the average person? I think, like, no, less. Again, hallucinate meaning, like-

Speaker 50:08 Yeah

Dwarkesh Patel 50:08 ... getting something wrong and then when they push them on it, they're like, "No," you know, whatever. And eventually they'll, like, accede if there's, they're clearly wrong. I think, like... And then they're actually more reliable than the average human.

Speaker 50:17 But so the thing about the average human is you can get the average human to not do that-

Dwarkesh Patel 50:21 Right

Speaker 50:21 ... with the right consequences.

Dwarkesh Patel 50:22 Yeah.

Speaker 50:23 And maybe AI we haven't found the right, like, reinforcement learning function or whatever to-

Dwarkesh Patel 50:27 Yeah

Speaker 50:27 ... get them to not do that.

Dwarkesh Patel 50:28 Right.

Erik Torenberg 50:29 Okay. Now let's get to the view that it's, it's 30 years away or the basic, um, what, what, what's that view?

Dwarkesh Patel 50:34 Oh, j- just this thing of, like, um, reasoning is relatively easy in comparison to... Forget about robotics, which is just, like, gonna be... You know, evolution spent billions of years trying to get, like, uh, robotics to work. Um, and but there's, like, other things involved with, like, tracking long-run state of, you know, a lion can-

Erik Torenberg 50:53 Right

Dwarkesh Patel 50:54 ... you know, follow prey for a month or something. But, uh, these models can't do a job for a month. And th- these kinds of things are actually much more complicated than even reasoning. Um, yeah.

Erik Torenberg 51:04 Hmm.

Speaker 51:05 And, and where you've netted out is it's either gonna happen in a few years or not for quite some time?

Dwarkesh Patel 51:10 Yeah.

Erik Torenberg 51:10 Why don't you explain this view?

Dwarkesh Patel 51:11 Basically, the progress in AI that we've seen over the last decade has been largely driven by stupendous increases in compute. So the compute used on training a frontier system has grown four X a year for I think, like, the last decade. And that just, like, you know, over four years is 160 X, right? So that's like, you know, over the course of a decade, that's, like, hundreds of thousands of times more compute. Um, that, uh, that physically cannot continue if you just, like, okay, what would it mean? Right now we're spending 1.2% of GDP or something on data centers. Um, not all of that is for training, of course. But what would it mean to continue this for another decade? Um, for maybe five more years you could have, you could keep increasing the share of energy that we're spending on, uh, uh, training data centers or the fraction of TSMC's leading-edge nodes, uh, wafers that we dedicate to making AI chips or the, even the fraction of GDP that we can dedicate to AI training. Um, but at some point, like, you, you can't keep this, like, four X trend going a year. And after that point, then it has to just, like, come from new ideas of like here's a new way we could train a model.

Speaker 52:16 And, and by the way, when I was writing that comparative advantage post and I was thinking about AI-specific constr- aggregate constraints, resource constraints-

Dwarkesh Patel 52:23 Right

Speaker 52:24 ... this is, that's what I was thinking of, actually.

Dwarkesh Patel 52:25 Yeah. Yeah.

Speaker 52:26 That, that, that expansion of compute has to slow down. But-

Dwarkesh Patel 52:29 Yeah

Speaker 52:29 ... I don't know how much it matters.

Dwarkesh Patel 52:30 Well, it, it, it, uh, that's for training and then, like, yeah, for the labor that will be, like, the inference will also use the same-

Speaker 52:36 It has the same bucket of compute

Dwarkesh Patel 52:38 ... inference as well. Um, it is the case that for the amount of compute it costs to train a system, uh, if you, like, set up a cluster to train a system, um, you can usually run 100,000 copies of that model at typical token speeds on that same cluster. Um, uh, that's still obviously not, like, billions. But, like, if we've got all this compute to be training these huge systems in the future, it would still allow us to sustain a population of hundreds of millions if not billions of AIs. Um, and, you know, at that point, maybe we s- obviously we'll still need, want more AIs, but-

Speaker 53:09 What, what does a, a single AI mean in this instance?

Dwarkesh Patel 53:11 Oh, like a copy of a, like, you know, when you're talking to Claude, it's, like, a single instance-

Speaker 53:16 Okay

Dwarkesh Patel 53:16 ... that's talking to you.

Speaker 53:17 I see.

Dwarkesh Patel 53:17 Yeah.

Speaker 53:18 So instances.

Dwarkesh Patel 53:18 Yeah, yeah.

Speaker 53:19 Okay.

Erik Torenberg 53:19 So what's gonna determine whether it's in a, in a few years or no?

Dwarkesh Patel 53:23 Oh, like, basically, um, right now we're, we're basically, uh, riding the wave of this extra compute. That's why AI is getting better every year mostly. Um, in terms of the contribution of new algorithms, it's a smaller fraction of the progress that's explained by that. So if this, we've just got this, like, s- rocket that's like, um, like how high will it take us? And does it get us to space or not? And if it doesn't, then, like, you know, um, then we just have to rely on the algorithmic progress, which has been a s-

Erik Torenberg 53:50 Slower.

Dwarkesh Patel 53:51 Yeah. I think it's, like, yeah.

Erik Torenberg 53:52 But you think it might get us to space?

Dwarkesh Patel 53:54 Yeah.

Erik Torenberg 53:54 The, the short-- Yeah.

Dwarkesh Patel 53:55 Yeah, yeah.

Erik Torenberg 53:55 So-

Dwarkesh Patel 53:56 I think there's, like, a chance that, like, oh, y- continual learning is also, like, uh, you know, I had this whole theory about, like, oh, it's so hard, and how do you slot it in? And they're like, "I fucking trained it to do this"-

Erik Torenberg 54:03 [laughs]

Dwarkesh Patel 54:03 ... like

Dwarkesh Patel 54:04 what we're talking about here.

Erik Torenberg 54:05 Yeah.

Speaker 54:05 Yeah. That, that leads into another thing that I've thought about, which is how poor our track record for making predictions about the future of AI-

Dwarkesh Patel 54:14 Yeah

Speaker 54:15 ... has, has been. Um, the first time you and I hung out, uh, I don't know if you remember this, was with Leopold.

Dwarkesh Patel 54:20 Yeah. Oh, really?

Speaker 54:21 Yeah.

Dwarkesh Patel 54:21 Yeah. I remember this.

Speaker 54:22 It was, it was at your old house.

Dwarkesh Patel 54:23 Yes.

Speaker 54:23 And, um, and Leopold was just pronouncing a whole bunch of pronouncements, uh, from the couch.

Dwarkesh Patel 54:28 Yeah.

Speaker 54:28 And, um, he released this big situational awareness thing.

Dwarkesh Patel 54:32 Right.

Speaker 54:32 And, um, I would say that that wasn't-- How long ago was that? A year and a half?

Dwarkesh Patel 54:36 Yeah.

Speaker 54:36 Yeah. I would say that already most of the things he predicted have been invalidated or made irrelevant-

Dwarkesh Patel 54:43 Really?

Speaker 54:43 ... uh, in the, in the last year and a half. Like, and especially in terms-- like, all the com- stuff about competition with China.

Dwarkesh Patel 54:49 Mm.

Speaker 54:49 Uh, you know, like it turns out filtration was able to get them a whole lot of things that he never predicted. It turns out that, like, so many of the things, other than just the idea that AI would keep getting better, which he predicts and a lot of people predict. But then it-- I feel like a lot of the specific predictions about US capabilities and Chinese capabilities, and what would be the bottlenecks and what would be the things that, you know, we had... Like, here's how we can compete-

Dwarkesh Patel 55:09 Yeah

Speaker 55:09 ... with China, that's all been proven wrong since.

Dwarkesh Patel 55:12 Um, I think this is actually an interesting trend in the history of science where, like, some of the scientists which were the sm- who were the smartest in thinking about the progression of the atom bomb or progression of physics just had these, like, ideas about, like, we'll have a wo- we- the only way we can sustain this is if we have one world government after, you know, after we have it after World War II.

Speaker 55:29 Right.

Dwarkesh Patel 55:29 Uh, there- there's no other way we can deal with this new technology. I do think relative to the technological predictions, uh, Leo- you know, like, I think the main way he-- in which he's been wrong is that, like, um, it didn't take some, like, s- breaking the l- breaking, breaking the servers in order to learn how o3 or something works. It was just the physical-- Sorry. The, um, just public-- Just see you being able to use the model, knowing that-

Speaker 55:54 You can talk to it and learn what it knows.

Dwarkesh Patel 55:55 T-t-like, just knowing a reasoning model works, and then you can, like, use it and you see, like, oh, what is the latency? Like, how fast is it operating tokens? That will teach you, like, how big is the model? Like, you learn a lot just from publicly using a model and, like, knowing a thing is possible. Um, he has been right in one big way, which is, like, he, he identified three key things that would be required to get us from GPT-4 to, like, a BBA GI kind of thing, which was, um, being able to think, so test-time compute, uh, onboarding, which-

Speaker 56:21 Did he talk about test-time compute in that document?

Dwarkesh Patel 56:23 Yeah, yeah.

Speaker 56:23 Oh, okay.

Dwarkesh Patel 56:23 It was, like, one of his, one of his three big unhobblings. Um, then, like, onboarding in terms of the workplace and then I think the final one was, uh, computer use and, like, look, one out of three. Like, and it was a big deal. Um, so yeah, I think, I think he got some things right, get some things wrong, but yeah.

Speaker 56:40 And, and what, what's your take on the model of automating A-AI research as, as the path to AGI?

Dwarkesh Patel 56:46 Uh, the Meteor Uplift paper, contrary to expectations, they found that whenever senior developers working in repositories that they understood well, uh, used AI, they were actually slowed down by twenty percent. Um-

Speaker 56:58 Yeah, I did see that.

Dwarkesh Patel 56:58 Yeah. Uh, where-whereas they themselves thought that they were sped up twenty percent and-

Speaker 57:03 Right

Dwarkesh Patel 57:04 ... um, and so there's a bunch of the-

Speaker 57:06 I'm getting things done. [laughs]

Dwarkesh Patel 57:09 This goes back to your theory about the phones are destroying us, you know? [laughs] Um, that is an update towards the idea that AI is not on this trend to be this super useful assistant that's helping us already make the sh- process of training AI are much faster, and this will just, like, be the succ- um, feedback loop and exponential. Um, I have other independent reasons. I'm like, I don't know, I'm, like, s- twenty percent that, like, we'll have some sort of intelligence

Dwarkesh Patel 57:37 explosion.

Speaker 57:38 One of the other Leopold predictions was, was nationalization.

Dwarkesh Patel 57:40 Yeah.

Speaker 57:40 Um, is, is that something you could potentially foresee in the next few years?

Dwarkesh Patel 57:45 I don't think it's, uh, politically plausible, um, especially given this administration. I don't think it's desirable. Um, at first I think it would, like, just drastically slow down AI progress because look, uh, this is not nineteen forty-five America, and also building an atom bomb is, like, a way easier project, uh, than building AGI.

Speaker 58:05 But China's quasi-nationalizing most of its in-- I mean, it-- And China doesn't control BYD's day-to-day decisions about what to build, but then if China says, you know, "Do this," BYD does it, as does every Chinese company.

Dwarkesh Patel 58:15 I mean, that's kind of the relationship American companies have with the US government as well.

Speaker 58:18 You think so?

Dwarkesh Patel 58:19 I mean, somewhat. I- also the, the big difference is w-what do we mean by nationalization? There's one thing which is like there's a party cadre who is, uh-

Speaker 58:28 In your company sitting-

Dwarkesh Patel 58:29 Exactly. There's another, which is that, um, each province is, like, just pouring a bunch of money into building their own comp-competitor to BYD, uh, in this, you know, b- potentially wasteful way. That, like, distributed competitive process is- seems like the opposite of nationalization to me. Like, when people imagine AGI nationalization, I don't think they're saying, like, Montana will have their AGI and Wyoming will have their AGI, and they'll all compete against each other. I think they imagine that, like, all the labs will merge, which is actually the opposite of how China does industrial policy.

Speaker 58:57 But then you do think that the American government, basically, if it, it says, "Do this," then, like, xAI and, and OpenAI will do it.

Dwarkesh Patel 59:04 Um, n-no. So actually, I think in that way, obviously, the Chinese system and the US system are different. Like, um, although it has been interesting to see that whenever, um... I don't know. You-- We've, we've noticed the way that different lab leaders have changed their tweets in the, in the aftermath of the election. I mean, also-

Speaker 59:20 Yeah

Dwarkesh Patel 59:20 ... I-th-

Speaker 59:21 More bullish up at source.

Dwarkesh Patel 59:22 Right.

Speaker 59:22 Yeah.

Dwarkesh Patel 59:23 And, uh, didn't, um, didn't Sam have a thing where, uh... I, I think previously he said that, um, AI will take jobs, how do we deal with this? And then didn't he recently say something at a panel where, like, "I think President Trump is c-correct, that AI will, like, you know, create jobs or something?" Where, like, I don't think in the long run you believe this.

Speaker 59:41 Um, but the reason why humans should be excited about even their jobs being taken is just they'll be so rich that why do they even need it?

Dwarkesh Patel 59:47 Yeah. Yeah.

Speaker 59:47 Much richer than they are now.

Dwarkesh Patel 59:49 Right. M-m-modulo, this redistribution slash not fucking it over with some, you know, guild-like thing.

Speaker 59:54 Yeah. The, um-- You mentioned the atomic bomb, and we also mentioned off, off camera that you don't think the nuke is a good comparison for what happens w- how does it play out when s- a lab figures out AGI? W-w-what, what then, then happens? Is, is there a huge advantage if one country has it first, or if one lab has it first, do they-

Dwarkesh Patel 1:00:10 Mm

Speaker 1:00:10 ... they dominate? W-what does it play out?

Dwarkesh Patel 1:00:12 I, I think it's c- less like the nuclear bomb, where there's a self-contained technology that is so obviously, u-um, relevant to specifically this, like, offensive capability. And you can say, well, like, there's nuclear power as well, but, like, neither of those two th-- Like, nuclear power is just, like, this very self-contained thing, whereas I think intelligence is much more like the Industrial Revolution, where there's not, like, this one machine that is the Industrial Revolution.Um, it is just this, like, broader process of growth and automation and, um, uh, and so forth. So I-- But, but that is, that being said-

Speaker 1:00:46 So Brad DeLong's right and Robert Gordon is wrong. Or if Robert Gordon said there's only, like, it's four things. It's just four big things.

Dwarkesh Patel 1:00:52 Oh, really? Interesting.

Speaker 1:00:52 And then Brad DeLong is like, "No, it's a process of discovery." So anyway.

Dwarkesh Patel 1:00:55 Interesting. Hmm.

Erik Torenberg 1:00:56 And what were Rob's four things again? Do you remember?

Speaker 1:00:57 Oh, uh, um, I mean, electricity-

Dwarkesh Patel 1:01:00 Test time computing.

Speaker 1:01:01 Uh-

Dwarkesh Patel 1:01:01 No, I'm kidding. [laughs]

Speaker 1:01:02 Test time compute. Uh, the internal combustion engine, um, uh, uh, like steam power, and then, like, um... What was the fourth one? Like, maybe, like plumbing-

Erik Torenberg 1:01:13 Right

Speaker 1:01:14 ... I think was the fourth one.

Dwarkesh Patel 1:01:15 Yeah. Or, or even in that case, maybe that actually is-- maybe that's closer to how I think about it. But then the, you need so many complementary innovations in order... So internal combust- uh, combustion engines, I think, invented in the 1870s. Drake finds the f- oil well in Pennsylvania in the 1850s. Uh, obviously, it takes, like, a bunch of complementary innovations before, like, these two things can merge. Before, they're just, like, using the oil for the kerosene to light lamps.

Speaker 1:01:37 Right.

Dwarkesh Patel 1:01:37 Um, uh, but regardless, so if it's this kind of process, it was the case that many countries achieved industrialization before other countries. And, um, you know, like, China was dismembered and went through a terrible century because of the Qing dynasty wasn't up to date on the industrialization stuff. Um, and much smaller countries were able to dominate it. Um, but that is not like we developed the atom bomb first, and now we can just, like, we have decisive advantage. Um, I think it'll be more-

Speaker 1:02:05 That's because it was us. If that had been Nazi Germany or the Soviet Union-

Dwarkesh Patel 1:02:07 Right. Agreed

Speaker 1:02:08 ... it would've gone differently.

Dwarkesh Patel 1:02:09 Yeah.

Speaker 1:02:09 How do you see the US-China competition playing out in terms of, of AI?

Dwarkesh Patel 1:02:13 I genuinely don't know. Yeah. I, I honestly don't know. I, I think it's compa- I think it's, like, possible that there could be some positive sum. Uh, uh, like both-- It's not like a nuclear weapon, where, like, both countries can just, you know, adopt AI and, um... There is this dynamic where if you have higher inference capacity, not only can you deploy AIs faster and you have more economic value that's generated, but, um, you ha- you can have a single copy, sorry, a single model, uh, learn from the experience of all of its copies, and you can have this basically broadly deployed intelligence explosion. So, um, I think it really matters to get to that discontinuity first. Um-

Speaker 1:02:53 Mm-hmm

Dwarkesh Patel 1:02:53 ... I don't have a sense of at what point, if ever, is it treated like the main geopolitical issue that countries are prioritizing. Um, I also from the misalignment, uh, stuff, the main thing I worry about is the AI playing us off each other rather than us playing the AIs off, off each other. Um-

Speaker 1:03:12 You mean AI, AI just, like, telling us all to hate each other the way, like, Russian trolls currently tell us all to hate each other.

Dwarkesh Patel 1:03:18 More so, like, the way that the East India Company was able to play different provinces in India off of each other, and to ultimately to the w- once y- at some point, you realize, okay, like, they control India. Um, and so you could have a scenario like, okay, think about the conquistadors, right? A couple hundred people show up to your border, and they take over an empire of 10 million people, and this happened not, like, once. It happened, like, two to three times. Um, and, like, okay, so why was this possible? Well, it's that the Aztecs, the Incas, um, weren't communicating with each other. Uh, like, they didn't even know the other empire existed, whereas Cortés learns from the subjugation of Cuba, and then he takes over the Aztecs. Uh, Pizarro learns from the subjugation of the Aztecs, then take- takes over the Incas. Um, and so they're able to, like, just learn about, like, okay, you take the emperor hostage, and then this is the strategy you employ, et cetera. Whereas-

Speaker 1:04:07 It's interesting the Aztecs and the Incas never met each other, and that worked both times, sort of.

Dwarkesh Patel 1:04:12 Yeah.

Speaker 1:04:13 Like, that's interesting that the, these totally disconnected civilizations both had the similar vulnerabilities.

Dwarkesh Patel 1:04:18 Yeah. It, it, I mean, it was, like, literally the exact same playbook. Um, the, the crucial thing that went wrong is that at this point in the 1500s, they, um, we don't, we, we actually don't have g- modern guns. We have arquebuses, but the main advantage that the Spanish had was they had horses, and then secondly, they had armor, and it was just incredibly-- You'd have thousands of, uh, warriors. If you were fighting on an open plain, the horses with armor will just, like, trounce all of them. Um, eventually, the Incas had this rebellion, and they learned they can, like, you know, roll, roll the rocks d- down hills, and the, the rebellion was moderately successful, even though it was eventually, well, we know who control, you know, we know what happened. Um-

Speaker 1:04:55 One could say that the Spanish on their side had guns, germs, and steel. [laughs]

Dwarkesh Patel 1:04:59 [laughs] Um, but so how could this have turned out differently? If, like, if the Aztecs had learned this and then had, like, told the Incas, I mean, they weren't in contact, but, like, if there was some way for them to communicate, like, "Here's how you take down a horse," you know? Um, I think the, what I would like to see happen between the US and China basically is, like, the equivalent of some red telephone during the Cold War, where, um, you can communicate, "Look, we noticed this," especially when AI becomes, uh, more integrated with, like, the economy and government, et cetera. Like, "We noticed this crazy attempt to, uh, do some sabotage." Like, be aware that this is a thing they can do, like train against it, et cetera.

Speaker 1:05:34 Right.

Dwarkesh Patel 1:05:35 Um-

Speaker 1:05:35 AI, AI is trying to trick you into doing this. Watch out.

Dwarkesh Patel 1:05:38 Yeah, exactly. Um, that would require a level of trust I'm not sure is plausible, but, like, that's, that's the optimal thing that would happen.

Erik Torenberg 1:05:45 At the lab level, do you think it's a multipolar, you know, or, or is there consolidation? And wh-wh-who's your bet to win?

Dwarkesh Patel 1:05:51 Um, I've been surpris- So you would expect over time, as the cost of competing at the frontier has increased, you would expect there to be fewer players at the frontier. This is what we've seen in semiconductor companies, right? That, like, you know, um-

Erik Torenberg 1:06:04 Sure

Dwarkesh Patel 1:06:04 ... uh, it gets more expensive over time. There's fewer, like, th- there's now maybe one company that's at the frontier in terms of, like, global semiconductor manufacturing. Um, uh, we've seen the opposite trend in AI, where there's, like, more competitors today than there were a year ago, even though it's gotten more expensive. I don't know where the equilibrium here is, um, because the cost of training these models is still much less than the value they generate. Um, so I think it'll, like, would still make sense to 10X the amount of invest- like, somebody new to come into this field and 10X the amount of investment. Do you have a take on where the equilibrium is?

Speaker 1:06:36 Oh. Um-Well, I mean, it has to do with entry barriers.

Dwarkesh Patel 1:06:42 Right.

Speaker 1:06:42 Basically, it's all about entry barriers.

Dwarkesh Patel 1:06:44 Yeah.

Speaker 1:06:44 It's the question of, like, if I just decide to plunk down this amount of money-- So if the only entry barrier is fixed costs, I'd say we have such a good system for, like, just loaning people money-

Dwarkesh Patel 1:06:54 Yeah

Speaker 1:06:54 ... that that's not gonna be that big a deal. But if there's entry barriers that have to do with, like, if you make the best AI, it gets even better, so, you know, why, why enter? That, that's the big question. I don't actually know the answer-

Dwarkesh Patel 1:07:07 Yeah

Speaker 1:07:07 ... to that question.

Dwarkesh Patel 1:07:08 Yeah, there's, there's a broad question we, we ask in general

Dwarkesh Patel 1:07:10 is, like, what are the network effects here?

Speaker 1:07:11 Right.

Dwarkesh Patel 1:07:11 Um, and, and what is the density? And, and it seems often-

Speaker 1:07:14 Network effects

Dwarkesh Patel 1:07:14 ... to be brand, uh-

Speaker 1:07:15 Yeah. Yeah. I, I mean, I'm not sure that's a network effect.

Dwarkesh Patel 1:07:18 Right. No, no. It isn't. [laughs]

Speaker 1:07:19 But, but, but brand, like, like, everybody just sort of, you know, OpenAI, ChatGPT is the Kleenex or, you know, of, of AI in that Kleenex is actually called a tissue, but we call it a Kleenex because there was a company called Kleenex.

Dwarkesh Patel 1:07:31 Where are you going with this?

Speaker 1:07:32 [laughs]

Dwarkesh Patel 1:07:32 Are we back to the Grog doing anything?

Speaker 1:07:34 The apocalypse. [laughs]

Speaker 1:07:35 Oh, no. Well, um, no-

Dwarkesh Patel 1:07:39 Sorry

Speaker 1:07:39 ... I'm, I'm just saying it's, uh, or, or what's a, what's another example? Xerox.

Dwarkesh Patel 1:07:42 Yeah.

Speaker 1:07:43 You, you make a, you Xerox this thing.

Dwarkesh Patel 1:07:44 Yeah.

Speaker 1:07:44 Xerox is just one company that makes a copier.

Dwarkesh Patel 1:07:46 Mm.

Speaker 1:07:46 Right? Uh, not even the biggest one. But like, but everybody knows that a Xerox thing.

Dwarkesh Patel 1:07:50 Right.

Speaker 1:07:50 And so, like, ChatGPT gets massive rents from the fact that everyone just says like, "I'll use AI. What's an AI? ChatGPT."

Dwarkesh Patel 1:07:59 Right.

Speaker 1:07:59 "I'll use it." And so, like, brand is, is the most important thing.

Dwarkesh Patel 1:08:02 But, but I think that's mostly due to the fact that-

Speaker 1:08:04 So far

Dwarkesh Patel 1:08:04 ... this, uh, key capability of learning on the job has not been unlocked. And so-

Speaker 1:08:08 Right

Dwarkesh Patel 1:08:09 ... for, um, I, I don't know-

Speaker 1:08:11 And I'm not saying that could be a technological network effect that could supersede the brand effect-

Dwarkesh Patel 1:08:15 Right

Speaker 1:08:15 ... possibly.

Dwarkesh Patel 1:08:15 Yeah. Yeah. And I think that that will have to be unlocked before most of the economic value of these models can be unlocked. And so by the point these labs are, like, worth hundred... They're already worth hundreds of billions of dollars. But by the point they're generating hundreds of billions of dollars a year, um, or maybe trillions of dollars a year, uh, they will have had to have come up with this thing, which will be a bigger advantage in my opinion than brand network effects.

Speaker 1:08:39 Uh, is Zuck throwing away money, wasting it on, oh, hiring all these-

Dwarkesh Patel 1:08:43 No, I think it's like, I mean, people have been saying like, "Look, uh, the messaging could've been better," or whatever. I mean, I think it's just much better to have worse messaging or something, but to not sleepwalk towards losing. Um, also if you're just thinking about like if you pay, uh, an employee $100 million, uh, and they're a great AI researcher, and they make your compute tr- uh, your training or your inference 1% more efficient, Zuck is spending on the order of, like, $80 billion a year on, uh, on compute. If that's made 1% more efficient, that's easily worth $100 million. Like, $100 million is below the break-even point for this extra researcher. So, um, the real question is, like, why we haven't we hit that, uh, that break-even point yet.

Speaker 1:09:25 Hmm.

Dwarkesh Patel 1:09:25 And if we as a pod- as podcasters encourage one researcher to join Meta, I mean, what's the- [laughing]

Dwarkesh Patel 1:09:31 How do you put a price on that?

Speaker 1:09:32 Yes. [laughs]

Speaker 1:09:33 Yeah.

Dwarkesh Patel 1:09:33 Is there anything we, we have, you wanna- Any last words for the audience based, based on our conversation? I don't know. I, I read your stuff a bunch, so it's great to actually just talk in person.

Speaker 1:09:41 Thanks, man. Yeah. I, um, I have to, to come out with an English language book so we, I can do the podcast. I, I've written a, a Japanese language book published in Japan.

Dwarkesh Patel 1:09:49 [laughs]

Speaker 1:09:49 But, uh-

Dwarkesh Patel 1:09:50 That's great

Speaker 1:09:50 ... I have to write my English language one-

Dwarkesh Patel 1:09:51 Nice. Nice

Speaker 1:09:51 ... and then I can do the Dwarkesh Podcast, one of my dreams.

Dwarkesh Patel 1:09:54 Amazing. Amazing.

Speaker 1:09:55 Noah Dwarkesh, thank you so much for coming on. It's been great.

Dwarkesh Patel 1:09:57 Awesome. Thanks, Erik. [outro music]