Dwarkesh PodcastDylan Patel – Two labs will soon control most of the world's workforce
EVERY SPOKEN WORD
80 min read · 16,272 words- 0:00 – 7:01
Two labs will soon control most of the world’s compute
- DPDwarkesh Patel
Okay, I'm back with Dylan Patel, founder of SemiAnalysis. Our version of a family Thanksgiving dinner [laughs] is a regular yearly podcast. We are not actually related.
- DPDylan Patel
Don't tell the people this.
- DPDwarkesh Patel
[laughs] It will destroy the myth. Um, walk me through-- Uh, so basically, where the world economy is headed is more and more becoming a function of where, like, lab economics are headed, where, like, the compute market is headed, et cetera. So I want to understand where the crazy future ends up within a few years. But, uh, let's start with just where we are today. So walk me through lab compute and lab revenue right now, and maybe projecting out a year or two.
- DPDylan Patel
Yeah. So when we go back to last year, even at the end of the year, most of GDP growth in America was just AI infrastructure. And as we look towards this year, about a third of the compute coming online is for the labs, for OpenAI and Anthropic. Now, it may be built by others and then rented to them, but it's... In, in... At the end customer, it's them. Um, as we go forward into the future, the, the numbers for compute are ballooning, right? We're at, you know, you know, a little bit over a trillion dollars of CapEx this year. As we go out into '28, it's gonna be more than two trillion dollars. Um, the labs are also taking an increasing percentage of this, and so ultimately, you've got a very interesting situation where the labs are going from companies that spend, you know, tens of billions of dollars a year to hundreds of billions of dollars a year to forecasting to spend trillions of dollars a year even, um, at, towards the end of the decade. And this is at least some of the contracts they've begun signing with their partners. And so e-this requires a big reshaping of what happens with their, um, economics, right? You know, so up until now, they have been companies that mostly lost money. Um, Anthropic started turning a profit in Q2. Um, it's believed at some point in Q3, OpenAI could potentially start turning a profit even, um, with the big rise of Codex and, and 5.6 and all this. Um, but if we go back a year ago, everything that they... All the money they had was venture-funded losses, right? If we go back to even the beginning of this year, it was venture-funded losses. Um, they've, they've now turned the corner and are actually starting to profit. Now, that doesn't mean they're not taking in new capital. The new capital's still coming in to accelerate the growth further. But ultimately, there's more and more of their business is being funded off of their own revenue rather than capital injections into them. Over the last, you know, year and a half, their margins have really skyrocketed. You know, the, the base cost of compute tends to be around ten or thirteen or fifteen million dollars per megawatt. The most interesting aspect about what's happening now is before, again, they were generating... If they served a model, right? GPT-4 being served on, um, you know, NVIDIA Hopper GPUs, was generating negative gross margin for OpenAI. But now, when OpenAI serves GPT 5.6 or Anthropic serves me- Opus-5 or Mythos, uh, Fable-5, their revenue generation has w-passed well beyond the sort of incremental ten, fifteen million dollars per megawatt. In the case of Anthropic, the, the revenue has gone as high as fifty, uh, million dollars per megawatt. Um, and, and what that now enables them to do is, "Hey, if I spend ten bucks on inference capacity, I actually generate fifty bucks of revenue, and then I can turn around and incrementally spend all of that profit on training."
- DPDwarkesh Patel
One thing I'm very interested in understanding is how you see the centralization of compute happening at the labs or the relative ratio of compute that goes to the world versus goes to the labs. Um, where if you say right now a third of marginal compute is going to the labs, by when is it over half of the incremental compute in the world is going to the labs? And by what point do the labs have basically a vast majority of the world's compute?
- DPDylan Patel
Yeah. So, so earlier this year, you know, the beginning of this year, Anthropic, OpenAI started at two for OpenAI and less than two for Anthropic. Um, end of this year, they're both above five. Um, so they've three, four X compute as a whole. Um, when you, when you look at the incremental compute added, that's about thirty percent of the compute added this year. And as we step forward to next year, given what's already been signed and penned and inked, you've got something even more dramatic, right? You've got Anthropic, OpenAI are taking, uh, as much as forty to fifty percent of compute, uh, next year. And the centralization doesn't look like it's slowing down or stopping. In fact, it looks like it's only accelerating.
- DPDwarkesh Patel
Right.
- DPDylan Patel
Now, who's building that compute for them will change. Um, you know, next year, a big new entrant is, for example, SpaceX is building a ton of compute, and they're actively going to lease quite a bit of it to Anthropic and OpenAI most likely, 'cause they're the ones who can... who have the marginal capability to pay the highest price. In addition, OpenAI and Anthropic are also starting to build their own compute. OpenAI with their own chips, Anthropic with TPUs that they're purchasing w- from Google and deploying with Fluid Stack. And so when you ask, "Hey, when does half of the world's incremental new compute go to just OpenAI and Anthropic?" I mean, it's, it's really by the end of next year-
- DPDwarkesh Patel
Mm.
- DPDylan Patel
-it's already half of the incremental compute is going to Anthropic and OpenAI.
- DPDwarkesh Patel
Be-because, uh, compute is growing so fast, incremental compute is gonna be basically most of compute. So it's very soon, you're saying maybe within a year and a half or two years, that most of the world's compute is owned by two labs, or at least is serving the demand from two labs. Um, how long do you think... So there's this trend where maybe world compute in gigawatts doubles every year, but the compute at the frontier labs triples every single year. But if you keep the current trend going, it goes from, like, two at the beginning of this year to close to, like, six at the end of this year. Just multiplying out by three. Eighteen by the end of twenty twenty-seven, fifty-four by the end of twenty twenty-eight. Are you like, "Okay, at that point, it, they simply can't continue tripling given the amount of world compute?" Or how, how do you see the world compute situation over the next few years?
- DPDylan Patel
Yeah. So if the incremental compute adds this year thirty gigawatts, next year fifty gigawatts, and the year after that seventy, roughly, you end up with this really interesting phenomenon, which is Okay, well, a, a new watt deployed this year is significantly more efficient than the watts deployed two years ago.
- DPDwarkesh Patel
Mm-hmm.
- DPDylan Patel
So actually, you know, a humongous percentage of the world's compute was deployed this year.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Even though it didn't double the number of watts deployed, I'm deploying GB300s and TPU v7s and Trainium 3s, which are way, way, way more efficient. You know, 3X, 5X more performance per watt than-
- DPDwarkesh Patel
Mm
- DPDylan Patel
... the prior generation chips. And so ultimately you've got a huge, um, ladder here. So if Anthropic and OpenAI take on, you know, 45% of compute next year, you've, you've got them in, let's say, December '27, they have taken on half of the world's incremental new compute. But that half of the world's new incremental compute is actually at a higher performance than everything else before it.
- DPDwarkesh Patel
Right.
- DPDylan Patel
So you've got an- another multiplier on that. So by the time you're in, like towards the end of 2028, if this trend continues, which I see nothing that's stopping it, um, you, you've got them just controlling most of the usable, you know, FLOPs in the world on their own.
- 7:01 – 13:08
$6 billion in fab capex enables $1t+ of end revenue
- DPDwarkesh Patel
The thing I'm confused about is why you think we only add 80 gigawatts in 2028 if we enter in a world in which the pri- the value of compute increases so much.
- DPDylan Patel
That's the upper bound, by the way [laughs] . That's the, that's the like, "I'm so fucking bullish."
- DPDwarkesh Patel
Right. Okay, so le- le- le- let's, le- let's do some chain of thought here. So when I interviewed you a few months ago, you said in order to make a gigawatt of, I think, Vera Rubins, you need... One sec. You need 55,000 N3 wafers, 6K N5 wafers, and 170K DRAM wafers. Um, I don't know if those numbers might have changed.
- DPDylan Patel
I'm gonna troll you, but the way you said "wafers" was so fucking Indian. Wafers. [laughs]
- DPDwarkesh Patel
[laughs] By the way, when we first, uh, when we first moved to the US, I had the, the V-W thing pretty bad, and I was a vegetarian. [laughs]
- DPDylan Patel
A vegetarian. I remember you told me about this. [laughs]
- DPDwarkesh Patel
[laughs] Inside, in North Dakota, I was in elementary school, and I'd be like [laughs] -
- DPDylan Patel
"Can I get a wedgie?"
- DPDwarkesh Patel
"Can I get some wedgies?" [laughs]
- DPDylan Patel
[laughs]
- DPDwarkesh Patel
Anyways, so that's for one gigawatt, right?
- DPDylan Patel
Yeah.
- DPDwarkesh Patel
Now, I had an LLM run your wafer fab equipment model and figure out how much, um, tooling, how much the tooling costs to produce a gigawatt of compute basically every single year. And it was at, like $3 to $4 billion. Now suppose you add in, you know, clean rooms and, uh, shell and everything else at the fab. So $6 billion of like fab capex produces every single year a gigawatt, and a gigawatt produces right now $100 billion [laughs] of revenue. But also that $6 billion in capex is producing a gigawatt every single year, and that gigawatt is producing $100 billion every single year. So even over the course of five years, so, you know, the, the first gigawatt is generated five years of profits. The second gigawatt that the fab has produced has generated four years of profits, and so on. Um, $6 billion of capex at the fab level will have generated over a trillion dollars of end rev- end AI revenue.
- DPDylan Patel
Yeah. There's a lot of opex along the way.
- DPDwarkesh Patel
Of course, of course.
- DPDylan Patel
There's a lot of, um, other capex, like the data center, the power-
- DPDwarkesh Patel
And you'd pay like a, you know, the OpenAI for the R&D-
- DPDylan Patel
Installation
- DPDwarkesh Patel
Exactly. Of course, of course.
- DPDylan Patel
There's a lot of different people who need money here
- DPDwarkesh Patel
So but, but take away, take away half-
- DPDylan Patel
But yeah, it's, there's a huge
- DPDwarkesh Patel
... take away half of it for all these pe- all these middlemen. That still means there's 100X discrepancy between fab capex and end revenue generated. More than that, actually, really, but, uh, we're just being very conservative. And as a result, this is capitalism, right? Like, you would imagine that people are gonna figure... Like, we're gonna be... You're, you're, you have this huge discrepancy where you can turn $1 into $100, and you're, they're not gonna figure out a way to make more mirrors?
- DPDylan Patel
I mean, they are.
- DPDwarkesh Patel
Right.
- DPDylan Patel
It's just these mirrors take some time to make, right? Like, you know?
- DPDwarkesh Patel
But the, but the, the emergency are so big where, like, Anthropic and OpenAI are like, "We could make $1 trillion right now, but we're just bottlenecked on the mirrors that go into the ASML machines." Like, they'd spend... Okay, how, how can we make more mirrors if we spend $100 billion on this, right? That's the situation we're gonna be in pretty soon. And I'm just like, we're not gonna be able to solve that supply constraint? That just seems quite hard to imagine.
- DPDylan Patel
No, there's definitely, um... You, you, you've seen people do funny arbitrages here where they buy, like turbines, and then they try and resell them.
- DPDwarkesh Patel
Right.
- DPDylan Patel
Because the value of a turbine is way more because it's the thing bottlenecking your data center.
- 13:08 – 18:22
Compute prices will rise if the labs outbid everyone
- DPDwarkesh Patel
the key question I really wanna understand is if, um, yeah, if the current consumption use would be like north of fifty gigawatts per lab by the end of twenty twenty-eight. So between them, they'd have a hundred gigawatts. Um, those gigawatts, as you're saying, drive many fold more throughput or more performance by twenty twenty-eight than they are now, right? Because the hardware's gotten better. So not only have like FLOPs per watt increased, but also the hardware gets better at s- um, working with AI workloads. Okay, so a hundred gigawatts for the labs end of twenty twenty-eight. How much is like world compute?
- DPDylan Patel
I think that may be a little difficult given twenty twenty-eight you start to have... They've taken seventy, eighty percent of incremental compute, and I'm not sure what happens to markets then.
- DPDwarkesh Patel
Mm.
- DPDylan Patel
Right? You know, how much does the price of compute skyrocket for them to actually be able to buy seventy, eighty percent of compute? Is, you know, Google or Meta or my Amazon willing to sell even that much? Um, also one caveat when we're sort of talking about these gigawatt numbers is, you know, when Amazon is serving Bedrock Anthropic models, that counts as Anthropic compute in sort of our-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... worldview because it is effectively, at the end of the day, counted as revenue for Anthropic-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... um, even though like there's a revenue share and credit back-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... all that. Um, but ultimately in twenty twenty-eight it's, it's, you know, if they get to a hundred gigawatts combined, they have done really disruptive things to the market because anyone can make money off of ten to fifteen million dollars per megawatt compute today. You literally-- Like I kid you not, it's not that hard. Go get a GB300 rack, go download the Kimi weights, go download VLM or SGLang, set it up. You know, Codex and Fable can actually help you do this. It's pretty simple. I, I m- mean, it's not like it's... You know, it's not trivial, but it's not like rocket science, and go put it on Open Router. It's very simple. Um, and, and you'll start generating more revenue than you're paying for the compute. Um, and so this is, this is sort of already led to this compute pricing ten to fifteen million dollars per megawatt start to inflect up. Um, and to get to that hundred gigawatts in twenty twenty-eight, you have to believe that the labs can outpay for compute, because anyone can make money at ten to fifteen. You, you know, does compute now get to twenty-five million dollars a megawatt? Does it get to forty million dollars a megawatt?
- DPDwarkesh Patel
But as you're saying, it's already the case that the labs are generating way more revenue per megawatt than everybody else. If they stay as far ahead as they are currently, you would expect that to be the continuing case. If there's like some kind of recursive self-improvement where the AI labs are like relatively uplifted, or they have models internally they're not releasing externally that are helping them make their next model better, you'd expect it- that to be even more the case. Aren't you already seeing this where like SpaceX or whoever is like slightly further behind will just sell compute to the highest bidder if they can't internally monetize it as well as the labs? So y- I feel like it's, it-- continue expecting them to be able to gobble up l-- like bid for larger and larger shares of the compute-
- DPDylan Patel
I, I think that is my worldview, that they will continue to gobble up more of the compute.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
But ultimately, they can't do it at, at current pricing or anywhere close to it.
- DPDwarkesh Patel
Sure, sure.
- DPDylan Patel
They, they do have to start paying twenty-five, thirty, fifty million dollars a megawatt to really gobble up seventy percent of the world's-
- DPDwarkesh Patel
Right
- DPDylan Patel
... compute in twenty twenty-eight-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... to get to that hundred gigawatts by twenty twenty-eight, which is a very sort of aggressive goal. The other aspect of this that's really challenging is we've already seen a huge slowdown for the AI labs, right? This, this regulation that they advocate for is actually slowing down the labs a lot more than it slows down, you know, sort of the open source Chinese language models. Um, you know, OpenAI not releasing, uh, Astra. OpenAI stopping training for two weeks. Anthropic not releasing what their safety assessment set as Model 2, which is widely believed to be the next version of Mythos. They're clearly not releasing their best models, and in which case, their revenue per megawatt stalls or even can start to decline again because other models are competitive again. Um, so it's not that they're falling behind, it's just that they're not releasing their best stuff.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
What if there is some regulatory impact that prevents them from releasing their best models? Now, their revenue per megawatt does not climb as fast, then their ability to d-- buy that incremental compute for a higher price than everyone else starts to diminish, and then maybe they can't get to that hundred gigawatts is, is sorta... I- in a world where safety doesn't matter, I, I do believe that's exactly what happens, right? They can start generating a hundred million dollars per megawatt or more, and they can pay fifty million dollars a megawatt, and no one else has any logical reason to do anything with their compute besides say, "Please, Dario, take everything off of my hands." Um, but there are, you know, forces at play, uh, that which we cannot describe [chuckles] , um, that, that would potentially slow this down.
- DPDwarkesh Patel
Yeah, yeah, yeah. I mean, I, I think a good intuition pump is just how-- what if the AI models were literally as good as a fully automated software engineer? They're not currently there yet, right? Like I think they're far from just being able to fully au-automate the job of like a full white collar worker. But white collar workers earn, you know, six figures or north of that a year. Um, and if you have a gigawatt that can sustain a population of like, say, a million of white collar workers, just let's, let's say roughly, right? Um, that's like-- You could then off the back of that- That would be 100 billion. That's actually surprisingly low. [laughs]
- DPDylan Patel
[laughs] Yeah, 100K per person, million population, yeah?
- DPDwarkesh Patel
Yeah, yeah, yeah.
- DPDylan Patel
Um-
- DPDwarkesh Patel
I don't know. But it would be many hundreds of billions of dollars if you get, like, full AGI,
- 18:22 – 25:40
Which layer will capture most of the surplus?
- DPDwarkesh Patel
uh, per, per gigawatt.
- DPDylan Patel
I, I think the other aspect of this is, and we've continued to see this, the most of the value capture is not happening, right? Like, most of the value that these models generate-
- DPDwarkesh Patel
For sure
- DPDylan Patel
... does not get given to OpenAI and Anthropic. Um, thankfully so far it is mostly just being given to the users.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
Right? Jane Street with their exclusive contract with OpenAI for GPT 5.6 ultra-fast mode or Jane Street, where they're, like, one of Anthropic's biggest customers, is generating way, way, way, way more value out of the tokens they're paying for than Anthropic is-
- DPDwarkesh Patel
Sure
- DPDylan Patel
... uh, generating in terms of profit, right? 'Cause they get to, you know, make money off of the market.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Um, or Meta, who at one point was, you know, rumored to be, you know, as much as 10% of Anthropic's business. Um, you know, they're generating way more efficiencies by optimizing their ad algorithms or what have you, and, and getting engagement time 5% longer and, you know, all these things. They're, they're making way more money off of using these models than, than Anthropic. And so the ultimately... You know, and that's, that's what's required. So sure, if you had a million new software engineers, the cost per software engineer would also fall.
- DPDwarkesh Patel
One thing I'm confused about is w- does the market come into equilibrium? And if it comes into equilibrium, would you just expect the price of compute to equal whatever Anthropic and OpenAI can generate from it or be very close to it with, like, a small amount of markup for Anthropic and OpenAI? Like, right now it's really weird that there's a four X or more difference between what compute sells for and how much money Anthropic can make from it. And in a world where the revenue per gigawatt continues to increase, if Anthropic's ability to monetize a gigawatt doubles or triples or something, it'd be weird if then the gap continued to increase. And so Anthropic just by, like, having some software, having some weights, can take something that cost them $10 and then turn, turn it into $100.
- DPDylan Patel
Yeah. So there is, there is a bit of, um... This is always a fun question, right, which is where does the value go in AI? AI's generating all this value. You've got, you know, the end user, which w- I think we all agree is generating more value than anyone else, hence they're paying-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... a lot for these models. But then you have, you know, the app layer. Well, so far the app layer has generated very little value. Um, then you've got the model layer, which again, up until bef- up until a year ago was generating negative gross margins and is now generating massive positive gross margins.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Um, and looks like it's on the path to generating, you know, $100 million per megawatt. Um, so turning, you know, $10, $15 into $100 as you said. Um, but if we go back again a year ago, the hardware supply chain was generating all this gross margin while literally everyone else was losing money on it. Um, OpenAI and Anthropic were just plowing VC money in. Um, and, and as were many other startups and, and many of these hyperscalers are building infrastructure without knowing if there was gonna be a payoff. Um, so ultimately you had this like, you know, negative value being created on the model layer almost, if you will. Um, because they were selling the tokens for less than it cost them on the infra side and all the value is being created, used at the chip, the fab. Initially in 2023, the memory guys were making no money off of, you know, HBM or memory for AI even though ge- theoretically their value they were delivering was humongous.
- DPDwarkesh Patel
Right.
- DPDylan Patel
Now you've got, well, actually KFC makes way less value than the memory guys. Um, is that actually how mu- You know, they're capturing less value, you know?
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
So, so the, the value capture has shifted around a lot-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... which is very fun for, um, people tracking the market or participating in the market like, like Jane Street as an example. [laughs] Um-
- DPDwarkesh Patel
[laughs] There's sponsors. You gotta unplug that.
- DPDylan Patel
This is not an ad. This is not an ad.
- DPDwarkesh Patel
You gotta unplug them that hard. [laughs]
- DPDylan Patel
[laughs] Um, so you know what happens, you know, going forward? Does Anthropic and OpenAI... You know, they've, they've, they've slowly started to balloon in value capture. Do they balloon and take all the value capture? Well, that was the thought and then, and then Elon showed, actually no, I can sell my compute for $25 million a megawatt or $40 million a megawatt to Anthropic and Google. Um, even if it's a short-term thing, I've sold it for this price, and I'll recoup my entire CapEx in a year.
- DPDwarkesh Patel
So what, what's your prediction of how much the relevant Toronto compute, like B300s or whatever that sold for 40B a gigawatt. That SpaceX sold for 40B a gigawatt to Google. What does that sell for at the end of next year?
- DPDylan Patel
I think most compute will still continue to transact at sub $20 billion a gigawatt.
- DPDwarkesh Patel
Even at the end of next year?
- DPDylan Patel
Because all of it has to be financed. For compute that you can build without financing, right? If, if Meta can build compute, Microsoft, Amazon, SpaceX can build compute without finding a customer just saying, "Fuck it, I'm gonna build this compute," and then turn around and wait till it's already built, they now con- control what's going on. So, so most compute is contracted well before it's built.
- 25:40 – 29:43
Will datacenter regulation slow down AI?
- DPDwarkesh Patel
What do you think their revenue per gigawatt is by the end of 2027? Like, for Anthropic or OpenAI by end of 2027?
- DPDylan Patel
I think, I think it's highly dependent on who has the best model, if they're allowed to keep releasing their best models, but I don't see why it wouldn't be $50-plus million a megawatt.
- DPDwarkesh Patel
By the end of '27?
- DPDylan Patel
Oh, by the end of '27?
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Um, that's where it gets more challenging, but I think, I think it could get to, you know, higher than that, to, like, seventy, eighty million dollars a megawatt, uh-
- DPDwarkesh Patel
This is low
- DPDylan Patel
... blended across a company.
- DPDwarkesh Patel
Th-
- DPDylan Patel
If not higher.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
Yeah. And so I think if, if that's the case, right, then what happens to the price of compute? Well, if I'm Anthropic, incremental compute is worth it. Maybe I spend forty million dollars a megawatt on SpaceX compute. Um, and if I'm SpaceX, you know, I, I look to the supply chain. I'm like, well, you know, I've struck this deal with Jensen where he's now all of a sudden using Twitter [laughs] . Um, and, and, you know, there's the, it-- Elon's saying they're exclusive to NVIDIA, but why doesn't Jensen raise his prices? And then, you know, h- SK Hynix and Micron and Samsung looked at NVIDIA and are like, "Well, why don't they raise their prices?" So I think, I think the value capture, there's a bullwhip effect here, right? Where, uh, just because someone has risen the prices doesn't mean the entire supply chain rebalances immediately.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
But over time, the supply chain will rebalance, and things will cost more and more. And, you know, to get that incremental capacity, you sort of have to, right? So TSMC raising prices very slowly but memory companies pr- raising prices very quickly, you know, substrate companies raising prices very quickly. Different parts of the supply chain raise, raise. You know, Elon wouldn't have sold if it was fifteen, but he's selling because it's twenty-five plus.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
So obviously, he rose his prices really quickly.
- DPDwarkesh Patel
Yeah. I, I'm sort of surprised you think, like, revenue per gigawatt doesn't increase way more than even, like, a hundred per-
- DPDylan Patel
Oh, I think, I think-
- DPDwarkesh Patel
... gigawatt by the end of next year.
- DPDylan Patel
When, when does RSI happen? When does take-off, right? I think, I think-
- DPDwarkesh Patel
Well, even if RSI doesn't happen, the current rate of progress continues. If you just look at how much progress have we made in, let's say, the last year and a half. Like, what was a model from a year and a half ago?
- DPDylan Patel
I, I mean-
- DPDwarkesh Patel
Let's say, like, Claude 3.5 or something
- DPDylan Patel
... my problem with this is the best model that exists in the world was trained in February.
- DPDwarkesh Patel
Okay, you're saying maybe we, we just weren't able, allowed to release the last, the best models.
- DPDylan Patel
Like, and OpenAI says they're not training models for two weeks, man. What the hell? [laughs]
- DPDwarkesh Patel
Yeah, yeah, yeah. I, I mean, there's another, there's one thing, like, internally are they getting enough use for it so they'll, like, bid up the price of compute? Or another is, like, does AI progress as a whole slow down because of regulation?
- DPDylan Patel
Yeah, but they're not even allowed to use this, like, new model intern-- Like, Astro's not widely deployed internally even.
- DPDwarkesh Patel
Right, right, right. Yeah, yeah. But still, I don't know. Just like the, the, uh, yeah, if you go, if you have, like, a model that is... What was the model released, like, let's say the beginning of last year? Like, GPT...
- DPDylan Patel
4.0? Was that 4.0?
- 29:43 – 33:27
Labs are shifting compute from inference to R&D
- DPDwarkesh Patel
As these companies go public and they're accountable to investors and they, let's say end of next year they have, I don't know, tw- tw- tw- close to twenty gigawatts, so, like, ten percent of the compute, two gigawatts. Let's say they want to go from sixty percent compute to, um, training to seventy percent compute to training, and their investors are like, "Well, if you're gonna be able to generate a hundred billion dollars per gigawatt, you're basically saying no to, like, two hundred billion dollars of revenue in order to increase your training compute." As investors are like, "What the fuck? You're already spending so much on training. Why are you spending even more on training?" As a public company, do you think- Yeah. What, what do you think would happen if they're just like, "No, we will keep increasing the share of compute we spent on training to offset the increase in revenue that each gigawatt of compute is giving us"?
- DPDylan Patel
Yeah, so, so this is sort of what I, I personally believe that the labs are gonna allocate less and less compute to inference-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... over time, which I think is very non-consensus, right? Everyone's sort of-- The standard belief of most people is, "Oh, most compute will go to inference." Um, most of it will go to forward passes for training, not maybe necessarily revenue-generating inference. But ultimately, you end up with, if they're generating, you know, thirty, forty million dollars per megawatt today, you allocate forty percent to inference. If you j- now get to gen-generating sixty, seventy million per megawatt, do you still allocate forty percent to inference and generate all this profit, and then do dividends and share buybacks? Or do you go build AGI?
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
And I think the obvious answer from Anthropic and OpenAI, and not just at the executive level, but also their board, is go build AGI 'cause it's way more profitable.
- DPDwarkesh Patel
Right.
- DPDylan Patel
Um, and, and so ultimately, you're going to see them ratchet up their percentage of compute dedicated to training-
- DPDwarkesh Patel
While each increment of compute is getting more and more profit generating if they had dedicated it to inference.
- DPDylan Patel
Right. And, and so the, the whole point is, well, okay, if I'm gener... If I'm selling tokens, is Anthrope-- is OpenAI releasing ultra-fast mode for just external, or are they doing it internally too? And it turns out, no, actually I'm gonna a-allocate it to internal and external because my internal, you know, value that I'm generating from super-fast AI or the best AI model is way more than what someone e-externally is.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
And so ultimately, sure, I could generate a hundred million dollars per megawatt, but if I turn that towards AI research, what is the incremental progress that I get, and then what does that do towards my future earnings potential, the discounted cash flows of whatever the hell I've done, right?
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
Um, and so, you know, they're not going through that calculation, but ultimately it's, it, it makes more sense to dedicate more and more compute internally. And the only reason to inc-- you know, have inference compute be so large is so you can grow your training fleet.
- DPDwarkesh Patel
Right, right, right. I think this is an interesting economics question that I, I feel like we can have the models digest of what is the-- at what, what would have to be true about a world where they reduce fraction of compute spent on inference?
- DPDylan Patel
I think they have been over the last three months already.
- DPDwarkesh Patel
Interesting.
- DPDylan Patel
I think, I think at s- parts of this year, they were increasing fraction of compute. So let's, let's just take month by month. You would agree that every month Anthropic has added more compute than the prior month. There might be some noise when they, like, sign a SpaceX deal or whatever. But in general, the amount of compute is, is a curve up. And so in January, they added less compute than December. And yet, their revenue adds, you know, skyrocketed, and then they've plat-- you know, sort of plateaued. They're only adding, you know, they're not adding twenty-five billion dollars-
- DPDwarkesh Patel
Mm
- DPDylan Patel
... of ARR every month now. And so that means the marginal megawatt they're getting is going higher percentage to R&D than it is inference.
- DPDwarkesh Patel
Interesting. Yeah.
- DPDylan Patel
And so they are factually increasing their compute towards R&D today.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
Yeah, I think this, this is, like, s-self-evident, um, if you, if you, like, look at what they're doing enough.
- DPDwarkesh Patel
Yeah.
- 33:27 – 48:48
China gets less than 10% of new compute, but its labs need less
- DPDwarkesh Patel
So I, if I look at the numbers you said of, like, how fast world compute grows, here, here are some things I wanna understand. So it seems like if I added the numbers you just said, it would be over two hundred gigawatts of world compute by the end of twenty twenty-eight, right?
- DPDylan Patel
Yeah, globally.
- DPDwarkesh Patel
Okay. And, uh, how fast can that continue growing, like, global, global AI compute after twenty twenty-eight?
- DPDylan Patel
Yeah, so thirty this year, fifty next year, seventy in 'twenty-eight. Um, 'twenty-nine should be, like, on the order of ninety to a hundred.
- DPDwarkesh Patel
Like, then just a hundred more every single year or something.
- DPDylan Patel
I think, I think the nu- the slope can continue to go upwards.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
I mean, it's hard to predict anything more than four years out-
- DPDwarkesh Patel
Yeah, yeah
- DPDylan Patel
... given, uh, [laughs] who knows what's, what's, you know, are we in RSI regime-
- DPDwarkesh Patel
Right
- DPDylan Patel
... or, you know, when is the world economy growing at ten percent a year?
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Uh, 'cause if you're at a hundred plus gigawatts a year, you're, you're at absurd revenue gr-- uh, uh, GDP growth.
- DPDwarkesh Patel
Right. If you think there's two hundred gigawatts globally in twenty twenty-eight, how much is in China by that point? And how does, like-- Yeah, how does Chinese compute continue increasing through this whole trend? Because if, if the RSI stuff kicks off in the West before China has a large amount of compute, maybe we're living in a different world than when it doesn't.
- DPDylan Patel
Yeah, so China today, um... So, so if we, if we sort of level set back to twenty twenty-two, the US was adding about forty-five to fifty percent of the world's compute. China was adding about thirty to thirty-five percent of the world's compute, and the rest being taken up by the rest of the world. Since twenty twenty-two, we've had big regulations against China, um, and a dramatic increase in America. So today, seventy percent of watts are being deployed in America. And, and, you know, China is, is really, uh, a very small number. It's sub-ten percent of watts being deployed for data center AI compute is in China. Um, and as we step forward, they're still at a very small number. Um, their domestic production is quite small. Their purchasing from Nvidia is still quite small. Um, and a lot of that ends up in, in other places as well, right? You know, Malaysia or what have you. Um, so ultimately, China domestically still continues to have ten per-- sub-ten percent of incremental new compute. So in twenty twenty-eight might start to inflect up, I think. Um, but it's pretty easy to say China will have, like, thirty gigawatts of AI compute or less.
- DPDwarkesh Patel
By twenty twenty-eight?
- DPDylan Patel
Yeah, in twenty twenty-eight.
- DPDwarkesh Patel
Okay. And then how fast does their hockey stick go up?
- DPDylan Patel
I, I do think in twenty twenty-eight, they have a big uplift in what compute they're able to deploy. Twenty twenty-six, they're still mostly relying on a lot of the smuggled chips. Um, you know, you know, a lot of the chips that, uh, TSMC made, uh, for companies that they thought weren't Huawei but ended up being Huawei or a lot of HBM that Samsung is shipping, you know, sort of. But in 'twenty-seven, fabs start to go up. In 'twenty-eight especially, fabs start to go up from SMIC and CXMT and such, where domestic production is actually reaching many millions of units a year. And now they're incrementally adding, you know, five, ten gigawatts in just twenty twenty-eight of domestically produced chips.
- DPDwarkesh Patel
Right. And what-
- DPDylan Patel
Those chips are definitely worse than the chips that- Nvidia will have in '28, or Google will have in '28, or OpenAI will have in 2028.
- DPDwarkesh Patel
So even, even the gigawatt number overstates things, you're saying. Like, uh, it's like thirty gigawatts, but it's really much worse chips. But then how-- Yeah, how does it... If you think the world is gonna add a hundred gigawatts the following year or some, you know-- I know you said you can't really say that far out. How much is China able to add the subsequent year? Basically, I wanna know, do they just hockey stick at the point at which they are able to start shipping large amounts of compute, or is it still gonna be less than U-US plus allies?
- DPDylan Patel
Um, there's a lot left to, you know, whether or not the US passes the MATCH Act, um, whether or not ex-tools continue to get export controlled, how fast China can build their new equipment that they're starting to be able to domest-- uh, produce, uh, domestically. Um, but ultimately, you know, China, China is definitely gonna hockey stick. If there's anything, uh, China's really good at is, is scaling manufacturing really, really quickly. Um, and, you know, I imagine, you know, China's-- China will start to be able to extract more and more purchasing of even foreign chips, um, into, into domestic China, or at least, uh, close the gap in what the US is allowing, uh, uh, you know, Nvidia to sell them or what have you.
- DPDwarkesh Patel
But, but do you think China could do adding fifty gigawatts by 2029? Marginal-- Incremental gigawatts in 2029.
- DPDylan Patel
I think that's, I think that's completely reasonable.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Um, and part of that could also be purchased from foreign.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Um, but yeah, I think it's completely reasonable that China in 2029 can do fifty gigs. Um, but if most of those are domestic chips, there is some factor there where that fifty gigawatts is really worth as much as twenty gigawatts in America, or-
- 48:48 – 1:07:52
Will AI cause a sovereign debt crisis?
- DPDwarkesh Patel
So you and I have been debating, uh, off air for the f- last few days whether there will be a sovereign debt crisis as a result of AI. And the logic is this: um, AI is... Y- you, you have a situation where, as we were mentioning, very little investment turns into a lot of money, right? So the rate of return-
- DPDylan Patel
What a fucking problem, dude.
- DPDwarkesh Patel
[laughs]
- DPDylan Patel
Oh, my God. [laughs] Can't believe it.
- DPDwarkesh Patel
[laughs] No, it is a huge problem for everybody else who can't turn a little money into a lot of money, right? Um, the- so the rate of return is incredibly high. Even at the data center level, you know, if you buil- if you, like, build a data center and you're, like, try- give- get, get rented out to Anthropic or an OpenAI for, like, 10X what it cost you on a depreciated basis to build it. It's fucking crazy. Um, and so you turn $1 into, like, $2 or $10 or something at the end of the year. That raises the rate of interest higher. Now, if the rate of interest goes higher, and if it does that for the entire economy and people are borrowing more and more money, they're competing against the other lending that the government would've done or that other companies would've done, or that you as a consumer or a mortgage buyer would've done, uh, then that's just making it m- basically more expensive for everybody else to borrow. This has huge implications for tons and tons of people. Sorry, I'm gonna go on a bit of a monologue here. Um, but we, we, we, we, we, we've been thinking about this together. So I think the US will be fine at the end of the day because they can... If the data centers are built in America, you can fundamentally just, like, tax the data centers. But the way the current tax system is set up, you know, corporate income is, like, less than 10% of federal revenues, and 80%-plus is payroll taxes and income taxes, which as more and more automation happens, will shrink. Um, at the same time, on the spending side, currently 20% of tax revenue spending goes towards paying... Servicing the debt, basically, paying interest payments on the debt. Um, now a lot of the debt is short duration, so it refurbishes every five years it rolls over. Why, why are you fucking laughing? [laughs]
- DPDylan Patel
'Cause, you know, it's like things we've lear- you've learned in the last [laughs] month.
- DPDwarkesh Patel
[laughs] Yeah, like it's any different for you. [laughs]
- DPDylan Patel
Yeah. I know.
- DPDwarkesh Patel
Like you got a degree in fucking financial economics.
- DPDylan Patel
I didn't. [laughs]
- DPDwarkesh Patel
[laughs]
- DPDylan Patel
I didn't. The internet thinks I'm a beekeeper.
- DPDwarkesh Patel
[laughs]
- DPDylan Patel
Few months, few months, few months.
- DPDwarkesh Patel
Yeah. Um. [laughs]
- DPDylan Patel
[laughs]
- DPDwarkesh Patel
This is our business, Dylan. [laughs]
- DPDylan Patel
I know, I know. Sorry, sorry.
- DPDwarkesh Patel
[laughs] Um, and, uh, so... [laughs]
- DPDylan Patel
[laughs]
- DPDwarkesh Patel
Now I'm self-conscious. Fuck. Um-
- DPDylan Patel
No, it's good. You're doing good.
- DPDwarkesh Patel
Yeah, yeah. [laughs]
- DPDylan Patel
I just think it's funny.
- DPDwarkesh Patel
Um-
- DPDylan Patel
Million people, listen to this guy who just learned about debt this month. [laughs]
- DPDwarkesh Patel
[laughs] Um, so you go from 20% of... 'Cause suppose interest rates rise, uh, 1%, then the, um, over a five-year basis, the amount of- the fraction of tax revenue that goes towards servicing the debt basically goes from 20% to 25%. If it rises five percentages, that would go towards, like, north of 40%. But if you take into account the fact that the government is borrowing $2 trillion every single year, then that goes from, like, 40% to, like, north of 60%. So 60% of tax revenue basically just goes towards paying interest payments on the debt. Now, I think the US is gonna be fine because also the tax base will increase if we let data centers get built in America. Um, other countries are absolutely fucked in my opinion. I was just looking at which countries have a lot of debt, have very little tax revenue, and also a lot of their debt is serviced quite often. And those countries, like Pakistan or Nigeria or something, I think are just gonna be very fucked in this new interest rate regime.
- DPDylan Patel
So, so, so this, this, this crowding out effect is actually, like, the thing that I've, like... is the reason it's not, like, YOLO one billion gigawatts.
- DPDwarkesh Patel
Yeah, yeah.
- DPDylan Patel
Right? Um, you've got, you've got all these industries and countries that use a lot of debt, whether it's, you know, all these impoverished countries that you mentioned earlier that are just gonna default. You've got, like, consumer packaged goods, right? Like, all of these, like, companies that make things you see at Trader Joe's or wherever use a lot of debt. All these telecom companies use a lot of debt. And banks use a lot of debt. And so if interest rates go up, um, in the market, not necessarily the government set interest rate, but in the market, the spread of interest rate, uh, between what the government says their federal rate is versus what everyone else is charging because Amazon wants to raise $100 billion of debt next year or whatever the hell the number is, um, you know, probably less. But, um, you end up with this, like, really challenging problem of where does the cash come from? Um, there is some level that is funded by cash flows, and cash flows keep going up. But the logical thing to do is to invest way more than your cash flows because then the returns in the future years-
- 1:07:52 – 1:16:52
Will the world's future workforce belong to a few companies?
- DPDwarkesh Patel
One thing I find crazy about these scenarios is just how much of the world's future labor supply ends up in, uh, very few companies, and also how fast that labor supply grows year over year. So if, like, compute at the frontier, you know, in FLOP terms is growing four or five x a year. And further, the compute required to achieve those capabilities is, like, decreasing three x a year. So the compute at the frontier, or the, basically the effective AI population size at the frontier lab is increasing ten x year over year. And so that doesn't really matter that much right now because AIs are not good enough to do full jobs or be, like, as autonomous as people in their capacity to do work or s- pull off schemes or whatever. But if the current trend continues, you have a world where OpenAI goes from having, say, ten million basically AI laborers this year to a hundred million the next year to a billion the year after that. And then pretty soon, even if compute scaling slows down, it doesn't take many more years before each company individually has more labor equivalents than there are people on Earth. Um, and I think that's, like, a thing that is very plausible by the end of this decade. That there's more AI labor, m- more effective population within a single lab, um, than there are people on Earth. So when we talk often about centralization of power because of nationalization or whatever, but we don't think enough about the fact that we're, we're actually moving very fast into a regime where most AI labor, or sorry, most, most people, like, in terms of, like, the, a work output or something, is just, like, concentrated within two labs who are consuming more and more of the world's compute. And so if, if these AIs are misaligned, then most of the world is misaligned, basically, 'cause, like, most of the world's minds are there. Um, but even if they're not, it just-- very few companies have, like, a lot of influence or a lot of control-
- DPDylan Patel
Yeah, it's, it's, it's sort of, um... There was the whole spat recently where it's like, um, I think Gavin Baker was like, Anthro-- Dario believes that there's only gonna be one company in the world.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
And then, you know, Sholto and Dario came out and were like, "No, no, no. We didn't say that." Um, but ultimately, you know, if you believe in RSI, you believe in the labs are the most effective, uh, user of compute and can generate the most, um, value from the compute, then the only thing that's gonna happen is centralization of compute. And if you believe in, you know, sort of AI researchers, RSI, AGI-
- DPDwarkesh Patel
So this is even true-
- DPDylan Patel
Then all of this exists. All of this is the base.
- DPDwarkesh Patel
This is even true if there's no RSI. The current effective, like, effective population of the frontier is currently increasing 10x year over year for a given level of capabilities, right? So if you get to the level of capabilities, which is a human, um, a very competent remote worker or, like, a very competent software engineer or a very competent researcher, that population of those are, like, 10x year over year at the current rate of capa- uh, current rate of capabilities grow.
- DPDylan Patel
I see. And w- without RSI.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Then once you have RSI, it's even crazier.
- DPDwarkesh Patel
Then it's, like, g- maybe growing, like, 100x a year or 1,000x a year. Or they're, like, intelligence is increasing, but the population isn't increasing, or some m- mixture of the two, right?
- DPDylan Patel
Um, yeah. I mean, I guess, I guess, like, what world do you see, Dwarkesh, where everything is not centralized? Um, because it seems to me that every force is screeching towards centralization.
- DPDwarkesh Patel
Right.
- DPDylan Patel
And that's scary as hell.
- DPDwarkesh Patel
Yeah.
- DPDylan Patel
Um, I don't, I don't... You know, I would love for it not to be centralized completely. Um, but maybe that's, that's the whole point of a machi- a, a machine that loves grace, right? Is, is it is everything, and it makes our lives great.
- DPDwarkesh Patel
Yeah, it's so hard to think about the future. Um, but I agree with you that-- I think the fundamental problem is that lab-- AI training has huge economies of scale because any effort you spend into training an AI for a specific skill or a specific set of knowledge gets amortized across billions of sessions or billions of users. Um, furthermore-- So that's, like, one effect. The other effect is if you're slightly ahead in the AI race and compute is in shortage, you can charge a much higher markup because you can better economize this scarce resource. So there's, like, two effects which are give more and more to the person who's, like, ahead in the AI race. There may be more, right? So if there's, if models are learning from deployment, and one model is, like, deployed much more widely than another one, and it's getting much more, like, real world data.
- DPDylan Patel
Yeah, your point, your point is taken that, like, whether it's user deployment and continual learning, um, whether it's, uh, training, having these economies of scales, um, whether it's the incremental progress that im- the best AI model helps you to make the next a-
- DPDwarkesh Patel
Yeah
- DPDylan Patel
... AI model, um, RSI, all of these things-
- DPDwarkesh Patel
Oh, so I didn't even mention RSI.
- DPDylan Patel
All of these things point to centralization.
- DPDwarkesh Patel
So I think one of the big intellectual projects, honestly, um, that, uh, yeah, we should spend some time thinking about, uh, Async, or at least I'll spend some time thinking about, is what is a vision of, like, a decentralized, broadly empowered future after AGI that takes these economies of scale seriously? The alternative vision is that the government controls it, and maybe you think that you can trust the government more because it's not a private corporation.
- DPDylan Patel
I don't trust the government, and I don't trust Dario, and I don't trust Sam. [laughs]
- DPDwarkesh Patel
Yeah, yeah. That's the problem, right? Um, but there's no, at least... Obviously, obviously, it's, like, very easy to be wrong about the future, and you don't anticipate a key effect or something that changes everything. But ex ante, it's very hard to see a reason why there... Or, like, how we avoid a scenario where we have to choose one source of centralization.
- DPDylan Patel
I mean, it's, it's why capitalism worked, right? It's the decentralized decision-making and decentralized power.
- DPDwarkesh Patel
Right.
- DPDylan Patel
And why super centralized capitalistic economies actually grew slower than super decentralized capitalist economies to some extent. You have to have rule of law and all this. But then AI flips all this on its head.
- DPDwarkesh Patel
Right.
- DPDylan Patel
And ultimately, you're like, actually, private ownership is probably not the most efficient economy, and therefore it grows slower than an AI economy, which is centralized.
Episode duration: 1:16:52
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode aV26V1UvkJw