Dylan Patel — The Single Biggest Bottleneck to Scaling AI Compute

All right, this is the episode where my roommate teaches me semiconductors. [laughs]

[laughs] It's also the send-off for this, uh, this current set.

It's fi-- yeah, you're-- you know, after you use it, I'm like, "I can't use this again." [laughs]

[laughs] Oh.

I gotta get out of here.

No, no sloppy seconds for Dwarkesh. [laughs]

[laughs] Okay, Dylan is the, uh, CEO of SemiAnalysis. Dylan, the burning question I have for you, um, if you add up the big four, Amazon, Meta, Google, Microsoft, their combined, uh, forecasted capex that y-you published recently this year is six hundred billion dollars. And given, uh, y-you know, yearly prices of renting that compute, that would be, like, close to fifty gigawatts. Now, obviously, we're not putting on fifty gigawatts this year, so presumably that's paying for compute that is gonna be coming online over the coming years. So I have a question about w-what-- how to think about the timeline ar-a-around when that capex comes online. Similar question for the labs, where, you know, OpenAI just announced that they raised a hundred and ten billion dollars. Anthropic just announced they raised thirty billion dollars. And if you look at the compute that they have coming online this year, um, you, you should tell me how much it is, but, like, is it not, is it in another four gigawatts total that they'll have this year? It feels like the cost to rent the compute that OpenAI and Anthropic will have this year to, like, sustain their compute spend at, you know, ten, thirteen billion dollars a gigawatt, th-those individual raises alone are, like, w-enough to cover their compute spend for the year, and then this is not even including the revenue that they're gonna earn this year. So help me understand first, when is the timescale at which the big tech capex is actually coming online? And two, what are the labs raising all this money for if, like, the, the yearly price of a, a one gigawatt data center is, like, thirteen billion dollars?

So when you talk about the capex of these hyperscalers, right, on the order of six hundred billion dollars, and you look at the cross, the rest of the supply chain, gets you to on the order of a trillion dollars. A portion of this is, you know, immediately for compute going online this year, right? The chips and the, uh, the, the other parts of capex that do get paid this year. But there's a lot of setup capex as well, right? So when we have-- when we're talking about twenty gigawatts this year in America, roughly-

Incremental

... incremental added capacity, a portion of this is not spent this year. A portion of that capex is actually spent the prior year. And so when you look at, hey, Google's got a hundred and eighty billion dollars, actually a ch- big chunk of that is spent on turbine deposits for '28 and '29. A chunk of that is spent on data center construction for '27. A chunk of that is spent on, you know, power purchasing agreements and down payments and all these other things that they're doing, uh, for further out into the future so that they can set up this super fast scaling, right? And, and, and this applies to all the hyperscalers and other people in the supply chain. And so, you know, twenty gigawatts roughly deployed this year, um, a big chunk of that being hyperscalers, chunk not being... And all of these companies, their biggest customers are Anthropic and OpenAI. Um, Anthropic and OpenAI are in the, you know, two gigawatt and, you know, two and a half gigawatt and one and a half gigawatts roughly right now. They're trying to scale to much larger, right? If you look at what Anthropic has done over the last few months, you know, four billion, six billion revenue added, and if we just draw a straight line, hey, yeah, they'll add another six billion dollars of revenue a month. Uh, people would argue that's bearish and that they should go faster. What that implies is that they're gonna add sixty billion dollars of revenue across the next ten months, right? And sixty billion dollars of revenue at the current gross margins that Anthropic had at least last, uh, reported by media, um, would imply that they have, you know, roughly forty billion dollars of compute spend for that inference, for that sixty bill of revenue. That forty billion of compute at roughly ten billion dollars a gigawatt, um, rental cost means that they need to add four gigawatts of inference capacity just to grow revenue, and that's saying that their research and development training fleet stays flat, right? So, you know, in a sense, Anthropic needs to get to well above five gigawatts by the end of this year, and it's gonna be really tough for them to get there, but it's possible.