Skip to content
Dwarkesh PodcastDwarkesh Podcast

Dylan Patel – Two labs will soon control most of the world's workforce

Had a lot of fun chatting again with my twin brother Dylan Patel. We went through lab economics over the next few years - the shift from inference to training as RSI draws near; and how Anthropic and OpenAI are on track to control most of the world’s usable FLOPs within the next few years (because they can monetize compute better and thus outbid everyone). And then we discuss whether the $10T+ of total AI capex we’ll see by the end of the decade will cause a sovereign debt crisis, where hyperscaler debt raises interest rates, drives non-AI exposed countries into bankruptcy, and crashes non-AI equities. One question we weren’t able to resolve is whether there’s anything that can counter all the forces barrelling towards centralization in this industry - the economies of scale in training, the scarcity of compute, and eventually continual learning and RSI. 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒 * Transcript: https://www.dwarkesh.com/p/dylan-patel-3 * Apple Podcasts: https://podcasts.apple.com/us/podcast/dylan-patel-anthropic-openai-will-have-most-of/id1516093381?i=1000785793715 * Spotify: https://open.spotify.com/episode/1chA0sqLyHUL684tUEE3ek?si=Dgplc0zpTz-H6O642qY5mA 𝐒𝐏𝐎𝐍𝐒𝐎𝐑𝐒 * Grok Bot has been quite helpful with my search for a new editor. I created a recruiter bot and described the type of editor I was looking for. That bot then spun up a handful of subagents that combed through my emails and X DMs, read the end credits of various documentaries I like, and figured out who edits for some of my favorite YouTubers. It took all of those results, and then delivered me a shortlist of candidates that matched my criteria. Try Grok Bot for yourself at https://x.ai/bot * Antithesis lets you add time travel to your software testing toolkit. Since the Antithesis platform is fully deterministic, everything that happens inside of it is perfectly reproducible. So if your software crashes, you can rewind to the exact right moment, freeze time, and investigate. Or you can test different hypotheses by perturbing the system: kill a node or disable a feature, see what happens, then reset the trajectory and try something else. Learn more at https://antithesis.com/dwarkesh * Jane Street is hiring for two separate ML internships right now, one focused primarily on research and one focused on engineering. In both cases, interns are expected to contribute to real work, not contrived exercises: one common project is adapting a frontier LLM paper to financial markets, which tend to come with a ton of different gnarly challenges. Importantly, you don’t need any finance background to apply. 2027 applications are open now at https://janestreet.com/dwarkesh To sponsor a future episode, visit https://dwarkesh.com/advertise. 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 00:00:00 – Two labs will soon control most of the world’s compute 00:07:01 – $6 billion in fab capex enables $1t+ of end revenue 00:13:08 – Compute prices will rise if the labs outbid everyone 00:18:22 – Which layer will capture most of the surplus? 00:25:40 – Will datacenter regulation slow down AI? 00:29:43 – Labs are shifting compute from inference to R&D 00:33:27 – China gets less than 10% of new compute, but its labs need less 00:48:48 – Will AI cause a sovereign debt crisis? 01:07:52 – Will the world's future workforce belong to a few companies?

Dwarkesh PatelhostDylan Patelguest
Aug 25, 20261h 16mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:007:01

    Two labs will soon control most of the world’s compute

    1. DP

      Okay, I'm back with Dylan Patel, founder of SemiAnalysis. Our version of a family Thanksgiving dinner [laughs] is a regular yearly podcast. We are not actually related.

    2. DP

      Don't tell the people this.

    3. DP

      [laughs] It will destroy the myth. Um, walk me through-- Uh, so basically, where the world economy is headed is more and more becoming a function of where, like, lab economics are headed, where, like, the compute market is headed, et cetera. So I want to understand where the crazy future ends up within a few years. But, uh, let's start with just where we are today. So walk me through lab compute and lab revenue right now, and maybe projecting out a year or two.

    4. DP

      Yeah. So when we go back to last year, even at the end of the year, most of GDP growth in America was just AI infrastructure. And as we look towards this year, about a third of the compute coming online is for the labs, for OpenAI and Anthropic. Now, it may be built by others and then rented to them, but it's... In, in... At the end customer, it's them. Um, as we go forward into the future, the, the numbers for compute are ballooning, right? We're at, you know, you know, a little bit over a trillion dollars of CapEx this year. As we go out into '28, it's gonna be more than two trillion dollars. Um, the labs are also taking an increasing percentage of this, and so ultimately, you've got a very interesting situation where the labs are going from companies that spend, you know, tens of billions of dollars a year to hundreds of billions of dollars a year to forecasting to spend trillions of dollars a year even, um, at, towards the end of the decade. And this is at least some of the contracts they've begun signing with their partners. And so e-this requires a big reshaping of what happens with their, um, economics, right? You know, so up until now, they have been companies that mostly lost money. Um, Anthropic started turning a profit in Q2. Um, it's believed at some point in Q3, OpenAI could potentially start turning a profit even, um, with the big rise of Codex and, and 5.6 and all this. Um, but if we go back a year ago, everything that they... All the money they had was venture-funded losses, right? If we go back to even the beginning of this year, it was venture-funded losses. Um, they've, they've now turned the corner and are actually starting to profit. Now, that doesn't mean they're not taking in new capital. The new capital's still coming in to accelerate the growth further. But ultimately, there's more and more of their business is being funded off of their own revenue rather than capital injections into them. Over the last, you know, year and a half, their margins have really skyrocketed. You know, the, the base cost of compute tends to be around ten or thirteen or fifteen million dollars per megawatt. The most interesting aspect about what's happening now is before, again, they were generating... If they served a model, right? GPT-4 being served on, um, you know, NVIDIA Hopper GPUs, was generating negative gross margin for OpenAI. But now, when OpenAI serves GPT 5.6 or Anthropic serves me- Opus-5 or Mythos, uh, Fable-5, their revenue generation has w-passed well beyond the sort of incremental ten, fifteen million dollars per megawatt. In the case of Anthropic, the, the revenue has gone as high as fifty, uh, million dollars per megawatt. Um, and, and what that now enables them to do is, "Hey, if I spend ten bucks on inference capacity, I actually generate fifty bucks of revenue, and then I can turn around and incrementally spend all of that profit on training."

    5. DP

      One thing I'm very interested in understanding is how you see the centralization of compute happening at the labs or the relative ratio of compute that goes to the world versus goes to the labs. Um, where if you say right now a third of marginal compute is going to the labs, by when is it over half of the incremental compute in the world is going to the labs? And by what point do the labs have basically a vast majority of the world's compute?

    6. DP

      Yeah. So, so earlier this year, you know, the beginning of this year, Anthropic, OpenAI started at two for OpenAI and less than two for Anthropic. Um, end of this year, they're both above five. Um, so they've three, four X compute as a whole. Um, when you, when you look at the incremental compute added, that's about thirty percent of the compute added this year. And as we step forward to next year, given what's already been signed and penned and inked, you've got something even more dramatic, right? You've got Anthropic, OpenAI are taking, uh, as much as forty to fifty percent of compute, uh, next year. And the centralization doesn't look like it's slowing down or stopping. In fact, it looks like it's only accelerating.

    7. DP

      Right.

    8. DP

      Now, who's building that compute for them will change. Um, you know, next year, a big new entrant is, for example, SpaceX is building a ton of compute, and they're actively going to lease quite a bit of it to Anthropic and OpenAI most likely, 'cause they're the ones who can... who have the marginal capability to pay the highest price. In addition, OpenAI and Anthropic are also starting to build their own compute. OpenAI with their own chips, Anthropic with TPUs that they're purchasing w- from Google and deploying with Fluid Stack. And so when you ask, "Hey, when does half of the world's incremental new compute go to just OpenAI and Anthropic?" I mean, it's, it's really by the end of next year-

    9. DP

      Mm.

    10. DP

      -it's already half of the incremental compute is going to Anthropic and OpenAI.

    11. DP

      Be-because, uh, compute is growing so fast, incremental compute is gonna be basically most of compute. So it's very soon, you're saying maybe within a year and a half or two years, that most of the world's compute is owned by two labs, or at least is serving the demand from two labs. Um, how long do you think... So there's this trend where maybe world compute in gigawatts doubles every year, but the compute at the frontier labs triples every single year. But if you keep the current trend going, it goes from, like, two at the beginning of this year to close to, like, six at the end of this year. Just multiplying out by three. Eighteen by the end of twenty twenty-seven, fifty-four by the end of twenty twenty-eight. Are you like, "Okay, at that point, it, they simply can't continue tripling given the amount of world compute?" Or how, how do you see the world compute situation over the next few years?

    12. DP

      Yeah. So if the incremental compute adds this year thirty gigawatts, next year fifty gigawatts, and the year after that seventy, roughly, you end up with this really interesting phenomenon, which is Okay, well, a, a new watt deployed this year is significantly more efficient than the watts deployed two years ago.

    13. DP

      Mm-hmm.

    14. DP

      So actually, you know, a humongous percentage of the world's compute was deployed this year.

    15. DP

      Yeah.

    16. DP

      Even though it didn't double the number of watts deployed, I'm deploying GB300s and TPU v7s and Trainium 3s, which are way, way, way more efficient. You know, 3X, 5X more performance per watt than-

    17. DP

      Mm

    18. DP

      ... the prior generation chips. And so ultimately you've got a huge, um, ladder here. So if Anthropic and OpenAI take on, you know, 45% of compute next year, you've, you've got them in, let's say, December '27, they have taken on half of the world's incremental new compute. But that half of the world's new incremental compute is actually at a higher performance than everything else before it.

    19. DP

      Right.

    20. DP

      So you've got an- another multiplier on that. So by the time you're in, like towards the end of 2028, if this trend continues, which I see nothing that's stopping it, um, you, you've got them just controlling most of the usable, you know, FLOPs in the world on their own.

  2. 7:0113:08

    $6 billion in fab capex enables $1t+ of end revenue

    1. DP

      The thing I'm confused about is why you think we only add 80 gigawatts in 2028 if we enter in a world in which the pri- the value of compute increases so much.

    2. DP

      That's the upper bound, by the way [laughs] . That's the, that's the like, "I'm so fucking bullish."

    3. DP

      Right. Okay, so le- le- le- let's, le- let's do some chain of thought here. So when I interviewed you a few months ago, you said in order to make a gigawatt of, I think, Vera Rubins, you need... One sec. You need 55,000 N3 wafers, 6K N5 wafers, and 170K DRAM wafers. Um, I don't know if those numbers might have changed.

    4. DP

      I'm gonna troll you, but the way you said "wafers" was so fucking Indian. Wafers. [laughs]

    5. DP

      [laughs] By the way, when we first, uh, when we first moved to the US, I had the, the V-W thing pretty bad, and I was a vegetarian. [laughs]

    6. DP

      A vegetarian. I remember you told me about this. [laughs]

    7. DP

      [laughs] Inside, in North Dakota, I was in elementary school, and I'd be like [laughs] -

    8. DP

      "Can I get a wedgie?"

    9. DP

      "Can I get some wedgies?" [laughs]

    10. DP

      [laughs]

    11. DP

      Anyways, so that's for one gigawatt, right?

    12. DP

      Yeah.

    13. DP

      Now, I had an LLM run your wafer fab equipment model and figure out how much, um, tooling, how much the tooling costs to produce a gigawatt of compute basically every single year. And it was at, like $3 to $4 billion. Now suppose you add in, you know, clean rooms and, uh, shell and everything else at the fab. So $6 billion of like fab capex produces every single year a gigawatt, and a gigawatt produces right now $100 billion [laughs] of revenue. But also that $6 billion in capex is producing a gigawatt every single year, and that gigawatt is producing $100 billion every single year. So even over the course of five years, so, you know, the, the first gigawatt is generated five years of profits. The second gigawatt that the fab has produced has generated four years of profits, and so on. Um, $6 billion of capex at the fab level will have generated over a trillion dollars of end rev- end AI revenue.

    14. DP

      Yeah. There's a lot of opex along the way.

    15. DP

      Of course, of course.

    16. DP

      There's a lot of, um, other capex, like the data center, the power-

    17. DP

      And you'd pay like a, you know, the OpenAI for the R&D-

    18. DP

      Installation

    19. DP

      Exactly. Of course, of course.

    20. DP

      There's a lot of different people who need money here

    21. DP

      So but, but take away, take away half-

    22. DP

      But yeah, it's, there's a huge

    23. DP

      ... take away half of it for all these pe- all these middlemen. That still means there's 100X discrepancy between fab capex and end revenue generated. More than that, actually, really, but, uh, we're just being very conservative. And as a result, this is capitalism, right? Like, you would imagine that people are gonna figure... Like, we're gonna be... You're, you're, you have this huge discrepancy where you can turn $1 into $100, and you're, they're not gonna figure out a way to make more mirrors?

    24. DP

      I mean, they are.

    25. DP

      Right.

    26. DP

      It's just these mirrors take some time to make, right? Like, you know?

    27. DP

      But the, but the, the emergency are so big where, like, Anthropic and OpenAI are like, "We could make $1 trillion right now, but we're just bottlenecked on the mirrors that go into the ASML machines." Like, they'd spend... Okay, how, how can we make more mirrors if we spend $100 billion on this, right? That's the situation we're gonna be in pretty soon. And I'm just like, we're not gonna be able to solve that supply constraint? That just seems quite hard to imagine.

    28. DP

      No, there's definitely, um... You, you, you've seen people do funny arbitrages here where they buy, like turbines, and then they try and resell them.

    29. DP

      Right.

    30. DP

      Because the value of a turbine is way more because it's the thing bottlenecking your data center.

  3. 13:0818:22

    Compute prices will rise if the labs outbid everyone

    1. DP

      the key question I really wanna understand is if, um, yeah, if the current consumption use would be like north of fifty gigawatts per lab by the end of twenty twenty-eight. So between them, they'd have a hundred gigawatts. Um, those gigawatts, as you're saying, drive many fold more throughput or more performance by twenty twenty-eight than they are now, right? Because the hardware's gotten better. So not only have like FLOPs per watt increased, but also the hardware gets better at s- um, working with AI workloads. Okay, so a hundred gigawatts for the labs end of twenty twenty-eight. How much is like world compute?

    2. DP

      I think that may be a little difficult given twenty twenty-eight you start to have... They've taken seventy, eighty percent of incremental compute, and I'm not sure what happens to markets then.

    3. DP

      Mm.

    4. DP

      Right? You know, how much does the price of compute skyrocket for them to actually be able to buy seventy, eighty percent of compute? Is, you know, Google or Meta or my Amazon willing to sell even that much? Um, also one caveat when we're sort of talking about these gigawatt numbers is, you know, when Amazon is serving Bedrock Anthropic models, that counts as Anthropic compute in sort of our-

    5. DP

      Yeah

    6. DP

      ... worldview because it is effectively, at the end of the day, counted as revenue for Anthropic-

    7. DP

      Yeah

    8. DP

      ... um, even though like there's a revenue share and credit back-

    9. DP

      Yeah

    10. DP

      ... all that. Um, but ultimately in twenty twenty-eight it's, it's, you know, if they get to a hundred gigawatts combined, they have done really disruptive things to the market because anyone can make money off of ten to fifteen million dollars per megawatt compute today. You literally-- Like I kid you not, it's not that hard. Go get a GB300 rack, go download the Kimi weights, go download VLM or SGLang, set it up. You know, Codex and Fable can actually help you do this. It's pretty simple. I, I m- mean, it's not like it's... You know, it's not trivial, but it's not like rocket science, and go put it on Open Router. It's very simple. Um, and, and you'll start generating more revenue than you're paying for the compute. Um, and so this is, this is sort of already led to this compute pricing ten to fifteen million dollars per megawatt start to inflect up. Um, and to get to that hundred gigawatts in twenty twenty-eight, you have to believe that the labs can outpay for compute, because anyone can make money at ten to fifteen. You, you know, does compute now get to twenty-five million dollars a megawatt? Does it get to forty million dollars a megawatt?

    11. DP

      But as you're saying, it's already the case that the labs are generating way more revenue per megawatt than everybody else. If they stay as far ahead as they are currently, you would expect that to be the continuing case. If there's like some kind of recursive self-improvement where the AI labs are like relatively uplifted, or they have models internally they're not releasing externally that are helping them make their next model better, you'd expect it- that to be even more the case. Aren't you already seeing this where like SpaceX or whoever is like slightly further behind will just sell compute to the highest bidder if they can't internally monetize it as well as the labs? So y- I feel like it's, it-- continue expecting them to be able to gobble up l-- like bid for larger and larger shares of the compute-

    12. DP

      I, I think that is my worldview, that they will continue to gobble up more of the compute.

    13. DP

      Yeah.

    14. DP

      But ultimately, they can't do it at, at current pricing or anywhere close to it.

    15. DP

      Sure, sure.

    16. DP

      They, they do have to start paying twenty-five, thirty, fifty million dollars a megawatt to really gobble up seventy percent of the world's-

    17. DP

      Right

    18. DP

      ... compute in twenty twenty-eight-

    19. DP

      Yeah

    20. DP

      ... to get to that hundred gigawatts by twenty twenty-eight, which is a very sort of aggressive goal. The other aspect of this that's really challenging is we've already seen a huge slowdown for the AI labs, right? This, this regulation that they advocate for is actually slowing down the labs a lot more than it slows down, you know, sort of the open source Chinese language models. Um, you know, OpenAI not releasing, uh, Astra. OpenAI stopping training for two weeks. Anthropic not releasing what their safety assessment set as Model 2, which is widely believed to be the next version of Mythos. They're clearly not releasing their best models, and in which case, their revenue per megawatt stalls or even can start to decline again because other models are competitive again. Um, so it's not that they're falling behind, it's just that they're not releasing their best stuff.

    21. DP

      Yeah, yeah.

    22. DP

      What if there is some regulatory impact that prevents them from releasing their best models? Now, their revenue per megawatt does not climb as fast, then their ability to d-- buy that incremental compute for a higher price than everyone else starts to diminish, and then maybe they can't get to that hundred gigawatts is, is sorta... I- in a world where safety doesn't matter, I, I do believe that's exactly what happens, right? They can start generating a hundred million dollars per megawatt or more, and they can pay fifty million dollars a megawatt, and no one else has any logical reason to do anything with their compute besides say, "Please, Dario, take everything off of my hands." Um, but there are, you know, forces at play, uh, that which we cannot describe [chuckles] , um, that, that would potentially slow this down.

    23. DP

      Yeah, yeah, yeah. I mean, I, I think a good intuition pump is just how-- what if the AI models were literally as good as a fully automated software engineer? They're not currently there yet, right? Like I think they're far from just being able to fully au-automate the job of like a full white collar worker. But white collar workers earn, you know, six figures or north of that a year. Um, and if you have a gigawatt that can sustain a population of like, say, a million of white collar workers, just let's, let's say roughly, right? Um, that's like-- You could then off the back of that- That would be 100 billion. That's actually surprisingly low. [laughs]

    24. DP

      [laughs] Yeah, 100K per person, million population, yeah?

    25. DP

      Yeah, yeah, yeah.

    26. DP

      Um-

    27. DP

      I don't know. But it would be many hundreds of billions of dollars if you get, like, full AGI,

  4. 18:2225:40

    Which layer will capture most of the surplus?

    1. DP

      uh, per, per gigawatt.

    2. DP

      I, I think the other aspect of this is, and we've continued to see this, the most of the value capture is not happening, right? Like, most of the value that these models generate-

    3. DP

      For sure

    4. DP

      ... does not get given to OpenAI and Anthropic. Um, thankfully so far it is mostly just being given to the users.

    5. DP

      Yeah, yeah.

    6. DP

      Right? Jane Street with their exclusive contract with OpenAI for GPT 5.6 ultra-fast mode or Jane Street, where they're, like, one of Anthropic's biggest customers, is generating way, way, way, way more value out of the tokens they're paying for than Anthropic is-

    7. DP

      Sure

    8. DP

      ... uh, generating in terms of profit, right? 'Cause they get to, you know, make money off of the market.

    9. DP

      Yeah.

    10. DP

      Um, or Meta, who at one point was, you know, rumored to be, you know, as much as 10% of Anthropic's business. Um, you know, they're generating way more efficiencies by optimizing their ad algorithms or what have you, and, and getting engagement time 5% longer and, you know, all these things. They're, they're making way more money off of using these models than, than Anthropic. And so the ultimately... You know, and that's, that's what's required. So sure, if you had a million new software engineers, the cost per software engineer would also fall.

    11. DP

      One thing I'm confused about is w- does the market come into equilibrium? And if it comes into equilibrium, would you just expect the price of compute to equal whatever Anthropic and OpenAI can generate from it or be very close to it with, like, a small amount of markup for Anthropic and OpenAI? Like, right now it's really weird that there's a four X or more difference between what compute sells for and how much money Anthropic can make from it. And in a world where the revenue per gigawatt continues to increase, if Anthropic's ability to monetize a gigawatt doubles or triples or something, it'd be weird if then the gap continued to increase. And so Anthropic just by, like, having some software, having some weights, can take something that cost them $10 and then turn, turn it into $100.

    12. DP

      Yeah. So there is, there is a bit of, um... This is always a fun question, right, which is where does the value go in AI? AI's generating all this value. You've got, you know, the end user, which w- I think we all agree is generating more value than anyone else, hence they're paying-

    13. DP

      Yeah

    14. DP

      ... a lot for these models. But then you have, you know, the app layer. Well, so far the app layer has generated very little value. Um, then you've got the model layer, which again, up until bef- up until a year ago was generating negative gross margins and is now generating massive positive gross margins.

    15. DP

      Yeah.

    16. DP

      Um, and looks like it's on the path to generating, you know, $100 million per megawatt. Um, so turning, you know, $10, $15 into $100 as you said. Um, but if we go back again a year ago, the hardware supply chain was generating all this gross margin while literally everyone else was losing money on it. Um, OpenAI and Anthropic were just plowing VC money in. Um, and, and as were many other startups and, and many of these hyperscalers are building infrastructure without knowing if there was gonna be a payoff. Um, so ultimately you had this like, you know, negative value being created on the model layer almost, if you will. Um, because they were selling the tokens for less than it cost them on the infra side and all the value is being created, used at the chip, the fab. Initially in 2023, the memory guys were making no money off of, you know, HBM or memory for AI even though ge- theoretically their value they were delivering was humongous.

    17. DP

      Right.

    18. DP

      Now you've got, well, actually KFC makes way less value than the memory guys. Um, is that actually how mu- You know, they're capturing less value, you know?

    19. DP

      Yeah.

    20. DP

      So, so the, the value capture has shifted around a lot-

    21. DP

      Yeah

    22. DP

      ... which is very fun for, um, people tracking the market or participating in the market like, like Jane Street as an example. [laughs] Um-

    23. DP

      [laughs] There's sponsors. You gotta unplug that.

    24. DP

      This is not an ad. This is not an ad.

    25. DP

      You gotta unplug them that hard. [laughs]

    26. DP

      [laughs] Um, so you know what happens, you know, going forward? Does Anthropic and OpenAI... You know, they've, they've, they've slowly started to balloon in value capture. Do they balloon and take all the value capture? Well, that was the thought and then, and then Elon showed, actually no, I can sell my compute for $25 million a megawatt or $40 million a megawatt to Anthropic and Google. Um, even if it's a short-term thing, I've sold it for this price, and I'll recoup my entire CapEx in a year.

    27. DP

      So what, what's your prediction of how much the relevant Toronto compute, like B300s or whatever that sold for 40B a gigawatt. That SpaceX sold for 40B a gigawatt to Google. What does that sell for at the end of next year?

    28. DP

      I think most compute will still continue to transact at sub $20 billion a gigawatt.

    29. DP

      Even at the end of next year?

    30. DP

      Because all of it has to be financed. For compute that you can build without financing, right? If, if Meta can build compute, Microsoft, Amazon, SpaceX can build compute without finding a customer just saying, "Fuck it, I'm gonna build this compute," and then turn around and wait till it's already built, they now con- control what's going on. So, so most compute is contracted well before it's built.

  5. 25:4029:43

    Will datacenter regulation slow down AI?

    1. DP

      What do you think their revenue per gigawatt is by the end of 2027? Like, for Anthropic or OpenAI by end of 2027?

    2. DP

      I think, I think it's highly dependent on who has the best model, if they're allowed to keep releasing their best models, but I don't see why it wouldn't be $50-plus million a megawatt.

    3. DP

      By the end of '27?

    4. DP

      Oh, by the end of '27?

    5. DP

      Yeah.

    6. DP

      Um, that's where it gets more challenging, but I think, I think it could get to, you know, higher than that, to, like, seventy, eighty million dollars a megawatt, uh-

    7. DP

      This is low

    8. DP

      ... blended across a company.

    9. DP

      Th-

    10. DP

      If not higher.

    11. DP

      Yeah, yeah.

    12. DP

      Yeah. And so I think if, if that's the case, right, then what happens to the price of compute? Well, if I'm Anthropic, incremental compute is worth it. Maybe I spend forty million dollars a megawatt on SpaceX compute. Um, and if I'm SpaceX, you know, I, I look to the supply chain. I'm like, well, you know, I've struck this deal with Jensen where he's now all of a sudden using Twitter [laughs] . Um, and, and, you know, there's the, it-- Elon's saying they're exclusive to NVIDIA, but why doesn't Jensen raise his prices? And then, you know, h- SK Hynix and Micron and Samsung looked at NVIDIA and are like, "Well, why don't they raise their prices?" So I think, I think the value capture, there's a bullwhip effect here, right? Where, uh, just because someone has risen the prices doesn't mean the entire supply chain rebalances immediately.

    13. DP

      Yeah.

    14. DP

      But over time, the supply chain will rebalance, and things will cost more and more. And, you know, to get that incremental capacity, you sort of have to, right? So TSMC raising prices very slowly but memory companies pr- raising prices very quickly, you know, substrate companies raising prices very quickly. Different parts of the supply chain raise, raise. You know, Elon wouldn't have sold if it was fifteen, but he's selling because it's twenty-five plus.

    15. DP

      Yeah, yeah.

    16. DP

      So obviously, he rose his prices really quickly.

    17. DP

      Yeah. I, I'm sort of surprised you think, like, revenue per gigawatt doesn't increase way more than even, like, a hundred per-

    18. DP

      Oh, I think, I think-

    19. DP

      ... gigawatt by the end of next year.

    20. DP

      When, when does RSI happen? When does take-off, right? I think, I think-

    21. DP

      Well, even if RSI doesn't happen, the current rate of progress continues. If you just look at how much progress have we made in, let's say, the last year and a half. Like, what was a model from a year and a half ago?

    22. DP

      I, I mean-

    23. DP

      Let's say, like, Claude 3.5 or something

    24. DP

      ... my problem with this is the best model that exists in the world was trained in February.

    25. DP

      Okay, you're saying maybe we, we just weren't able, allowed to release the last, the best models.

    26. DP

      Like, and OpenAI says they're not training models for two weeks, man. What the hell? [laughs]

    27. DP

      Yeah, yeah, yeah. I, I mean, there's another, there's one thing, like, internally are they getting enough use for it so they'll, like, bid up the price of compute? Or another is, like, does AI progress as a whole slow down because of regulation?

    28. DP

      Yeah, but they're not even allowed to use this, like, new model intern-- Like, Astro's not widely deployed internally even.

    29. DP

      Right, right, right. Yeah, yeah. But still, I don't know. Just like the, the, uh, yeah, if you go, if you have, like, a model that is... What was the model released, like, let's say the beginning of last year? Like, GPT...

    30. DP

      4.0? Was that 4.0?

  6. 29:4333:27

    Labs are shifting compute from inference to R&D

    1. DP

      As these companies go public and they're accountable to investors and they, let's say end of next year they have, I don't know, tw- tw- tw- close to twenty gigawatts, so, like, ten percent of the compute, two gigawatts. Let's say they want to go from sixty percent compute to, um, training to seventy percent compute to training, and their investors are like, "Well, if you're gonna be able to generate a hundred billion dollars per gigawatt, you're basically saying no to, like, two hundred billion dollars of revenue in order to increase your training compute." As investors are like, "What the fuck? You're already spending so much on training. Why are you spending even more on training?" As a public company, do you think- Yeah. What, what do you think would happen if they're just like, "No, we will keep increasing the share of compute we spent on training to offset the increase in revenue that each gigawatt of compute is giving us"?

    2. DP

      Yeah, so, so this is sort of what I, I personally believe that the labs are gonna allocate less and less compute to inference-

    3. DP

      Yeah

    4. DP

      ... over time, which I think is very non-consensus, right? Everyone's sort of-- The standard belief of most people is, "Oh, most compute will go to inference." Um, most of it will go to forward passes for training, not maybe necessarily revenue-generating inference. But ultimately, you end up with, if they're generating, you know, thirty, forty million dollars per megawatt today, you allocate forty percent to inference. If you j- now get to gen-generating sixty, seventy million per megawatt, do you still allocate forty percent to inference and generate all this profit, and then do dividends and share buybacks? Or do you go build AGI?

    5. DP

      Yeah, yeah.

    6. DP

      And I think the obvious answer from Anthropic and OpenAI, and not just at the executive level, but also their board, is go build AGI 'cause it's way more profitable.

    7. DP

      Right.

    8. DP

      Um, and, and so ultimately, you're going to see them ratchet up their percentage of compute dedicated to training-

    9. DP

      While each increment of compute is getting more and more profit generating if they had dedicated it to inference.

    10. DP

      Right. And, and so the, the whole point is, well, okay, if I'm gener... If I'm selling tokens, is Anthrope-- is OpenAI releasing ultra-fast mode for just external, or are they doing it internally too? And it turns out, no, actually I'm gonna a-allocate it to internal and external because my internal, you know, value that I'm generating from super-fast AI or the best AI model is way more than what someone e-externally is.

    11. DP

      Yeah, yeah.

    12. DP

      And so ultimately, sure, I could generate a hundred million dollars per megawatt, but if I turn that towards AI research, what is the incremental progress that I get, and then what does that do towards my future earnings potential, the discounted cash flows of whatever the hell I've done, right?

    13. DP

      Yeah, yeah.

    14. DP

      Um, and so, you know, they're not going through that calculation, but ultimately it's, it, it makes more sense to dedicate more and more compute internally. And the only reason to inc-- you know, have inference compute be so large is so you can grow your training fleet.

    15. DP

      Right, right, right. I think this is an interesting economics question that I, I feel like we can have the models digest of what is the-- at what, what would have to be true about a world where they reduce fraction of compute spent on inference?

    16. DP

      I think they have been over the last three months already.

    17. DP

      Interesting.

    18. DP

      I think, I think at s- parts of this year, they were increasing fraction of compute. So let's, let's just take month by month. You would agree that every month Anthropic has added more compute than the prior month. There might be some noise when they, like, sign a SpaceX deal or whatever. But in general, the amount of compute is, is a curve up. And so in January, they added less compute than December. And yet, their revenue adds, you know, skyrocketed, and then they've plat-- you know, sort of plateaued. They're only adding, you know, they're not adding twenty-five billion dollars-

    19. DP

      Mm

    20. DP

      ... of ARR every month now. And so that means the marginal megawatt they're getting is going higher percentage to R&D than it is inference.

    21. DP

      Interesting. Yeah.

    22. DP

      And so they are factually increasing their compute towards R&D today.

    23. DP

      Yeah, yeah.

    24. DP

      Yeah, I think this, this is, like, s-self-evident, um, if you, if you, like, look at what they're doing enough.

    25. DP

      Yeah.

  7. 33:2748:48

    China gets less than 10% of new compute, but its labs need less

    1. DP

      So I, if I look at the numbers you said of, like, how fast world compute grows, here, here are some things I wanna understand. So it seems like if I added the numbers you just said, it would be over two hundred gigawatts of world compute by the end of twenty twenty-eight, right?

    2. DP

      Yeah, globally.

    3. DP

      Okay. And, uh, how fast can that continue growing, like, global, global AI compute after twenty twenty-eight?

    4. DP

      Yeah, so thirty this year, fifty next year, seventy in 'twenty-eight. Um, 'twenty-nine should be, like, on the order of ninety to a hundred.

    5. DP

      Like, then just a hundred more every single year or something.

    6. DP

      I think, I think the nu- the slope can continue to go upwards.

    7. DP

      Yeah.

    8. DP

      I mean, it's hard to predict anything more than four years out-

    9. DP

      Yeah, yeah

    10. DP

      ... given, uh, [laughs] who knows what's, what's, you know, are we in RSI regime-

    11. DP

      Right

    12. DP

      ... or, you know, when is the world economy growing at ten percent a year?

    13. DP

      Yeah.

    14. DP

      Uh, 'cause if you're at a hundred plus gigawatts a year, you're, you're at absurd revenue gr-- uh, uh, GDP growth.

    15. DP

      Right. If you think there's two hundred gigawatts globally in twenty twenty-eight, how much is in China by that point? And how does, like-- Yeah, how does Chinese compute continue increasing through this whole trend? Because if, if the RSI stuff kicks off in the West before China has a large amount of compute, maybe we're living in a different world than when it doesn't.

    16. DP

      Yeah, so China today, um... So, so if we, if we sort of level set back to twenty twenty-two, the US was adding about forty-five to fifty percent of the world's compute. China was adding about thirty to thirty-five percent of the world's compute, and the rest being taken up by the rest of the world. Since twenty twenty-two, we've had big regulations against China, um, and a dramatic increase in America. So today, seventy percent of watts are being deployed in America. And, and, you know, China is, is really, uh, a very small number. It's sub-ten percent of watts being deployed for data center AI compute is in China. Um, and as we step forward, they're still at a very small number. Um, their domestic production is quite small. Their purchasing from Nvidia is still quite small. Um, and a lot of that ends up in, in other places as well, right? You know, Malaysia or what have you. Um, so ultimately, China domestically still continues to have ten per-- sub-ten percent of incremental new compute. So in twenty twenty-eight might start to inflect up, I think. Um, but it's pretty easy to say China will have, like, thirty gigawatts of AI compute or less.

    17. DP

      By twenty twenty-eight?

    18. DP

      Yeah, in twenty twenty-eight.

    19. DP

      Okay. And then how fast does their hockey stick go up?

    20. DP

      I, I do think in twenty twenty-eight, they have a big uplift in what compute they're able to deploy. Twenty twenty-six, they're still mostly relying on a lot of the smuggled chips. Um, you know, you know, a lot of the chips that, uh, TSMC made, uh, for companies that they thought weren't Huawei but ended up being Huawei or a lot of HBM that Samsung is shipping, you know, sort of. But in 'twenty-seven, fabs start to go up. In 'twenty-eight especially, fabs start to go up from SMIC and CXMT and such, where domestic production is actually reaching many millions of units a year. And now they're incrementally adding, you know, five, ten gigawatts in just twenty twenty-eight of domestically produced chips.

    21. DP

      Right. And what-

    22. DP

      Those chips are definitely worse than the chips that- Nvidia will have in '28, or Google will have in '28, or OpenAI will have in 2028.

    23. DP

      So even, even the gigawatt number overstates things, you're saying. Like, uh, it's like thirty gigawatts, but it's really much worse chips. But then how-- Yeah, how does it... If you think the world is gonna add a hundred gigawatts the following year or some, you know-- I know you said you can't really say that far out. How much is China able to add the subsequent year? Basically, I wanna know, do they just hockey stick at the point at which they are able to start shipping large amounts of compute, or is it still gonna be less than U-US plus allies?

    24. DP

      Um, there's a lot left to, you know, whether or not the US passes the MATCH Act, um, whether or not ex-tools continue to get export controlled, how fast China can build their new equipment that they're starting to be able to domest-- uh, produce, uh, domestically. Um, but ultimately, you know, China, China is definitely gonna hockey stick. If there's anything, uh, China's really good at is, is scaling manufacturing really, really quickly. Um, and, you know, I imagine, you know, China's-- China will start to be able to extract more and more purchasing of even foreign chips, um, into, into domestic China, or at least, uh, close the gap in what the US is allowing, uh, uh, you know, Nvidia to sell them or what have you.

    25. DP

      But, but do you think China could do adding fifty gigawatts by 2029? Marginal-- Incremental gigawatts in 2029.

    26. DP

      I think that's, I think that's completely reasonable.

    27. DP

      Yeah.

    28. DP

      Um, and part of that could also be purchased from foreign.

    29. DP

      Yeah.

    30. DP

      Um, but yeah, I think it's completely reasonable that China in 2029 can do fifty gigs. Um, but if most of those are domestic chips, there is some factor there where that fifty gigawatts is really worth as much as twenty gigawatts in America, or-

  8. 48:481:07:52

    Will AI cause a sovereign debt crisis?

    1. DP

      So you and I have been debating, uh, off air for the f- last few days whether there will be a sovereign debt crisis as a result of AI. And the logic is this: um, AI is... Y- you, you have a situation where, as we were mentioning, very little investment turns into a lot of money, right? So the rate of return-

    2. DP

      What a fucking problem, dude.

    3. DP

      [laughs]

    4. DP

      Oh, my God. [laughs] Can't believe it.

    5. DP

      [laughs] No, it is a huge problem for everybody else who can't turn a little money into a lot of money, right? Um, the- so the rate of return is incredibly high. Even at the data center level, you know, if you buil- if you, like, build a data center and you're, like, try- give- get, get rented out to Anthropic or an OpenAI for, like, 10X what it cost you on a depreciated basis to build it. It's fucking crazy. Um, and so you turn $1 into, like, $2 or $10 or something at the end of the year. That raises the rate of interest higher. Now, if the rate of interest goes higher, and if it does that for the entire economy and people are borrowing more and more money, they're competing against the other lending that the government would've done or that other companies would've done, or that you as a consumer or a mortgage buyer would've done, uh, then that's just making it m- basically more expensive for everybody else to borrow. This has huge implications for tons and tons of people. Sorry, I'm gonna go on a bit of a monologue here. Um, but we, we, we, we, we, we've been thinking about this together. So I think the US will be fine at the end of the day because they can... If the data centers are built in America, you can fundamentally just, like, tax the data centers. But the way the current tax system is set up, you know, corporate income is, like, less than 10% of federal revenues, and 80%-plus is payroll taxes and income taxes, which as more and more automation happens, will shrink. Um, at the same time, on the spending side, currently 20% of tax revenue spending goes towards paying... Servicing the debt, basically, paying interest payments on the debt. Um, now a lot of the debt is short duration, so it refurbishes every five years it rolls over. Why, why are you fucking laughing? [laughs]

    6. DP

      'Cause, you know, it's like things we've lear- you've learned in the last [laughs] month.

    7. DP

      [laughs] Yeah, like it's any different for you. [laughs]

    8. DP

      Yeah. I know.

    9. DP

      Like you got a degree in fucking financial economics.

    10. DP

      I didn't. [laughs]

    11. DP

      [laughs]

    12. DP

      I didn't. The internet thinks I'm a beekeeper.

    13. DP

      [laughs]

    14. DP

      Few months, few months, few months.

    15. DP

      Yeah. Um. [laughs]

    16. DP

      [laughs]

    17. DP

      This is our business, Dylan. [laughs]

    18. DP

      I know, I know. Sorry, sorry.

    19. DP

      [laughs] Um, and, uh, so... [laughs]

    20. DP

      [laughs]

    21. DP

      Now I'm self-conscious. Fuck. Um-

    22. DP

      No, it's good. You're doing good.

    23. DP

      Yeah, yeah. [laughs]

    24. DP

      I just think it's funny.

    25. DP

      Um-

    26. DP

      Million people, listen to this guy who just learned about debt this month. [laughs]

    27. DP

      [laughs] Um, so you go from 20% of... 'Cause suppose interest rates rise, uh, 1%, then the, um, over a five-year basis, the amount of- the fraction of tax revenue that goes towards servicing the debt basically goes from 20% to 25%. If it rises five percentages, that would go towards, like, north of 40%. But if you take into account the fact that the government is borrowing $2 trillion every single year, then that goes from, like, 40% to, like, north of 60%. So 60% of tax revenue basically just goes towards paying interest payments on the debt. Now, I think the US is gonna be fine because also the tax base will increase if we let data centers get built in America. Um, other countries are absolutely fucked in my opinion. I was just looking at which countries have a lot of debt, have very little tax revenue, and also a lot of their debt is serviced quite often. And those countries, like Pakistan or Nigeria or something, I think are just gonna be very fucked in this new interest rate regime.

    28. DP

      So, so, so this, this, this crowding out effect is actually, like, the thing that I've, like... is the reason it's not, like, YOLO one billion gigawatts.

    29. DP

      Yeah, yeah.

    30. DP

      Right? Um, you've got, you've got all these industries and countries that use a lot of debt, whether it's, you know, all these impoverished countries that you mentioned earlier that are just gonna default. You've got, like, consumer packaged goods, right? Like, all of these, like, companies that make things you see at Trader Joe's or wherever use a lot of debt. All these telecom companies use a lot of debt. And banks use a lot of debt. And so if interest rates go up, um, in the market, not necessarily the government set interest rate, but in the market, the spread of interest rate, uh, between what the government says their federal rate is versus what everyone else is charging because Amazon wants to raise $100 billion of debt next year or whatever the hell the number is, um, you know, probably less. But, um, you end up with this, like, really challenging problem of where does the cash come from? Um, there is some level that is funded by cash flows, and cash flows keep going up. But the logical thing to do is to invest way more than your cash flows because then the returns in the future years-

  9. 1:07:521:16:52

    Will the world's future workforce belong to a few companies?

    1. DP

      One thing I find crazy about these scenarios is just how much of the world's future labor supply ends up in, uh, very few companies, and also how fast that labor supply grows year over year. So if, like, compute at the frontier, you know, in FLOP terms is growing four or five x a year. And further, the compute required to achieve those capabilities is, like, decreasing three x a year. So the compute at the frontier, or the, basically the effective AI population size at the frontier lab is increasing ten x year over year. And so that doesn't really matter that much right now because AIs are not good enough to do full jobs or be, like, as autonomous as people in their capacity to do work or s- pull off schemes or whatever. But if the current trend continues, you have a world where OpenAI goes from having, say, ten million basically AI laborers this year to a hundred million the next year to a billion the year after that. And then pretty soon, even if compute scaling slows down, it doesn't take many more years before each company individually has more labor equivalents than there are people on Earth. Um, and I think that's, like, a thing that is very plausible by the end of this decade. That there's more AI labor, m- more effective population within a single lab, um, than there are people on Earth. So when we talk often about centralization of power because of nationalization or whatever, but we don't think enough about the fact that we're, we're actually moving very fast into a regime where most AI labor, or sorry, most, most people, like, in terms of, like, the, a work output or something, is just, like, concentrated within two labs who are consuming more and more of the world's compute. And so if, if these AIs are misaligned, then most of the world is misaligned, basically, 'cause, like, most of the world's minds are there. Um, but even if they're not, it just-- very few companies have, like, a lot of influence or a lot of control-

    2. DP

      Yeah, it's, it's, it's sort of, um... There was the whole spat recently where it's like, um, I think Gavin Baker was like, Anthro-- Dario believes that there's only gonna be one company in the world.

    3. DP

      Yeah.

    4. DP

      And then, you know, Sholto and Dario came out and were like, "No, no, no. We didn't say that." Um, but ultimately, you know, if you believe in RSI, you believe in the labs are the most effective, uh, user of compute and can generate the most, um, value from the compute, then the only thing that's gonna happen is centralization of compute. And if you believe in, you know, sort of AI researchers, RSI, AGI-

    5. DP

      So this is even true-

    6. DP

      Then all of this exists. All of this is the base.

    7. DP

      This is even true if there's no RSI. The current effective, like, effective population of the frontier is currently increasing 10x year over year for a given level of capabilities, right? So if you get to the level of capabilities, which is a human, um, a very competent remote worker or, like, a very competent software engineer or a very competent researcher, that population of those are, like, 10x year over year at the current rate of capa- uh, current rate of capabilities grow.

    8. DP

      I see. And w- without RSI.

    9. DP

      Yeah.

    10. DP

      Then once you have RSI, it's even crazier.

    11. DP

      Then it's, like, g- maybe growing, like, 100x a year or 1,000x a year. Or they're, like, intelligence is increasing, but the population isn't increasing, or some m- mixture of the two, right?

    12. DP

      Um, yeah. I mean, I guess, I guess, like, what world do you see, Dwarkesh, where everything is not centralized? Um, because it seems to me that every force is screeching towards centralization.

    13. DP

      Right.

    14. DP

      And that's scary as hell.

    15. DP

      Yeah.

    16. DP

      Um, I don't, I don't... You know, I would love for it not to be centralized completely. Um, but maybe that's, that's the whole point of a machi- a, a machine that loves grace, right? Is, is it is everything, and it makes our lives great.

    17. DP

      Yeah, it's so hard to think about the future. Um, but I agree with you that-- I think the fundamental problem is that lab-- AI training has huge economies of scale because any effort you spend into training an AI for a specific skill or a specific set of knowledge gets amortized across billions of sessions or billions of users. Um, furthermore-- So that's, like, one effect. The other effect is if you're slightly ahead in the AI race and compute is in shortage, you can charge a much higher markup because you can better economize this scarce resource. So there's, like, two effects which are give more and more to the person who's, like, ahead in the AI race. There may be more, right? So if there's, if models are learning from deployment, and one model is, like, deployed much more widely than another one, and it's getting much more, like, real world data.

    18. DP

      Yeah, your point, your point is taken that, like, whether it's user deployment and continual learning, um, whether it's, uh, training, having these economies of scales, um, whether it's the incremental progress that im- the best AI model helps you to make the next a-

    19. DP

      Yeah

    20. DP

      ... AI model, um, RSI, all of these things-

    21. DP

      Oh, so I didn't even mention RSI.

    22. DP

      All of these things point to centralization.

    23. DP

      So I think one of the big intellectual projects, honestly, um, that, uh, yeah, we should spend some time thinking about, uh, Async, or at least I'll spend some time thinking about, is what is a vision of, like, a decentralized, broadly empowered future after AGI that takes these economies of scale seriously? The alternative vision is that the government controls it, and maybe you think that you can trust the government more because it's not a private corporation.

    24. DP

      I don't trust the government, and I don't trust Dario, and I don't trust Sam. [laughs]

    25. DP

      Yeah, yeah. That's the problem, right? Um, but there's no, at least... Obviously, obviously, it's, like, very easy to be wrong about the future, and you don't anticipate a key effect or something that changes everything. But ex ante, it's very hard to see a reason why there... Or, like, how we avoid a scenario where we have to choose one source of centralization.

    26. DP

      I mean, it's, it's why capitalism worked, right? It's the decentralized decision-making and decentralized power.

    27. DP

      Right.

    28. DP

      And why super centralized capitalistic economies actually grew slower than super decentralized capitalist economies to some extent. You have to have rule of law and all this. But then AI flips all this on its head.

    29. DP

      Right.

    30. DP

      And ultimately, you're like, actually, private ownership is probably not the most efficient economy, and therefore it grows slower than an AI economy, which is centralized.

Episode duration: 1:16:52

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode aV26V1UvkJw

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.