The Twenty Minute VCGroq Founder, Jonathan Ross: OpenAI & Anthropic Will Build Their Own Chips & Will NVIDIA Hit $10TRN
EVERY SPOKEN WORD
150 min read · 30,017 words- 0:00 – 1:10
Intro
- JRJonathan Ross
The countries that control compute will control AI, and you cannot have compute without energy.
- HSHarry Stebbings
So I'm thrilled to welcome Jonathan Ross, founder and CEO at Grok, back to the hot seat.
- JRJonathan Ross
And now we're going to be able to add more labor to the economy by producing more compute and better AI. That has never happened in the history of the economy before. What is that gonna do? I personally would be surprised if in five years NVIDIA wasn't worth 10 trillion, but I can't predict the outcome. The demand for compute is insatiable. If OpenAI were given twice the inference compute that they have today, if Anthropic was given twice the inference compute that they have today, within one month from now, their revenue would almost double.
- HSHarry Stebbings
I'm sorry, can you unpa that for me? (dramatic music) Ready to go? Jonathan, you've just been told by our team that our last show was the most successful of, uh, the year when it came out, so there's no pressure at all that this is gonna be the most successful of this year. But welcome to the studio, man. (laughs)
- JRJonathan Ross
Thank you.
- HSHarry Stebbings
It's great to have you here, dude.
- 1:10 – 3:33
Analyzing the Current Market Landscape
- HSHarry Stebbings
Now, I, I wanted to start with a understanding of where we are. It seems the world moves faster than ever before, and honestly, I think a lot of us are trying to understand where everyone lies in a new market. If we'd look at the current state of the market today, how do you analyze it?
- JRJonathan Ross
Are you asking is there a bubble?
- HSHarry Stebbings
Relatively.
- JRJonathan Ross
Okay. So, (laughs) um, in terms of whether or not there's a bubble, uh, my answer is if you ask a question and you keep not getting an answer, maybe you should ask a different question. And so instead of asking, "Is there a bubble?" you should ask, "What is the smart money doing?" So what is Google doing? What is Microsoft doing? Amazon? What are some nations doing? And they're all doubling down on AI. They're spending more. Um, y- like, every time they make an announcement on how much they're spending, it goes up the next time. And one of the best examples of the value that's coming from this spend, Microsoft in one quarter deployed a bunch of GPUs and then announced that they weren't going to make them available in Azure because they made more money using them themselves than renting them out. So there's real money in the market, and the best way that I, I think to explain this market is like the early days of oil drilling, a lot of dry holes and a couple of gushers. I think the stat that I heard was, um, 35, uh, companies or 36 companies are responsible for 99% of the revenue, uh, or at least the token spend, um, in AI right now. Yeah. It's very lumpy. And so-
- HSHarry Stebbings
I'm surprised it's not less.
- JRJonathan Ross
(laughs)
- HSHarry Stebbings
When you look at... No, but I mean, seriously, NVIDIA really, you know, having concentration of revenue with two clients so heavily.
- JRJonathan Ross
Yeah, and maybe NVIDIA represents 98% of that. (laughs) But, um, when it's that lumpy, what that's an indication of is it's like the early days of the oil drilling where people didn't know how to find oil. They were going off of instinct, uh, you know, almost vibe investing, um, and, uh, people who had a good instinct would make a fortune and everyone else would lose their shirts. Over time, it becomes a science. It becomes very predictable, and there's less, uh, lumpiness. Um, there's more predictability. But, um, investors make less money at that point. The, the, the good investors make less money. So right now is the best time for investors. Right now, people are making more money than they're spending. It's just very lumpy.
- 3:33 – 12:35
Why the Hyperscalers Have to Keep Spending Recklessly on AI
- JRJonathan Ross
- HSHarry Stebbings
I'm sorry. They're making more money than they're spending?
- JRJonathan Ross
Oh, as an aggregate. Plenty of people are gonna lose their shirts, but overall, less money is gonna go in than is gonna come out.
- HSHarry Stebbings
But when we look at the capex spend today by the big providers-
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
... everyone is going, "Okay, okay, okay," because there's something coming at the end of it.
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
And the trouble is the capex spend is going up and up and up.
- JRJonathan Ross
Okay. So y- you're thinking of it purely financially, and I think that the financial returns will be positive. But that's not why people are motivated. So I was in Abu Dhabi at the inaugural Goldman Sachs Abu Dhabi event, and, um, I went and, and, um, you know, as you now know, uh, we're sponsoring McLaren, and so, um, Zak Brown was talking, I was talking, and it was a fun event. But I was asked, uh, a similar question, like, "Is AI a bubble?" And, um, I asked the following question. Everyone... So this is, like, a bunch of people who manage 10 billion plus in AUM, right? The entire o- like, 50 plus people who manage 10 billion plus. I'm like, "Who here is 100% convinced that in 10 years AI won't be able to do your job?" No hands went up. I'm like, "Great." That's how the hyperscalers feel. So of course they're gonna be spending like drunken sailors, because the alternative is that they're completely locked out of their business. So it's not a purely economical framework that they're using. It's a, "Do we get to maintain our leadership?" Now, when you look at it the next step, there are these, um, you know, scale law sort of outcomes. You want to remain in the top 10, right? We keep talking about the Mag Seven. If you're not a member of the Mag Seven, (clears throat) you're not gonna be able to get anywhere near the valuation. And so what do you do to stay there? You spend. And it's worth it because the stock value stays up because you're in the top seven or 10.
- HSHarry Stebbings
At some point, the returns have to be delivered, though.
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
The spend has to materialize into actual tangible revenue back, and if it doesn't, whether you're in the Mag Seven or not doesn't, doesn't matter. Correct?
- JRJonathan Ross
That's correct. But, um, right now, AI's returning massive value already. It's very lumpy in the allo- uh, in the applications, but it's returning massive amounts of value. Let me talk about an example that actually happened for us. So I've tried a little bit of vibe coding. Um, I'm not the best in the world at it. We've got some, uh, interns who are amazing at it. And we, we had this customer visit us, or, um, and, and I had a meeting with them.And so they asked for a feature, and I spec'd it out, very high level, vibey. Um, so I was prompt engineering the engineers. And four hours later, it was in production. Not a single line of code was written by a human being. There was no debugging done by a human being. It was all prompting. Um, I think we even have Slack integration now, where you push in, like, you commit things through Slack. So all that was done. Four hours later, it's in production. Think about the value there. But now, imagine, fast-forward six months from now when that could happen before the customer meeting's over. It's a qualitative difference. It's not even just a dollar amount difference. Yes, um, you know, when you're able to do it that fast, you spend less to get the feature into production. That's real, uh, ROI. However, qualitatively, when you can do that before the customer meeting is over, you're gonna be able to win deals that your competitors won't.
- HSHarry Stebbings
Can I ask you, just going back to the Mag 7, to stay in the Mag 7, do you think everyone realizes that they will need to move into the chip layer and own the full vertical end-to-end?
- JRJonathan Ross
Um, I don't think you're gonna see too many successfully moving into the chip layer. So w- people look at the TPU as a big success. And what they don't realize is that there were about three chip efforts at Google at the same time, uh, and only one of them ended up, uh, outperforming GPUs. And when you look around the industry, you've got a bunch of people building chips. Some of them are getting canceled, like Dojo recently got canceled. Building chips is hard. I, going off and saying, "I'm gonna build my own AI chip to compete with NVIDIA," it's a little bit like saying, you know, "That Google search is pretty nice. Let's go replicate it." It's insane. Like, the level of optimization, the level of design and engineering that goes into that, um, you're not gonna be able to replicate it with a high probability of success. However, if there's a bunch of players out there trying to do it, and you have optionality, and one of them succeeds, then you have another chip.
- HSHarry Stebbings
We mentioned earlier that you have to spend if you want to stay in Mag 7.
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
NVIDIA are investing $100 billion into OpenAI for OpenAI just to go and buy back NVIDIA chips.
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
Is this not just an infinite money loop?
- JRJonathan Ross
Um, that would be the case if they weren't spending it with suppliers to build those chips. It's not round tripping if actual productive outcomes are occurring. So, um, think of it this way. How, what percentage of the spend is going to building that infrastructure? 40%? So at least 40% of those dollars are actually going out into the ecosystem. So that is not an infinite loop.
- HSHarry Stebbings
Okay, so it's a partial loop. 60%...
- JRJonathan Ross
It's a partial loop, yeah.
- HSHarry Stebbings
60%'s going back to NVIDIA?
- JRJonathan Ross
Sure.
- HSHarry Stebbings
And then they get a bump in their stock price of a couple of hundred billion dollars?
- JRJonathan Ross
Yes.
- HSHarry Stebbings
How did you analyze that?
- JRJonathan Ross
Here's... So let's analyze it in a couple of different ways. From, from an economic point of view, makes perfect sense. Why not do that all day long? Um, the value accrues if there is lock-in, right? That's where, like, when revenue increases result in stock price increases that are greater than the amount of the revenue, it's because you believe that that revenue is gonna continue. And that's the belief. And I would actually say with NVIDIA, that's probably true. However, it's not just because NVIDIA's good. And NVIDIA is very good. It's also because there isn't enough compute in the world. There isn't. It's insa- the demand for compute is insatiable. Um, I would wager that if OpenAI were given twice the inference compute that they have today, if Anthropic was given twice the inference compute that they have today, that within one month from now, their revenue would almost double.
- HSHarry Stebbings
I'm sorry, can you unpack that for me?
- 12:35 – 18:14
Why OpenAI and Anthropic Will Have to Build Their Own Chips
- JRJonathan Ross
- HSHarry Stebbings
Do you think OpenAI will be able to move into the chip layer? At some point, NVIDIA must be concerned that the OpenAI will want to verticalize and own the chip layer as well. Do you think they will be able to make that successful transition?
- JRJonathan Ross
Um, I think one of the problems in building your own chip is it's really f- first of all, everyone thinks that building the chip is the hard part. Uh, and then as you do it, you start to realize building the software is the hard part. And then as you do it, you- you realize keeping up with where everything is going starts to become the hard part. I have no doubt that, uh, OpenAI will be able to build its own chips. I have no doubt that, um, eventually Anthropic will be building their own chips, that every hyperscaler will build their own chip. (clears throat) One of the things that, um, w- I- I had this experience when I was at Google, where, um, I- I got a lab tour. And this was before AMD was doing a great job, right? AMD was struggling for a little while and now they're doing great. But, um, they had built 10,000 servers and those 10,000 servers of AMD chips, I was walking through the lab and they were pulling the servers out of the racks, taking the AMD chip, popping it off, and throwing it in a trash can. And the funny thing was, it was almost preordained because everyone knew that in that generation Intel was gonna win. So, why did Google build 10,000 servers? Because they wanted to, um, get a discount on the Intel chips they bought. And when you're at that scale, the cost to design your own server, 'cause they had to design their own motherboard in order to fit the AMD chip, uh, and- and to build that out and test it, versus the discount that you get, totally worth it. So, you have to think of what all the motivations are when people are building their own chips. It's not just because they're gonna deploy that chip in mass production. (clears throat) The- the thing is NVIDIA effectively has a monopsony on HBM. A monopsony is the opposite of a monopoly, so when you're a single buyer. And there's a finite amount of HBM capacity, which is the high bandwidth memory that goes into the GPUs. The- the GPU itself is made using the same process that's used to build the chip that's in your mobile phone. If NVIDIA wanted to, they could build 50 million of those GPU die per year. But they're gonna build about 5.5 million GPUs this year. And the reason is because of that HBM, because of the interposer that it goes on, and, uh, there's just a finite capacity. So, what happens is a hyperscaler comes in and says, "I want a million GPUs." And NVIDIA's like, "Sorry, I've got other customers." And the hyperscaler says, "No problem, I'm gonna build them myself." And then all of a sudden those GPUs are found by NVIDIA to give to the hyperscaler. Um, there is just a finite amount of capacity. By building your own chip, what you really get isn't your own chip, it's that you get control over your own destiny. That's the unique selling point of building your own chip. And so, um, it-
- HSHarry Stebbings
What does that mean, control over your own destiny?
- JRJonathan Ross
Uh, NVIDIA can't tell you what your GPU allocation is. It may cost you more to deploy your own chip, because it's not going to be quite as good as NVIDIA's. Let's think about why NVIDIA's GPUs, with a slight edge over AMD's GPUs, dominate. If your total cost to deploy is a huge multiple of the cost of the chips in the systems, then a small percentage increase in the cost of the chip i- is negligible. So, (clears throat) think about it this way, if I'm gonna deploy a CPU and that CPU is 20% of the BOM, and I get a 20% increase in the speed of the chip, that is a 20% value increase in the entire system versus the 20%, you know, you know, increase in the- the chip cost, right? It's negligible. So, you get these huge multiples when you improve the- the chip performance. So, small dif- differences in performance make a huge difference in the value of the product. So, a small edge gives you a, um, massive edge in selling that product.
- HSHarry Stebbings
Can I ask you, you- you mentioned a mono- monopsony?
- JRJonathan Ross
Yes.
- HSHarry Stebbings
Yeah. Is it possible for OpenAI, Anthropic, any of the Mag 7, any of the other providers, to move into the chip layer if there is a monopsony on the HBM market?
- JRJonathan Ross
It's very hard. However, there is an incentive from those building HBM to spread that around, because NVIDIA gets to negotiate very good rates because they're such a large buyer. However, if you were building an HBM fab, um, and, um, packaging house and all of this other, you know, part of the ecosystem, if NVIDIA comes in and writes a big check, then you're gonna build the fab for them. So NVIDIA's always gonna get the amount of supply that they want, uh, in advance. The- the problem is you have to, um, write that check more than two years in advance. And so the... Where AI's gone, you know, just absolutely hocking- hockey sticking, um, even when you have the cashflow of NVIDIA, it's hard to actually write the checks for the amount of demand that there's going to be in advance. So, there is going to be a supply constraint, and it's not purely based on being a monopsony. Part of it is based on just the sheer capital cost, and the memory, um, suppliers are very conservative. There's also, um, this situation where the margin on HBM is so high that no one wants to actually increase the supply, because then the margin goes down.
- 18:14 – 28:20
OpenAI and Anthropic Will be $5BN Companies: The Bull Case
- HSHarry Stebbings
I- I totally understand that. Can I ask you, when you look at that, and when you look at OpenAI, when you look at Anthropic, having their own chips, how, is that why they're raising the money they are? Sam said they're going to need hundreds of billions of dollars. Is that factoring that in?
- JRJonathan Ross
No. M- most of the spend ... So, so, um, buying a system is expensive. Buying a data center's more expensive. The reason is, you're amortizing that data center over a longer period of time. So even if a data center was going to be one-third of your cost per year, if you're amortizing that data center over 10 years, and the chip's over three to five years, the data center's gonna end up costing you more. Per year. So when you hear the hyperscalers talking about that, you know, $75 billion to $100 billion a year investment, because they're building out the capacity for data centers, they're putting a lot of money up, uh, for returns that they're expecting over the next 10-plus years. So it's actually not that much money when you think about it.
- HSHarry Stebbings
Are we thinking about amortization in the right way, in a three to five-year cycle-
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
... if chip cycles are actually faster than that?
- JRJonathan Ross
Uh, I think that the amortization ... Like, people are definitely thinking about it over a longer period than I would. We use a more conservative number, uh, internally. Um, I think five to six years-
- HSHarry Stebbings
Which would be like three years.
- JRJonathan Ross
W- uh, a little bit less.
- HSHarry Stebbings
Yeah.
- JRJonathan Ross
W- w- we're, we're looking at, um, upgrading chips about once a year. Yeah. Now, y- here's the, the way to think about it. There's, there's two phases of the, the value of a chip. There's the, "Am I willing to buy it and deploy it?" and there's, "Am I willing to keep it running?" They're two very different calculations. And so when you deploy it, you have to be able to cover the CapEx. When you keep it running, you just have to beat the OpEx. So if I deploy a chip today, I have to beat the CapEx. I have to earn all my CapEx back and make a profit, and produce a return. Once I've deployed it, as long as I'm beating my operational costs, I'm gonna keep that thing in production. So you're okay with the, the price, th- the value of that chip going down over time. Now the bet that everyone is making is that those new chips that come out aren't gonna reduce the value of the old chips below the OpEx.
- HSHarry Stebbings
That's it.
- JRJonathan Ross
That's right. And in our case, we actually don't think that five years makes any sense.
- HSHarry Stebbings
Because they will be so much less performant that actually the value will be lower than the operating cost.
- JRJonathan Ross
For the electricity and for paying for the data center.
- HSHarry Stebbings
So what happens then? We just have this excess supply of wasted chips-
- JRJonathan Ross
No.
- HSHarry Stebbings
... which are going-
- JRJonathan Ross
Because a lot of these people have entered into really long contracts, and so they have a third point where they have to consider their calculation, which is, "Breaking this contract, is that cheaper than running the chip at a loss?" Yeah.
- HSHarry Stebbings
Happen- s- Can you see this?
- JRJonathan Ross
(laughs)
- HSHarry Stebbings
Um, so what happens then?
- JRJonathan Ross
Um, then, uh ... I can't tell you what happens, because we're trying to avoid that situation, so, um, by having a much faster payback period in all of our, um, calculations. Uh, I would not want to make a bet that long out. Th- the shorter the timeframe that you're making the bet, the clearer your outcome is.
- HSHarry Stebbings
So essentially, you want to minimize payback period as much as possible-
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
... and then minimize operating cost so that you can shed less performant chips faster.
- JRJonathan Ross
Yes, but a- a- also here's another crazy part, which is when you look at the math this way, you're, like if I'm approaching it as an accountant, I'm gonna be like, "This is a terrible idea." But if I look at it empirically, people are still renting H100s. How old are those chips? They're, they're getting close to five years old. Um, and they're still operating well ab- uh, they're still earning more than their operating cost by quite a bit. You would never deploy an H100 today, but they're still profitable to run, right? They're in that second phase. And the reason is, people can't get enough compute. If that wasn't the case, H100s would be renting for a fraction of what they're renting for today. And as long as you can't get enough compute, that's going to be true. The question is, is there an alternative out there that isn't a supply constraint? And so this is where we're hoping to come in. So let's, let's talk about our value proposition. Um, so you started off asking me about speed. Do you know how many customers come to us asking for speed?
- HSHarry Stebbings
No.
- JRJonathan Ross
100%. Do you know how many customers keep asking about that once they realize, um, uh, the supply constraint out there? None. So they start with speed. That, that, they know the value of that to their end customer. And then they're like, "Oh, wait a second. I can't even get enough compute." The real value prop is, can you provide more compute capacity? So two weeks ago, we had a customer come to us and ask for 5X our total capacity. They couldn't get that capacity from any hyperscaler. They couldn't get it from anyone else. We couldn't give it to them. No one can. And so we couldn't get that customer. The hyperscalers couldn't get that customer. There isn't enough compute. So when you're in a market where there isn't enou- so your, your choice is, "I buy this compute and I get the customer." This is where I was going to you when I said, you know, if OpenAI or Anthropic were to double their compute, they would double their revenue, right? So if you're someone who can't get enough compute to serve your customer, then you're gonna be willing to pay whatever it takes to get those customers, 'cause you feel that there's lock-in value by getting that customer now.
- HSHarry Stebbings
Mm-hmm.
- JRJonathan Ross
And so the number one value prop that we have is that our supply chain is not like a GPU supply chain. You, you have to write a check two years in advance to get GPUs. For us-You write us a check for a million LPUs, and the first of those LPUs starts showing up six months later.
- 28:20 – 34:37
Why China is Behind the US in AI and Deepseek is More Expensive to Run
- JRJonathan Ross
- HSHarry Stebbings
Totally understand that. But bluntly, the, the assumption when you look at GPT-5 and the focus on efficiency is that SAM transitioned from performance to efficiency, because compute does not equal a parallel level of performance improvement.
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
Do you think that is fair and true, and does that not go against what you just said?
- JRJonathan Ross
No, and w- you have to think of the different outcomes that they're looking for. So if you are OpenAI, you have moved into markets that are incredibly cost sensitive. Let's talk about India for a second. So if you wanna go in India, what's the one thing you need? 99 rupees a month. That's about $1.13 with current conversion rates. You need to charge your customer $1.13 for your product. So they're going after a market whose alternative is, "I have no AI."
- HSHarry Stebbings
You've got Open. I mean, they can use DeepSeek.
- JRJonathan Ross
This is another misconception in the market. Let's just start busting every misconception.
- HSHarry Stebbings
Sure, great.
- JRJonathan Ross
All right.
- HSHarry Stebbings
Love that.
- JRJonathan Ross
So when the Chinese models came out, everyone reacted by saying, "Oh my God, they've trained models that are almost as good as the US models." And we had, we had a, a podcast on this, right?
- HSHarry Stebbings
Mm-hmm.
- JRJonathan Ross
Uh, and even I was, uh, snookered a little bit at first. Um, and, "Oh my gosh, aren't these models so much cheaper to run?" Um, eh, now that I know more about the foundation models that people are using, uh, versus the Chinese models, no, they're not cheaper to run. They're about 10X as expensive. Actually, let's just take the GPT-OSS model that was released. It's optimized for something different than the Chinese models, but the quality is very high, uh, and I would argue clearly is a better model for what it focuses on than the Chinese models. Now, the Chinese models focus on different things. However, the cost to run the OSS model is about, uh, 1/10 that of the Chinese models. So why was everyone charging less-Well, when you have a, um, sort of a, a captive market for a model, because people say, "I want this model," and there's only one provider of it, you can charge 10 times as much. The price was higher, and people were confusing the cost with the price. So the Chinese models were optimized to be cheaper to train as opposed to be cheaper to run. And when you see how much, um, intelligence has been squeezed into the OSS model versus the equivalent, uh, Chinese models, it's clear that the US still has a training advantage. And the economics work out such that you have to amortize that training over, um, every inference, which means that you want to charge more. And so there's still a balance there, but as you scale out into larger and larger numbers of people, being able to afford to train a model starts to be a payoff. As you deploy more inference capacity, you want to spend a l- bit more on the training to get your inference cost down. In the US we have a, a massive compute advantage, and so people train the models harder, bringing the cost down.
- HSHarry Stebbings
Why do we have a compute advantage in the US, just in terms of access to chips?
- JRJonathan Ross
That's correct.
- HSHarry Stebbings
Yeah.
- JRJonathan Ross
Yeah. Um, and so-
- HSHarry Stebbings
Will, will China not just subsidize the inference and the running, though?
- JRJonathan Ross
Yes.
- HSHarry Stebbings
I understand what-
- JRJonathan Ross
Yes, so here's-
- HSHarry Stebbings
So, so, so does it matter? If their cost of running is higher, but the Chinese, the CCP will just subsidize it, does it matter?
- JRJonathan Ross
There's a home game and there's an away game. Um, the home game is we want to, um, build enough, uh, compute for the United States. The away game is we want to build it for our allies, right? Europe, South Korea, Japan, um, India, and so on. And the advantage that the U... So China can, can win their own home game. They're going to build 150 nuclear reactors so they're going to have enough energy, even though their chips aren't as energy efficient. Uh, and they can subsidize, as you mentioned. But the away game is different. If a country only has 100 megawatts of power, what are they going to do? Build another nuclear power plant? Like, that's just not a realistic thing. You can do that in China, you can't do that elsewhere. So having a better chip gives you an advantage in the away game. So my expectation is that right now, for the next two to three years, the United States has a clear advantage in that away game over China. And if we move very quickly, then we're going to be able to bring a bunch of allies into the AI race.
- HSHarry Stebbings
Do you think we should have open models to allow for China to distill in the effective ways that they have done already?
- JRJonathan Ross
I think the model itself is not a clear advantage. So, the, the first time you had me on your podcast, I predicted that OpenAI was about to open source their model.
- HSHarry Stebbings
Mm.
- JRJonathan Ross
You remember that?
- HSHarry Stebbings
Mm.
- JRJonathan Ross
Yeah. And my prediction was based on their branding strength. Frankly, OpenAI could probably op- uh, like use, um, LLaMA 2, the old model from how long ago? Like, uh, two years ago?
- HSHarry Stebbings
Yeah.
- 34:37 – 37:21
How Europe Could Compete in AI and Why the US is More Risk Averse Than Europe
- JRJonathan Ross
- HSHarry Stebbings
Totally get that. Can I ask you? Uh, there's so many different areas I want to take this, but, you know, we said that just build as much compute as possible. The energy requirements are intense. Is the only way to provide the energy required for this compute wave, tsunami, whatever you want to call it, is the only way nuclear?
- JRJonathan Ross
No. No, no, no. Um, so nuclear is efficient and, and cost-effective, but, um, uh, renewables are efficient and cost-effective. I'll give you my, my simple hack. Um, so all, all the allies in the United States have to do in order to have more energy than China is to be willing to locate their compute where energy is cheap. So right now... L- l- okay, let's compare Europe to the United States. The United States is incredibly risk averse compared to Europe.
- HSHarry Stebbings
Wow.
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
In energy?
- JRJonathan Ross
No, no, no, in, in everything. But you have to ask what kind of risk. There's two kinds of risk. There's, um, mistakes of commission, where you do something that's a mistake, and then there's mistakes of omission, where you don't do something and it's a mistake. And the United States is terrified of making mistakes of omission. When you are in a massive growth economy, missing out is more expensive than fumbling something. And so the- Europe is incredibly willing to embrace the risk of omission.So, the way that Europe is trying to compete is through legislation, by saying things like, "I want to keep this data in Europe," or, "I want to keep this data in this country." If Europe wanted to compete in AI, all you'd need to do is say, "Norway, please deploy an enormous number of wind turbines." Why? Norway has about an 80%, uh, utilization rate of wind, so like 80% of the time, you can be generating energy. Um, they have enough hydro that if you deployed an, uh, 5X the wind power of the hydro, Norway itself could provide as much energy as the United States and could do it consistently. The entire United States. That's one country in Europe. How much other energy is there out there that could be unlocked that
- 37:21 – 48:16
Why We Have to Have Nuclear Energy and How to Bring it Back
- JRJonathan Ross
isn't nuclear? And by the way, let's also deploy nuclear. Nuclear is incredibly safe these days.
- HSHarry Stebbings
Why do we not, then?
- JRJonathan Ross
Fear.
- HSHarry Stebbings
Is that really it?
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
When you speak to European governments, what do they say to you?
- JRJonathan Ross
I don't bring up nuclear, because I'm not gonna push an energy source that everyone's gonna push back on. But, um, when I was in Japan recently, they were talking about bringing their nuclear reactors back online. Look, Japan, um, has a reputation of being, uh, very slow. There's a lack of subtlety and nuance in that, um, perception. The reality is, Japan is slow to make a decision, but when they decide something, they move really fast. Um, let's take an example. Japan decided to build a two-nanometer fab. Um, when I was there last, they were showing off these two-nanometer wafers that they had produced. Now, the yield's not where it needs to be. This is not production grade. But they built a two-nanometer fab, and they are producing wafers out of it. And they're gonna start getting that defect density down, they're gonna move quickly. Uh, they've allocated $65 billion for AI, and they're gonna spend it, and they're gonna spend it quick. They're gonna turn their nuclear reactors back on. When Japan is gonna turn their nuclear reactors back on, Europe needs to listen to that and go, "Gosh, we need to catch up in energy."
- HSHarry Stebbings
Catch up is exactly what I was thinking, because what I'm thinking is, the speed it takes to build out, you said about kind of Norway's c- like, latent capacity of wind and how we could utilize it.
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
Dude, it takes years to build huge, huge supply of turbines. Like-
- JRJonathan Ross
Does it?
- HSHarry Stebbings
Yeah. You really-
- JRJonathan Ross
Why?
- HSHarry Stebbings
Y- you think y- the Norwegian government is gonna, like, shell out and have 10,000 wind turbines on the go?
- JRJonathan Ross
Why does the Norwegian government need to pay for it?
- HSHarry Stebbings
Who should?
- JRJonathan Ross
How about the hyperscalers? How about, um, other governments that want to locate there? In Saudi Arabia, there are gigawatts of power, and they're building out data centers for that. Why doesn't Europe work with Saudi Arabia to say, "You know what?" So, Saudi Arabia wants to do a program of data embassies, where you have sovereign, um, oversight over your data, but you get to use their energy. Why not use that? Problem solved. They're gonna build out three to four gigawatts in the very near future.
- HSHarry Stebbings
So, the hyperscalers would pay Norway to use their renewable energy sources, and then leverage that?
- JRJonathan Ross
The complaint that the hyperscalers have is all of the, th- the, um, paperwork and the slowness. I was talking to someone who was on the board of a major energy company that builds nuclear power plants. He said they spend three times as much on the permitting in the United States than on the nuclear power plant. And I don't know about Europe, but typically, the United States is better than Europe on this. How much does it cost to build a nuclear power plant in Europe, versus, like, uh, versus the actual cost of the infrastructure versus the permitting? Here, here's what everyone needs to walk away from this with. The countries that control compute will control AI, and you cannot have compute without energy.
- HSHarry Stebbings
How far behind is Europe, and is there a way for us to get back? Like, is it too late? I don't want to be negative, I'm not overly pessimistic, but is there a chasm which we can catch up on?
- JRJonathan Ross
I don't think there's a problem right now if, if Europe acts now. I mean, um, China is ahead in action, but there are 500 million people in Europe. There's over 300 million in the US, and if you start bringing all the allies together, South Korea, who, by the way, knows how to build nuclear power plants. The power plant in, um, the UAE was built by South Korea. They could build power plants here. France knows how to build power plants. How about a little bit of a Manhattan Project for building enough energy? Um, when I'm walking around in Europe in the summer, it's incredibly hot, and when I'm walking around in the winter, it's incredibly cold. That is not an experience you have anywhere else in the world. Build more energy.
- HSHarry Stebbings
I'm with you, Jonathan, but I'm also realistic. I know how slow we are as governments, both singular and in collaborating together. It's not gonna happen at the speed of which this needs to be done. What happens if that does not happen in the speed with which it needs to be done?
- JRJonathan Ross
Um, then Europe's economy is gonna be a tourist economy. People are gonna come here to see the quaint old buildings, and that's gonna be it. You, you cannot, um, you cannot compete in a new economy if you don't have the resources that the new economy is built on. And the new economy is going to be AI, and it's going to be built on compute.
- HSHarry Stebbings
Is model sovereignty enough to win? If you look at a provider-
- JRJonathan Ross
No, because if you don't have compute, you can't run the AI.It doesn't matter how good your model is. Um, you could have a model that is 10 times smarter than OpenAI's model, and if you have 10 times the compute, OpenAI's model's going to be better.
- HSHarry Stebbings
So for, uh, Mistral, if you say, "Hey, we're going to have sovereignty within Europe, and the German healthcare system and the Croatian transport ministry are going to use Mistral because we're a European alternative," that's not a reason to win.
- JRJonathan Ross
What's the USP? What's the unique selling point?
- HSHarry Stebbings
It's a European model, and it doesn't have ownership in the US under a Trump administration.
- JRJonathan Ross
What does that have to do with giving you enough compute? What, what you're solving for there is removing someone's else, someone else's ability to control you.
- HSHarry Stebbings
Yeah.
- 48:16 – 51:54
Deflationary Pressures and New Job Markets
- JRJonathan Ross
improves.
- HSHarry Stebbings
You, you said the economy gets stronger. When we think about kind of what that's predicated on, that's predicated on the $10 trillion labor spend in GDP, uh, shifting...Um, to AI, and us taking a portion of that. Do you think that we will see significant shifts in the GDP or the spend on labor moving towards AI in the next five years?
- JRJonathan Ross
I believe that AI is going to cause massive labor shortages. Yeah, I- I don't think we're gonna have enough people to fill all the jobs that are gonna be created. There's- there's three things that are gonna happen because of AI. The first is massive deflationary pressure. Um, this cup of coffee is going to cost less. Your housing is going to cost less. Everything is going to cost less, which means people are going to need less money.
- HSHarry Stebbings
So how is it going to cost less to have a cup of coffee because of AI?
- JRJonathan Ross
Because you're going to have, uh, robots that are going to be farming the coffee more efficiently. You're going to have better supply chain management. You're going to, um, it's just going to be across the entire supply chain. Um, you're going to be able to genetically engineer the coffee so that you get more of it per, um, watt of sunlight, right? Just across the entire spectrum. So you're going to have massive deflationary pressure. That's number one. And what that means is people will need to work less. And that's going to lead you to number two, which is people are going to opt out of the economy more. They're going to work fewer hours, they're going to work fewer days a week, and they're going to work fewer years. They're going to retire earlier, because they're going to be able to support their lifestyle working less. And then number three is we're going to create new jobs and new, uh, company, uh, new industries that don't exist today. Um, think about 100 years ago. 98% of the workforce in the United States was in agriculture. 2% did other things. When we were able to reduce that to 2% of the population working in agriculture, we found things for those n- other 98% of the population to do. The jobs that are going to exist 100 years from now, we can't even contemplate. 100 years ago, the idea of a software developer made no sense. 100 years from now, it's going to make no sense, but in a different way because everyone's going to be vibe coding, right? Um, and influencers, that wouldn't have made sense 100 years ago. Uh, but now that's a real job. People make millions of dollars off of it. So what jobs are going to exist 100 years? So number one, deflationary pressure. Number two, opting out of the workforce because of that deflationary pressure. And number three, uh, jobs and companies that couldn't exist today that are gonna exist and are gonna need labor. We're not gonna have enough people.
- HSHarry Stebbings
It's fascinating, the counter-narrative, isn't it?
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
Everyone being like, "Oh, millions and millions of people will be unemployed." And you're like, "Nope, we're actually not gonna have enough people for the jobs."
- JRJonathan Ross
Well, what was the famous, um, prog- prognostication 100 years ago that there was going to be massive famine because we weren't going to be able to feed ourselves? People always underestimate what's gonna change in the economy when you improve technology.
- HSHarry Stebbings
When you think about the requirements from an energy perspective, and then also what you just said there about kind of labor, do you think Trump and the Trump administration is doing more to help or to hurt the advancement of AI in the US?
- JRJonathan Ross
Um, definitely help. Uh, all of the moves that have been made are things that are gonna help with AI. Um, for example, um, you know, the permitting issues, right? Um, overall, it's been a very positive experience
- 51:54 – 53:40
The Future of Vibe Coding
- JRJonathan Ross
on AI.
- HSHarry Stebbings
You mentioned vibe coding. I do just have to ask about it. Do you think this is an enduring and sustainable market? When you look at, um, a lot of the use, uh, use cases today, they're quite transient.
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
Do you... How do you analyze the future of the vibe coding market, having played with it a little bit, and having seen also interns, as you said, who are very good at it internally use it well?
- JRJonathan Ross
Um, vibe coding is gonna be... So let, let's take reading. Reading used to be a ca- reading and writing used to be a career. If you were a scribe, you were one of the small percentage of people who knew how to read and write, and people would hire you just to record things. And you, you did much better than the average person in the economy because of that, 'cause it was a specialized skill. Coding has been the same thing. Very small percentage of the population did it, took, you know, a couple years to learn how to do it well. Uh, some people were really good at it. Now everyone reads, everyone writes. It's not a special skill. It's expected in every job. And coding is gonna become the same thing. For you to be in marketing, you're gonna have to be able to code. For you to be, um, in customer service, you're gonna have to be code, uh, be able to code. Uh, I was having dinner with someone who runs a chain of 25 coffee shops, has never coded in their life, and they've vibe coded a, a supply chain tool that allowed them to check inventory. They didn't write a single line of code. They got it to work. And it was funny because they discovered all the problems that we software engineers, uh, discover over time. They started getting feedback from their employees, like, "This feature doesn't work." It doesn't, "This thing doesn't work when I do this." All the little edge cases. And then he just started fixing them, and all through vibe coding.
- 53:40 – 58:46
Why AI Companies Should Strive to Have Low Margins
- JRJonathan Ross
- HSHarry Stebbings
Do margins matter in a world of exponential growth? When we look at the demand for your products, when we look at the demand for a Lovable or a Ratplit, uh, both bluntly have bad margins. Does it matter having bad margins when growth demands are so high?
- JRJonathan Ross
Um, I- I would say that margins... First of all, you do have to have profitability in the end, or at least break even, right? To be an ongoing concern. At some point, you can't just keep raising money. Even Amazon had to start making some money. But the real reason why you need higher margins is volatility. Because if you have a razor thin margin and the market moves, you may not be able to raise more money, you may not be able to get a loan. And so what a margin does is it gives you stability and staying power in the market. On the other hand, um...What it does is it also gives competition the ability to enter, right? Your margin is my opportunity. And so what you're trading is stability, um, for, um, for a competitive mode. That's the decision that you have to make.
- HSHarry Stebbings
How do you think about margin internally today?
- JRJonathan Ross
Um, I think you want the ability to have margin, and you wanna give it to your customers, and you wanna give them an advantage. And if you have the ability to take that margin when it's needed, then, um, you're in a great position. So I, I remember talking to a ... So we hired this amazing CFO recently, but I remember talking to a, a previous candidate, and when we were talking about, um, m- margin, they said that we should price, uh, so that our supply met our, our demand. In other words, they wanted to increase the price in order for the demand to come down.
- HSHarry Stebbings
Makes sense.
- JRJonathan Ross
Does it?
- HSHarry Stebbings
Economic sense, yeah.
- JRJonathan Ross
Economic sense?
- HSHarry Stebbings
Logically and rationally, yes.
- JRJonathan Ross
But then logically, uh, why not, um, use up your brand equity? Why not use, like, the trust that your customers have to sell them things that aren't good? Brand value, brand equity has value. You wanna keep your brand equity as high as possible because trust pays interest. And similarly, you wanna keep your margins low enough that you're building up this sort of equity value with your customers, where they know that you are giving them a good deal. When you charge a high margin, you are at odds with your customer, and you wanna do everything that you possibly can to align with your customer. I want my margin to be as low as I possibly can make it while keeping my business stable, and I'm gonna make my cashflow by increasing the volume. And one of the things that I love about the compute business is that the need for compute is insatiable. It's Jevons paradox. If we produce 10X the compute, we will have 10X the sales. That's just the way it works. As long as we keep bringing the cost down, people are gonna buy more. And so I want to keep bringing that cost down, I wanna keep increasing the volume, and I wanna keep selling more for less so that people get more value out of their business and they buy more and that cycle continues.
- HSHarry Stebbings
How far are we on the journey to bring the cost down? You know, I remember, uh, I- I look back at some of the shows, dude, and I, I cringe at myself because I'm talking about like, "Oh, Canva implementing AI, and it's hurting their margins because they're implementing AI, and it's gonna cost them more." And it's just such a naive approach to ask that question even because now the cost of implementation's gone down by 98%. How far are we in terms of that cost reduction cycle?
- JRJonathan Ross
Well, let's step back and, and use your Canva example.
- HSHarry Stebbings
Yeah.
- JRJonathan Ross
Um, successful businesses don't watch the bottom line. They watch their customers. They, they solve problems that their customers have. If you are competing, you are doing it wrong. You wanna differentiate. You wanna solve a problem that your customer has not solved yet and can't solve any other way, and then they're happy to pay you money. And that's how it works. You solve their problem, and then your cashflow is solved. So someone's spending on AI, if you just look at the balance sheet, that doesn't make sense. But when the customer is very happy and they're solving a problem that they couldn't solve otherwise, first of all, you're increasing the TAM usually with AI, because it makes the product so much easier to use. Did you use Photoshop two years ago? Impossible. Now, if you wanna generate an image, you just explain what you want. That increases the TAM. You may be able to charge less per photo, but your total revenue increases. Your total market increases.
- 58:46 – 1:06:44
S&P 7000, Mag 7 & Market Choppiness
- JRJonathan Ross
- HSHarry Stebbings
Forgive me for this financial question, but we see the S&P about to hit 7,000. We see this ripping of the Mag 7, like we haven't seen a concentration of value in, in many, many years. And people suddenly start to feel like, "Wow, it's getting toppy." I listen to you and I hear all of this, and I think, "Pff, it's just the start." How should I think about the duality of those two thoughts?
- JRJonathan Ross
There's two components to the value. Um, one is the weighing machine, and one is the popularity contest. And there are some products that are pure popularity contests, like crypto. I have never bought a Bitcoin. You know, I missed out. Why? Because I can't play in the popularity contest. I'm not good at it. I don't know what's gonna be popular and what isn't. All I can do is I can see value. When I look at AI, I see real value being delivered. Best example, PE firms are all over us. They want access to cheap AI compute, because every time they get more cheap AI compute, they can bring the, the ... They can change the bottom line of their businesses. It has real value. When PE firms go after something and see value in it, it's not a popularity contest. It's pure value. And so what happens is, the, the reason companies get a large multiple is people see that the valuation is ... Th- the actual value is going to accrue, or they get hype-cycled on it and it, it's pure popularity contest. And there are different participants in the market. Some of them are just playing the popularity contest. Others are looking at the value, and they may come to the same conclusion for different reasons.Coming at it from the value point of view, the weighing machine point of view, the most valuable thing in the economy is labor, and now we're going to be able to add more labor to the economy by producing more compute and better AI. That has never happened in the history of the economy before. What is that going to do?
- HSHarry Stebbings
Do you worry that if we have a speed bump in the short term, it will derail significant parts of the economy given the concentration of value? Everyone rips today, but if NVIDIA, Meta, Google, Microsoft suddenly hit speed bumps, then the AI speed train is just slowed down, the consequent multiplier effect is mega. Do you worry about that?
- JRJonathan Ross
Yeah, and, and this is, this is independent of the value of AI. This is the sort of control system, um, theory of what's going on, right? So-
- HSHarry Stebbings
Mm-hmm.
- JRJonathan Ross
... a stock market could inherently be on an upward trajectory. It can overheat, and that overheating causes it to run away. People bid things up, they realize they've made a mistake, and then it has to come back down, and then it dips below where it should be, spending, um, retreats, and then people don't have the funds they need to build their businesses. Uh, a lot of good businesses can die during one of these downward trends. But this is also where the best businesses are made. How many times do you see a downturn, um, and a ton of amazing businesses come out of it?
- HSHarry Stebbings
Do you think we will have a downturn in the next year?
- JRJonathan Ross
I, I can't predict whether or not there'll be a downturn. There are things... So the ability to predict something is largely dependent on whether or not predictions affect predictions. If a prediction affects the prediction, you cannot predict it, because you are pred- whatever your prediction is changes the outcome. The only things that are predictable are things where the predictions don't change the outcome. If an asteroid is headed towards the Earth and we see that, um, if we don't have the technology to stop it, then it's gonna happen. But if we see that happen and we can predict it, then we might develop the technology to stop it. Do y- do you see the problem?
- HSHarry Stebbings
I do.
- JRJonathan Ross
And so in the economy, you don't have to do anything other than move dollars around, so you have these very sort of fast twitches in the economy based on people's ability to predict, which makes it unpredictable. I can't tell you what's going to happen in the economy. All I can tell you is that right now, the biggest problem I see in AI is if y- you see a good engineer, one that you would have hired before, they can go out and they can raise 10, 20, 100 million, a billion dollars, and then rather than contributing to one of the other AI startups, they go create their own. Which means that you have difficulty in getting critical mass of talent in any one of these AI startups. On the other hand, AI's making everyone at one of these startups more productive. So in terms of whether or not the, the economy is overheated, I think one of the best predictors of that is, is the economy getting in the way of the success of the companies? If it's not getting in the way, then I don't think it's overheated.
- HSHarry Stebbings
Do you not think it is getting in the way? Because fundamentally, the capital supply side-
- JRJonathan Ross
Yes, a-
- HSHarry Stebbings
... is so, uh, uh, large that we are actually preventing you from being able to get great engineering teams together because we're funding talent to the extreme where they can raise huge amounts of money rather than joining Rock.
- JRJonathan Ross
Yes, please stop doing that. Um-
- HSHarry Stebbings
Yeah.
- JRJonathan Ross
No, no, no. But, but AI is making people more productive. So it might be possible for, um, the economy to keep ripping and for all of the companies to continue being very successful. We don't know. We've never been through this before.
- HSHarry Stebbings
Is the war for talent insane today?
- JRJonathan Ross
It's, it's definitely, um, uh, much more aggressive than it's ever been in history, uh, but only in tech. When you look at sports, um, sports have always been insane, or at least recently been insane. Like, you look back 20 years, 30 years ago in sports, the salaries looked a lot like tech salaries.
- HSHarry Stebbings
Sure.
- JRJonathan Ross
People are just realizing the value. The problem is, in, in sports, um, y- you have a limited number of, um, s- teams. You have a l- you know, uh, you might even institute a salary cap and things like this. In technology, we're not doing that, and y- you have an unlimited number of teams, an unlimited number of startups, right? Just imagine if anyone could go create their own football team. What would that do to salaries? What would... And, and what would that do to the, the value of the franchise?
- HSHarry Stebbings
Which incumbent are you most impressed by, and which are you most worried or concerned for?
- JRJonathan Ross
Um, eh, I would say Google has probably done the biggest turnaround, and they had a structural advantage in that. So Google historically has depended more on their engineers to come up with good ideas, and as long as management gets out of the way, great things happen at Google. And so I just think from a cultural perspective, that's a systemic advantage. Um, and for, for them to get-
- HSHarry Stebbings
Do you think Gemini has been a, a success for them ultimately?
- JRJonathan Ross
I do. I mean, y- you just look at the numbers of ad- and the adoption. It's been great.
- HSHarry Stebbings
How do you feel about the implementation into consumer products?
- JRJonathan Ross
Um, eh, less so. I mean, you, you see, like, random Gemini introduction into each product. It's like, it's in Gmail, but it's practically unusable. It's in, uh, pretty much every product, and it, it seems thrown in kind of, like, half thought through. But you shouldn't judge that yet, because at least they're getting exposure to how people are using it, and they can use that to figure out what they should actually do. I mean, what happened with Google Chrome, right? Like, it was originally Google TV, it was a total flop, and then they iterated and they turned it into Google Chrome. Uh, this is the classic, um, problem where, where someone puts something out there, everyone throws darts at it, and you don't realize that they're just willing to take those darts in order to build
- 1:06:44 – 1:13:35
Why OpenAI and Anthropic are so Undervalued
- JRJonathan Ross
a better product.
- HSHarry Stebbings
And it's fine to take those darts as long as the window of distribution advantage remains, but what's challenging is it doesn't.OpenAI has closed that chasm so significantly.
- JRJonathan Ross
That's true. Um, Google may be too late.
- HSHarry Stebbings
Do you see what I mean?
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
It's, it's like a classic, like, you know, can the incumbent attain, uh, innovation before the startup acquires distribution? And it's like, the startups acquired distribution 10% of the world. It's pretty impressive.
- JRJonathan Ross
Yeah, at this point it would be hard to imagine a scenario where OpenAI goes away. I just, I don't see how that happens. And so at the very least, you have two competitors from this point on going at it. But-
- HSHarry Stebbings
Which is OpenAI and Anthropic, or OpenAI and Google?
- JRJonathan Ross
OpenAI and Google. Anthropic does something different. Anthropics doing coding, right? Um, OpenAI is doing a chatbot, Google's doing a chatbot. Google's also doing coding. Google's doing everything.
- HSHarry Stebbings
Well, I mean, OpenAI is doing coding too.
- JRJonathan Ross
That's... Well, um, yes, and actually, um, our engineers recently started using Codex more than using the Anthropic tools.
- HSHarry Stebbings
Wow.
- JRJonathan Ross
Yeah. And it's funny because it's almost on a monthly basis. So we have a philosophy, we don't tell our engineers what tools to use. We do tell them, "You must use AI," because otherwise you're just not going to be competitive. Um, but we saw them, um, using Sourcegraph. We saw them then using Anthropic. We saw them then using Codex. Next month it'll probably be Sourcegraph again. It just keeps going around and around in a circle.
- HSHarry Stebbings
Do any of these have enduring value then if the switching cost is so low, and if they're just bluntly being used so promiscuously? (laughs)
- JRJonathan Ross
Um, our engineers are cutting-edge engineers who will switch to the best tool the moment it's the best tool. Not everyone is like that.
- HSHarry Stebbings
A lot are like that though. Engineers-
- JRJonathan Ross
A lot, a lot of the people you interact with are like that. Enterprises make these long-term deals and they stick with whatever their deal they made a year ago.
- HSHarry Stebbings
Mm-hmm. Would you rather invest in OpenAI at 500 billion or Anthropic at 180?
- JRJonathan Ross
I'd want to invest in both.
- HSHarry Stebbings
Would you?
- JRJonathan Ross
Yeah. They're both undervalued. Highly undervalued. You're, you're still... Okay, you're still looking at them as if they're competing in a finite market for a finite, uh, outcome, when they're actually increasing the value of the market with the more R&D that they do.
- HSHarry Stebbings
Play this out for me then. If we do the bull case for them, what does that look like?
- JRJonathan Ross
I think the current tech companies can increase their value significantly, but I don't know why they couldn't increase their value significantly while the AI labs catch up to where those current A-, you know, AI, uh, the current technology leaders are. The Mag Seven is going to increase in value, and what's going to happen is, the, the AI labs are gonna achieve the same amount of value as the current Mag Seven, but the Mag Seven is going to be more valuable. The question is, will the AI labs overtake the Mag Seven?
- HSHarry Stebbings
What will determine that?
- JRJonathan Ross
I don't know. I... Frankly, I think they're just going to become the Mag Nine, the Mag 11, the Mag 20.
- HSHarry Stebbings
Do you think the AI labs move very significantly into the application layer and subsume the majority of it?
- JRJonathan Ross
That is the natural tendency of a very successful tech company. Um, they start to do what their customers do, and they move up the stack and then they cre- they subsume what their customers did, and then there are new people who build on top of them, right? And, um, OpenAI, uh, you know, I think on your show, Sam Altman said something about how, um, if you're just, uh, doing something, uh, like a small refinement on top of OpenAI, you're, you're gonna get overrun or whatever. Um, he was just being very honest. That's what they do. In our case, um, we found an area where we will not compete with our customers, which is we will not create our own models. So we just won't do it. And by putting that line in the sand, we're saying it's safe to build on our infrastructure, right? Because we're not going to go after what you do. And that may be the wrong call. We may find that we're subsumed by one of our customers. Um, but it also means that you can trust that you can build on us. I could be making a huge mistake on that call.
- HSHarry Stebbings
You could be. You would also need a lot of cash to do that, to build your own models.
- JRJonathan Ross
To, to build our own models?
- HSHarry Stebbings
Yeah. And speaking of cash, how much did you just raise?
- 1:13:35 – 1:21:51
The Chip Market in 5 Years
- JRJonathan Ross
- HSHarry Stebbings
Can you help me understand what the chip market looks like in a five-year timeline? You said there we'll have OpenAI, we'll have Anthropic, we'll have all the pr- providers having their own chip infrastructure. You'll also have NVIDIA. They'll also be... What does that look like?
- JRJonathan Ross
My prediction is that in five years, NVIDIA will still have over 50% of the revenue. However, they will have a minority of the chips sold. They might ha- uh, you know, minority share. They might have, um, 51% of the revenue, and they might have 10% of the chips sold. Um-
- HSHarry Stebbings
Can you help me understand that?
- JRJonathan Ross
Yeah. There, there is huge value in being a brand. Um, you get to charge more. However, uh, it makes you, uh, less hungry, and you're, you're gonna start charging high margins. And some people are gonna pay it because no one's gonna get fired for buying from NVIDIA. It's a great place to be in. That business is gonna remain incredibly valuable. Um, if you're invested in NVIDIA, you're probably gonna do okay. However, um, if you're looking at it from the customer point of view, when you have customer concentration like we're seeing, where, you know, 35, 36 customers are 90%, 99% of the total spend in the market, they're gonna make decisions less on brand, and they're gonna make decisions more on what makes their business successful, because they're gonna have more power to make those decisions. So, you're gonna see other chips being used, because those companies are gonna have enough power to make decisions themselves.
- HSHarry Stebbings
You said you won't do badly if you're an NVIDIA investor. One of my, um, friends says, "The thing I love about Harry is that, you know, he's wonderfully charming, but at the end of the day, he goes, 'That's great, that's great, but what about me?'" (laughs) Which is very true. Over/under on NVIDIA in a five-year timeline, 10 trillion.
- JRJonathan Ross
I personally would be surprised if in five years NVIDIA wasn't worth 10 trillion. The question you should ask is, will Groq be worth 10 trillion in five years? Possible. We don't have the same supply chain constraints. We can build more compute than anyone else in the world. The, the most finite resource right now, compute, the thing that people are bidding up and paying these high margins for, we can produce nearly unlimited quantities of.
- HSHarry Stebbings
What do you think the market does not understand about Groq that you think they should understand?
- JRJonathan Ross
Oh, it changes every month. Um, it used to be we couldn't have, um, uh, uh ... it used to be we couldn't have multiple users. Um, and then we demoed multiple users to people, uh, on the, on the same hardware, right? They used to think that we were-
- HSHarry Stebbings
And this is 'cause of the SRAM structure.
- JRJonathan Ross
'Cause of the SRAM. Actually, here's another one. I still get asked this.
- HSHarry Stebbings
Are you not impressed with my learning-
- JRJonathan Ross
Yeah. (laughs)
- HSHarry Stebbings
... from last time? Thank you so much. Yeah, I, I l- dude, I learned so much from you, genuinely. I was like, genuinely learning so much. But, okay, n-
- JRJonathan Ross
The, the, the question I get asked the most is, "Isn't SRAM more expensive than DRAM?"
- HSHarry Stebbings
Hm.
- JRJonathan Ross
The answer is yes. Um, SRAM, uh, a good way to think of it is SRAM is inherently three to four times as expensive per bit, inherently, putting all the like, you know-
- HSHarry Stebbings
And just for anyone who doesn't know, again, SRAM is versus DRAM, super simple.
- JRJonathan Ross
So, um, SRAM, I'll, I'll keep it super simple, but this isn't technically accurate. SRAM is the memory inside of a chip. DRAM is the external memory. It really has more to do with how you design it, um, but, but anyway, um, so SRAM has three to four times as many transistors or, or capacitors, uh, just transistors for SRAM than DRAM. DRAM is a capacitor and a transistor. SRAM is six to eight, uh, transistors. And so SRAM is inherently larger per bit, which means it uses more silicon, therefore it's more expensive. You're also deploying it on a more expensive chip, like a three nanometer chip. So, it costs you more per unit of area than DRAM. So it- so there's a multiple. Maybe it's 10 times as expensive, uh, per bit. The thing is, when we're running a model like Kimi, and we're running it on 4,000 of our chips, and you're running that Kimi model on eight GPUs, we're using 500 times as many chips, which means the GPUs have 500 copies of that model, which means they're using 500 times as much memory, which means that their cost is higher, because they, even if it- the SRAM is 10 times more expensive, they're using 500 times as much memory in the DRAM. So, this is one of those classic problems of looking at it from a chip point of view rather than a system point of view. Everything that we did was actually system point of view, and now it's world point of view. We actually load balance things across our data centers. We're now at 13 data centers. We have data centers in the United States, in Canada, in Europe, uh, in the Middle East. When you have a world-scale distribution, you don't just make decisions at the data center level. We actually will have, um, more...... um, instances of some models in some data centers with different, um, compile optimizations for input or output based on what's going on in a geography. We may not even have an instance of a model in a particular data center, and we may have it elsewhere, and we can load balance that. And so we're optimizing at the world level, not at the data center level.
- HSHarry Stebbings
What would you do if you weren't scared, Jonathan?
- JRJonathan Ross
I'll, I'll rephrase that to, where could I increase risk in the business-
- HSHarry Stebbings
Yeah, same question.
- JRJonathan Ross
... and where we haven't? Um, we could double our, our orders in our supply chain. Yes, we have a six-month supply chain, so we can respond to the market faster than anyone else, um, but-
- HSHarry Stebbings
How overweight demand are you than supply?
- JRJonathan Ross
Uh, like I said, last week someone came to us and asked for five times our total capacity. Th- here's the only reason we don't just completely double down on the amount-
- HSHarry Stebbings
If you're not supply constrained, why can't you just do that?
- JRJonathan Ross
Because, um, there are thresholds. So for example, if we had doubled the capacity, we wouldn't have won that customer. They needed 5X. So it's not enough to have twice as much. We have to have enough. And so if we double the capacity, do we have enough for those customers?
- HSHarry Stebbings
And so what you- the risk that you could take is to what, sorry? Just specifically.
- JRJonathan Ross
We could just double, um, the rate at which we're building out supply. I mean, with this fundraise, we, we ended up raising, um, you know, more than twice what we were, you know, expecting to raise, and then we were 4X oversubscribed over, um, over what we did raise. And so we could've raised a lot more money; it would've been more dilutive. Um, and I'm trying to be dilution-sensitive for investors and everyone else, um, but on the other hand, we could've just raised more money, and we could've just built a ton of compute. Um, the other advantage that we have is, versus anyone else, our cost per token, especially given a, uh, given, uh, speed, um, is very advantageous. So we know that we can charge less than the rest of the market, um, which matters when you're trying to build these businesses, not because people are, are spend-conscious. If we lower, um, what we charge 50%, people are gonna buy twice as much. They're spending as much as they're making because whatever they spend increases the quality of the output.
- HSHarry Stebbings
Do you think about going public at all?
- 1:21:51 – 1:24:53
Quick-Fire Round: Biggest Fear, Nvidia: $10TRN, Zuck Buying AI: Work or Not
- HSHarry Stebbings
I do wanna discuss, um, a quick fire-round. So I say-
- JRJonathan Ross
Okay.
- HSHarry Stebbings
... a short statement, you give me your immediate thought. Does that sound okay?
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
What's the biggest misconception about NVIDIA today?
- JRJonathan Ross
That NVIDIA's software is a moat.
- HSHarry Stebbings
CUDA lock-in is bullshit.
- JRJonathan Ross
Yeah. Um, it's true for training, but it's not true for inference. I mean, we have 2.2 million developers on us now. That's how many have signed up.
- HSHarry Stebbings
Wow.
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
How many do CUDA have?
- JRJonathan Ross
They claim 6 million.
- HSHarry Stebbings
If you were founding Groq today with NVIDIA at 4 trillion and the AI boom in full swing, what would you do differently?
- JRJonathan Ross
I wouldn't do chips. (laughs) That, that ship has already sailed. It takes too long to build a chip. The, the bet that we-
- HSHarry Stebbings
Does it? So, so-
- JRJonathan Ross
It does.
- HSHarry Stebbings
... for the chip providers today that are coming out, and we, we are seeing new chip providers come out where they're raising, like, a lot of money-
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
... from good people, it's too late?
- JRJonathan Ross
Yeah, so the reason that I decided to go into chips, so, um, I did do Google TPU, but, um, also before I left, I set a record on the, uh, best classification model, like ResNet-50, um, uh, with someone in Google Brain. We, we didn't experiment. We, we beat everything. Um, and so I could've gone on the algorithm side. And the reason that, um... And, and actually when we were fundraising, I wasn't even 100% sure that I was gonna do chips. I was, like, thinking maybe we'd, we'd do something on the algorithm side, especially in formal reasoning, uh, which is (laughs) good that I didn't. But, um, the, the main motivation to go into chips was the, the moat, the, the temporal moat. So a question we get asked by VCs a lot is, "What prevents someone from copying what we're doing?" And the answer to that is, if you copy what we do, you're three years behind us because it takes that long to go from the design of a chip to a chip in production if you execute perfectly. I've done, um, three chips now that are, um, in production or ramping to production. All three were A0 silicon. Only 14% of chips that are taped out the first time work, or the first time, are A0 silicon. So that means there's an 86% chance each time that you're gonna have to re-spin it. When we built our V2 chip, we actually, um, uh, already scheduled a re-spin for it, and we ended up not having to do it because, to our shock, the first one worked. Like, you, you shouldn't expect that. So that three years is if everything goes perfect. NVIDIA, um, typically takes three to four years per chip, and they just have multiple being done at a time. Uh, Groq is now in a one-year cycle. So a year after our V2 is our V3, and a year after that is our V4.
- HSHarry Stebbings
How do you evaluate the meteoric re-acceleration rise of Larry Ellison and Oracle?
Episode duration: 1:31:19
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode VfIK5LFGnlk
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome