EVERY SPOKEN WORD
125 min read · 24,714 words- 0:02 – 7:35
From gaming GPUs to simulating reality: NVIDIA’s new ambition
- BGBen Gilbert
Still got Swedish House Mafia, Greyhound in my head from the pump up.
- DRDavid Rosenthal
Nice. Nice. [laughing]
- BGBen Gilbert
It is funny how all like GPU companies, like I was watching a bunch of NVIDIA keynotes and AMD keynotes to get ready for this, and everyone is so like techno, neon lighting.
- DRDavid Rosenthal
[chuckles]
- BGBen Gilbert
Like, it's like crypto before crypto.
- SPSpeaker
[singing] Who got the truth? Is it you? Is it you? Is it you? Who got the truth now? Is it you? Is it you? Is it you? Sit me down, say it straight. Another story on the way. Who got the truth?
- BGBen Gilbert
Welcome to Season Ten, Episode Six of Acquired, the podcast about great technology companies and the stories and playbooks behind them. I'm Ben Gilbert, and I am the co-founder and managing director of Seattle-based Pioneer Square Labs, and our venture fund, PSL Ventures.
- DRDavid Rosenthal
And I'm David Rosenthal, and I am an angel investor based in San Francisco.
- BGBen Gilbert
And we are your hosts. When I was a kid, David, I used to stare into backyard bonfires and wonder if that fire flickering was doing so in a random way, or if I knew about every input in the world, all the air, exactly the physical construction of the wood, all the variables in the environment, if it was actually predictable. And I don't think I knew the term at the time, but modelable. If I could know what the flame could look like if I knew all those inputs. And we now know, of course, it is indeed predictable, but the data and compute required to actually know that is extremely difficult. But that is what NVIDIA is doing today.
- DRDavid Rosenthal
Ben, I love that intro. That's great!
- BGBen Gilbert
[laughing]
- DRDavid Rosenthal
I was thinking, like, "Where is Ben going with this?"
- BGBen Gilbert
And this was occurring to me as I was watching Jensen sharing the omniverse vision for NVIDIA, and realizing NVIDIA has really built all the building blocks: the hardware, the software for developers to use that hardware, all the user-facing software now, and services to simulate everything in our physical world with their unbelievably efficient and powerful GPU architecture. And these building blocks, listeners, aren't just for gamers anymore. They are making it possible to recreate the real world in a digital twin, to do things like predict airflow over a wing, or simulate cell interaction to quickly discover new drugs without ever once touching a petri dish, or even model and predict how climate change will play out precisely. And there is so much to unpack here, especially in how NVIDIA went from making commodity graphics cards to now owning the whole stack in industries from gaming to enterprise data centers, to scientific computing, and now even basically off-the-shelf, self-driving car architecture for manufacturers. And at the scale that they're operating at, these improvements that they're making are literally unfathomable to the human mind. And just to illustrate, if you are training one single speech recognition machine learning model these days, one, just one model, the number of math operations, like adds or multiplies, to accomplish it is actually greater than the number of grains of sand on the Earth.
- DRDavid Rosenthal
I know exactly what- [chuckles]
- BGBen Gilbert
[chuckles]
- DRDavid Rosenthal
-part of the research you got that from, 'cause I read the same thing, and I was like: "You gotta be freaking kidding me." [chuckles]
- BGBen Gilbert
Isn't that nuts? I mean, there's just nothing better in all of the research that you and I both did, I don't think, to better illustrate just the unbelievable scale of data and compute required to accomplish the stuff that they're accomplishing, and how unfathomably small all of this is, the fact that that happens on one graphics card.
- DRDavid Rosenthal
Yep. So great.
- BGBen Gilbert
Many of you already know this. Many of you have already RSVP'd, but if you have not, we would love to see you at our Arena show in Seattle. That's gonna be on May 4th at five PM. It's gonna be an awesome show. We've announced we'll have Jim Weber there, who's the CEO of Brooks Running, which is now amazingly a billion-dollar revenue business inside of Berkshire. We'll have other announcements coming as well. Go to acquired.fm/arenashow, or click the link in the show notes to RSVP. All proceeds are going to charity. Our huge thanks to our friends at PitchBook Data for putting this on with us. That's acquired.fm/arenashow, and we hope to see you there.
- DRDavid Rosenthal
I get giddy every time you say that URL. [laughing]
- BGBen Gilbert
[chuckles] Yes, indeed, it is real. All right, now, before we dive in, we have a fun little Q&A from our presenting sponsor, Vanta, the leader in automated cloud security and compliance. We are huge fans of Vanta and their approach. They do everything from SOC 2, to HIPAA, to GDPR, and more, and we are back with CEO and co-founder Christina Cacioppo to talk about it. So Christina, based on our last few conversations, I'm getting the sense Vanta is becoming a lot more than just a SOC 2 compliance company. Is that true, and how do you think about that?
- SPSpeaker
Yes. I mean, the joke about Vanta, that's not a joke, is that we're a security company masquerading as a compliance company. And it comes from the early founding days when we wanted to start a security company and went around and asked all of our friends at startups, you know, what security problems they had. And they did have some, and then we're like: "Oh, you know, what happens if we solve them for you?" And our friends were like: "I'm not gonna use that, 'cause I have ten other problems ahead of that one. I just feel badly. I know I should be doing it, but it is hard for me to prioritize. I often need to get customers, so that's what I do." Which was a little demoralizing [chuckles] when you're trying to come up with a startup idea. But then we actually realized that compliance can be like the security feature someone is asking for, right? So generally, if you're selling, someone doesn't say: "Hey, will you be more secure?" They say: "Hey, are you SOC 2 compliant?" And so if we could build a product that help folks get more secure, help them prepare for and get through a compliance audit, it would accelerate their business, because they have their compliance certification, and leave them more secure.... One of our product team's goals is to help our users fix security issues we surface faster. So we can surface lots of misconfigurations, like proverbial doors and windows you leave open, but if no one fixes them, then we're not having the impact we're looking for, and we're not ultimately securing the company. And so we just have a substantial portion of our product team focused on building features that help companies fix misconfigurations faster. So the whole set of work that falls under that, but it's one of the things I'm most excited about.
- BGBen Gilbert
Thanks, Christina, and thank you to Vanta, the leader in automated security and compliance software. If you are looking to join Vanta's two thousand-plus customers and get compliance certified in weeks instead of months, click the link in the show notes or go to vanta.com/acquired for a sweet ten percent discount.
- DRDavid Rosenthal
Woo!
- BGBen Gilbert
And after you finish this episode, come join the Slack acquired.fm/slack, and talk about it with us. And if you're dying for even more Acquired before we, uh, come back with our next season episode, search for Acquired LP Show in the podcast player of your choice. Our latest installment was a very fun, deep dive and close to home for me, diving in with Nick and Lauren, the creators of Trova Trip, on travel for the creator economy, where they talk about a very interesting business model that they have on their hands and a space that David and I know well in creator things.
- DRDavid Rosenthal
Indeed.
- BGBen Gilbert
All right, David, without further ado, take us in. And as always, listeners, this is not investment advice. David and I may hold positions in securities discussed, and, uh, please do your own research.
- 7:35 – 14:55
Context recap: the early foundations NVIDIA built (and why they mattered later)
- DRDavid Rosenthal
That's good. I was gonna make sure that you said that this time, because we're gonna talk a lot about investing and investors in Nvidia stock over the years. It has been a wild, wild journey. So last we left our plucky heroes, Jensen Huang and Nvidia, in the end of our Nvidia, the GPU company years, ending kind of roughly, you know, 2004, 2005, 2006. They had cheated death, not once, but twice. The first time in the super overcrowded graphics card market when they were first getting started, and then once they sort of, you know, jumped out of that frying pan into the fire of Intel, now gunning for them, coming to commoditize them, like all the other, you know, PCI chips that plugged into the Intel motherboard back in the day. And they bravely fend them off. They team up with Microsoft. They make the GPU programmable. This is amazing. They come out with programmable shaders with the GeForce 3. They power the Xbox. They create the CG programming language with Microsoft. And so here we are. It's now 2004, 2005, and this is a pretty impressive company. Public company, stock is high-flying after the tech bubble crash. They've conquered the graphics card market. Of course, there's ATI out there as well, which will come up again. But there's three pretty important things that I think the company built in the first ten years. So one, we talked about this a lot last time, the six-month ship cycles for their chips. We talked about that, but we didn't actually say [chuckles] the rate at which they ship these things. I actually wrote down, like, a little list. So in the fall of 1999, they shipped the first GeForce card, the GeForce 256. In the spring of 2000, GeForce 2. In the fall of 2000, GeForce 2 Ultra. Spring of 2001, GeForce 3, that's the big one with the programmable shaders. Then six months later, the GeForce 3 Ti 500.
- BGBen Gilbert
I mean, the normal cycle, I think we said, was two years, maybe eighteen months, for most of their competitors, who just got largely left in the dust.
- DRDavid Rosenthal
Well, I was just thinking, you know, yeah, the competitors are gone at this point, but I'm thinking about Intel. How often did Intel ship new products, let alone fundamentally new architecture? You know, there was the 286, and then the 386, and the Pentium, and it got up to Pentium, I don't know, five, whatever.
- BGBen Gilbert
Dude, I feel like the Intel product cycle is approximately the same as a new body style of cars.
- DRDavid Rosenthal
Yes, exactly!
- BGBen Gilbert
Every five, six years, there seems to be a meaningful new architecture change.
- DRDavid Rosenthal
And Intel is the driver of Moore's Law, right? Like, these guys ship and bring out new architectures at warp speed, and they've continued that through to today. Two, one thing that we missed last time that is super important and becomes a big foundation of everything Nvidia becomes today that we're gonna talk about, they wrote their own drivers for their graphics cards. And we owe a big thank you for this and many other things to a great listener, a very kind listener named Jeremy, who reached out to us in Slack and pointed us to a whole bunch of stuff, including, um, the Asianometry YouTube channel.
- BGBen Gilbert
So good. I've probably watched, like, twenty-five Asianometry videos this week. [chuckles]
- DRDavid Rosenthal
So, so good. Huge shout-out to them. But all the other graphics cards companies at the time, and most peripheral companies, they let the further downstream partners write the drivers for what they were doing. Nvidia was the first one that said, "No, no, no, we wanna control this. We wanna make sure consumers who use Nvidia cards have a good experience on whatever systems they're on." And that meant, A, that they could ensure quality, but, B, they start to build up in the company this, like, base of really nitty-gritty, low-level software developers in this chip company, and there are not a lot of other chip companies that have capabilities like this.
- BGBen Gilbert
No, and what they're doing here is taking on a bigger fixed cost base. I mean, it's very expensive to employ all the people who are writing the drivers for all the different operating systems, all the different OEMs, all the different boards that it has to be compatible with. But they viewed it as it's kind of an Apple-esque view of the world. We want the control or as much control as we can get over making sure that people using our products have a great user experience, so they were sort of willing to, uh, take the short-term pain of that expense for the long-term benefit of that improved user experience with their products.
- DRDavid Rosenthal
That their users-... high-end gamers that want the best experience, you know, they're gonna go out, they're gonna spend at the time, three, four, or five hundred dollars on an NVIDIA top-of-the-line graphics card. They're gonna drop it into the PC that they built. You know, they want it to work. I remember [chuckles] messing around with drivers back in the day and things not working, like, this is super important. So all this is focused, and then, of course, they have the third advantage in the company is programmable shaders. You know, which ATI copies as well, but, like, they innovated, like, they've, you know, done all this. So all of this at this time, it's all in service of the gaming market.
- BGBen Gilbert
And one seed to plant here, David, when you say the programmable shaders developers, the notion of a NVIDIA developer did not exist until this moment. It was people who wrote software that would run on the operating system, and then from there, maybe it would get- that compute load would get offloaded to whatever the graphics card was. But it wasn't like you were developing for the GPU, for the graphics card, with a language and a library that was specific to that card. So for the very first time now, they start to build a real direct relationship with developers so that they can actually start saying, "Look, if you develop for our specific hardware, there are advantages for you."
- DRDavid Rosenthal
And really, our specific gaming card. It's like everything we're talking about, these developers, they're game developers. [chuckles] All of this stuff, it's all in service of the gaming market. So, you know, again, they're a public company. They have this great deal with Microsoft. They bring out CG together. They're powering the Xbox. You know, Wall Street loves them. They go from sub a billion-dollar market cap company after the tech crash, up to five to six billion dollars kind of by two thousand and four, two thousand and five. Stock keeps going on a tear. By mid-two thousand and seven, the stock reaches just under twenty billion dollar market cap. You know, this is great, and this, all the story is like, this is pure play gaming. These guys have built such a great advantage and a developer ecosystem in a large and growing market, clearly, which is video games.
- BGBen Gilbert
Which on its own, that would be a great wave to surf. I mean, I think, what's the gaming market today? A hundred and eighty billion or something, and when we talked to Trip Hawkins, who sort of like helped invent it, or Nolan Bushnell, you know, it was zero then. And so NVIDIA is sort of like on a wave that's at an amazing inflection point. They could totally just ride this gaming thing and be an important company.
- DRDavid Rosenthal
It's not running out of steam. I mean, like, how could you not be, not just satisfied, but, like, more than satisfied with this as a founder?
- BGBen Gilbert
Yes.
- DRDavid Rosenthal
You're like, "Yes, I am the leading company in this major market, this huge wave that I don't see ending anytime soon." [chuckles] You know, ninety-nine point nine percent of founders who are themselves as a class, like, you know, very ambitious, are gonna be satisfied with that. [chuckles]
- BGBen Gilbert
But not Jensen.
- 14:55 – 23:12
The radical bet: general-purpose GPU computing and the birth of CUDA (2006+)
- DRDavid Rosenthal
But not Jensen. [laughing] So while all this is happening, he starts thinking about, well, what's the next chapter? You know, [chuckles] I'm dominating this market. I wanna keep growing. I don't want NVIDIA to be just a gaming company. So we ended last time with the little, you know, almost a surely apocryphal story of a Stanford researcher, you know, sends the email to Jensen, and it's like, "Ah, you know, thanks to you, my son told me to go buy off-the-shelf, you know, GeForce cards at the local Fry's Electronics, and I stuffed them into my PC at work, and, you know, I ran my models on, on this." Uh, he's a-- I think it was a quantum chemistry researcher, supposedly. "It was ten times faster than the supercomputer I was using in, in the lab, and so thank you. I can get my life's work done in my lifetime." [chuckles]
- BGBen Gilbert
And Jensen loves that quote. It comes out at every GTC.
- DRDavid Rosenthal
So that story, if you're a, uh, skeptical listener, might beg two questions. First is a practical one. You know, we just said everything's about gaming here, and here's, like, a researcher, like a scientific researcher, doing, you know, chemistry modeling using GeForce cards for that. What's he writing this in? Well, it turns out-
- BGBen Gilbert
Programmable shaders, right?
- DRDavid Rosenthal
Yeah. [chuckles] They were shoehorning CG, which was built for graphics. They were translating everything that they were doing into graphical terms, even if it was not a graphical problem they were trying to solve, and writing it in CG. This is not for the faint of heart, so to speak.
- BGBen Gilbert
Right. So everything is sort of metaphorical. He's a quantum chemistry researcher, and he's basically telling the hardware, "Okay, so imagine this data that I'm giving you is actually a triangle, and imagine that this way that I wanna transform the data is actually like applying a little bit of lighting to the triangle, and then I want you to output something that you think is the right color pixel, and then I will translate it back into the result that I need for my quantum chemistry." Like, you can see why that's suboptimal.
- DRDavid Rosenthal
Yep. So he thinks this is an interesting market. He wants NVIDIA to serve it. If you really wanna do that right, it is a massive undertaking. It was ten-plus years to get to the company to this point. You know, what CG was is like a small sliver of the stack of what you would need to build for developers to use GPUs in a general purpose way, like we're talking about. You know, it's kind of like, um... They worked with Microsoft to make CG. It's like the difference between working on CG and, like, Microsoft building the whole .NET framework for developing on Windows, [chuckles] you know? Or today, even better, Apple, right? Like, everything Apple gives to iOS and Mac developers-
- BGBen Gilbert
Right
- DRDavid Rosenthal
... to develop on Mac.
- BGBen Gilbert
Right. Yeah, the analogy's not perfect, but it's like instead of just Apple saying, "Okay, Objective-C is the way that you write code for our platforms. Good luck," they're like: "Okay, well, well, you need UI framework, so how about AppKit and Cocoa Touch? And how about all these other SDKs and frameworks like ARKit and like StoreKit and like HomeKit?" It's basically, you need the whole sort of abstraction stack on top of the programming language to actually make it very accessible to write software for domains and disciplines that you know are gonna be really popular using that hardware.
- DRDavid Rosenthal
Exactly. So-... When Jensen commits himself and the company to pursuing this, he's biting off a lot. Now, we talked about they've been writing their own drivers, so they have actually a lot of very low-level, and I don't mean low-level like bad, I mean low-level like infrastructure, like close, very difficult systems-oriented programming talent within the company. So that kind of enables them to start here, but, like, still, this is big. [chuckles] So then the second question, if you're a discerning investor, particularly in Nvidia, that you want to ask at this point in time, is like: "Okay, Jensen, you're committing the company to a big undertaking. What's the business case for that? Show me the market." I mean, Don Valentine, at this point, would be sitting there listening to Jensen and being like, "Show me the market."
- BGBen Gilbert
And not only is it show me the market, but it's how long will the market take to get here? And it's how long is it gonna take us, and how many dollars and resources is it gonna take us to actually get to something that's useful for that market when it materializes? Because while CUDA development began in 2006, that was not a useful, usable f- platform for six-plus years at Nvidia.
- DRDavid Rosenthal
Yep. This is closer to on the order of the Microsoft development environment [chuckles] or the Apple development environment than what Nvidia was doing before, which was like, "Hey, we made some APIs and worked with Microsoft so that you can program for my thing." [chuckles]
- BGBen Gilbert
Right. I'm gonna flash way forward just to illustrate the l- insane undertaking of this. I searched LinkedIn for people who work at Nvidia today and have the word "CUDA" in their title. There are eleven hundred employees dedicated specifically to the CUDA platform.
- DRDavid Rosenthal
I'm surprised it's not eleven thousand.
- BGBen Gilbert
Yeah.
- DRDavid Rosenthal
Okay, [chuckles] so, like, where's the market for this? Yes, Ben, you asked the, you know, the third question, which is, okay, the intersection of what does this take to do this, and when is the market gonna get there, and time and cost and all that, but even just put that aside. Is there a market for this? Is the first order question. And the answer to that is probably no, [chuckles] at this point in time.
- BGBen Gilbert
And what they're aiming at is scientific computing, right? It's researchers who are in science-specific domains, who right now need supercomputers or access to a supercomputer to run some calculation that they think is gonna take weeks or months, and wouldn't it be nice if they could do it cheaper or faster? Is that kind of the market they're looking at?
- DRDavid Rosenthal
Yeah, they're attacking, like, the Cray market, like Cray supercomputers, like, that kind of stuff. You know, great company, right? But, like, that's no Nvidia today. [chuckles]
- BGBen Gilbert
Right.
- DRDavid Rosenthal
And they were dominating the market. You know, yeah, it's scientific research, computing, you know, it's drug discovery. It's probably a lot of this work, they're thinking, "Oh, maybe we can get into more professional, like Hollywood and architecture and other professional graphics domains." Yeah, yeah, yeah, sure. But, you know, you sum all that stuff up, and, like, maybe you get to a couple billion-dollar market, maybe, like, total market.
- BGBen Gilbert
Mm-hmm.
- DRDavid Rosenthal
And not enough to justify the time and the cost of what you're gonna have to build out to go after this, to any rational person. [chuckles] So, you know, here we come. Jensen and Nvidia, like, they are doing this. He is committed. He's drunk the Kool-Aid. 2006, 2007, 2008, they are pouring a lot of resources into building what will become CUDA, that we'll get to in a second. Um, like, it already is CUDA at this point in time.
- BGBen Gilbert
And I think Jensen's psychology here is sort of twofold. One is, he is enamored with this market. He loves the idea that they can develop hardware to accelerate specific use cases in computing that he finds sort of fanciful, and, and he likes the idea of making it more possible to do more things for humanity with computers. But the other part of it is certainly a business model realization, where he has spent the last, gosh, at this point, thirteen, fourteen years being commoditized in all these different ways, and I think he sees a path here to durable differentiation, where he's like, "Whoa!"
- DRDavid Rosenthal
To own the platform.
- BGBen Gilbert
You know, it's kind of the Apple thing again, to own the platform and to build hardware that's differentiated by not only software, but relationships with developers that use that custom software. Like, then I can build a really sort of like, a, a company that can throw its weight around in the industry.
- DRDavid Rosenthal
A hundred percent. Jensen, I don't know if he used it at the time, because he probably would've gotten pilloried, but maybe he did. I don't think he cared. [chuckles] Uh, he certainly, uh, has used it since. You know, the way he thought about this was, uh, it wasn't just like, if we build it, they will come, which is what was going on. The phrase he uses is: "If you don't build it, they can't come." [chuckles]
- BGBen Gilbert
[chuckles]
- DRDavid Rosenthal
So it's not even like, "Yeah, I'm pretty sure if we build it, they will come." It's one step removed from that. It's like, "Well, if we don't build it, they can't even possibly come. I don't know if they will come, but they can't come if we don't build it." [chuckles]
- BGBen Gilbert
[chuckles]
- 23:12 – 26:00
Competitive pressure and the 2008 crash: AMD-ATI and NVIDIA’s stock collapse
- DRDavid Rosenthal
So Wall Street is mostly willing to ignore this in 2006, 2007, 2008. The company's still growing really nicely. They had this great market cap run leading up to right before the financial crisis. But then, you know, you mentioned last time, I think it gets announced in 2006, maybe, and closes in 2007, AMD acquires ATI.
- BGBen Gilbert
Yep.
- DRDavid Rosenthal
And ATI was a very legit competitor. It was the only standing legit competitor to Nvidia through its whole life. But now, AMD acquired it, and I think they acquired it for what? Six, seven billion dollars, something like that?
- BGBen Gilbert
Something like that.
- DRDavid Rosenthal
So it was a lot of money, and then they put a lot of resources. Like, they weren't just acquiring this to, you know, get some talent. Like, they're like: "No, no, no, this is gonna be a big product line for us. We're putting a lot of weight behind this."
- BGBen Gilbert
We haven't done the research into AMD the way we have into Nvidia, but the AMD Radeon line, which used to be the ATI Radeon line, that is how you think about AMD as a company, is that they make these GPUs mostly for the gaming use case.
- DRDavid Rosenthal
Yep. Before the acquisition, I think the first PC I built in, like, end of high school, beginning of college, I think I had a Radeon-... card in it. I think I was probably in the minority. I think Nvidia was bigger, but for whatever reason, I liked ATI at that point in time, so, like, they were legit. Well, so here's Nvidia now focusing on this whole other thing, and you're still in the gaming market, which, like we said, is like a massive rising tide. Your competitor now has all these resources, and AMD that's fully dedicated to going after it. Mid-2008, Nvidia whiffs on earnings. [chuckles] Like, this is natural. They took their eye off the ball. Of course, they did, and, uh, the stock gets just hammered.
- BGBen Gilbert
Because in anything that CUDA empowers is not yet a revenue driver, and they've totally taken their eye of- off of gaming.
- DRDavid Rosenthal
Yes. So, you know, we said the high was around a twenty billion dollar market cap. It drops eighty percent, eight zero. [chuckles] This isn't just the financial crisis. It's almost quaint, I think, you know, for me, thinking back on the financial crisis now and, like, people freaking out, the Dow, you know, or the S&P dropping five percent in a day, and like, oh, that's a Thursday these days, [laughing] you know?
- BGBen Gilbert
It is literally the Thursday that we are recording.
- DRDavid Rosenthal
Yes. For a company's stock to drop eighty percent, a technology company's stock, even during the financial crisis, they're not just in the penalty box, they're, like, getting kicked to the curb.
- BGBen Gilbert
Right. Are they done? The headlines at this point are, "Is Nvidia's run over?"
- DRDavid Rosenthal
If you're most CEOs at this point in time, you're probably calling up Goldman or, you know, Allen & Company or Frank Quattrone, and you're shopping this thing, 'cause how are you gonna recover?
- BGBen Gilbert
But not Jensen.
- 26:00 – 35:46
CUDA as a platform: full-stack tooling, parallel programming, and the Apple-like model
- DRDavid Rosenthal
But not Jensen, obviously. So instead, he goes and builds CUDA and continues to build CUDA. And, um, this is, you know, just to set context, like, we get excited about a lot of stuff on Acquired, but I think CUDA is, like, one of the greatest business stories of the last ten years, twenty years, more? I don't know. What do you think, Ben?
- BGBen Gilbert
I mean, I'd say it's one of the boldest bets we've ever covered, but so were programmable shaders, and so was Nvidia's original attempt to make a more efficient quadrilateral-focused-
- DRDavid Rosenthal
Yeah
- BGBen Gilbert
... graphics.
- DRDavid Rosenthal
Those were big bets. I think this is, this is a bet on another scale, though. This is a bet that we don't cover that often on Acquired.
- BGBen Gilbert
Those were big bets relative to the company's size at the time, but this bet is like an iPhone-sized bet.
- DRDavid Rosenthal
That's exactly what this is. It's an iPhone-sized bet.
- BGBen Gilbert
It is a bet the company when you are already a several billion dollar company.
- DRDavid Rosenthal
Yes. An attempt to create something that if they are successful and this market materializes, this will be a generational company.
- BGBen Gilbert
Yep.
- DRDavid Rosenthal
So [chuckles] what is CUDA? It is Nvidia's Compute Unified Device Architecture. It is, as we've referred to, you know, thus far throughout the episode, a full, and I mean full, development framework for doing any kind of computation that you would want on GPUs.
- BGBen Gilbert
Yeah, and in particular, it's interesting because I've heard Jensen reference it as a programming language. I've heard him reference it as a computing platform. It is all of these things. It's an API. It is an extension of C or C++, so there's a way that it's sort of a language, but importantly, it's got all these frameworks and libraries that live on top of it, and it enables super high-level application development, you know, really high abstraction layer development for hundreds of industries at this point to communicate down to CUDA, which communicates down to the GPU and everything else that they have done at this point.
- DRDavid Rosenthal
Ah, this is what's so brilliant. So right after we released, right, the same day that we released part one-
- BGBen Gilbert
Yep
- DRDavid Rosenthal
... the first Nvidia episode we did a couple weeks ago, Ben Thompson had this amazing interview with Jensen on Stratechery. And, uh, Jensen, in this interview, I think, puts what CUDA is and, and how important it is, I think, better than I've seen anywhere else. So this is Jensen speaking to Ben: "We've been advancing CUDA and the ecosystem for fifteen years and counting. We optimize across the full stack, iterating between GPU, acceleration libraries, systems, and applications continuously, all while expanding the reach of our platform by adding new application domains that we accelerate. We start with amazing chips, but for each field of science, industry, and application, we create a full stack. We have over a hundred and fifty SDKs that serve industries from gaming and design to life and earth sciences, quantum computing, AI, cybersecurity, 5G, and robotics." And then he talks about what it took to make this. This is, like, the point we were trying to, like, hammer home here. He says, "You have to internalize that this is a brand-new programming model, and everything that's associated with being a program processor company or a computing platform company had to be created. So we had to create a compiler team. We had to think about SDKs. We had to think about libraries. We had to reach out to developers and evangelize our architecture and help people realize the benefits of it. And we even had to help them market this vision so that there would be demand for their software that they write on our platform," and on, and on, and on.
- BGBen Gilbert
It's crazy. It's amazing. And when he says that it's a whole new programming, I think he says maybe paradigm or way of programming, it is literally true because most programming languages up to this point, and most computing platforms, primarily contemplated serial execution of programs. And what CUDA did was it said, "You know what? The way that our GPUs work and the way that they're gonna work going forward is tons and tons of cores all executing things at the same time, parallel programming, parallel architecture." Today, there's over ten thousand cores on their most recent consumer graphics card. So insanely, uh, or dare I say, embarrassingly parallel, and CUDA is designed for-... parallel execution from the very beginning.
- DRDavid Rosenthal
That's the, like, catchphrase in the industry of [chuckles] embarrassingly parallel.
- BGBen Gilbert
And it's actually kind of a technical term.
- DRDavid Rosenthal
I don't know why it's embarrassing.
- BGBen Gilbert
It's basically the notion that this software is so parallelizable, which means that all of the computations that need to be run are independent. They don't depend on a previous result in order to start executing. It's sort of like it would be embarrassing for you to execute these instructions in order instead of finding a way to do it parallel.
- DRDavid Rosenthal
Ah, it's not that it's parallel that's embarrassing, it's embarrassing if you were to do it the old way on CPUs, serially. [chuckles]
- BGBen Gilbert
I think that's the implication.
- DRDavid Rosenthal
Got it. Got it.
- BGBen Gilbert
This is so obvious that it's embarrassingly parallel.
- DRDavid Rosenthal
Okay, now it makes sense. Now, here's the coup de grâce. We're gonna spend a few minutes talking about how brilliant this was. Everything we just described, this whole undertaking, like, it's like building the pyramids of Egypt-
- BGBen Gilbert
Wow
- DRDavid Rosenthal
... or something here. It is entirely free. [chuckles] Nvidia, to this day... Now, this may be changing, we'll talk about this at the end of the episode, has never charged a dollar for CUDA.
- BGBen Gilbert
But-
- DRDavid Rosenthal
Anyone can download it, learn it, use it, you know, blah, blah, blah, you know, all of this work, stand on the shoulders of everything Nvidia has done. But, Ben, what is the but?
- BGBen Gilbert
It is closed source and proprietary exclusively to Nvidia's hardware.
- 35:46 – 42:50
A detour that didn’t save them: Tegra and the mobile misadventures
- DRDavid Rosenthal
So it's 2008. What's going on in 2008, you know, in the tech world? It's mobile. So in 2008, they launch the Tegra chip and platform within Nvidia.
- BGBen Gilbert
This may not be what saved the company. [chuckles]
- DRDavid Rosenthal
This is not what saved the company. [laughing]
- BGBen Gilbert
[laughing]
- DRDavid Rosenthal
This is more, uh, uh, clown car style. Uh, that's- maybe that's too rough on Nvidia, but what was Tegra? Uh, people might recognize that name. It was a full-on system on a chip for smartphones, competing directly with Qualcomm, with Samsung. Like, it was a processor, like an Arm-based CPU, plus all of the other stuff you would need for a system on a chip to power-... Android headsets. [chuckles] Uh, I mean, this is like a wild departure for-- it leverages none of Nvidia's core skill sets, except maybe graphics being part of smartphones. But, like, come on, if there's ever a use case for integrated graphics, it's smartphones. [laughing]
- BGBen Gilbert
Right. Right, low power, smaller footprint.
- DRDavid Rosenthal
Yep, totally.
- BGBen Gilbert
Yeah.
- DRDavid Rosenthal
Do you know... [chuckles] This is one of my favorite parts about the whole research. Do you know what the first product was that shipped using a Tegra chip?
- BGBen Gilbert
Uh, no.
- DRDavid Rosenthal
It was the Microsoft Zune HD media player. [laughing]
- BGBen Gilbert
[laughing]
- DRDavid Rosenthal
Uh, that just tells you pretty much everything you need to know. [laughing] Uh, it did, though, the Tegra system, it is still around, sort of, to this day. It powered the original Tesla Model S touchscreen. So, like, before any of the autopilot, autonomous driving stuff, they were the processor powering just the infotainment, the touchscreen infotainment in the Model S. And I think that actually starts to help Nvidia get into the automotive market. The Tegra platform still to this day is the main processor of the Nintendo Switch.
- BGBen Gilbert
Oh, they repurposed it for that?
- DRDavid Rosenthal
Yeah, for that. And they s-- I think they still have their Nvidia Shield proprietary gaming device stuff that I don't know that anybody buys those.
- BGBen Gilbert
Oh, this makes so much sense because they basically have walked away from every console since the PlayStation 3.
- DRDavid Rosenthal
Yep.
- BGBen Gilbert
And so it's interesting that they have this thriving gaming division that doesn't power any of the consoles except the Nintendo Switch, and I always sort of wondered, like, why did they take on the Switch business? 'Cause they kinda already had it done.
- DRDavid Rosenthal
It's not for the graphics cards. It was as somewhere to put the Tegra stuff.
- BGBen Gilbert
Fascinating. Uh, quick aside, it's funny how these GPU companies have not been good at transitioning to mobile. There's, like, a funny naming thing-
- DRDavid Rosenthal
Yeah
- BGBen Gilbert
... but do you know what happened to-- so there's the ATI Radeon, which became the AMD Radeon desktop series. They tried to make mobile GPUs. It didn't go great, and they ended up spinning that out and selling all that IP to another company. Do you know the company?
- DRDavid Rosenthal
Ooh, I do not. Was it Apple?
- BGBen Gilbert
It is Qualcomm.
- DRDavid Rosenthal
Oh.
- BGBen Gilbert
And it today is Qualcomm's mobile GPU division, and Qualcomm's good at mobile, and so it's a natural home for it. Do you know what that line of mobile GPU processors is called?
- DRDavid Rosenthal
No.
- BGBen Gilbert
It is the Ardeno, A-R-D-E-N-O, processors. And do you know why it's called the Ardeno or Ardeno?
- DRDavid Rosenthal
No. That sounds super familiar, but no.
- BGBen Gilbert
The letters are rearranged from Radeon.
- 42:50 – 48:55
The miracle moment: ImageNet, AlexNet, and deep learning meets CUDA (2012)
- DRDavid Rosenthal
Jensen did not plan AlexNet or see it coming, 'cause nobody saw AlexNet coming. So in 2009, a Princeton computer science professor and also undergrad alum of Princeton, just like yours truly, woo, wonderful place, named Fei-Fei Li, their specialty is artificial intelligence and computer science, starts working on an image-classifying project that she calls ImageNet. Now, the inspiration for this was actually a way old project from, I think, the '80s at Princeton called WordNet, that was, like, classifying words, but this is classifying image, ImageNet. And her idea is to create a database of millions of labeled images, like images that they have the correct label applied to them, like, this is a dog, or this is a strawberry, or something like that. And that with that database, then artificial intelligence image recognition algorithms could run against that database and see how they do. So like, "Oh, look at this image of..." You know, you and I would look at it and be like, "That's a strawberry," but you don't give the answer to the algorithm, and the algorithm figures out if it thinks it's a strawberry or a dog or whatever. [chuckles] So she and her collaborators start working on this. It's super cool. They build the database. They use, uh, Mechanical Turk, Amazon Mechanical Turk, to build it. And then one of them, I'm not exactly sure who, if it was Fei-Fei or somebody else, has the idea of like, "Well, you know, we've got this database. We want people to use it. Well, let's make a competition." This is, like, a very standard thing in computer science academia of like, "Let's have a competition, an algorithm competition." So we'll do this annually, and anyone, any team, can submit their algorithms against the ImageNet database, and they'll compete, like, who can get the lowest error rate, like, the most number of images, percentage of the images correct. And, uh, this is great, so it brings her great renown, becomes popular in the AI research community. She gets poached away by Stanford the next year. I guess that's okay, 'cause I went there, too, so that's fine. [chuckles] And, uh-
- BGBen Gilbert
David!
- DRDavid Rosenthal
... She's still there to this-- I know.
- BGBen Gilbert
[laughing]
- DRDavid Rosenthal
I, I couldn't resist. I couldn't resist. I was just... She's like a kindred spirit to me. Do you know... I know you do know, but I bet most listeners do not know, what her endowed tenure chair is at Stanford today?
- BGBen Gilbert
I do. She is the Sequoia Chair.
- DRDavid Rosenthal
Yes, the Sequoia Capital Professor of Computer Science at Stanford. [inhales] So cool. Why did she become the Sequoia Capital chair, and what does all this have to do with NVIDIA? Well, in the 2012 competition, a team from the University of Toronto submits an algorithm that wins the competition, and it doesn't just win it by, like, a little bit. It wins it by a lot. So the way they measure this is, the hundred percent of the images in the database, what percentage of them did you get wrong? So it wins it by over ten percent. I think it had a fifteen percent error rate or something, and the next-
- BGBen Gilbert
Like, all the best previous ones have been, like, twenty-five-point-something percent.
- DRDavid Rosenthal
Yes.
- BGBen Gilbert
This is like someone breaking the four-minute mile. Uh, actually, in some ways, it's more impressive than the four-minute mile thing because they just didn't brute force their way all the way there. They, like, tried a completely different approach-
- DRDavid Rosenthal
Yes
- BGBen Gilbert
... And then, boom, showed that we could get way more accurate than anyone else ever thought.
- DRDavid Rosenthal
So what was that approach? Well, they called the team, which was composed of Alex Krizhevsky, uh, was the primary, uh, lead of the team, he was a PhD student, in collaboration with Ilya Sutskever and Geoff Hinton. Uh, Geoff Hinton was the PhD advisor of, of Alex. They call it AlexNet. What is it? It is a convolutional neural network, which is a branch of artificial intelligence called deep learning. Now, deep learning is new for this use case, but, Ben, as-- you weren't exactly right. It had been around for a long time, a very long time, and deep learning neural networks, this was not a new idea. The algorithms had existed for many decades, I think, but they were really, really, really computationally intensive. They required, to train the models to do a, a deep neural network, you need a lot of compute, like, on the order of, you know, like, grains of sand that exist on Earth. [chuckles] It was completely impossible with a traditional computer architecture that you could make these work in any practical applications.
- BGBen Gilbert
And people were forecasting, too, like, when with Moore's Law, when will we be able to do this? And it still seemed like the far future, because not only did Moore's Law need to happen, but you also needed the NVIDIA approach of massively parallelizable architecture, where suddenly you could get all these incredible performance gains, not just because you're putting, you know, more transistors in a given space, but because you're able to run programs in parallel now.
- DRDavid Rosenthal
Yes. So AlexNet took these old ideas, and it implemented them on GPUs. And to be very specific, it implemented them in CUDA on NVIDIA GPUs. We cannot overstate the importance of this moment, not just for NVIDIA, but for, like, computer science, for technology, for business, for [chuckles] the world, for us staring at the screens of our phones all day, every day. This was the Big Bang moment for artificial intelligence, and NVIDIA and CUDA were right there. [chuckles]
- BGBen Gilbert
Yep. It's funny, there's another example within the next couple of years, 2012, 2013, where NVIDIA had been thinking about-... this notion of general purpose computing for their architecture for a long time. In fact, they even thought about, "Should we relaunch our GPUs as GP GPUs, general purpose graphics processing units?" And of course, they decided not to do that, but just built CUDA.
- DRDavid Rosenthal
Which is code word for, like, "We've been searching for years for a market for this thing. We can't find a market, so we'll just say, 'You can use it for anything!'" [chuckles]
- 48:55 – 54:32
Productizing deep learning: cuDNN and the hyperscaler adoption wave
- BGBen Gilbert
Right. And so deep learning's generating a lot of buzz, you know, a, a lot from this AlexNet competition. And so in 2013, Brian Catanzaro, who's a research scientist at NVIDIA, published a paper with some other researchers at Stanford, which included Andrew Ng, where they were able to take this unsupervised learning approach that had been done inside the Google Brain team, where they had sort of- the Google Brain team had sort of published their work on this, and it had 1,000 nodes. And, you know, this is a big part of the sort of early neural network hype cycle of people trying cool stuff, and this team was able to do it with just three nodes. So totally different model, super parallelized, lots of compute for a super short period of time in a really high-performance computing way, or HPC, as it would sort of become known. And this ends up being the very core of what becomes cuDNN, which is the library for deep neural networks that's actually baked into CUDA, that makes it easy for data scientists and research scientists everywhere who aren't hardware engineers or software engineers to just pretty easily write high-performance deep neural networks on NVIDIA hardware. So this AlexNet thing, plus then Brian and Andrew Ng's paper, it just collapses all these sort of previously thought to be impossible lines to cross and just makes it way easier and way more performant and way less energy-intensive for other teams to do it in the future.
- DRDavid Rosenthal
Yep, and specifically to do deep learning. So I think at this point, like, everybody knows that this is [chuckles] pretty important, but it's not that much of a leap to say if you can train a computer to recognize images on its own, that you can then train a computer to see on its own, to drive a car on its own, to play chess, to play Go, to make your photos look really awesome when you take them on the latest iPhone, even if you don't have everything right.
- BGBen Gilbert
To eventually let you describe a scene and then have a transformer model paint that scene for you in a way that is unbelievable that a human didn't make it.
- DRDavid Rosenthal
Yep, and then most importantly, for the market that Jensen and NVIDIA are looking for, you can use this same branch of AI to predict what type of content you might like to see next show up in your feed [chuckles] of content and what type of ad might work really, really, really well on you. [chuckles] So basically, all of these people we were just talking about, I bet a lot of you recognize their names, they get scooped up by Google. Fei-Fei Li goes to Google.
- BGBen Gilbert
Brian went to Baidu, and he's back at NVIDIA now, doing applied AI.
- DRDavid Rosenthal
Brian went to Baidu. Jeff Hinton goes to Facebook. So, you know, all the other markets-- Like, even throw out-- Say you don't believe in self-driving cars, you don't think it's gonna happen or any of this other stuff. Like, it just doesn't matter. Like, [chuckles] the market of advertising, of digital advertising that this enables is a freaking multi-trillion dollar market.
- BGBen Gilbert
And it's funny 'cause like, that feels like, ooh, that's the killer use case, but that's just the easiest use case. That's the most, like-
- DRDavid Rosenthal
Yes
- BGBen Gilbert
... obvious, well-labeled data set, that these models don't have to be amazingly good 'cause they're not generating unique output. They're just assisting in making something more efficient. But then, like, flash forward ten more years, and now we're into these crazy transformer models with, I don't know if it's hundreds of millions or billions of parameters. Things that we thought only humans could do are now being done by machines, and it's like it's happening faster than ever.
- DRDavid Rosenthal
Yep.
- BGBen Gilbert
So I think to your point, David, it's like, oh, there was this big cash cow enabled by, you know, neural networks and deep learning in advertising. Sure, but that was just the easy stuff.
- DRDavid Rosenthal
Right, but that was necessary, though. This was finally the market that enabled the building of scale [chuckles] and the building of technology to do this, and, uh-
- BGBen Gilbert
Yes
- DRDavid Rosenthal
... in the Ben Thompson, um, Jensen interview, Ben actually says this when he's sort of realizing this, talking to Jensen, he says... This is Ben talking: "The way value accrues on the internet in a world of zero marginal costs, where there's just an explosion in abundance of content, that value accrues to those who help you navigate the content." And he's talking about aggregation theory, duh. And then he says, "What I'm hearing from you, Jensen, is that, yes, the value accrues to people that help you navigate that content, but someone has to make the chips and the software so that they can do that effectively." [chuckles] And it's like it sort of used to be with Windows was the consumer-facing layer, and Intel was the other piece of the Wintel monopoly. This is Google and Facebook and a whole host of other companies on the consumer side, and they're all dependent on NVIDIA, and that sounds like a pretty good place to be. [chuckles] And indeed, it was a pretty good place to be.
- BGBen Gilbert
Amazing place to be.
- DRDavid Rosenthal
Oh, my gosh. The thing is, like, the market did not realize this for years, and, I mean, I didn't realize this, and I- you probably didn't realize this. We were the class of people working in tech as venture capitalists that should have.
- BGBen Gilbert
Ooh, do you know the Marc Andreessen quote?
- DRDavid Rosenthal
Ooh, no.
- BGBen Gilbert
Oh, this is awesome. Okay, so it's a couple years later, so it's, like, getting more obvious, but it's 2016, and Marc Andreessen gave an interview, and he said, "We've been investing in a lot of companies applying deep learning to many areas, and every single one effectively comes in building on NVIDIA's platforms. It's like when people were all building on Windows in the '90s or all building on the iPhone in the late 2000s." And then he says, "For fun, our firm has an internal game of what public companies we'd invest in if we were a hedge fund. We'd put in all of our money to NVIDIA."
- 54:32 – 1:05:43
From underappreciated to obvious: stock cycles, crypto mining, and renewed volatility
- DRDavid Rosenthal
This is like, uh... It was Paradigm, right? That called all of their capital in one of their funds and put it into Bitcoin when it was, like, three thousand dollars a coin or something like that.... We all should have been doing this. So literally, Nvidia's stock in twenty, like recent, like this is now known, twenty twelve, thirteen, fourteen, fifteen, it doesn't trade above, like, five bucks a share. And Nvidia today, as we record this, is I think about two twenty a share. The high in the past year has been well over three hundred. Like, if you realized what was going on, and, and, and again, in a lot of those years, it was not that hard to realize what was going on. Wow! Like, [chuckles] it was huge.
- BGBen Gilbert
It's funny. So there was even-- and we'll get to what happened in twenty seventeen and twenty eighteen with crypto in a little bit, but there was a massive stock run-up to like sixty-five dollars a share in twenty eighteen. And even a- as late as, I think, the very beginning of twenty nineteen, you could have gotten it... I tweeted this, and we'll put the graph on the screen in the YouTube version here. You could have gotten it in that crash for thirty-four bucks a share.
- DRDavid Rosenthal
[exhales] In twenty nineteen!
- BGBen Gilbert
If you zoom out on that graph, which is the next tweet here, then you can see that, like, in retrospect, that little crash just looks like nothing. You don't even pay attention to it in the crazy run-up that they had to three fifty or whatever their, their all-time high was.
- DRDavid Rosenthal
Yeah, it's wild. And a few more wild things about this: It's not until twenty sixteen, again, AlexNet happens in twenty twelve. It's not until twenty sixteen that Nvidia gets back to the twenty billion dollar market cap peak that they were in two thousand and seven when they were just a gaming company. [chuckles] That's almost ten years.
- BGBen Gilbert
I really hadn't thought about it the way that you're describing it, but the breakthrough happened in twenty ten, twenty eleven, twenty twelve. Lots of people had the opportunity, especially because freaking Jensen's talking about it on stage, he's talking about it at earnings calls at this point.
- DRDavid Rosenthal
He's not keeping this a secret.
- BGBen Gilbert
No, he's, like, trying to tell us all that this is the future, and people are still skeptical. Everyone's not rushing to buy the stock. We're watching this freaking magic happen, using their hardware, using their software on top of it, and, like, even semiconductor analysts, who are like students of listening to Jensen talk and following the space very closely, sort of think he sounds like a crazy person when he's up there espousing that the future is neural networks, and we're gonna go all in, and we're not pivoting the business, but from the amount of attention that he's giving in earnings calls to this versus gaming, I mean, everyone's just like, "Uh, are you off your rocker?"
- DRDavid Rosenthal
Well, I think people had just lost trust and interest, [chuckles] you know, after, like... There were so many years of, like, they were so early with CUDA and early tech, and they didn't even know that this-- like, they didn't know AlexNet was gonna happen.
- BGBen Gilbert
Right.
- DRDavid Rosenthal
Jensen felt like the GPU platform could enable things that the CPU paradigm could not, and he, like, had this faith that something would happen. But, like, he didn't know this was gonna happen. [chuckles] And so for years, he was just saying that, like: "We're building it, they will come," you know? And-
- BGBen Gilbert
And to be more specific, it was that... Well, look, the GPU has accelerated the graphics workload, so we've taken the graphics workload off of the CPU. The CPU is great, it's your primary workhorse for all sorts of flexible stuff, but we know graphics needs to happen in its own separate environment and have all these fancy fans on it and get super cooled, and it needs these matrix transforms. The math that needs to be done is matrix multiplication, and there was starting to be this belief that, like, oh, well, because the, you know, professor, the apocryphal professor, told me that he was able to use these program, the matrix transforms, to work for him, you know, maybe this matrix math is really useful for other stuff. And sure, it was for scientific computing. And then, honestly, like, it fell so hard into Nvidia's lap that the thing that made deep learning work was massively parallelized matrix math, and they're like, Nvidia's just, like, staring down at their GPUs, like, "Uh, I think we have exactly what you are looking for." [laughing]
- DRDavid Rosenthal
Yes. [laughing] There's, uh, that same, uh, interview with Brian Catanzaro. He says about when all this happened, he says, the deep learning happened to be the most important of all applications that need high-throughput computation. Understatement of the century. [chuckles] And so once Nvidia saw that, it was basically instant. The whole company just latched onto it. There's so many things to laud Jensen for. You know, he was painting a vision for the future, but he was paying very close attention, and the company was paying very close attention to anything that was happening, and then when they saw that this was happening, they were not asleep at the switch.
- BGBen Gilbert
Yeah, hundred percent. It's interesting thinking about the fact that i- in some ways, it feels like an accident of history, in some ways, it feels so intentional, that graphics is an embarrassingly parallel problem because every pixel on a screen is unique. I mean, they don't have a core to drive every pixel on the screen. There's only ten thousand cores on the most recent Nvidia graphics cards, but there's not... Which is crazy, right?
- DRDavid Rosenthal
[chuckles]
- BGBen Gilbert
But there's way more pixels on a screen.
- DRDavid Rosenthal
Right.
- BGBen Gilbert
So, you know, they're not all doing every single pixel at the same time, every clock iteration. But it worked out so well that neural networks also can be done entirely in parallel like that, where every single computation that is done is independent of all the other computations that need to be done, so they also can be done on this super parallel set of cores. It's just, you gotta wonder, like, when you kind of reduce all this stuff to just math, it is interesting that these are two very large applications of the same type of math. In the search space of the world, of what other problems can we solve with parallel matrix multiplication, there may be more. There may even be bigger markets out there.
- DRDavid Rosenthal
Totally. Well, I think there probably will be. [chuckles] A big part of Jensen's vision that he paints for Nvidia now, which we'll get to in a sec, is this is just the beginning. [chuckles] You know, there's robotics, there's autonomous vehicles, there's the omniverse. It's all coming.... It's funny, we just joked about how, like, nobody saw this before the run-up in twenty sixteen, twenty seventeen. There were all these years where, like, Marc Andreessen knew, [chuckles] you know, whether he made money in his personal account or not, you know-
- BGBen Gilbert
[chuckles]
- DRDavid Rosenthal
-we'll have to ask him. But then in twenty eighteen, another class of problems that are embarrassingly parallelizable is, of course, cryptocurrency mining. [chuckles] And so a lot of people were going out and buying consumer NVIDIA, you know, graphics cards and using them to set up crypto mining rigs in twenty sixteen and twenty seventeen. And then when the crypto winter hit in twenty eighteen and the end of the ICO craze and all that, the mining rig demand it fell off, and this had become so big for NVIDIA that their revenue actually declined.
- BGBen Gilbert
Right. Yeah, so a couple interesting things here. Let's talk about technically why. So the way crypto mining works is effectively guess and check. You're effectively brute forcing an encryption scheme, and when you're mining, you know, you're trying to discover the answer to something that is hard to discover. So you're guessing. If that's not the right thing, you're incrementing, you're guessing again, and that's a vast oversimplification and not technically exactly right, but that's the right way to think about it. And if you were gonna guess and check at a math problem, and you had to do that on the order of a few million times in order to discover the right answer, you could very unlikely discover the right answer on the first time, but, you know, that probabilistically is only gonna happen to you once, if ever. And so well, the cool thing about these chips is that, A, they have a crap ton of cores, so the problem like this is massively parallelizable because instead of guessing and checking with one thing, you can guess and check with ten thousand at the same time, and then ten thousand more, and then ten thousand more. And the other thing is, it is matrix math. So yet again, there's this third application beyond gaming, beyond neural networks. There's now this third application in the same decade for the two things that these chips are uniquely good at. And so it's interesting that, like, you could build hardware that's better for crypto mining or better for AI, and both of those things have been built by NVIDIA and their competitors now, but the sort of like general purpose GPU happened to be pretty darn good at both of those things.
- DRDavid Rosenthal
Well, at least way, way, way better than a CPU.
- BGBen Gilbert
Yeah. As some of NVIDIA's startup competitors put it today, and Cerebras is the one that I'm thinking of, they sort of say, "Well, the GPU is a thousand times better or, you know, much, much better than a CPU for doing this kind of stuff, but it's like a thousand times worse than it should be." There exists much more optimal solutions for, you know, doing some of this, this AI stuff.
- DRDavid Rosenthal
Interesting. Really begs the question of like, how good is good enough in these use cases? [chuckles]
- BGBen Gilbert
Right. And now, I mean, uh, to flash way forward, the game that NVIDIA and everyone else, all these upstarts are playing, is really... It's still the accelerated computing game, but now it's how do you accelerate workloads off the GPU instead of off the CPU?
- DRDavid Rosenthal
Interesting. [chuckles] Well, back to crypto winter-
- BGBen Gilbert
[chuckles]
- DRDavid Rosenthal
... just 'cause this is so funny. Uh, A, well, crypto itself became like a real industry. [chuckles] I don't think that's a controversial statement at this point. Maybe it is, maybe it isn't, but certainly it's less controversial than it was in twenty eighteen.
- BGBen Gilbert
Yes.
- 1:05:43 – 1:16:50
Data center dominance: enterprise AI economics and the Mellanox expansion
- DRDavid Rosenthal
report financials a couple different ways, but one of the ways they break it out is a few different segments. There's the gaming consumer segment, and then their data center segment, and it's like data center. Like, what do they do in the data center? Well, all the use cases for-
- BGBen Gilbert
Data centers AI.
- DRDavid Rosenthal
[chuckles] Right. All of the stuff we're talking about, it's all done in the data center. Like, Google isn't going and buying, [chuckles] you know, a bunch of NVIDIA GPUs and hooking them up to the laptops of their software engineers like-
- BGBen Gilbert
Is Stadia still a thing? Like, I think that's used for cloud gaming and some-- like, there, there are-
- DRDavid Rosenthal
Oh, yeah, but it's all happening in the data center is, uh, my point.
- BGBen Gilbert
Right, right. I guess what I'm saying, and my argument is, every time I see data center revenue, I, in my mind, I sort of make it synonymous with, this is their ML segment.
- DRDavid Rosenthal
Ah, yes, yes, that's what I'm saying. I, I agree.
- BGBen Gilbert
Yeah.
- DRDavid Rosenthal
Now, the data center, this is really interesting, again, because they used to sell these cards that would get packaged, put on a shelf, a consumer would buy them. Yeah, they made some specialty cards for the scientific computing market and stuff like that, but this data center opportunity, like, man, do you know the prices that you can sell gear to data centers for? Like, it makes the RTX 3090 look like a pence.
- BGBen Gilbert
... And the RTX 3090, which is their most expensive high-end graphics card that you can buy as a consumer, was $3,000, now it's like $2,000. But if you're buying, I don't know, what's the latest? It's not the A100, it's the H100?
- DRDavid Rosenthal
Uh, so the A100, they just announced the H100.
- BGBen Gilbert
And that's what? Like 20 or 30 grand in order to just get one card.
- DRDavid Rosenthal
Yeah. [chuckles] And people are buying a lot of these things.
- BGBen Gilbert
Yeah, it's crazy. It's crazy.
- DRDavid Rosenthal
It's funny, I tweeted about this, and I was sort of wrong, but then, like everything, there's nuance. You know, Tesla has announced making their own hardware. They're certainly doing it for the on-the-car, the inference stuff, like the full self-driving computer on Teslas, they now make those chips themselves. The Tesla Dojo, which is the training center that they announced, they announced they were also gonna make their own silicon for that. They actually haven't done it yet, so they're still [chuckles] using NVIDIA chips for their training. The current compute cluster that they have, that they're still using, I wanna say I did the math and, like, assumed some pricing, I think they spent between $50 and $100 million, that they paid NVIDIA for all of the compute in that-
- BGBen Gilbert
Wow!
- DRDavid Rosenthal
... cluster. [chuckles]
- BGBen Gilbert
That's one customer.
- DRDavid Rosenthal
That's one customer for one use case at that one customer.
- BGBen Gilbert
Crazy. I mean, you see this show up in their earnings. So we're at the part of the episode where we're close enough to today that it's best illustrated by the today numbers, so I'll, I'll just flash forward to what the data center segment looks like now. So two years ago, they had about $3 billion of revenue, and it was only about half of their gaming revenue segment. So gaming, you know, through all this, through 2006 to AlexNet, all the way, you know, another decade forward to 2020, gaming is still king. It generates almost $6 billion in revenue. The data center revenue segment was $3 billion, but had been pretty flat for a couple years. So then insanely, over the last two years, it 3X'd. The data center segment 3X'd, and is now doing over $10.5 billion a year in revenue, and it's basically the same size as the gaming segment. It's nuts. It's amazing how it was, like, sort of obvious in the mid-2010s, but when the enterprises really showed up and said, "We're buying all this hardware and putting it in our data centers," and whether that's the hyperscalers, the, like, cloud folks, Google, Microsoft, Amazon, putting it in their data centers, or whether it's companies doing it in their own private clouds or whatever they wanna call it these days, on-prem data centers, everyone is now using machine learning hardware in the data center.
- DRDavid Rosenthal
Yep. And NVIDIA is selling it for very, very, very healthy [chuckles] gross margins. Apple-level gross margins.
- BGBen Gilbert
Yes, exactly.
- DRDavid Rosenthal
So speaking of the data center, couple things. One, in... [chuckles] This is so NVIDIA. In 2018, they actually do change the terms of the user agreements of their consumer-
- BGBen Gilbert
[chuckles]
- DRDavid Rosenthal
... cards, of GeForce cards, that you cannot put them in data centers anymore. [laughing]
- BGBen Gilbert
They're like-
- DRDavid Rosenthal
Of course
- BGBen Gilbert
... "Uh, we really do need to start segmenting a little bit here, and, uh, we know that the enterprises have much more willingness to pay," and it, it is worth it. I mean, you buy these crazy data center cards, and they have, like, twice as many transistors, and actually, they don't even have video outputs. Like, you can't use the data center GPUs, like the A100 does not have video out, so they actually can't be used as graphic cards.
- DRDavid Rosenthal
Oh, yeah, there was a-- there's a cool, um, Linus Tech Tips video about this, where they get a hold of an A100 somehow, and then they run some benchmarks on it, but they can't actually, like, drive a game on it.
- BGBen Gilbert
Oh, fascinating.
- 1:16:50 – 1:30:00
Owning the whole stack (almost): the Arm acquisition attempt and Grace/Hopper
- DRDavid Rosenthal
That one more thing is, uh... It's easy to forget now. I know because we've just been deep on this. NVIDIA was gonna buy Arm. [chuckles] Do you remember this?
- BGBen Gilbert
Yes, they were. And in fact, this is gonna be like a corporate communications nightmare. Everyone out there, Jensen, their IR person, different tech people who are being interviewed on various podcasts, were talking about the whole strategy and how excited they are to own Arm, and how NVIDIA's gonna be... You know, it's good on its own, but it could be so much better if we had Arm, and here's all the cool stuff we're gonna do with it, and then it doesn't happen.
- DRDavid Rosenthal
They were talking about it like it was a done deal.
- BGBen Gilbert
And now you've got dozens of hours of people talking about the strategy, so you're almost like-- It's funny that now after listening to all that, I'm sort of, like, disappointed with NVIDIA's ambition on its own without having the strategic assets of Arm. [chuckles]
- DRDavid Rosenthal
Yeah. We should revisit Arm at some point. We did do the SoftBank acquiring Arm episode years and years ago now. But, you know, you think Arm, like, they are a CPU architecture company [chuckles] whose primary use case is mobile and smartphones, right? So, like, everything that Intel screwed up on [chuckles] back in the misguided mobile era, now they're going and buying, like, the most important company in that space. You know, and it's interesting, like, again, in the Ben Thompson interview, Jensen talks all about this, and maybe this is just justifying in retrospect, but I don't think so. He's like: Look, it was about the data center. Yeah, like, everything Arm does is, like, great, and that's fine, but, like, we wanna own the data center. When we say we wanna own the data center, we wanna own everything in the data center, and we think Arm chips, Arm CPUs, can be really, a really important part of that. Arm is not focusing right now enough on that. Why would they? Their core market is mobile. We want them to do that. We think there's a huge opportunity. We wanna dial them in and do that. And indeed, this year, NVIDIA announced they are making a data center CPU, an Arm-based data center CPU called Grace, to go with the new Hopper architecture for their latest GPU. So there's Grace and Hopper. Uh, of course, the rear admiral, Grace Hopper, I think?
- BGBen Gilbert
I think that's right. Yeah.
- DRDavid Rosenthal
She was in the Navy. Uh, great computer scientist, pioneer.... So yeah, like da- data center. [chuckles] It's, it's big.
- BGBen Gilbert
It's interesting. So the objectors to that acquisition, and it's a good objection, and this is ultimately, I think, why they abandoned it, 'cause they got the regulatory pressure on this, is Arm's business is simple. They make the IP, so you can license one of two things from them. You can license the instruction set, so even Apple, who designs their own chips, is licensing the Arm instruction set. And so in order to use that, I don't know what it actually is, 20 keywords or so, that, that can get compiled to assembly language to run on whatever the chip is, you know, if you wanna use these instructions, you have to license it from Arm, great. And if you don't wanna be Apple, and you don't wanna go build your own chips, or you don't wanna be NVIDIA or whatever, but you wanna use our- that instruction set, you can also license these off-the-shelf chip designs from us, and we will never manufacture any of them. But you take one of these two things you license from us, you have someone like TSMC make them, great, now you're a fabless semiconductor company. And they sell to everyone. And so of course, a regulatory body is gonna step in and being like, "Wait, wait, wait. So NVIDIA, you're a fabless chip company. You're a vertically integrated business model. Are you gonna stop allowing Arm licenses to other people?" And NVIDIA goes, "Oh, no, no, no, no. Of course, we would never do that." Over time, they might do some stuff like that. But the thing that they were sort of like... Which, which is believable, beating the drum on that the strategy was going to be, is right now, our whole business's strategy is that CUDA and everything built on top of it, our whole software services ecosystem, is just for our hardware. And how cool would it be if you could use that stuff on Arm-designed IP, either just the, using the ISA or also using the actual designs that people license from them? How cool would it be if, because we were one company, we were able to make all of that stuff available for Arm chips as well?
- DRDavid Rosenthal
Yep.
- BGBen Gilbert
Plausible, interesting, but no surprise at all that they faced too much regulatory pressure to go through with this.
- DRDavid Rosenthal
No. But clearly, that idea rattled around in Jensen's head a bunch [chuckles] and in NVIDIA's head, because, um... Well, let's catch us up to today. So they just did GTC at the end of March, the big, uh, developer- the big GPU developer conference that they do every year, that they started in 2009 as part of building the whole CUDA ecosystem. Um, it's so freaking impressive [chuckles] now. Like, there are now three million registered CUDA developers, 450 separate SDKs and models for CUDA. They announced 60, six-zero, new ones at this GTC. Yeah, we talked about the next-generation GPU architecture with Hopper, and then the Grace CPU to go along with it. I think Hopper, I could be wrong on this, I think Hopper is gonna be the world's first four-nanometer process chip using TSMC's new four-nanometer process, which is-
- BGBen Gilbert
I think that's right
- DRDavid Rosenthal
... amazing. To talk a lot about Omniverse, we're gonna talk about Omniverse in a second, but you mentioned this licensing thing. They usually do their investor day, their analyst day, at the same time as GTC. [chuckles] And in the analyst day, Jensen gets up there... It's just so funny, I've been going through the whole history of this now, of, like, looking for a market, trying to find some market of any size, and he's like, "We are targeting a trillion-dollar market." [chuckles] He's like a startup raising a seed round, walking in with a pitch deck.
- BGBen Gilbert
We'll put this graphic up on the screen for those watching the video. It's a articulation of what the segments are of this trillion-dollar addressable opportunity that NVIDIA has in front of it. My view of this is, if their stock price wasn't what it was, there's no way that they would try to be making this claim that they're going after a trillion-dollar market. I think it's squishy.
- DRDavid Rosenthal
Oh, there's a lot of squish in there. [chuckles]
- BGBen Gilbert
But the fact that they're valued today... I mean, what's their market cap right now? Something like-
- DRDavid Rosenthal
About half a trillion.
- BGBen Gilbert
Half a trillion dollars. They need to sort of justify that, unless they are willing to have it go down, and so they need to come up with a story about how they're going after this ginormous opportunity. Which maybe they are, but it leads to things like an investor day presentation of, "Let us tell you about our trillion-dollar opportunity ahead." And the way that they actually articulate it is, "We are going to serve customers that represent a $100 trillion opportunity, and we will be able to capture about 1% of that."
- DRDavid Rosenthal
[chuckles] God, it's just like a freaking seed company pitch deck. [chuckles]
- BGBen Gilbert
If we just get 1% of the market. [chuckles]
- DRDavid Rosenthal
Well, and that's the thing. We're gonna talk about this in narratives in a minute, but this is a generational company. This is unbelievable. This is amazing. There's so much to admire here. This company did, what, like 20-something billion in revenue last year and is worth half a trillion dollars?
- BGBen Gilbert
They did $27 billion last year in revenue.
- DRDavid Rosenthal
Google AdWords revenue in the fourth quarter of 2021 [chuckles] was 43 billion.
- BGBen Gilbert
[laughs]
- DRDavid Rosenthal
Google as a whole did 257 billion in revenue. So, like, you gotta believe if you're an NVIDIA shareholder.
- BGBen Gilbert
Right. They're the eighth largest company in the world by market cap, but these revenue numbers, you know, are in a different order of magnitude.
- DRDavid Rosenthal
You gotta believe it's on the come.
- BGBen Gilbert
Yeah, you do. I mean, NVIDIA has literally three times the price-to-sales ratio of Apple, or price to revenue as Apple, and nearly 2X Microsoft, and that's on revenue. I mean, fortunately, this NVIDIA story is not speculative in the way that an early-stage startup is speculative. Like, even if you think it's overvalued, it is still a very cash-generative business.
- DRDavid Rosenthal
Yes.
- BGBen Gilbert
They generate eight billion of free cash flow every year.
- 1:30:00 – 1:42:27
New frontiers: gaming innovations (ray tracing, DLSS), automotive, and Omniverse digital twins
- BGBen Gilbert
Inflation be damned. Okay, a couple other things about specific segments of the business that I think are pretty interesting. So they have not slept on gaming. Like, we keep beating this Nvidia data center, enterprise-
- DRDavid Rosenthal
Yeah
- BGBen Gilbert
... machine learning argument.
- DRDavid Rosenthal
Yeah, we haven't even talked about ray tracing and-
- BGBen Gilbert
Right. Yeah, this RTX set of cards that they came out with. The fact that they could do ray tracing in real time, holy crap! For anyone who's looking for sort of a fun dive on how graphics works, go to the Wikipedia page for ray tracing. It's very cool. You model where all the light sources are coming from, where all the paths would go in 3D. The fact that Nvidia can render that in real time at sixty frames a second or whatever, while you're playing a video game, is nuts. And one of the ways that they do that, they invented this new technology that's extremely cool. It's called DLSS, deep learning super sampling, and this, I think, is, like, where Nvidia really shines, bringing machine learning stuff and gaming stuff together.... where they basically have faced this problem of, well, we either could render stuff at low resolution with less frames, 'cause, uh, we can only render so much per amount of time, or we could render really high-resolution stuff with less frames. And nobody likes less frames, but everyone likes high resolution, so what if we could cheat death, and what if we could get high resolution and high frame rate? And they're sitting around thinking, "How on earth can we do that?" And they're like, "You know what? Maybe this 15-year bet that we've been making on deep learning can help us out."
- DRDavid Rosenthal
Mm.
- BGBen Gilbert
And what they discovered here and, and invented in DLSS, and AMD does have a competitor to this, it's a similar sort of idea, but this DLSS concept is totally amazing. So what they basically do is they say, "Well, it's very likely that you can infer what a pixel is going to be based on the pixels around it."
- DRDavid Rosenthal
Ah, that's awesome.
- BGBen Gilbert
"It's also pretty likely you can infer what a pixel is gonna be based on what it was in the previous frames. And so let's actually render it at a slightly lower resolution, so we can bump up the frame rate, and then when we're outputting it to screen, we will use deep learning to artificially- "
- DRDavid Rosenthal
Ah, at the final stage of the graphics pipeline.
- BGBen Gilbert
Yes.
- DRDavid Rosenthal
Yo, that's awesome.
- BGBen Gilbert
It's really cool, and when you watch the side by side on all these YouTube videos, it looks amazing. I mean, it does involve really tight, embedded development with the game developers. They have to sort of do stuff to make it DLSS-enabled, but it just looks phenomenal, and it's so cool that when you're looking at this 4K or even 8K output of a game at, you know, full frame rate, you're like, "Whoa! In the middle of the graphics pipeline, this was not this resolution, and then they magically upscaled it." It's basically making the, like, enhance joke, like a real thing.
- DRDavid Rosenthal
That's so awesome. [chuckles] I'm remembering back to the Riva 128 in the beginning of- when they went to game developers, and they were like, "Yeah, yeah, yeah, all the blend modes in, in DirectX," you know, "You don't need all of them, just use these." [chuckles]
- BGBen Gilbert
Yes, exactly. Exactly. And they have the power to do it. I mean, they have the stick and the carrot with game developers to do it.
- DRDavid Rosenthal
Oh, I mean, at this point, no game developer is not gonna make their games optimized for the latest NVIDIA hardware.
- BGBen Gilbert
The other thing that is funny that's within the gaming segment, because they didn't want to create a new segment for it, is crypto. So because they have poor visibility into it, and before they weren't liking the fact that it was actually reducing the amount of cards that were available to the retail channel for their gamers to go and buy, what they did was they artificially crippled the card to make it worse at crypto mining.
- DRDavid Rosenthal
And then they came out with a dedicated crypto mining card. [chuckles]
- BGBen Gilbert
Yes. And so, like, the charitable PR thing from NVIDIA is, "Hey, you know, we really did- we love gamers, and we didn't want to make it so that the gamers couldn't get access to-
- DRDavid Rosenthal
[chuckles]
- BGBen Gilbert
... you know, all the cards they want." But really, they're like, "Hmm, people are just, like, straight-up performing an arbitrage by crypto mining on these cards. Let's make that more expensive on the cheap cards, and let's make dedicated crypto hardware for them to buy to do those."
- DRDavid Rosenthal
Let's make that our arbitrage. [chuckles]
- BGBen Gilbert
Yes.
- DRDavid Rosenthal
Your arbitrage is my opportunity. [chuckles]
- BGBen Gilbert
So magically, their revenue is more predictable now, and they get to make more money because, much like their sort of terms of service data center thing, they terms of serviced their way to being able to create some segmentation, and thus, more profitability.
- DRDavid Rosenthal
Love it.
- BGBen Gilbert
Evil, evil, genius laugh. [chuckles] The last thing that you should know about NVIDIA's gaming segment is this really weird concept of add-in board partners. So we've been oversimplifying in this whole episode, saying, "Oh, you know, you go, and you buy your RTX 3090 Ti at the store, and you run your favorite game on it." But actually, you're not buying that from NVIDIA the vast majority of the time. You are going to some third-party partner, ASUS, MSI, uh, ZOTAC is one. They've-- there's also, like, a bunch of really low-end ones as well, who NVIDIA sells the cards to, and those people install the cooling and the branding and all this stuff on top of it, and you buy it from them, and it's really weird to me that NVIDIA does that.
- DRDavid Rosenthal
I love how consumer gaming graphics cards have become the modern-day equivalent of a hot rod. [chuckles]
- BGBen Gilbert
Oh, dude, as you can imagine, for this episode, I've been hanging a lot on the NVIDIA subreddit, and, like, it's not actually about NVIDIA or NVIDIA the company or NVIDIA the strategy. It's like, show off your sick photos of your glowing rig, [chuckles] which is pretty funny. But, like, it feels like a remnant of old NVIDIA that they still do this. Like, they do make something called the Founders Edition card, and it's basically a reference design where you can buy it from NVIDIA directly, but I don't think the vast majority of their sales actually come from that.
- DRDavid Rosenthal
Oh, it's like, um... What are the Android phones that Google makes, Pixel?
- 1:42:27 – 1:54:28
Moats, powers, and the forward-looking debate: bull/bear, valuation, and platform lock-in
- BGBen Gilbert
All right, you wanna talk bear and bull case on the company?
- DRDavid Rosenthal
Let's do it. [sighs] Analysis.
- BGBen Gilbert
So, I mean, they paint the bull case for us when they say, "There's a $100 trillion future. We're gonna capture 1% of it. There's $300 billion from automotive. Here's the four or five segments that add up to a trillion dollars of opportunity." Sure, that's, like, a very neat way with a bow on it and a very wishy-washy, hand-wavy way of articulating it. So the question sort of becomes: Where does AMD fall in all this? They're a legitimate second-place competitor for high-end gaming graphics, and I think will continue to be. That feels like a place where th- these two are gonna keep going head-to-head. The bear case is that there's a TikTok rather than a durable, competitive advantage for Nvidia, but most high-end games you can play on both AMD and Nvidia hardware at this point. The question for the data center is...... Is the future these general purpose GPUs that NVIDIA continues to modify the definition of GPU to include specialized, you know, functions as well, all this other stuff they're putting on their, in their hardware? Or is there someone else who is coming along with a completely different approach to accelerated computing, where they're accelerating workloads off the GPU onto something new, like a Cerebras or like a Graphcore, that is gonna eat their lunch in the enterprise AI data center market? [chuckles] That's an open question.
- DRDavid Rosenthal
You know, it's interesting. Like, people have been talking about that for a while. The other big bear case that people have been talking about, again, for a while now, is, you know, the big, big customers of NVIDIA that are paying them a lot of money, the Teslas, the Googles, the Facebooks, the Amazons, the Apples. And not just paying them a lot of money and getting, you know, assets of value of that, they're paying high gross margin dollars to [chuckles] NVIDIA for what they're getting, that those companies are gonna wanna say, "You know, it's not that hard to design our own silicon to bring all this stuff in-house. We can tune it to exactly our use cases," sorta similar to the Cerebras, uh, Graphcore bear case on NVIDIA. I think in both of these cases, you know, it hasn't happened yet. [chuckles]
- BGBen Gilbert
Well, there have been a lot of people who have made a lot of noise-
- DRDavid Rosenthal
Yes.
- BGBen Gilbert
-but there have been few that have executed on it. Like, Apple has their own GPUs on the M1s. Tesla's switching... hasn't happened yet, but switching the, to their own, for the full self-driving, they're s- they're doing their own stuff on the car, and they're switching.
- DRDavid Rosenthal
Yep, that is switch- on the inference side-
- BGBen Gilbert
Yes
- DRDavid Rosenthal
... on-device, yes, that has happened. But look, NVIDIA's probably strong in that, but I think the real thing to watch is the data center.
- BGBen Gilbert
And Google is probably the biggest bear case there.
- DRDavid Rosenthal
Yeah.
- BGBen Gilbert
It's interesting to talk about these companies, and particularly Cerebras, 'cause what they're doing is such a gigantic swing and a totally different take than what everyone else has done. For folks who hasn't sort of followed the company, they're making a chip that's the size of a dinner plate. Everyone else's chip is like a thumbnail, but they're making a dinner plate-sized chip. And, you know, the yields on these things kinda suck, so, like, they need all the redundancy on those-
- DRDavid Rosenthal
Oof
- BGBen Gilbert
... huge chips to make it so that-
- DRDavid Rosenthal
Oh, my God, the amount of expense to do that.
- BGBen Gilbert
Right, and you can put one on a wafer.
- DRDavid Rosenthal
Oof.
- BGBen Gilbert
These wafers are crazy expensive to make.
- DRDavid Rosenthal
Wow, so you get poor yields in the wrong places on a wafer, and, like, that whole wafer is toast.
- BGBen Gilbert
Right. So a big part of the design of Cerebras is this sort of redundancy and the ability to turn off different pieces that aren't working. They draw 60 times as much power. They're way more expensive. Like, if NVIDIA is gonna sell you a $20,000 or $30,000 chip, Cerebras is gonna sell you a $2 million chip to do AI training. And so it is this bet in a big way on hyper-specialized hardware for enterprises that wanna do these very specific AI workloads. And it's d- deployed in these beta sites in research labs right now, and, you know, not there yet, but it'll be very interesting to watch if they're able to meaningfully compete for what everyone thinks will be a very large market, these enterprise AI workloads. I mentioned Google, that made a bunch of noise about making their own silicon in the data center and then stayed the course and stayed really serious about it with their TPUs. Their business model is different, so nobody knows what the bill of materials is to create a TPU. Nobody knows really what they cost to run. They don't retail them. They're only available in Google Cloud. And so Google is sort of counter-positioned against NVIDIA here, where they're saying, "We wanna differentiate Google Cloud with this offering, that depending on your workload, it might be much cheaper for you to use TPUs with us than for you to use NVIDIA hardware with us or anyone else." And they're probably willing to eat margin on that in order to grow Google Cloud's share in the cloud market.
- DRDavid Rosenthal
Mm-hmm. Interesting.
- BGBen Gilbert
So it's kind of the Android strategy but run in the data center.
- DRDavid Rosenthal
One thing we haven't mentioned, but we should, is, uh, cloud is also part of the NVIDIA story, too. Like, you can get NVIDIA GPUs in AWS and Azure and, and Google Cloud, and that is part of the growth story for NVIDIA, too. [chuckles]
- BGBen Gilbert
And NVIDIA's starting their own cloud. You can get direct from NVIDIA, cloud-based GPUs.
- DRDavid Rosenthal
Data center GPUs. Interesting.
- BGBen Gilbert
Yeah. It'll be very interesting to see how this all shakes out with, uh, with NVIDIA, the startups, and with Google.
- DRDavid Rosenthal
[exhales] I mean, all that said, though, like, I think... But look, NVIDIA is very, very, very richly [chuckles] valued on a valuation basis right now. [chuckles] Very, with another very in there.
- BGBen Gilbert
It depends if you think their growth will continue. Are they a 60% growing company year over year over year for a while? Then they're not richly valued. But if you think it's a COVID hiccup or a crypto hiccup-
- DRDavid Rosenthal
But to the, the bull bear case and kinda both the startups and the big tech companies doing this stuff in-house, uh, it's not so easy. You know, like, yeah, Facebook and Tesla and Google and Amazon and Apple are capable of doing a lot, but we just told this whole story. This is 15 years of CUDA [chuckles] and the hardware underneath it and the libraries on top of it that NVIDIA has built. To go recreate that and surpass it on your own is such an enormous, enormous bite to bite.
- 1:54:28 – 2:15:21
Wrap-up: playbook lessons, capital efficiency, and closing announcements
- DRDavid Rosenthal
Let's move to playbook. So, man, I have-- I just wrote down in advance one that is [chuckles] such a big one for me, and I'm biased because I, I, I try to think about this in investing, particularly in public markets investing. But, like, man, you really, really want to invest in whoever is selling the picks and the shovels in a gold rush.
- BGBen Gilbert
Hmm.
- DRDavid Rosenthal
The AI, you know, ML, deep learning gold rush, uh, those years, gosh! Oh, my gosh, like, we should all, all be kicking ourselves of twenty twelve, thirteen. Maybe not twenty twelve, but certainly twenty fourteen, twenty fifteen into twenty sixteen. Like, duh. [chuckles] You know, Marc Andreessen saying, "Every startup that comes in here wants to do AI and deep learning, and they're all using Nvidia. Like, maybe we should [chuckles] have bought Nvidia." Like, I don't know if any one of those startups, any given one, is gonna succeed, but I'm pretty sure Nvidia was gonna succeed back then.
- BGBen Gilbert
Yeah, it's such a good point. Kicking myself.... One I have is, uh, being willing to expand your mission. So it's funny how, uh, Jensen, early days, would talk about to enable graphics to be a storytelling medium. And, of course, this led to the invention of the pixel shader and the idea that everybody can sort of tell their own visual story their own way in a social, networked, real-time way. Very cool. And now it's much more that wherever there is a CPU, there is an opportunity to accelerate that CPU, and Nvidia will bring accelerated computing to everyone. And we will make all the best hardware, software, and services solutions to make it so that any computing workload runs in the most efficient way possible through accelerated computing. That's pretty different than enable graphics as a storytelling medium, but also, they need [chuckles] to sell a pretty big story around the TAM that they're going after.
- DRDavid Rosenthal
I think there's also something to, uh, the whole Nvidia story, you know, across the whole arc of the company of, you know, it's sort of a trite, cliché thing at this point in startup land, but so few companies and founders can actually do it: just not dying.
- BGBen Gilbert
Yeah.
- DRDavid Rosenthal
They should have died at least four separate times, and they didn't. And part of that was brilliant strategy, part of that was things going their way, but I think a large part of it, too, was just the company and Jensen, particularly in this, these most recent chapters, where they're already a public company, just being like, "Yeah, I'm willing to just sit here and endure this pain [chuckles] and I have confidence that, like, we will figure it out. The market will come. I'm not gonna declare game over."
- BGBen Gilbert
One that I have is, we mentioned at the top of the show, but the scale of everything involved in machine learning at this point, and anything semiconductors, is kind of unfathomable. You and I mentioned falling down the YouTube rabbit hole with that Asianometry channel, and I was watching a bunch of stuff on how they make the silicon wafers, and my God, floor planning is this just unbelievable exercise at this point in history, uh, especially with the way that they sort of overlay different designs on top of each other on different layers of the, the chip.
- DRDavid Rosenthal
Yeah. Say more about what floor planning is. I bet a lot of listeners won't know.
- BGBen Gilbert
So it's funny how they keep appropriating these sort of real-world, large-scale analogies to chips. So floor planning, the way that an architect would lay out the 15 rooms in a house or five rooms in a house or two rooms in a house, on a chip, is laying out all of the circuitry and wires on the actual chip itself, except, of course, there's, like, ten million rooms, and so it's incredibly complex. And the stat that I was gonna bring up, which was just mind-bending to think about, is that there are dozens of miles of wiring on a GPU.
- DRDavid Rosenthal
Wow, that is mind-bending. 'Cause these things are like, you know, I don't know, they're less than the size of your palm, right?
- BGBen Gilbert
Right! And it obviously is not wiring in the way you think about, like, a wire. I'm gonna reach down and pick up my Ethernet cable, but it's wiring in the EUV-etched substrate on chip... Exposure is probably the term that I'm looking for here, photolithography exposure. But it is just so tiny. I mean, you can say four nanometers all you want, David, but that won't register with me how freaking tiny that is until you're sort of faced with the reality of dozens of miles of, quote, unquote, "wires" on this chip.
- DRDavid Rosenthal
Yeah, it's not like-- To me, that registers as like, "Oh, yeah, that's like a decal I put on my hot rod. Four nanometers!" [chuckles]
- BGBen Gilbert
[chuckles] Yes.
- DRDavid Rosenthal
"I got the S version." [chuckles]
- BGBen Gilbert
[chuckles]
- DRDavid Rosenthal
But yeah, like, that's what that means.
- BGBen Gilbert
Okay, here's one that I had that we actually even talked about, which I think will be fun. So I generated a CapEx graph.
- DRDavid Rosenthal
Ooh, fun!
- BGBen Gilbert
We'll show it on screen here for those watching on video. Obviously, there's a very high-looking line for Amazon 'cause building data centers and fulfillment centers is very expensive, especially in the last couple of years when they're doing this massive build-out. But imagine without that line for a minute. Nvidia only has a billion dollars of CapEx per year.
- DRDavid Rosenthal
Hmm, and this is relative, for people listening on audio, relative to a bunch of other, you know, FAANG-type companies?
- BGBen Gilbert
Yeah. So Apple has ten billion dollars of spend on capital expenditures per year. Microsoft and Google have twenty-five billion. TSMC, who makes the chips, has thirty billion. What a great capital-efficient business that Nvidia has on their hands, only spending a billion dollars a year in CapEx. It's like it's a software business, and it basically is.
- DRDavid Rosenthal
Well, it is, right? Like, TSMC does the fabbing, Nvidia makes software and IP.
- BGBen Gilbert
Yep. So here, this is the best graph for you to very clearly see the magic of the fabless business model that Morris Chang was so gracious to invent when he grew TSMC.
- DRDavid Rosenthal
Thank you, Morris.
- BGBen Gilbert
Another one that I wanted to point out: it's a freaking hardware company-- I know we didn't... They're not a hardware company, but they're a hardware company with thirty-seven percent operating margins, so this is even better than Apple. And for non-finance folks, operating margins... So we talked about their sixty-six percent gross margin, that's like unit economics, but that doesn't account for all the headcount and the leases and just all the fixed costs in running the business. Even after you subtract all that out, thirty-seven percent of every dollar that comes in gets to be kept by Nvidia shareholders. It's a really, really, really cash-generative business, and so if they can continue to scale and can keep these operating margins or even improve them, 'cause they think they can improve them, that's really impressive.
- DRDavid Rosenthal
Wow, I didn't realize that's better than Apple's.
- BGBen Gilbert
Yeah. I think it's not as good as, like, Facebook and Google because they just run these, like-
- DRDavid Rosenthal
Well, those are digital monopolies. Like, come on. [chuckles]
- BGBen Gilbert
... Basically zero-cost [chuckles] digital monopolies in some of the largest markets in history, but, uh, it's still very good. All right, well, let's do grading. And before we actually grade, we wanna tell you about another one of our friends. For our final sponsor-... Let's talk about the SoftBank Latin America Fund. So you know this by now, these folks created the fund with a simple thesis: the region of Latin America was overflowing with innovative founders and great opportunities, but short on the ingredient of capital. SoftBank has invested eight billion dollars in seventy plus companies, and they have one gigantic takeaway. And I, I can't say this enough, because I think it's like you can keep hearing it, but I think the important thing is sort of like internalizing it, that technology in Latin America is not about disruption, it's about inclusion. So when you're thinking about economic opportunities in this region, you don't have to think like, "Ooh, how can we overthrow the incumbent?" It really is like if you're used to living your life or doing business in North America in a lot of the like, ways that you feel are, quote, unquote, "modern," a lot of these business models and a lot of this technology just has not happened yet to serve the vast populations in Latin America. You sort of have a case study in some businesses that have worked, and now you get to go and bring it to the masses. So just amazing, uh, opportunity for inclusion here. The vast majority of the population is underserved by every category, from banking to transportation to e-commerce. Businesses are not served by modern software solutions, as I was saying, and we want to highlight a great portfolio company, VTEX. Now, this is a crazy story. Speaking of high-growth companies recently, they saw ninety-eight percent growth during the pandemic as companies looked to VTEX for their digital commerce, native marketplace, and order management capabilities. Today, VTEX powers over three thousand online storefronts for global brands like Walmart, Coca-Cola, Nestle, and as we mentioned on our Sony episode, Sony. They were recently named the world's fastest growing e-commerce platform, and they are just one example of how SoftBank is partnering with great founders and bringing them the capital and expertise they need to bring the future and build it in Latin America now. To learn more, you can click the link in the show notes or go to latinamericafund.com.
Episode duration: 2:15:21
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode xU_rLZqlca4
