EVERY SPOKEN WORD
70 min read · 13,995 words- 0:00 – 1:22
Introduction
- SGSarah Guo
(instrumental music plays) Hi, listeners, and welcome to No Priors. Today, we're here again, one year since our last discussion with the one and only Jensen Huang, founder and CEO of NVIDIA. Today, NVIDIA's market cap is over three trillion dollars, and it's the one literally holding all the chips in the AI revolution. We're excited to hang out in NVIDIA's headquarters and talk all things frontier models and data center-scale computing, and the bets NVIDIA is taking on a 10-year basis. Welcome back, Jensen. 30 years in to NVIDIA and looking 10 years out, what are the big bets you think are, are still to make? Is it all about scale up from here? Are we running into limitations in terms of how we can squeeze more compute memory out of the architectures we have? What are you focused on?
- JHJensen Huang
Well, if we take a step back and, and think about what we've done, we went from coding to machine learning, from writing software tools to creating AIs, and all of that running on CPUs that was designed for human coding to now running on GPUs designed for, um, AI coding basically. Machine learning. And so th- the world has changed. Th- the way we do computing, the whole stack has changed. And as a result, the scale of the problems we could address has changed a lot
- 1:22 – 2:28
NVIDIA's 10-year bets
- JHJensen Huang
because we could... If you could parallelize your software on one GPU, you've set the foundations to parallelize across a whole cluster or maybe across multiple clusters or multiple data centers. And so I think we, we've set ourselves up to be able to scale computing, uh, at a level and develop software at a level that nobody's ever imagined before. And so we're at the beginning of that. Um, uh, over the next 10 years, uh, our hope is that we could double or triple performance every year at, at scale. Not at chip, at scale. And to be able to therefore drive the cost down by a factor of two or three, drive the energy down by a factor of two or three every single year. When you do that every single year, when you double or triple every year, in just a few years, it adds up. (laughs) And so it compounds really, really aggressively. And so I wouldn't be surprised if, you know, the way people think about Moore's Law, which is, uh, uh, 2X every couple of years, um, you know, we're gonna be on some kind of a hyper Moore's Law curve, and, um, I, I fully
- 2:28 – 3:42
Outpacing Moore’s Law
- JHJensen Huang
hope that we continue to do that.
- EGElad Gil
What, what do you think is the driver of making that happen even faster than Moore's Law?
- JHJensen Huang
Well-
- EGElad Gil
'Cause I know Moore's Law was sort of self-reflexive, right? It was something that he said and then they, people kind of implemented it to make it happen.
- JHJensen Huang
Yeah, yeah. The two fundamental, um, technical pillars, one of them was Dennard scaling and the other one was Carver Mead's VLSI scaling, and both of those techniques were rigorous techniques, um, but, uh, those, those techniques have really run out of steam, and, and, uh, so now we need a new way of doing scaling. Uh, you know, obviously the new way of doing scaling are, are all kinds of things associated with co-design. Unless you can modify or change the algorithm to reflect the architecture of the system, or change, and then change the system to reflect the architecture of the new software and go back and forth-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... unless you can con- control both sides of it, you have no hope. But if you can control both sides of it, you can do things like move from FP64 to FP32 to BF16 to-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... FP8 to, you know, FP4 to who knows what, right?
- EGElad Gil
Mm-hmm.
- JHJensen Huang
And so, and so I think that, that co-design is a very big part of that. The second part of it, we call it full stack. The second
- 3:42 – 7:16
Data centers and NVLink
- JHJensen Huang
part of it is, uh, data center scale. You know, unless you could treat the network as a compute fabric and, and, uh, push a lot of the work into the network-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... push a lot of the, the work into the fabric, and as a result, you, you're compressing, you know, doing compressing at very large scales.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
And so that, that, that's the reason why we bought Mellanox and started fusing InfiniBand and, and VLink-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... um, in such an aggressive way. And now look where MVLink is gonna go. You know, the, the compute fabric is going to, going to, um, uh, uh, scale out, uh, what appears to be one incredible processor called a GPU, and now we're getting hundreds of GPUs that are gonna be working together. You know, most, most of these, these computing challenges that we're dealing with now, one of the, the, the most exciting ones, of course, is, is, uh, inference time scaling. It has to do with essentially, uh, generating tokens at incredibly low latency.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
Because you're self-reflecting as you, as you just mentioned. I mean, you're gonna be, you're gonna be, uh, uh, doing tree search, you're, you're gonna be doing chain of thought, you're gonna be doing probably some amount of simulation in your head. You're gonna be reflecting on your own answers. While, you're gonna be prompting yourself and generating text to your, in-, you know, silently, um, and still respond, hopefully in a second. Well, uh, the only way to do that is if your latency low- your latency is extremely low. Meanwhile, the data center is still about producing high throughput tokens because, you know, you still wanna keep the cost down, you wanna keep the throughput high, you want it, right, you know-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... generate a return. And so these two fundamental things about a factory, low latency and high throughput, th- they're at odds with each other. And so, so in order for us to create something that is really great at in- in- in both, um, we have to go invent something new, and MVLink is really our way of doing that. We... Now you have, now you have a virtual GPU that has incredible amount of f- flops because you need it for context, you need a huge amount of memory, working memory, and still have incredible bandwidth for token generation all at the same time. That's the, one of the big ideas.
- EGElad Gil
And I guess in parallel you also have all the people building the models actually also optimizing things pretty dramatically. Like, uh, David on my team pulled data where over the last 18 months or so, the cost of, um, a million tokens going into a GPT-4-equivalent model has basically dropped 240X.
- JHJensen Huang
Yeah.
- EGElad Gil
And so there's just massive...... uh, optimization and compression happening on that side as well.
- JHJensen Huang
Just in our layer, just on the layer that we work on. You know, one of the things that, that, that we care a lot about, of course, is the ecosystem of our stack and the productivity of our software. You know, people forget that, that because you have CUDA foundation, and that's a solid foundation, everything above it can change.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
If everything... If, if the foundation's changing underneath you, it's hard to build a building on top, it's hard to create anything in- interesting on top. And so, so CUDA made it possible for us to iterate so quickly. Just in the last year, I think we just went back and benchmarked, uh, when LLaMA first came out. We've improved the performance of Hopper by a factor of five-
- EGElad Gil
Hmm.
- JHJensen Huang
... without the algorithm, without the layer on top-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... ever changing. Now, well, a factor of five in one year is impossible using traditional computing approaches. But accelerated computing and using this way of, of, um, uh, code, code, co-design, uh, we're able to invent all kinds of new
- 7:16 – 10:40
Infrastructure flexibility for large-scale training and inference
- JHJensen Huang
things, yeah.
- SGSarah Guo
How much, uh, are, um, you know, your biggest customers thinking about the, uh, interchangeability of their infrastructure between large-scale training and, uh, inference?
- JHJensen Huang
Well, you know, infrastructure is disaggregated these days. Sam was just telling me that he, he had decommissioned Volta just recently. They have Pascals, they have Ampers, all different configurations of Blackwall coming. Some of it is optimized for air cool, some of it's optimized for liquid cool. Your services are gonna have to take advantage of all of this. The advantage that NVIDIA has, of course, is that the, the infrastructure that you built today for training, um, will just be wonderful for inference tomorrow. And most of ChatGPT, I believe, are inferenced on the same type of systems that were trained on just recently. And so if you can train on it, you can inference on it. And so you're leaving, you're leaving a trail of, of infrastructure that you know is gonna be incredibly good at inference, and, and you have complete confidence that you can then take that return on, on the investment that you've had and put it into a new infrastructure to go scale, scale with. You know you're gonna leave behind something of use. And you know that, that NVIDIA and the rest of the ecosystem are gonna be working on improving the algorithm so that the rest of your infrastructure improves by a factor of five, you know, in just a year. And so that, that motion will never, never change. And so the way that, the way that people will think about the infrastructures, yeah, even though I built it for, for training today, it's gotta be great for training, we know it's gonna be great for inference. Um, inference is gonna be multi-scale. I mean, you're gonna take... First of all, in order to distill a smaller model, it's good to have a larger model to distill from. And so, (laughs) so you're still gonna create these incredible frontier models. They're gonna be used for, of course, the, the groundbreaking work, you're gonna use it for synthetic data generation, you're gonna use the models, the big models to teach smaller models and distill down to smaller models. And so there's a, there's a whole bunch of, uh, different things you could do. But in the end, you're gonna have giant models all the way down to little tiny models. The little tiny models are gonna be quite effective. You know, not as generalizable, but quite effective. And so, you know, they're gonna perform very specific stunts incredibly well, that one task. And we're gonna s- we're gonna see superhuman task in one, one little tiny domain from a little tiny, tin- tiny model, maybe, you know, it's not a small language model, but, you know, tiny language model, TLMs, or, you know-
- EGElad Gil
Yeah.
- JHJensen Huang
... whatever. Yeah, so, so I think we're gonna see all kinds of sizes, and we hope. Is that right? Just kind of like software is today.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
Y- yeah, I think in a lot of ways, artificial intelligence allows us to break new ground in, in how easy it is to create new applications. But everything about computing has largely remained the same. For example, uh, the cost of maintaining software is extremely expensive. And once you build it, you would like it to run on a large of an install base as possible. You would like not to write the same software twice. I mean, a- you know, a lot of people still feel the same way, and you like to take your engineering and move 'em forward. And, and so to the extent that, to the extent that, that the architecture allows you to, on one hand, um, create software today that runs even better tomorrow with new hardware, that's great. Or software that you create tomorrow, AI that you create tomorrow runs on a large install base, you think that that's great. That, that way of thinking about software is not gonna change.
- 10:40 – 13:30
Building and optimizing data centers
- JHJensen Huang
- SGSarah Guo
NVIDIA has moved into larger and larger, let's say, like unit of support for customers.
- JHJensen Huang
Mm-hmm.
- SGSarah Guo
I think about it going from single chip to, you know-
- JHJensen Huang
Yeah.
- SGSarah Guo
... server to rack, NVL 72.
- JHJensen Huang
Yeah.
- SGSarah Guo
How do you think about that progression? Like, what, what's next?
- JHJensen Huang
Uh-huh.
- SGSarah Guo
Like, should NVIDIA do-
- JHJensen Huang
That's great.
- SGSarah Guo
... full data center?
- JHJensen Huang
Uh, in fact, we build full data centers. The way that we build everything, unless you're building... If you're developing software, you need the computer in its full manifestation. Um, we don't, we don't build PowerPoint slides and ship the chips. And we build a whole data center. And until we get the whole data center built up, how do you know the software works until you get the whole data center built up? How do you know your, you know, your fabric works, and all the things that you expect it, the efficiencies to be? How do you know it's gonna really work at scale? And, and that's the reason why, that's the reason why it's not unusual to see somebody's actual performance be dramatically lower than their peak performance as shown in PowerPoint slides.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
And, and, and, and it's... Computing is just not used to... It's not what it used to be. You know, I say that the new unit of computing is the data center. That's, to us-
- SGSarah Guo
So that's what you have to deliver.
- JHJensen Huang
That's what we build.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Now, we build the whole thing like that, and then we... For every single thing that, every combination, uh, air cooled, x86, liquid cooled, grace, ethernet, InfiniBand, NVLink, no NVLink, you know what I'm saying? We build every single configuration. We have five super computers in our company today. Next year, we're gonna build easily five more. So if you're serious about software, you build your own computers.... if you're serious about software, then you're gonna build your whole computer, and we build it all at scale. This is the part that, that is really interesting. We build it at scale and we build it, uh, ver- vertically integrated. We optimize it, um, full stack, end to end. And then we disaggregate everything and we sell it in parts. That's the part that is completely, utterly remarkable about what we do.
- EGElad Gil
Mm-hmm.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
The complexity of, of that is just insane. And the reason for that is we wanna be able to graft our infrastructure into GCP, AWS, Azure, OCI. All of their control planes, security planes are all different, and all of the way they think about their cluster sizing, all different. And, um, uh, but yet we make it possible for them to all accommodate NVIDIA's architecture, so that CUDA could be everywhere. That's really, really in the end, the, the singular thought, you know, that we would like to have a computing platform that developers could use that's largely consistent, modular, you know, 10% here and there because people's infrastructure are slightly optimized differently, and modular 10% here and there, but, but everything they, they build will run everywhere. This is kinda the, one of the principles of software that should never be gi- given up, and it... and, and we,
- 13:30 – 15:00
Maintaining software and architecture compatibility
- JHJensen Huang
we, we protect it quite dearly. Uh, it makes it possible for our software engineers to build once, run everywhere. And, and that's because we recognize, uh, that the investment of software is the most expensive investment, and it's easy to test. Uh, look at the size of the whole hardware industry, and then look at the size of the world's industries. There's $100 trillion on top of this $1 trillion industry, and that tells you something. The software that you build, you have to... you know, you basically maintain for as long as you shall live. We've never given up on a piece of software. The reason why CUDA is used is because, you know, I told everybody, "We will, we will... we will maintain this for as long as we shall live," and we're serious. And we still maintain... uh, (laughs) I just saw a review the other day, uh, NVIDIA SHIELD, our Android TV. It's the best Android TV in the world. We shipped it seven years ago. It is still the number one Android TV that, that people... you know, anybody who, who enjoys TV. Uh, and we just updated the software just this last week, and people wrote a new story about it. GeForce, we have 300 million gamers around the world. We've never l- stranded a single one of them. And so, the fact that our architecture is, uh, compatible across all of these different areas makes it possible for us to do it. Otherwise-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... we would be su- we would be... we would have, uh... you know, we would have software teams that are 100 times the size of our company is today, if not for this architectural compatibility. So, we're very serious about that, and that translates to benefits to cu- you know, the developers.
- EGElad Gil
One impressive substantiation of that recently was how quickly you brought up a cluster for
- 15:00 – 18:55
X.AI’s supercluster
- EGElad Gil
x.ai.
- JHJensen Huang
Yeah.
- EGElad Gil
And if you wanna talk about that, 'cause tha- that was striking in terms of both the scale and the speed with which you did that.
- JHJensen Huang
You know, uh, a lot of that credit, you gotta give to Elon. I think the, um... uh, f- first of all, to, uh, decide to do something, select the site, um... uh, bring cooling to it, uh, power-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... and then, and then, uh, decide to build this 100,000 GPU supercluster, which is, you know, the largest of its kind in, in one unit. Um, and then working backwards, you know, uh, uh, we started planning together, uh, the date that he was gonna stand everything up. And the date that he was gonna stand everything up was determined, um, you know, quite... a few months ago.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
And so all of the components, all the OEMs, all the systems, all the software integration we did with their team, all the network simulation... We simulate all th- all the... all the networ- network configurations.
- EGElad Gil
That makes sense. Yeah.
- JHJensen Huang
We, we pre... I mean, it's not like... we pre-staged everything as a digital twin.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
We, we pre-st... uh, we, we, uh, pre-staged all of his supply chain. Uh, we pre-staged all of the wiring of the networking. We even, we even set up a small version of it, uh, kind of a... you know, just a first instance of it, um... uh, you know, ground truth, if you... reference zero-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... you know, system zero, uh, uh, before everything else showed up. So by the time that everything showed up, everything was staged, uh, all the practicing was done, all the simulations were done, and then, you know, the massive integration. Even then, the massive integration was a... was a monument of, you know-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... gargantuan teams of humanity, (laughs) you know-
- EGElad Gil
Yeah.
- JHJensen Huang
... crawling over each other, wiring everything up 24/7. And, and within a few weeks, uh, the clusters were up. I mean, it's, it's really-
- EGElad Gil
Insane.
- JHJensen Huang
Yeah. It's really a testament to, to, uh, his willpower and, and, um, uh, how he's able to think through mechanical things, electrical things, and, and overcome what is apparently, you know, extraordinary obstacles. I mean, what was done there is the first time that a com- a computer of that large scale has ever been done at that speed.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
Unless our two teams are working from the networking team, the compute team, the software team, the training team, the... you know, and the infrastructure team, the people that... th- the electrical engineers to the... you know, to the software engineers all working together, yeah, it's really quite a f- feat to watch.
- SGSarah Guo
Was there a challenge that felt most, um, likely to be blocking from an engineering perspective?
- JHJensen Huang
Just the tonnage of electronics that had to come together. I mean, it'd probably be worth just to measure it. I mean, it's, uh... you know, it, it, it... tons and tons of equipment, it's just abnormal.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
You know, usually, usually a supercomputer system like that, um, you plan it for a couple of years, uh, from the moment that the first systems come on... come delivered to the time that you probably submitted everything for some serious work. Don't be surprised if it's a year.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
You know? I mean, uh, th- that happens all the time. It's not abnormal. Now, we, we couldn't afford to do that. So we, we created... you know, uh, uh, a few years ago, uh, there was an initiative in our company that's called data center as a product. We don't sell it as a product, but we have to treat it like it's a product. Everything about...... planning for it, and then standing it up, optimizing it, tuning it, keep it operational. The goal is that it should be, you know, kind of like opening up your beautiful new iPhone, and you open it up and everything just kind of works. Now, of course, it's a miracle of technology making it that, like that, but we now have the skills to do that. And so, if you're interested in a data center, you just have to give me a space and some power, some cooling, you know, and, uh, we'll, we'll help you set it up within, call it, 30 days. I mean, it's-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... pretty extraordinary.
- 18:55 – 20:39
Challenges of super scaling data centers
- SGSarah Guo
to 200,000, 500,000, a million, um, in a supercluster, or whatever you call it, at that point-
- JHJensen Huang
Mm-hmm.
- SGSarah Guo
... um, what do you think is the biggest blocker? Capital? Energy? Supply in one area?
- JHJensen Huang
Everything. Nothing about what you just, the scales that you talked about, the, nothing is normal.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Uh, yeah.
- SGSarah Guo
But nothing is impossible.
- JHJensen Huang
Nothing is, yeah, n- no laws of physics limits. Um, but everything is gonna be hard. And, and of course, you know, i- is it worth it? Um, like you can't believe. You know, to, to get to something that we would recognize as, as a computer that, that, um, uh, so easily and so able to do what we ask it to do, what, you know, otherwise, uh, general intelligence of some kind. Uh, a- and even, you know, even, even if we could argue about, is it really general intelligence? Just getting close to it is going to be a miracle. We know that. And so I think the, there, there are five or six endeavors to try to get there, right? I think, um, of course, OpenAI and, and, uh, Anthropic and X and, uh, you know, of course, Google and Meta and, uh, Microsoft and, um, you know, they're, they're, they're ... this, this frontier, the next couple of clicks up that mountain are just so vital.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Uh, who doesn't want to be the first on that, on that, on that mountain? I think the, the, uh, the prize, uh, for reinventing, uh, intelligence altogether, uh, is just, it's, it's too consequential not to attempt it. And so I think the, there are no laws of physics. Everything is gonna be hard.
- SGSarah Guo
A year ago, uh, when we spoke together, you
- 20:39 – 22:23
AI’s role in chip design
- SGSarah Guo
talked about, w- we asked, like, what applications you got most excited about that NVIDIA would serve next in AI r- and otherwise. And you talked about how you let your most extreme customers sort of lead you there.
- JHJensen Huang
Yeah.
- SGSarah Guo
Um, and, and about some of the scientific applications. I think that's become, like, much more, uh, mainstream of you over the last year. Uh, is it still, like, science and AI's application of science that most excites you?
- JHJensen Huang
I love the fact that we have digital ... we have AI chip designers.
- SGSarah Guo
Here at NVIDIA?
- JHJensen Huang
Yeah. I, I love that we have AI software engineers.
- SGSarah Guo
Mm-hmm. How effective are AI chip designers today?
- JHJensen Huang
Super good. We can't, we couldn't have built, we couldn't have built Hopper without it. And the reason for that is because they could explore a much larger space than we can. And, uh, because they have infinite time.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
They're running on a supercomputer. Uh, we have so little time using, using human engineers that, that, um, we don't explore as much of the space as we should, and we also can't explore it combinatorially. I can't explore my space while including your exploration and your exploration and ... And so, you know, our chips are so large, it's not like it's designed as one chip. It's designed almost like 1,000 chips.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
And we have to ex- we have to optimize each one of them kind of in isolation. You really wanna optimize a lot of them together and, and, um, you know, cross-module co-design and, and optimize across a much larger space. Now, obviously we're gonna be able to find, find, you know, local maximums that are hidden behind local minimums somewhere and so, so clearly we can find better answers. Um, y- you can't do that without AI engineers. Just simply can't do it. We just don't have enough time.
- EGElad Gil
One other thing that's changed, um, since we last spoke,
- 22:23 – 27:03
NVIDIA's market cap surge and company evolution
- EGElad Gil
uh, collectively, and I, I looked it up. Um, at the time NVIDIA's market cap was about 500 billion. It's now over three trillion. So over the last 18 months, you've added two and a half trillion plus of market cap, which effectively is $100 billion plus a month, or two and a half Snowflakes or, you know, a Stripe plus a little bit or however you wanna (laughs) think about it.
- SGSarah Guo
A country or two.
- EGElad Gil
A country or two. Um, obviously a lot of things have stayed consistent in terms of focus on what you're building and-
- JHJensen Huang
Mm-hmm.
- EGElad Gil
... et cetera, and, you know, walking through here earlier today, I felt the buzz like when I was at Google 15 years ago.
- JHJensen Huang
Mm-hmm.
- EGElad Gil
It was kind of, you felt the energy of the company and the vibe-
- JHJensen Huang
Mm-hmm.
- EGElad Gil
... of excitement. What has changed during that period, if anything? Or how, what, what is different in terms of either how NVIDIA functions or how you think about the world or the size of bets you can take or ...
- JHJensen Huang
Mm-hmm. Well, our company can't change as fast as the stock price. Let's just be clear about that. (laughs)
- SGSarah Guo
Yeah. (laughs)
- JHJensen Huang
(laughs) And so in a lot of ways, we haven't changed that much. I think the, um, the thing to do is to take a step back and ask ourselves what, what, what are we doing?
- EGElad Gil
Mm-hmm.
- JHJensen Huang
I think that that's really the big, you know, the big observation, realization, awakening for, uh, companies and countries, is what's actually happening.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
I think what we were talking about earlier, um, from our industry perspective, we've reinvented computing. Now, it hasn't been reinvented for 60 years. That's how big of a deal it is.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
That we've driven down the, the marginal cost of computing down probably by a million X in the last 10 years to the point that we just, "Hey, let's just let the computer go exhaustively write the software." That's the big realization.
- EGElad Gil
Mm-hmm.
- JHJensen Huang
And that, that, in a lot of ways, I was kinda s- we were kinda saying the same thing about chip design. We would love for the computer to go discover something about our chips that we otherwise couldn't have done ourselves, explore our chips and optimize it in a way that we couldn't do ourselves. Right? In, in the way that we would love for digital biology or-
- EGElad Gil
Mm-hmm.
- JHJensen Huang
... you know, any other, any other field of science. And so I, I think people are starting to realize, one, we reinvented, reinvented, uh, computing. But what does that mean even?And as we, all of a sudden, we created this thing called intelligence, and- and what happened to computing? Well, we went from data centers, data centers are multi-tenant, stores our files. These new data centers we're creating are not data centers. They don't, they're not multi-tenant. They tend to be single tenant. They're not storing any of our files. They're just, they're producing something. They're producing tokens. And these tokens are re- reconstituted into what appears to be intelligence, isn't that right? Mm-hmm. And intelligence of all different kinds. You know, it could be articulation of robotic motion, it could be, um, uh, sequences of- of amino acids, it could be, you know, chemical chains. It could be all kinds of interesting things, right? So what are we really doing? We've created a new instrument, a new machinery, that- that in a lot of ways is the noun of the adjective generative AI. You know? Instead of generative AI, now it's, it's an AI factory. Mm-hmm. It's a factory that generates AI. And we're doing that at extremely large scale. And- and what people are starting to realize is, you know, maybe this is a new industry. It generates tokens, it generates numbers, but these numbers constitute in a way- Mm-hmm. ... that is fairly valuable. And- and, um, what industry would benefit from it? Then you take a step back and you ask yourself again, you know, what's going on NVIDIA? On the one hand, we reinvented computing as we know it, and so there's a trillion dollars worth of infrastructure that needs to be modernized. Mm-hmm. That's just one layer of it. The big layer of it is that there's this instrument that, that we're building is not just for data centers, which we sh- we're modernizing, but you're using it for producing some new commodity. And how big can this new commodity industry be? Hard to say. Mm-hmm. But it's probably worth trillions. And so that I think is, is kind of the, if you were s- to take a step back, you know, we, we don't build computers anymore. We build factories. Mm-hmm. And every country's gonna need it. Every company's gonna need it. You know, give me an example of a company who, or industry has says, "You know what? We don't need to produce intelligence. We got plenty of it." Mm-hmm. And so, (laughs) so that's the big idea, I think, you know, and- and that's kind of an abstracted industrial view. And, you know, someday, someday people will realize that in- in a lot of ways, the- the semiconductor industry wasn't about building chips. It was build- it was about building the- the foundational fabric for society, and then all of a sudden everybody goes, "Oh, I get it." You know, "This is a big deal. This is not just about chips." Mm-hmm.
- SGSarah Guo
How do you think about embodiment now?
- JHJensen Huang
Mm.
- 27:03 – 28:33
Embodied AI
- JHJensen Huang
Well, the thing I- I, uh, I'm super excited about is in a lot of ways, we've, we're close to artificial general intelligence, but we're also close to artificial general robotics. Tokens are tokens. I mean, the question is, can you tokenize it? You know, of course token is, tokenizing things is not easy, uh, as you guys know. But if you were able to tokenize things, um, align it with large l- large language models and other modalities, if I can generate a video that has Jensen reaching out to pick up the coffee cup, why can't I prompt a robot to generate the tokens to pick up the ro- it, you know? Mm-hmm. And so intuitively you would think that the problem statement is rather similar for a computer. And- and so I, I think that we're that close. That's incredibly exciting. Now, the- the two, the two brownfield, uh, robotic systems, brownfield meaning that you don't have to change the environment for, is, uh, self-driving cars and- and, um, with digital chauffeurs and embodied robots, right? Between the cars and the human robot, uh, we- we could literally, um, bring robotics to the world without changing the world because we built the world for those two things. Mm-hmm. It's probably not a coincidence that- that Elon's focused on those two forms of robotics because, uh, it is likely to have the larger potential scale. And- and so I- I think that- that's exciting. But the digital version of it i- is equally
- 28:33 – 31:25
AI employees
- JHJensen Huang
exciting. You know, when we're talking about digital or AI employees, there's no question we're gonna have AI employees of all kinds. And our outlook will be some biologics and some artificial intelligence, and, uh, we will prompt them in the same way, isn't that right? Yeah. Mostly I prompt my employees, I, you know, provide them context, um, ask them, uh, to perform a mission. They go and, uh, recruit other team members, uh, they come back and- Mm-hmm. ... and we're going back and forth. Uh, how is that gonna be any different with digital and AI employees of all kinds? So we're gonna have AI marketing people, AI chip designers, AI supply chain people, AI, you know, and- and I'm- I'm hoping that NVIDIA is someday, um, uh, biologically bigger, um, but also, uh, from an artificial intelligence perspective, much, much bigger. Mm-hmm. That's our future company.
- SGSarah Guo
If we came back and talked to you a year from now, what part of the company do you think would be, um, most artificially intelligent?
- JHJensen Huang
I'm hoping it's chip design.
- SGSarah Guo
Okay.
- JHJensen Huang
And Most important part. (laughs) And the r- that's right, because it-
- SGSarah Guo
Yeah.
- JHJensen Huang
... because I should start, I should start where it moves the needle most. Also, where we can make the biggest impact most. You know, it's such an ins- insanely hard problem. I work with, uh, Susheen at- at Synopsis and Ruud at- at Cadence. Um, I totally imagine them having Synopsis chip designers that I can rent. And they- they know something about a particular module, their- their- their tool, and- and, uh, they trained an AI th- to be incredibly good at it. And we'll just hire a whole bunch of them whenever we need, we're in that phase of that chip design, you know, I might- might rent a million Synopsis engineers to come and help me out. And then go rent a million Cadence engineers to help me out. And that, what an- what an exciting future for them, that they have all these agents that- that sit on top of their tools platform, that use the tools platform in other, and collaborate with- with other platforms. And you'll do that for, you know, Christian will do that at SAP and Bill will do that at ServiceNow. You know, people- people say that these SaaS platforms are gonna be disruptive. I- I actually think the opposite.... that they're sitting on a goldmine. That, uh, that they're gonna be, uh, this flourishing of agents that are gonna be specialized in Salesforce, specialized in, you know, well, Sp- Salesforce, I think they call Lightning, and, uh, SAP has a BAAP and everybody's got their own language. Is that right? And we've got CUDA, and we've got OpenUSD for Omniverse and, and who's gonna create an AI agent that's awesome at OpenUSD? We are. You know? Because nobody cares about it more than we do. Right? And so, so I, I think in a lot of ways, these platforms are gonna be flourishing with agents, and we're gonna introduce them to each other, and they're gonna collaborate-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... and solve problems.
- SGSarah Guo
You see a wealth of different people working in every domain in AI. What do you think is, um, under-noticed or that people ... that
- 31:25 – 35:40
Impact of AI on science and engineering
- SGSarah Guo
you want more entrepreneurs or engineers or business people to go work on?
- JHJensen Huang
Well, first of all, I think what, what is misunderstood and, and, and, um, uh, misunderstood, maybe, maybe underestimated, is the, the, um, under the, under-the-water activity, under-the-surface activity of, uh, groundbreaking science, computer science, to science-
- SGSarah Guo
Hmm.
- JHJensen Huang
... and engineering that is being affected by AI and machine learning. I think y- you just can't walk into a science department anywhere, theoretical math department-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... anywhere, where AI and machine learning and the type of work that we're talking about today is gonna transform tomorrow. If they are ... if, if you take all of the engineers in the world, all of the scientists in the world, and you say that th- the way they're working today is early indication of the future, because obviously it is-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... then you are gonna see a, a tidal wave of generative AI, a tidal wave of AI, a tidal wave of machine learning, change everything that we do in some short period of time. Now remember, uh, I, I saw the early indications of, of computer vision and, and the work with, with, um, uh, Alex and Ilya and, and Hinton at, at, at, uh, in Toronto and, um, uh, uh, Yann LeCun and, and of course Andrew Ng here in Stanford and, you know, I saw the early indications of it, um, and we were, we were, we were fortunate to have extrapolated from what was observed to be detecting cats-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... into a profound change in computer science, in computing altogether. That extrapolation was fortunate for us. And now of course, we, we were, we were, uh, so excited by, so inspired by it that we changed everything about how we did things. But that took how long? It took, uh, literally six years from observing that toy, AlexNet, which I think by today's standards would be con- considered a toy-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... to superhuman levels of capabilities in, in object recognition. Well, that was only a few years. Uh, what is happening right now, the groundswell in all of the fields of science, not one field of science left behind. I mean, just to be-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... very clear.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Okay? (laughs) Everything from quantum computing to quantum chemistry, you know, every field of science i- is involved in, in the, the approaches that we're talking about. If we give ourselves ... And they've been at it for a couple, two, three years. If we give ourselves another co- couple, two, three years, the world's gonna change. There's not gonna be one paper, there's not gonna be one breakthrough in science, one breakthrough in engineering where generative AI isn't at the foundation of it. I'm fairly certain of it now.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
And so I, I think, I think, um, uh, you know, there's a lot of questions about ... You know, I, every so often I hear about wh- whether this is a fad.
- SGSarah Guo
(laughs)
- JHJensen Huang
Um, uh, computer ... Y- you just gotta go back to first principles and observe what is actually happening. The computing stack, the way we do computing has changed. If the way you write software has changed, I mean, that is pretty core.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Software is how humans encode knowledge. This is how we encode our, you know, our algorithms. We encode it in a very different way now. That's gonna affect everything. Nothing else will ever be the same. And so I, I think the, the, uh, uh, I think I'm, I'm talking to the converted here, and, and we all see the same thing in all the startups that, that, you know, you guys, you guys work with, and the scientists I work with, and the engineers I work with. Nothing will be left behind. I mean this, we're gonna take everybody with us.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Yeah.
- SGSarah Guo
I think one of the most exciting things coming from, like, the computer science world and looking at all these other fields of science is, uh, like I can go to a robotics conference now-
- JHJensen Huang
Yeah.
- SGSarah Guo
... a material science conference-
- JHJensen Huang
Oh yeah.
- SGSarah Guo
... a biotech conference, and like, I'm like, "Oh, I understand this."
- JHJensen Huang
Yeah.
- 35:40 – 36:48
Jensen’s personal use of AI tools
- JHJensen Huang
I'm using it myself every day, you know? I don't know about you guys, but it's my tutor now. I mean, I, I, I don't do ... I, I don't learn anything without first going to an AI.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
You know? Why learn the hard way?
- SGSarah Guo
(laughs)
- JHJensen Huang
(laughs) Just, just go directly to an AI.
- SGSarah Guo
Good success, yeah.
- JHJensen Huang
I just go directly to ChatGPT or, you know, sometimes I do Perplexity just depending on just the, the formulation of my questions, and I just start learning from there, and then you can always fork off and go deeper if you like. Um, but, but holy cow, it's just incredible. And, and almost everything I know, I ch- I double-check.
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
Even though I know it to be a fact, you know, what I consider to be ground truth-
- SGSarah Guo
Mm-hmm.
- JHJensen Huang
... I'm the expert. I'll still go to AI and check, make ... double-check. (laughs)
- SGSarah Guo
Yeah.
- JHJensen Huang
Yeah, it's so great. Uh, almost everything I do, I involve it. Yeah.
- SGSarah Guo
I think it's a great note to stop on. Yeah, thanks so much for your time today.
- JHJensen Huang
Yeah, really enjoyed it. Nice to see you guys.
- SGSarah Guo
Thanks, Jensen. (instrumental music plays) Find us on Twitter @nopriorspod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.
Episode duration: 36:48
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode hw7EnjC68Fw
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome