The Twenty Minute VCSatya Nadella: Whether to Trust Frontier Models | Why Companies Need to Own Their Own Intelligence
EVERY SPOKEN WORD
25 min read · 4,657 words- 0:00 – 1:23
Intro
- SNSatya Nadella
And Dario and Ilya and Sam and Greg and Elon were all, I guess, one happy family. They were really the believers in scaling laws, and that's what I believed in back in 2018, 2019. I think the biggest change now I have is I used to always think that alignment will just fall out. I no longer believe that.
- HSHarry Stebbings
Now, this show honestly is quite an emotional one for me. I started 20VC from my bedroom in London. I was 18 years old. I had no money. I had no contacts in my phone book. And today, I have the chance to sit down with Satya Nadella, CEO of Microsoft, one of the greatest companies of the last three or four decades. And I will just say on Satya, the warmth that he was able to imbue in this interview in such a short space of time was almost unlike anything I've seen before. The clarity of his arguments and his ideas, his ability to go very deep very quickly, it was almost unlike anything I've ever seen before. It was a show I'll always remember, and I really hope you enjoy it. Ready to go? [upbeat music] Satya, thank you so much for doing this with us. Um, I would say, you know, thank you for coming to the studio, but you're actually hosting us here, so really appreciate you taking the time.
- SNSatya Nadella
No, like thank you so much, uh, for having me on the show, Harry. It's been a long time coming, but I'm finally glad we were able
- 1:23 – 2:01
Why Satya Says We Should Assume AI Models Are Compromised
- SNSatya Nadella
to do it.
- HSHarry Stebbings
Now, you wrote a seminal piece, uh, 10 days ago or so, and you said that we should assume a model is compromised, and I just wanted to expand on that. Does that mean that we can't or shouldn't rely on the disclosures of the model provider companies?
- SNSatya Nadella
No, I think the, the challenge the way I see it right now is the models are becoming more and more capable, right? Scaling laws are working, except one of the assumptions I had was that you'll have alignment just fall out of scaling, and it doesn't seem to be. Um, and there's more science, uh, challenges there that need to be tackled, right? And mechanistic understanding is also elusive.
- 2:01 – 4:45
Why AI Needs a “Zero Trust” Architecture
- SNSatya Nadella
And so given that, I think the best posture to have if you want to continue to use these sophisticated models is to treat these models in a trust architecture. I mean, after all, zero trust is not something new, right? We've had zero trust architectural approaches. So the claim at least I'm making is let's create an engineered zero trust environment that can contain any malfunctioning agent, um, before it does any harm, right? So that's the point because if you really going to have agents that are more capable, uh, and yet less, uh, aligned, uh, then you need to have a trust system, uh, that you can rely on.
- HSHarry Stebbings
Can I ask, what does that trust system look like? Who's the architecture of it? Who's the arbiter of it?
- SNSatya Nadella
It's a great point. And so the... That's why I kind of look at it and say, "You leave AI safety o- outside of this," right? Because that alignment, for example, at least my t- take is that it's g- it's, it's really mostly has to show up in training, right? In, in fact, maybe even in pre-training. I don't think you can even RL your way to alignment. You have to, uh, truly get alignment, uh, to be something that truly gets embedded, uh, in the weights, uh, right at the very beginning of it. And so that's the, the model itself hopefully at some point will get aligned. So if you leave AI safety, then the trust architecture is going to be just, you know, what a company like Microsoft does or all the cyber folks do. So what we have done, for example, is we built these two things, right? Agent 365 and Perception, right? What does it do? It gives every agent an identity. Um, it then logs all of its activity, all its tool calls, in fact, every model call and the CoT unit. In fact, that's where I'm also asking that all the CoT better be in, uh, English and not in new release, uh, and then you can check using multiple models. Uh, it better be tamper-proof, right? You can't have the agent go modify its own, um, uh, logs. Um, and then on that you set policy, and then you behaviorally monitor the activity of the agents so they're conformed to the policy. So that's the architecture.
- HSHarry Stebbings
We don't have that architecture today. Is that just symbolic of where we are in the progression?
- SNSatya Nadella
No, that architecture exists. So there, there are two... I think people conflate what I would call an engineered security Zero Trust architectural system for containment-
- HSHarry Stebbings
Yeah
- SNSatya Nadella
... and AI safety, right? Because I think people say, "Well, AI safety is hard and elusive and, uh, not yet achieved, so therefore there must be nothing we can do on security."
- HSHarry Stebbings
Mm-hmm.
- SNSatya Nadella
And I'm not saying that, um, that these two things are all gotta come together at the same time. In fact, I would just say, "Hey, we have to deal with the situation as it exists, so let's build a trust system outside, in fact, of the model, uh, and then just be super aggressive in, um, setting policy and monitoring for policy
- 4:45 – 5:41
Why Chain of Thought Is Not Reliable
- SNSatya Nadella
deviation."
- HSHarry Stebbings
On the monitoring side-
- SNSatya Nadella
Mm-hmm
- HSHarry Stebbings
... if we think about Chain of Thought, how reliable is that?
- SNSatya Nadella
It's not, right. We know that the Chain of Thought itself can be unfaithful, and, and, and in fact it can be made worse when the model company says, "Well, it's unfaithful, and I'm gonna hide it."
- HSHarry Stebbings
[laughs]
- SNSatya Nadella
Um, and, uh... And so it becomes even more, uh, problematic. And that's where I think a little bit of the push for standardization, uh, and the CoT at least for trusted partners being available as is so that it's inspectable, it's auditable, and I can check using multiple models and what have you, right? So that's gonna be one of the... In fact, that's one of the things that may even drive more open weights, uh, quite frankly, because if you can't tru- It's funny. Closed-weight models are more powerful, uh, but if they kind of don't recognize the need to be more transparent, uh, especially as their trusted partners use it, then we may have a problem.
- 5:41 – 7:04
Will Open-Weight Models Win Because They’re More Transparent?
- HSHarry Stebbings
Can I ask you, when we think about that rise of open source, I completely agree with you in terms of closed being more powerful. Open source today is predominantly, uh, Chinese models. Is that a concern then when we think about safety being baked into pre-training, when we think about disclosures, as we said there, and then juxtaposed with the majority of open source being Chinese?
- SNSatya Nadella
Yeah. I mean, I think there are two sets of things. I do believe if you sort of step back somehow, AI safety not being there, um, is not a good thing for China, it's not a good thing for the United States, it's not a good thing for the UK. So I think that there's a part of everyone... Because this is not something that somehow an unaligned open-weight Chinese model will probably attack even China. So it's not like, uh, you can sell, the rest think that this will only be used as an attack vector. So setting that aside, I'm pretty pleased with some of the progress even the open-weight models, uh, in the West are making, and I'm, you know, obviously, you know, Jensen's working on it. The other folks are working on it. Uh, we will have even the MAI models, uh, that will be not open-weight. We'll talk... I, I think of it more as licensed weights. Think of it that way. I want to turn this into just a simple, straightforward software business as opposed to a n- network effects intelligence business. So I think there will be different business models, and I ultimately trust in, uh, your ability to use those models and control those models is what's going to determine what
- 7:04 – 9:06
Open vs Closed Models: Where Will AI Traffic Go?
- SNSatya Nadella
gets used.
- HSHarry Stebbings
As a venture investor, I have to predict where traffic will go today between closed, open. When you think about the two different dimensions, which is like traffic and dollars, if we look forward, say, 24 months, what percent of traffic goes through open versus closed, do you think?
- SNSatya Nadella
It's a great question. I, I, you know, I think it depends on, on what the frontier looks like, right? Which is I think most things seem to get saturated, and when it gets saturated, open-weights or, um, or even just cheaper models of even closed weights, uh, seems to work. Uh, because especially one of the things is if you can train on your own traces, um, then you really don't need the latest model, uh, to be able to solve that same problem if the problem has not changed, right? So if you're ba- you know, if you have an eval that you want to continuously keep moving forward, uh, then you may want to pay. But even there, by the way, I'm very thrilled about this Hydro Diffusion stuff we now have in GitHub, uh, where, you know-
- HSHarry Stebbings
I just saw this.
- SNSatya Nadella
Yeah, because the cool thing there is it's a learned model. It now, it's... Anyway, we are now going into these agent swarms and multi- multiple sub-agents. So I can call a lot of sub-agents that even happen to be local models. Um, and so I think my suspicion is that there will always be a market for frontier. Uh, there will always be some new thing. Science may be the next thing that, you know, frontier is being set at. Uh, but I think predominantly token usage will be, uh, commodity token usage.
- HSHarry Stebbings
Does that mean it's open source predominantly?
- SNSatya Nadella
It, it may be. It may be open weights, or as I said, it does, it has, it can also be cheap clo- closed weights, right? Because there will be lots of price competition, let me put it that way, right? Because if you look at my own usage inside a copilot, we use a lot of closed source, cheaper models along with, uh, open-weight models.
- HSHarry Stebbings
You said about kind of the importance of owning your own intelligence-
- SNSatya Nadella
Yeah
- HSHarry Stebbings
... for some of your amazing customers that you have. You were talking about some in the UK. Is that the model environment that we have moving forward, specialized intelligence for companies
- 9:06 – 10:43
Why Every Company Needs to Own Its Own Intelligence
- HSHarry Stebbings
with their own data?
- SNSatya Nadella
Yeah. So it's-- I, I think this is, to me, one of the most important things, right? Which is what's the future of a firm-
- HSHarry Stebbings
Yeah
- SNSatya Nadella
... in an, uh, era of intelligence, right? Because if you go back even to Hayek, uh, his fundamental description of a economy is local intelligence. Tacit knowledge is distributed throughout the economy, right? Without that, you know, there can't be only one firm, uh, and it can't be one intelligence, and so that's sort of the foundational thing. In fact, capitalism works because of that. Price discovery works because of that, um, or pricing mechanism works because of that. So therefore, I think it'll be important for every firm to be able to accumulate, uh, knowledge, uh, at increasing scales using AI versus leak. That's why I called it the reverse information paradox, right? Which is, um, you know, the, you know, Kenneth Arrow kind of described this, um, um, paradox of information where if you're a seller and you disclose the secret without a patent or a trade secret protection or copyright protection, you leak the value.
- HSHarry Stebbings
Yeah.
- SNSatya Nadella
Now for the first time, the buyer needs the protection because the buyer, even it's, and most people think of it as, "Oh, my data," it's even my exhaust of using it, right? Thumbs up, thumbs down signal. Like, who, like, who... You're creating value, uh, for the model company versus capturing that value for yourself. So I think one of the things that at least, uh, I ascribe is that we need to have, for this to be stable, more of that surplus that gets created by use of intelligence to remain at the
- 10:43 – 12:16
Why Private Evals Could Become a Company’s Most Valuable IP
- SNSatya Nadella
firm. And by the way, it doesn't need to be only by embedding your knowledge in weights. It can be in context. It can be in your evals, right? You may have a benchmark. In fact, one of the things that I'm very gung ho about is we all become, every firm will have a private eval benchmark, which is their core IP. Uh, and every model comes in and tries to sort of hill climb on that eval, and that to me is how a firm works.
- HSHarry Stebbings
I, I love this. Private evals, benchmarks for each company. This is for a rarefied air of companies, though, it would seem, your Harveys or your Lagoras or the kind of very forward-thinking AI companies. Will that disperse to every company in the world as you mentioned with your partners?
- SNSatya Nadella
Yeah, it happens. Yeah, it happens, Har- uh, Harry, always, right? Which is, you know, there were, there was a day when I remember in the late '80s and the early '90s that the few companies that had deployed PCs and were doing-
- HSHarry Stebbings
[laughs]
- SNSatya Nadella
... for- forecasting, which felt magical. They were able to put it in Excel spreadsheets, send it in email, and avoid the fax machine and interoffice memos, and it felt like, "Oh, wow, when will this happen?" And it took 10 years maybe, and it spread all throughout. This time around, what is considered a, "Oh, I'm an AI native company, and I do this," I think, you know, five years from now it'll be pretty passe, I, I would imagine. And then that's kind of what... I mean, these tech cycles always start like that, right? Which is even cloud. I remember in the beginning of cloud, it was like the big deal was, "Oh, who's cloud native and who's not?" And you know, if you're not cloud native, and now everybody is, right? Even Lloyds, you know, which is, what, 500 years old, is cloud native. And so
- 12:16 – 14:11
Why AI Will Create a New Implementation Economy
- SNSatya Nadella
it happens.
- HSHarry Stebbings
Do we move into an implementation economy then? 'Cause when we think about these customers, say, as you said, some of your very big customers, they will need heavy FD support, much more so than they have done in traditional years, within the last 10 years. Will we have a very heavy implementation economy to transition existing customers into this?
- SNSatya Nadella
Yeah, this, it's, it's interesting, right? The, the implementation economy, like, systems integrators have always existed, right? You, you know, we've depended on, uh, implementation, uh, economy for lots of other technology diffusion. So this one, I think, has different contours, right, because it's the combination of tech and business, if I could say. It's ki- but, but it's not classic management consulting, it's not classic systems integration, and it's not classic sort of technology support. It's sort of, uh... Like, for example, think about this, right? You wanna start. Like, when I look at my FDE engagements, right? Even take the Unilever example I talked about at the keynote today. It's fascinating. They, they built an agentic system to say, "Hey, we wanna really have order to cash, uh, automated using an agentic system," plus their trade claims. Um, and by the way, th- they started, uh, using a frontier model, and then they substitute, uh, for... In fact, I think they're using currently an MAI model, uh, for all the trade claims because it's sort of a lower cost, uh, and it's, by the way, they control it, right? Because they're able to train on their traces. Um, and so we had some FDEs, but a lot of, uh, who got deployed, and now their, their IT team itself is, you know, uh, managing it all. Um, but it started with what's the business, what's the eval, what's the way to conceptualize the agentic architecture? So it's sort of business consulting combined with technical consulting, and I think, yeah, you're right in saying that there will be a new set of skills that'll get built, uh, in the ecosystem. Uh, today I think all of us have, quote-unquote, "our own FDEs," but FDEs will become like, you know, traditional systems integration will become new FDE.
- 14:11 – 15:52
Does AI Require Full-Stack Vertical Integration?
- HSHarry Stebbings
You've mentioned obviously MIA a lot, uh, MAI.
- SNSatya Nadella
Yeah.
- HSHarry Stebbings
MIA.
- SNSatya Nadella
Yeah.
- HSHarry Stebbings
Gosh. Um, but when we look at a lot of different players, it's kind of this realization that you need to be full stack from chips to models to application. Is that the future? You have to have vertical ownership? And what does that mean?
- SNSatya Nadella
Yeah, it all depends on the industry structure, right? Um, you know, I think that, I would say today, you know, there'll be, who knows, five, six open-weight mo- you know, closed models all competing, maybe five, six clo- you know, open-weight models all competing. And so therefore at some level, if you're building an application, like agentic system, you could even have something like what I said, which is a router with even a decision engine built into it, and then you just route to the best model for a given task. So that's a possibility. Uh, of course there's always the risk that there could be one vertically integrated model, uh, and a app, app tier and the, the, uh, and the s- all the way to the silicon and system. It's unlikely to me, right? I, I thought at some point that that might be the case, that there will be one. It's, if anything, it's going the other way, which is every day you wake up and you say, "Wow, there's more competition in every layer of the stack," whether it's in silicon, whether it's in systems to manage the silicon, uh, in the model layer, in the app layer. And so I think that's a healthy sort of direction to travel, but you never know, right? Because this is one of those things where it all depends on, uh, what becomes the dominant design. Um, but my hope is that this becomes a little bit of a horizontal specialization with multiple winners in each layer, and then the ability for people to compose.
- 15:52 – 16:00
Why Inference Demand Is Bigger Than Anyone Expected
- HSHarry Stebbings
The thing I didn't see is the depth of each layer.
- SNSatya Nadella
Yeah.
- HSHarry Stebbings
Like, what I know now that I wish I'd known 12 months ago is just, like, the infinite demand for inference.
- SNSatya Nadella
Yeah. [laughs] That's a great
- 16:00 – 17:54
Can the World Support $100BN AI Training Runs?
- SNSatya Nadella
point. Yeah.
- HSHarry Stebbings
And I wish I'd known it 12 months ago, I would've done some very different investing. W- speaking of that, we've had, like, Mustafa Suleyman saying, "Training runs of 100 billion are coming, and only a handful of labs can pay for that." $100 billion training runs. Do we have the energy infrastructure and power to enable this scale across the stack, but especially when we do-
- SNSatya Nadella
Yeah, I mean, the way I come at it is, um, ultimately it has to come from the world GDP inflecting, um, and then that allowing, uh, someone to then have as a percentage of R&D, right? Because, uh, when somebody says training run, it's just a percentage of R&D, right? Sort of companies that are heavy R&D spend what? 15, 20%, um, you know, um, on R&D. And so therefore, yeah, at some point if you have, uh, enough revenue, you can spend enough on training. Um, and then of course you'll have enough revenue if you're contributing to the world economy growing so that you can get your fair share, so to speak. And then y- you also bring up the other side, which is the supply side, which is we have power shortages, we have silicon shortages, we have, you know, all kinds of things today. They will all... They're long lead items, right? You just can't, you know, snap your finger and have power generation or you can't have a new fab. Um, and so now interestingly enough, given the demand, everyone's building. Um, and so the question is, when will all of this come online? What will be the demand supply? No one knows, right? Quite frankly, any one of us who are sitting here today will just be speculating. But my feeling is that if you start by thinking of ultim- the stable state would be just like today, right? Income statements are suddenly not gonna be different, right? For a company. It's going to be, hey, I have some revenue, I have some cost of revenue, and I have to have some profit. Um, and if that is going to be true, then, uh, the question is what is the world GDP growth rate? That's the bigger thing for me.
- 17:54 – 19:09
Can AI Meaningfully Accelerate Global GDP Growth?
- HSHarry Stebbings
In terms of the inflation of GDP growth rate, the world has kind of stabilized at 2% a year for a very large amount of time. Does AI have the ability to, to seriously inflate that in a short amount of time?
- SNSatya Nadella
I think so, um, and I somewhat hope so, [laughs] uh, because that is the point, right? When I look at... Let's take even, um, as a percentage of any of our GDPs, the amount of money we spend in our healthcare, right? So that's why I look at even the deployment of Copilot in NHS and say, "Wow, um, if it can improve patient outcomes and reduce cost in the healthcare, that is by definition, uh, you know, helping the GDP of the United Kingdom."
- HSHarry Stebbings
Mm-hmm.
- SNSatya Nadella
And so I think that there are these pockets of, uh, the economy which are ripe for adoption of AI and really cr- in fact, there's the least amount of displacement, the most amount of benefit Um, and so that I think is where I think we'll have to find the best fit. Um, software development is another one, right? Which is we- i- it's fascinating to see how much the demand is elastic for software development, right? Which is, uh, that means the IT backlogs were huge, and none of us even recognized
- 19:09 – 21:34
Which Industries Will See the Biggest AI Productivity Gains?
- SNSatya Nadella
it.
- HSHarry Stebbings
Satya, I was taught that you had to be the top one or two in a market to win in venture. We're in, you know, Factory, which is fifth, and it's winning. Fifth.
- SNSatya Nadella
Yeah.
- HSHarry Stebbings
This breaks all rules of venture.
- SNSatya Nadella
Yeah.
- HSHarry Stebbings
A question that I have, again, slightly naughty, not on schedule, but it's just like, will we see the tipping in terms of capability advancement in alternative functions, finance, sales, marketing, that we've seen in coding? Because coding's been the mother lode.
- SNSatya Nadella
I think that that's a good point. It- it, you know, people talk a lot about, you know, do you have a verifiable reba- reward and, you know, it's easy to do these, because a lot of the magic has been RL on top of these pre-trained models. That is what, you know, has helped coding. Uh, can we crack that in knowledge work seems much m- harder. Uh, but again, all the examples I talked about, sort of, you know, in contact centers, in the supply chain systems, in these, uh, marketing slash or cash to order systems, I think it's- it's possible, uh, and we're seeing it. Um, is the demand as elastic as coding? I think it may be science may be as elastic, right? Or engineering in general. Like, by the way, you know, if I now look at even what's happening inside, not just in software development, let's take silicon development. That's another thing that's going through a radical shift, right? Your ability to tape out much faster, simulate everything in silico is much better. Your first spin can completely come out, uh-
- HSHarry Stebbings
But even just, like, optimization of maintenance reports-
- SNSatya Nadella
Oh, yeah
- HSHarry Stebbings
... like you said with Rolls-Royce-
- SNSatya Nadella
Yeah, the Rolls-Royce one is-
- HSHarry Stebbings
Which I would never cons-
- SNSatya Nadella
Oh, it's unbelievable.
- HSHarry Stebbings
Yeah.
- SNSatya Nadella
Like, I mean, it's a thing that they've been dealing with forever, just schedule maintenance, and I had not realized that, wow, that's a big optimization problem that you can go now run.
- HSHarry Stebbings
So I've, I've watched, yeah, many, many of your shows, and we're gonna do a quick fire round.
- SNSatya Nadella
Yep.
- HSHarry Stebbings
Um, and people haven't asked some of these questions, which I've always wanted to know, and I think everyone wants to know, but yeah, people haven't asked them. What's the exercise routine given the travel schedule, Satya?
- SNSatya Nadella
Well, I mean, you know-
- HSHarry Stebbings
You're in good shape.
- SNSatya Nadella
Well, I, I try to be. I don't know whether it's all well hidden. [laughs]
- HSHarry Stebbings
[laughs]
- SNSatya Nadella
But, uh, you know, the one thing I've gotten into the habit, you know, this has been now maybe 25 years even on, which is every morning I get up and, you know, I used to be able to run. Unfortunately, my knees have gone [laughs] away, and so now at least-
- HSHarry Stebbings
Me too
- SNSatya Nadella
... I walk with incline. Uh, even this morning I was able to get a half hour in, and so that's what... It, it, uh, in, in spite of any jet lag, wherever I am, I get, hit the gym every day.
- HSHarry Stebbings
Love it. Uh, I too have bad knees. It's, it's my age as well. Uh, what have you changed your mind on in the last 12 months?
- 21:34 – 22:10
Why Satya No Longer Believes Alignment Will “Just Fall Out” of Scaling
- SNSatya Nadella
Last twel- It might be, because after all, I was one of the first, yeah, uh, uh, obviously the, when, uh, when Dario and Ilya and Sam and Greg and Elon were all, I guess, one happy family. They were really the believers, uh, in scaling laws, and that's what I believed in, uh, back in 2018, 2019. Um, and, uh, I think the biggest change now I have is I used to always think that, you know, alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder, different problem. Um, and so, uh, yeah, that's the one thing I've changed my mind
- 22:10 – 22:36
Why Every Company Could Become an AI Benchmark
- SNSatya Nadella
on.
- HSHarry Stebbings
What looks crazy today, but will be incredibly common and accepted in five years?
- SNSatya Nadella
You know, it might be that, um, that thing that we just talked about, those evals, right? Which, uh, what is a firm? I think a firm would be an, an AI benchmark.
- HSHarry Stebbings
Final one for you. There are many things to be excited about over the next 10 years. For you specifically, like for me it's, uh, my mother's got MS. I'm so excited for the drug development that could be capable.
- SNSatya Nadella
Yeah.
- HSHarry Stebbings
Um, what are
- 22:36 – 23:37
What Satya Is Most Excited About for the Next Decade
- HSHarry Stebbings
you most excited for?
- SNSatya Nadella
I mean, that's a great one. I think one of the things that I've always been, I would say, been looking forward to is a general purpose technology, even as powerful as this, and people say, "Oh, this is not, uh, just another general purpose technology," whatever this is. I hope this spreads, diffuses, uh, throughout the world faster than anything else, and throughout all sectors of the economy, unlike anything else. And truly for once, there is a real shift in the frontier, uh, of that global growth, um, that happens, where it's not about just counting winners and losers, but it's kind of like everyone gets lifted. Like, that's been my dream. In fact, my father was a development economist and, you know-
- HSHarry Stebbings
[laughs]
- SNSatya Nadella
... it was always about convergence growth. When will the Global South get something that just suddenly boosts that? I hope this is it.
- HSHarry Stebbings
Satya, this has been such a joy to do. Thank you so much for taking the time.
- SNSatya Nadella
Absolutely. It's a pleasure, Harry.
Episode duration: 23:48
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 24XUcWhqY_4