AI Fund’s GP, Andrew Ng: LLMs as the Next Geopolitical Weapon & Do Margins Still Matter in AI?

(instrumental music plays) In my career working in AI, (computer data sound effects) I have yet to meet a single person that ever felt like they had enough compute.

I could not ask for a better guest. Andrew Ng, globally recognized leader in AI.

Data centers are the critical infrastructure for building the digital economy. I think that open-way models is a tremendous source of geopolitical influence. (computer data sound effects) The work ethic, the velocity, when China's government makes a all-nation commitment, it's all-industrial commitment, that's actually a very powerful force that I wouldn't underestimate.

Ready to go? (instrumental music plays) Andrew, I've been an admirer for a long time, so I've been really looking forward to making this happen, so thank you so much for joining me today.

No, yeah. Thank you, Harry. I watch a bunch of shows. I really enjoyed your recent one with, uh, my friend, Martin, Martín Casado as well. That was very memorable. So actually thrilled to be, uh, uh, here.

I d- I love Martín. Very, very special man. I, I wanna start with something that you've said before. You said AI is the new electricity, and when I think about electricity and where we are today, I wanna understand the bottlenecks. And everyone seems to suggest that it really is about data, compute, and algorithms. Is that the three parameters to which we should think about bottlenecks, and if so, which one do you think is the biggest bottleneck?

I would say the two biggest bottlenecks right now, um, it may be, uh, uh, I, I think electricity is one of them. Uh, so in the US, I am honestly worried that, uh, many data center operators were stuck in kind of permitting and, you know. And, and I know that local community support is important and some people don't want a data center there, but, um, once we build roads and railways as the infrastructure for a certain generation, data centers are the critical infrastructure for building the digital economy, and so lack of electricity in, in America and in number of Western countries is a problem. And in contrast, I see China building power plants left and right, including nuclear, so that will be interesting dynamic. And semiconductors is another bottleneck. Um, but AI is so complicated. I think we also need more data. We also need more, um, better algorithms. You know, all of it is worth working on, but in the short term, some constraints with electricity and, and, and semiconductors.

Can you talk to me about the c- constraints around semiconductors that you think are most pressing that most people don't realize?

First, in my career working in AI, I have yet to meet a single AI person that ever felt like they had enough compute. So, um, you know, give us any amount of compute, we will use it all up and say, "We still don't have enough." So this is a constraint for the last 20 years or so. But what I'm seeing is, um, with the rise of gen AI, there are very valuable workloads, uh, for example, AI-assisted coding. You know, it's fantastic. It's making us so much more productive. But if you use cloud code enough, sometimes you get rate-limited, and I find that many companies have, really have excess demand, which is a very rare problem to have, but so many people want more LM inference, want more tokens generated, and we just don't have the semiconductors and data centers and electricity to meet the demand. But, you know, there's a lot we could do with AI, um, token generation, uh, and it's frustrating when we can't, uh, when on supply side, we can't supply enough to people that want it. On the demand side, you know, you, you get rate-limited if you, if you, if you use too much.