No Priors Ep. 128 | With Andrew Ng, Managing General Partner at AI Fund

Sarah Guo 0:00 (instrumental music plays) Hi, listeners. Welcome back to No Priors. Today, Elade and I are here with Andrew Ng. Andrew is one of the godfathers of the AI revolution. He was the co-founder of Google Brain, Coursera, and the venture studio AI Fund. More recently, he coined the term agentic AI and joined the board of Amazon. Also, he was one of the very first people a decade ago to convince me that deep learning was the future. Welcome, Andrew. Andrew, thank you so much for being with us.

Andrew Ng 0:30 No, always great to see you.

Sarah Guo 0:32 I'm not sure where we should begin because you, you have such a broad view of these topics. But I feel like we should start with the biggest question, which is, um, you know, if you look forward at capability growth from here, uh, where does it come from? Does it come from more scale? Does it come from data work?

Andrew Ng 0:46 Multiple vectors of progress. So I think, um, there is probably a little bit more juice out of the scalability lemon that we squeeze, so hopefully you can see many products there, but it's getting really, really difficult. Um, society's perception of AI has been very skewed by the PR machinery of a handful of companies with amazing PR capabilities and because that number of companies drove scales and narrative, people think of scale first of, as a vector progress. But I think, you know, agentic workflows, um, uh, the way we build multimodal models, we have a lot of work to build concrete applications. I mean, there are multiple vectors of progress as well as wild cards like brand new technologies like diffusion models which are used to generate images for the most part. Will that also work for generating text? I think that's exciting. So I think there'll be multiple ways for AI to make progress.

Sarah Guo 1:29 You actually came up with the term agentic AI. What did you mean then?

Andrew Ng 1:32 So when I, uh, decided to start talk about agentic AI, which wasn't a thing when I started to use the term, my team was slightly annoyed at me. One of my team members that I won't name, he actually said, "Andrew, the world does not need you to make up another term." But I decided to do it anyway and for whatever reason, it stuck. And the reason I started to talk about agentic AI was because, um, uh, like couple years ago, I saw people were spending a lot of time debating, "Is this an agent? Is this not an agent? What is an agent?" And I felt there's a lot of good work, and there was a spectrum of degrees of agency where there are highly autonomous agents that could plan, take multiple steps if using, do a lot of stuff by themselves, and then things that were lower degrees of agency where it would prompt an LLM for effect in its output. And, and I felt like rather than debating this is an agent or not, let's just, um, say the degrees of agency and say it's all agentic so we can spend our time actually building this. So I started to push the term agentic AI. What I did not expect was that, uh, several months later, a bunch of marketers would get ahold of this term and use it as a sticker to stick it on everything in sight. And so I think the term agentic AI really took off. I feel like the marketing hype has gone like that insanely fast, but the real business progress has also been, you know, rapidly growing, but maybe not as fast as the marketing

Andrew Ng 2:44 hype.

Elad Gil 2:44 What do you think are the biggest obstacles right now to true agents actually being implemented as AI applications? Because to your point, I think we've been talking about it for a little while now. There's certain things that were missing initially that are now in place in terms of everything from certain forms of inference time compute on through to forms of memory and other things that allow you to maintain some sort of state against what you're doing. What do you view are the things that are still missing or need to get built or will sort of foment progress on that end?

Andrew Ng 3:06 I think the technology component level, there's stuff that I hope will improve. For example, computer use, you know, kind of works, often doesn't work. Um, I think... So the guard rails, evals is a huge problem. How do we quickly evaluate these things and drive eval? So I think the, the component is there's room for improvement. But what I see as the single biggest barrier to getting more, uh, agentic AI workflows implemented is, is actually talent. Uh, so when I look at the way many teams build agents, the single biggest differentiator that I see in the market is, does the team know how to drive a systematic error analysis process with evals? So you're building the agents by analyzing at any moment in time what's working, what's not working, what do you improve, as opposed to, uh, less experienced teams who kind of try things in a more random way that just takes a long time. And we're looking across a huge range of businesses, small and large, it feels like there's so much work that can be automated through agentic workflows, but, you know, the talent and the skills and maybe the software tooling, I don't know, just isn't there to drive that disciplined engineering process to get this stuff built.

Sarah Guo 4:05 How much of that engineering process could you imagine being automated with AI?

Andrew Ng 4:10 You know, it turns out that a lot of this process of building agentic workflows, it requires ingesting external knowledge, which is often locked up in the heads of people. So until and unless we built, you know, AI avatars that can interview employees doing the work, and that's a visual AI that can look at the computer monitor. I think, uh, maybe eventually, you know, but I think at least right now for the next year or two, I think there's a lot of work for human engineers to do, um, to build more agentic workflows.

Sarah Guo 4:40 What's-

Elad Gil 4:40 And so that's more the kind of, uh, collection of data feedback, et cetera for certain loops that people are doing. Is that... Other, other things that... I- I'm sort of curious like what that translates into tangibly versus-

Andrew Ng 4:51 Yeah, so one example. So I see a lot of workflows like, um, you know, maybe a customer emails you a document. You're going to convert the document to text, then maybe do a web search for some compliance reason to see if you're working with a vendor you're not supposed to, and then look up a database record, see if the pricing is right, save it somewhere else and so on. There's multi-step agentic workflows, kind of next gen robotic process automation. So we implement this and it doesn't work, you know, is it a problem? Have you got the invoice date wrong? Is that a problem or not? Or have you routed a message to the wrong person for verification? So when all of... When you implement these things, you know, almost always it doesn't work the first time. But then to know what's important for your business process and is it okay that, I don't know, I bothered the CEO of the company too many times or is the CEO maybe doesn't mind verifying some invoices. So all that external contextual knowledge, um, often, at least right now, I see thoughtful human product managers or human engineers having to just think through this and make these decisions. So can an AI agent do that someday? I don't know. Seems pretty difficult right now. Maybe someday.

Sarah Guo 5:51 But it's not in the internet pre-training dataset and it's not in a manual that we can automatically extract?

Andrew Ng 5:58 I feel like for a lot of work to be done building agentic workflows, that dataset is proprietary. It, it's just knowl- it's, it's not general knowledge on the internet. So figuring that out is still, still exciting work to do.

Sarah Guo 6:08 What is the...

Sarah Guo 6:09 Um, if you just look at the spectrum of agentic AI, what's the strongest example of agency you've seen?

Andrew Ng 6:15 I feel like-Bleeding edge of Agentic AI, I've been really impressed by some of the AI coding agents. Um, so I think in terms of economic value, I feel like there are two very clear and very apparent buckets. One is answering people's questions. Uh, probably, you know, OpenAI ChatGPT seems to mark a leader with that, with real ex- real takeoff, liftoff velocity. The second massive bucket of economic value is, uh, coding agents, where coding agents, like my, my, my personal favorite cloud

Elad Gil 6:43 Mm-hmm.

Andrew Ng 6:43 ... developer too right now is Cloud Code. Maybe it will change at some point, but I, I, I, I, I just use it, love it. Uh, highly autonomous in terms of planning out, you know, what to do to build this software, building a checklist, going through it one at a time. So this ability to plan a multi-step thing, execute the multiple steps of the plan, uh, is one of the most highly autonomized agents out there being used that, that actually works. Uh, there's other stuff that I think doesn't work, like some of the computer use stuff, like, you know, go shop for something for me and browse online. Those, some of those things are really nice demos, but, but not yet production ready, so-

Elad Gil 7:18 Do you think that's because of source or criteria in terms of what needs to be done and more variability around actions? Or do you think there's a, a better training set or sort of set of outputs for coding? I'm sort of curious, like, why does one work so well or almost feels magical at times, and the others are, you know, really struggling as use cases so far?

Andrew Ng 7:35 I think, you know, engineers really good at getting all sorts of stuff to work, but, um, uh, the economic value of coding is just clear and apparent and massive. So I think the sheer amounts of resources dedicated to this has led to a lot of smart people for whom they themselves are the user. So also good instinct on product, building really amazing coding agents. Uh, and then I think, I don't know-

Elad Gil 8:00 You don't think it's a fundamental research challenge, you just think it's like capitalism at work and then domain knowledge in a lab? (laughs)

Andrew Ng 8:05 Oh, I think capitalism is great at solving fundamental research problems. Yeah.

Elad Gil 8:09 (laughs) At what point do you think, um,

Elad Gil 8:12 models will effectively be bootstrapping themselves in terms of, you know, 99% of the code of a model will be written by agentic coding agents? Or the error analysis in-

Andrew Ng 8:22 Oh, I, I, I, I feel like we're, I, I, I'm, I, I certainly suspect we're slowly getting there. So some of the leading foundation model companies are clearly, well, they've said publicly they're using AI to write a lot of the codes. Um, I mean, one thing I find exciting is, uh, AI models using agentic workflows to generate data for the next generation-

Elad Gil 8:40 Mm-hmm.

Andrew Ng 8:40 ... of models. So I think, I think the LLaMA research paper talked about this, but older version of LLaMA would be used to think for a long time to generate puzzles, that then you train the next generation of the model to try to solve really quickly without needing to think as long. So I find that exciting too. Um, yeah, multiple vectors of progress. It feels like, it feels like, you know, AI is not just one way to make progress. There's so many smart people pushing forward in, in, in so many different ways.

Elad Gil 9:04 Mm-hmm. I think you

Elad Gil 9:05 have rejected the term vibe coding in, in favor of AI-assisted coding. Like what, what's the difference?

Andrew Ng 9:11 You know, um, I, I, I, I know that-

Elad Gil 9:13 I'm assuming you do the latter, you're not vibing.

Andrew Ng 9:15 Oh, yeah. Vibe coding leads people to think, you know, like, "I'm just gonna go with the vibes and accept all the changes that cursor suggests or whatever." And it's fine that sometimes you could do that and it works, but I wish it was that easy. So when I'm coding for a day or for an afternoon, I'm not like, "Going with the vibes." This is like, a deeply intellectual exercise. And I think the term vibe coding makes people think it's easier than it is. So frankly, after a day of using AI-assisted coding, like, I'm exhausted mentally, right? So I think of it as rapid engineering, where AI is letting us build serious systems, build products much faster than ever before, but it is, you know, engineering just done really rapidly.

Elad Gil 9:55 Do you think

Elad Gil 9:56 that's changing the nature of startups, how many people you need, how you build things, how you approach things? Or do you think it's still the same old kind of approach, but you just have people that get more leverage because they have these tools now?

Andrew Ng 10:05 So, you know, AI founder, we built startups, and it is really exciting to see how, uh, uh, rapid engineering, AI-assisted coding, um, is changing the way we build startups. So there's so many things that, you know, would've taken a team of six engineers, like, three months to build that now today, one of my friends or I, we just build on a weekend.

Elad Gil 10:24 (laughs)

Andrew Ng 10:24 And the fascinating thing I'm seeing is, um, if we think about building a startup, the, the core loop of what we do, right? We wan- I want to build a product that users love. And so the core iteration loop is write software, you know, it's a software engineering work, and then the product managers maybe go do user testing, look at it, go by gut, whatever, to decide how to improve the product. So when we go look at this loop, the speed of coding is accelerating and the cost is falling. And so increasingly, the bottleneck is actually product management. Uh, so, so the product management bottleneck is now that we can build whatever we want much faster, well, the bottleneck is deciding what do we actually want to build. And previously, if it took you, say, three weeks to build a prototype, if you need a week to get user feedback, it's fine. But if you now build a product in a day, then boy, if you have to wait a week for user feedback, that's really painful. So I find my teams, um, frankly, increasingly relying on gut because, um, we go and collect a lot of data that informs our very human mental model, our brain's mental model of what the user wants. And then we often, you know, have to have deep customer empathy, so we can just make product decisions like that, right, really, really fast in order to drive progress.

Elad Gil 11:34 Have you seen anything

Elad Gil 11:35 that actually automates some aspects of that? I know that there have been, um, some versions of things where people, for example, are trying to generate market research by having a series of bots kind of react in real time, and that, that almost forms your market or your user base as a simulated environment of users. Have you seen any tool like that work or take off? Or do you think that's coming? Or do you think that's too hard to do?

Andrew Ng 11:54 Yeah, so there's been a bunch of tools to try to speed up product management. Um, I feel like, uh, well, uh, the, the recent Figma IPO is one, you know, great example of design. AI

Elad Gil 12:03 Mm-hmm.

Andrew Ng 12:03 ... high DNA, you know, it didn't do a great job. Um, then there are these tools that, uh, are trying to use AI to help interview prospective users. And as you say, we looked at some of the scientific papers on using a flock of AI agents...

Elad Gil 12:15 Uh-huh.

Andrew Ng 12:15 ... to simulate, you know, a group of users and how to calibrate that. It all feels promising and early and hopefully wildly exciting in the future. I don't think those tools are accelerating product managers nearly as much as coding tools are accelerating software engineers. So this does create more of the bottleneck on the product management side.

Elad Gil 12:32 It doesn't make sense to me that my partner Mike has this idea that I think is broadly applicable in a different, a couple different ways of, like, computers can now interrogate humans at scale. Um, and so there's companies like ListenLabs working on this for, like, consumer research-type tasks, right? But you could also use it to-

Andrew Ng 12:48 Mm-hmm.

Elad Gil 12:48 ... you know, understand tasks for training.

Sarah Guo 12:50 ... or for, um, you know, the data collection piece that you described. When you think about

Sarah Guo 12:55 your teams that are in this iteration loop, has like the founder profile that makes sense changed over time?

Andrew Ng 13:01 Uh, to me, there are so many things that the world used to do in 2022 that just do not work in 2025. So in fact, often I- I- I- I ask myself, "Is there anything we're doing that today that we're also doing in 2022? And if so, let's take a look and see if it still even makes sense today." Because a lot of stuff, a lot of workflows in 2020 don't make sense today. So I think today, um, the technology is moving so fast, founders that are on top of gen AI technology, that's, you know, uh, tech-oriented product leaders, I think are much more likely to succeed than someone that maybe is more business oriented, more business savvy, but is not... doesn't have a good feel for where AI is going. I think unless you have a good feel for what this technology can and cannot do, it's really difficult to think about strategy, whether, whether lead the company.

Sarah Guo 13:48 We believe this too.

Andrew Ng 13:49 Yeah. Cool. Yeah. Yeah. Yeah.

Elad Gil 13:51 I think that's like old school Silicon Valley even. Like if you, uh, if you look at, uh, Gates or Steve Jobs/Wozniak or a lot of the really early pioneers of the semiconductor, computer, um, early internet era, they were all highly technical.

Andrew Ng 14:05 Yeah.

Elad Gil 14:05 And so I almost feel like we kind of lost that for a little bit of time and now it's, it's very clear that, uh, you need technical leaders for technology companies.

Andrew Ng 14:12 I think we used to think, "Oh, you know, they've had one exit before or so, or two exits even, so let's just back that founder again." But I think if that founder has stayed on top of AI, then that's fantastic. But if, you know... A- a- and I think part of it is, um, in moments of technological disruption, which AI rapidly changing, that's the real knowledge. So, so actually take, take mobile technology. You know, like everyone kind of knows what a mobile phone can and cannot do, right? What a mobile app is, there's GPS, all that. Everyone kind of knows that, so you don't need to be very technical to have a gut for, can I build a mobile app for that?

Sarah Guo 14:43 Mm-hmm.

Andrew Ng 14:44 But AI is changing so rapidly, what could you do with a voice app, what engineering workflows do, how rapidly foundation models, uh, w- what was an easy model.

Sarah Guo 14:50 Mm-hmm.

Andrew Ng 14:50 So having that knowledge is a much bigger differentiator, whereas, you know, knowing what a mobile app can do, to build a mobile app, right?

Elad Gil 14:57 Yeah. It's an interesting point 'cause when I look at the biggest mobile apps, they were all started by engineers. So WhatsApp was started by an engineer, Instagram was started by an engineer. I think Travis at Uber was, was technical-ish. Uh-

Sarah Guo 15:08 Technically adjacent.

Elad Gil 15:09 Technical adjacent. Um, Instacart, Apoorva was an engineer at Amazon.

Andrew Ng 15:13 Yeah. A- a- and Travis had the insight that GPS enabled a new thing.

Elad Gil 15:18 Yeah.

Andrew Ng 15:18 But so you had to be one of the people that saw GPS on mobile coming early-

Elad Gil 15:21 Yeah.

Andrew Ng 15:21 ... to go and do that.

Sarah Guo 15:22 Yeah. You have to be like really aware of the capabilities.

Elad Gil 15:24 Yeah, you have to know the technology.

Sarah Guo 15:26 That's what I mean, yeah.

Elad Gil 15:26 Yeah. It's super interesting. What, what other characteristics do you think are, are common? I mean, I know, um, people have been talking about, for example, uh, it almost felt like there was an era where being hardworking was kind of poo-poo-ed. Or do you think founders have to work hard? Do you think people who succeed we- I'm just sort of curious, like, uh, aggression, uh, hours work, like what else may correlate or not correlate in your mind?

Andrew Ng 15:47 You know, I work very hard. The periods in my life where, you know, I- I encourage others that want to have a great career and impact, like work hard. But e- even now I feel like a little bit of nervousness saying that because in some parts of society it's considered not politically correct-

Elad Gil 16:04 Mm-hmm.

Andrew Ng 16:04 ... to say, "Well, working hard probably correlates with your personal success." Um, I think it's just a reality.

Elad Gil 16:09 Yeah.

Andrew Ng 16:09 I- I know that not everyone at every point in their life is in the time where they work hard. You know, when my kids were first born-

Elad Gil 16:15 Mm-hmm.

Andrew Ng 16:15 ... that week I did not work very hard.

Elad Gil 16:16 Yeah.

Andrew Ng 16:16 It was fine, right? So acknowledging that not everyone is in circumstances where they work hard.

Elad Gil 16:21 Mm-hmm.

Andrew Ng 16:21 Just a factual reality is people that work hard accomplish a lot more. Um, but of course you need to respect people that aren't in a phase where they-

Sarah Guo 16:29 Yeah, I'd say something maybe a little less correct, which is I, uh... Less politically correct, which is like, I think there was an era where people thought s- like there was a, there was a statement that startups are for everyone. And like, I do not believe that's true, right? I think like, you know, you're trying to do a very unreasonable thing, which is like-

Elad Gil 16:45 Yeah.

Sarah Guo 16:45 ... create a lot of value impacting people very quickly. And when you're trying to do an unreasonable thing, you probably have to work pretty hard, right? And so I think people... I think that got very... The sort of work ethic required to like move the needle in the world very quickly disappeared.

Andrew Ng 16:59 Yeah. Tho- there was a, there was, there was a quote, I- I'm, I wish I remember who said this, but, uh, was it, "The only people that would change the world are the ones crazy enough to think they can." I think it does take someone with the boldness, the decisiveness to go and say, "You know what? This is the state of the world, I'm gonna take a shot at changing it." And, and, and there's only, uh, people with that, um, conviction, uh, uh, that, uh, I think can do this.

Elad Gil 17:22 Strikes me as being true in any endeavor. You know, I used to work as a biologist, and I think it's true in biology, I think it's true in technology, I think it's true in, um, almost every field that I've seen is i- it's the people who work really hard do very well. And then in startups at least, the thing I tended to forget for a while was just how important competitiveness or people who really wanted to compete and win mattered. And sometimes people come across as really low key, but they still have that drive and that urge and they, they want to be the ones who are the winners. And so I think that matters. And similarly, that was kind of put aside for a little bit, at least, uh, from a societal perspective relative to companies.

Andrew Ng 17:55 Actually, I think I've seen, I feel like I've seen two types. One is they really want their business to win. That's fine. Some do great. Some are, they really want their customers to win.

Elad Gil 18:04 Mm-hmm.

Andrew Ng 18:04 And they're so obsessed with serving the customer that that works out, uh. I- I should say early days at Coursera, you know, yes, I knew about competition, blah, blah, blah, but I was really obsessed with, you know, learnedness with the customers, and that drove a lot of my behaviors, yeah.

Elad Gil 18:17 No, that, that's a really good framework. And when I say competition, I don't mean necessarily with other companies, but it's almost like with whatever metric you set for yourself or whatever thing you want to win at or be the best at.

Andrew Ng 18:28 Well, one thing I've found is, in a software environment, you just got to make so many decisions every day. Um, you just have to go by gut a lot of the time, right? I, I, I feel like, you know, building a startup feels more like playing tennis than solving calculus problems. Like you just don't have the time to think, you just got to make a decision. And I feel like, um, so this is why people that obsess day and night with the customer, with the company, uh...... think really deeply and have that conceptual knowledge that when, when someone says, "Do I ship product- feature A or feature B?"

Sarah Guo 18:57 I feel like you know, yeah.

Andrew Ng 18:57 Like, you just gotta know, a lot of the time, not always. And, and turns out there are so many, um, to use Jeff Bezos' term, like two-way doors in startups because frankly, you know, you have, you have very little to lose.

Sarah Guo 19:08 Mm-hmm.

Andrew Ng 19:08 So just make a decision. If it's wrong, change it a week later, it's fine. So I find... But, but to be really decisive and move really fast, you need to have obsessed usually about the customer, maybe the technology, to have that state of knowledge to make really rapid decisions-

Sarah Guo 19:21 Okay.

Andrew Ng 19:21 ... and still be right most of the time.

Sarah Guo 19:22 How do you

Sarah Guo 19:23 think about that bottleneck, uh, in terms of product management that you mentioned, or people who have good product instincts? Because I was talking to, uh, one of the best-known sort of tech public company CEOs, and his view was that in all of Silicon Valley, or in all of tech kind of globally, there's probably a few hundred, at most, great product people.

Andrew Ng 19:41 Hmm.

Sarah Guo 19:41 Do you think that's true, or do you think there's a broader swath of people who are very capable at it? And then, how do you find those people? 'Cause I think that's actually a very rare skill set in terms of the people who are, you know... Just like there's a 10X engineer, there's 10X product insights, it feels?

Andrew Ng 19:54 Boy, that's a great question. I, I, I feel like it's gotta be more than a few hundred great product people. Maybe just as I think there are way more than a few hundred great AI people. You know, I think there are. But, but I think one, one thing I find is very difficult is, um, that user empathy or that customer empathy because, you know, to form a model of the user or the customer, there's so many sources of data. You know, you run surveys, you talk to a handful of people, you read market reports, uh, you look at people's behavior on other parallel or competing apps or whatever. But there's so many sources of data, but to take all this data and then to, you know, get out of your own head to form a mental model for what your, right, maybe ideal customer profile or some, some user you wanna serve, uh, think and act so you can very quickly make decisions to serve them better. That human empathy... One of my failures, one, one of, one of the things I did not do well, early phase of my career, um, for, for some dumb reason, I tried to make a bunch of engineers product managers. I gave them product manager training, and I found that I just foolishly made a bunch of really good engineers feel bad for not being good product managers, right? Which... But, but, but I found that one correlate for whether someone, you know, would have good product instincts is that very high human empathy where you can synthesize lots of signals to really put yourself in the other person's shoes, to then very rapidly make product decisions on how to serve them.

Sarah Guo 21:12 You know, going back to,

Sarah Guo 21:14 um, coding assistance, it's really interest-... I think it is, like, reasonably well-known that the, um, Cursor team, like, they make their decisions actually very, uh, instinctively, uh, versus spending a lot of time talking to users. And I think that makes sense if you are the user, and then, like, your mental model of, like, yourself and what you want is actually applicable to a lot of people. And similarly, like, I, I think, uh, I mean, you know, these things change all the time, but, uh, I don't think Cloud Code incorporates, despite, you know, scale of usage, feedback data today, um, from, like, a training loop perspective, and I think that surprises people because it is really just like, "What do we think the product should be at this stage?"

Andrew Ng 21:57 So it turns out one advantage that startups have is, um, uh, while you're early, you can serve kind of one user profile. Uh, today, you know, if, if you're, I don't know, like Google, right? Google serves such a diverse set of user personal analysis, you really have to think about a lot of different user personal analysis, and that adds complexity to the product changes. But when you're a startup trying to get your initial wedge in the market, you know, if you pick even one human, uh, that is representative enough of a broad set of users, and you just build a product for one user that you have, or one ideal customer profile, one, you know, hypothetical person, then you, you actually go quite far. And I think that, uh, for some of these businesses, be it Cursor or Cloud Code or something, if they have internal the, uh, mental picture of a user that is close enough to a very large group of prospective users, uh, that you can actually go really far that way.

Sarah Guo 22:46 The other thing

Sarah Guo 22:47 that I've observed, I'm curious if you guys see this in some of our companies, is just like the floor is lava, right? The ground is changing in terms of capability all the time, and the competition is also very fierce in the categories that are already obviously important and have multiple players. So, leaders who are really effective in companies a generation ago, um, are not necessarily that effective when recruited in- r- to these companies as they're scaling. Like, because the pace of... You know, there's a velocity of operation or the pace of change. It's interesting to see you say like, "I'm looking at what I was doing in, like, today and in 2022," and saying like, "Is that still right?" Versus if, you know, if you're an engineering leader or a go-to-market leader and you've like, built your career being really great at how that's done, that may not be applicable anymore.

Andrew Ng 23:32 I think it's a challenge for a lot of people. I know many great leaders in lots of different functions still doing things the way they were in 2022, and I think it, it's, it, it, it's just got to change. Wh- when, when new technology comes... I mean, you know, once upon a time there was no such thing as web search. Today, we... Uh, who are you... Would you hire anyone for any role that doesn't know how to search the web, right? And I think we're well past the point that for a lot of job roles if you can't use LMs in an effective way, you're just much less effective than someone that can. And it turns out, um, everyone in my team at AIFund knows how to code. Uh, and for everyone has a GitHub account. And I see for a lot of my team members, you know, when my, I don't know, um, uh, assi- assistant general counsel or my CFO or my front desk operator, when they learn how to code, they're not software engineers but they do their job function better because by learning the language of computers, they can now tell a computer more precisely what they want it to do for them, what the computer will do for them, and this makes them more effective at jo- their job function.

Sarah Guo 24:30 Mm-hmm.

Andrew Ng 24:30 I think the rapid pace of change is, uh, disconcerting to a lot of people. Uh, but I guess...

Sarah Guo 24:37 Mm-hmm.

Andrew Ng 24:37 I don't know. I, I, I feel like when the world is moving at this pace, we just have to change at the wor- at the pace in the world events-

Sarah Guo 24:43 Yeah, I've seen that, uh, to your point, show up in, um, hires particularly around product. So, uh... Or product and design. So one sort of later stage AI company I'm involved with-

Elad Gil 24:53 They were doing a search for somebody to run product and somebody to run design, and in both cases, they selected for people who really understood how to use some of the vibe coding/AI-assisted coding tools.

Andrew Ng 25:02 Mm-hmm.

Elad Gil 25:03 Because they're, they, they said your point, it's like you can prototype something so rapidly and if you can't even just mock it up really quickly to show what it could look like or feel like, or do in a very simple way, you're wasting an enormous amount of time talking and writing up the product requirements document and everything else. And so I do think there's a shift in terms of how do you even think about what processes do you use to develop a product or even pitch it, right? Like, what should you show up with to a meeting when you're talking about a product for the first time?

Sarah Guo 25:26 The whole thing, apparently.

Elad Gil 25:26 It's completely changed. Yeah, no, you should, you should have a prototype in some cases.

Andrew Ng 25:29 I should just give you an example. (...) engineers for a role and hired their, uh, interviewed someone with about 10 years of experience, you know, full stack, very good resume. Also interviewed a fresh college grad. Um, but the difference was the person with 10 years of experience had not used AI tools much at all. Fresh college grad had, and my assessment was the fresh college grad that knew AI would be much more productive and I decided to hire them instead. Turned out to be a great decision. Uh, now the flip side of this is the best engineers I work with today are not fresh college grads. They're people with, you know, 10, 15 or more years of experience, but they're also really on top of AI tools and that, those engineers are just completely in a class of their own. So I feel like I, I, I actually think software engineering is a harbinger of what will happen in other disciplines because the tools are most advanced in software engineering.

Elad Gil 26:16 It's interesting, one company that I guess both of us are involved with is called Harvey, and I led their series B. And when I did that, um, I called a bunch of their customers. And the thing that was most interesting to me about some of those customer calls was 'cause legal is notorious as being a tough profession for adopting new technology, right? There aren't a dozen great legal software companies. Those customers that I called, which were big law firms or people who were, you know, quite far along in terms of adopting Harvey, they all thought this was the future. They all thought that AI was really gonna matter for their vertical. And the main thing they would raise is questions like, "In a world where this is ubiquitous, suddenly instead of hiring a hundred associates, I only hire 10. And how do I think about future partners and who to promote if I don't have a big pool?" And so I thought that mindset shift was really interesting. And to your point, I feel like it's percolating into all these markets or industries and it's sort of slowly happening, but as industry by industry, people are starting to rethink aspects of their business in really interesting ways. And it'll take a decade, two decades for this transformation to happen. But as, uh, it's compelling to kind of see how people, like the earliest adopting verticals and something that the people were thinking deepest about it.

Andrew Ng 27:19 That should be really interesting. I th- I think, um, yeah, we should have a, uh, legal startup called AI that AI fund helped build. It's doing very well as well. Um, I think, I think the na- the nature of work in the future will be very interesting. So I feel like a lot of teams, um, wound up, you know, outsourcing a lot of work, right? Partly because of, uh, the costs. Um, but with AI and a- AI assistance, part of me wonders is a really small, really skilled team-

Elad Gil 27:46 Mm-hmm.

Andrew Ng 27:47 ... um, with lots of AI tools, is that gonna outperform a much larger, you know, and maybe lower cost team that, that may or may not be-

Sarah Guo 27:54 And they have less coordination cost.

Andrew Ng 27:56 Yeah. So a- actually, so, so some of the most productive teams I'm on, you know, that I'm a part of now is s- some of the smallest teams than, than... very small teams of-

Elad Gil 28:06 Mm-hmm.

Andrew Ng 28:06 ... really good engineers with lots of AI enablement, um, and very low (inaudible) as well together in person. So we'll see, we'll see how the world evolves. Too early to make a call, but you can see where, um, maybe thinking the world may or may not be headed.

Sarah Guo 28:20 I, uh,

Sarah Guo 28:21 work with several teams now, um, one of which is called Open Evidence and has like a pretty good penetration, like 50% of doctors in the US now, where it's an explicit objective in the company to try to be as small as possible-

Andrew Ng 28:34 Mm-hmm.

Sarah Guo 28:34 ... um, as they grow impact. And, you know, we'll, we'll see where these companies land because, you know, there's lots of functions that need to grow in a company over time. But that certainly wasn't an objective for, like-

Elad Gil 28:45 I've heard that objective a lot.

Sarah Guo 28:46 ... five years ago.

Elad Gil 28:46 I've actually, I heard that objective a lot in the 2010s and there's a bunch of companies that I actually think underhired pretty dramatically or stayed profitable and would brag about being profitable, but gross wasn't as strong as it could be. So I actually feel like that's a trap, partly from-

Sarah Guo 28:58 How would you calibrate them?

Elad Gil 28:59 ... talks about this.

Sarah Guo 28:59 Yeah.

Elad Gil 29:00 Um, it's basically really, it's, it's almost are you being, um, lackadaisical or too accepting of the progress that your company's making 'cause it's going just fine. It could be going much better, but it's still going great on a relative basis. And so you're like, "Oh, I'll keep the team small. I'll be super lean. I won't spend any money. Look at me, how profitable I am." And sometimes it's amazing, right? Capital efficiency is great, but sometimes you're actually missing the opportunity or not going as fast as you can. And, um, usually I think what happens is in the early stage of a startup life, you're competing with other startups. And if you're way ahead, it feels great, but eventually if there are incumbents in your market, they come in and the faster you capture the market and move up market, the less time you give them to sort of realize what's going on and catch on. And so often five, six, seven years in the life of a startup, you're actually competing with incumbents suddenly and they just kill you with distribution or other things. And so I think people really miss the mark. And you could argue that was kind of Slack versus Teams. That was, um, you know, there's a few companies I won't name, but I feel like they're so proud of their profitability and they kind of blew up. I guess on the design side that was Sketch, right, remember? They were-

Sarah Guo 29:59 Yeah. Bohemian Coding. Yeah.

Elad Gil 30:00 You know, they, they were based in the Netherlands. They were super happy. They were profitable. They were doing great. And then the Figma wave kind of came.

Sarah Guo 30:07 Do you think your companies stay this small?

Andrew Ng 30:09 What? Do I think what?

Sarah Guo 30:09 Do you think your teams stay this small?

Andrew Ng 30:11 Do I think my team stay this small?

Sarah Guo 30:12 Yeah.

Andrew Ng 30:13 What do you mean?

Sarah Guo 30:14 In terms of just efficiency of like-

Andrew Ng 30:17 Oh.

Sarah Guo 30:17 ... can, can you actually get to, you know, affect millions and billions of people with 10, 50, 100 person teams?

Andrew Ng 30:23 I think teams can definitely be smaller now than they used to be. But, uh, are we over-investing or under-investing? And then also, I think to your point, the, the, the analysis of market dynamics, right? If, if it's a... if it looks like a winner-take-all market, then the incentives just-

Elad Gil 30:37 Got to go.

Andrew Ng 30:38 Yeah. It's got to go.

Elad Gil 30:38 You got to move.

Andrew Ng 30:39 Yeah.

Elad Gil 30:39 Minecraft, I think when it sold to Microsoft was how many people? Like five people or something. And it sold for a few billion dollars and it was massively used. I think people forget all these examples, right?

Sarah Guo 30:48 Yeah.

Elad Gil 30:48 It's just this, oh, suddenly you can do things really lean. You could always do something, things lean before. The real question is how much leverage did you have in headcount? How did you distribute?What did you actually need to invest money behind? And then I would almost argue that one of the reasons small teams are so efficient with AI is because small teams are efficient in general. They didn't hire 30 extra crufty people who get in the way, and I think often people do that. If you look at the big tech companies, for example, right now-

Sarah Guo 31:11 Mm-hmm.

Elad Gil 31:12 ... many, not all of them, but many of them could probably shrink by 70% and be more effective, right? And so I do think people also forget the fact that, A, there's AI efficiency, B, there's sort of, um, high value capital being arbitraged into markets that normally wouldn't have them. Legal is a good example. Great engineers didn't want to work in legal, now they do because of things like Harvey. And then-

Sarah Guo 31:29 Or healthcare.

Elad Gil 31:30 Or healthcare, which again-

Sarah Guo 31:31 Yeah.

Elad Gil 31:31 ... suddenly you have these great people showing up. But I think also the other part of it is just small teams tend to be more effective, and AI helps you argue other reasons to keep teams highly small in performance, which I think is kind of under-discussed.

Andrew Ng 31:44 I feel like one of the... There's this other reason why that AI instinct is so important. Every one week, um, I had two conversations with two different team members. Uh, one person came to me to say, "Hey, Andrew, I'm going to do this. Can you give me some more headcount to do this?"

Elad Gil 31:56 Mm-hmm.

Andrew Ng 31:56 I said, "No." Later that week, I think independently, someone else, uh, very similar said, "Hey, Andrew, can you give me some budget to hire AI to do this?"

Elad Gil 32:05 Yeah.

Andrew Ng 32:05 I said, "Yes."

Elad Gil 32:06 (laughs)

Andrew Ng 32:06 And so that realization as you're hiring AI-

Elad Gil 32:08 Yeah.

Andrew Ng 32:08 ... not, you know, a lot more humans with this, you just got to have those instincts.

Elad Gil 32:11 Yeah, it's interesting.

Sarah Guo 32:12 If you

Sarah Guo 32:13 think of, um, what's happening in so- happening in software engineering as the harbinger for, like, the next industry transformations, uh, you spend a lot of time investing at the application level or, like, building things there, what, what do you think is next? Or what do you want to be next?

Andrew Ng 32:28 I feel like there's a lot of, uh... At the tooling level, I feel like-

Sarah Guo 32:31 I actually prefer a ranked list (laughs) -

Andrew Ng 32:32 Oh. Yeah.

Sarah Guo 32:33 ... you know, we're all-

Andrew Ng 32:33 Yeah.

Sarah Guo 32:33 ... investing in this stuff.

Andrew Ng 32:35 Th- you know, there's actually one, there's actually one thing I find really interesting which is where the economists doing a lot of studies-

Sarah Guo 32:40 Mm-hmm.

Andrew Ng 32:40 ... on whether the jobs, you know, at highest risk of AI disruption. I think, I think you, you are skeptical. I actually look at them sometimes for inspiration for, you know, where we should-

Elad Gil 32:49 Yeah.

Andrew Ng 32:49 ... find ideas to do our projects.

Elad Gil 32:50 That's true.

Andrew Ng 32:51 One, one of my friends, Erik Brynjolfsson, right, his, his, he, he and his company worked which we're involved in, is very insightful in the nature of work.

Sarah Guo 32:57 Yeah, I like him. Yeah.

Andrew Ng 32:58 Yeah. Good. I mean, good. So, so I, I, I find talking to that sometimes useful, although actually one of the lesson I've learned though is, uh, in view of top-down market analysis, I think AI will environment. There's so many ideas that no one's working on yet because the tech of it is still new. So one thing I've learned is, um, at AI Fund, we have an obsession with speed. All my life, I've always had an obsession with speed, but now we have tools to go even faster than we could. And so one of the lessons I've learned is, um, we really like concrete ideas. So if someone says, "I did a market analysis, AI will transform healthcare," that's true, but I don't know what to do with that.

Elad Gil 33:32 Mm-hmm.

Andrew Ng 33:33 But if someone, a subject matter expert or an engineer comes and says, "I have an idea. Look at this part of healthcare operations-

Elad Gil 33:38 Yeah.

Andrew Ng 33:38 "... and drive efficient all this." Like, "Oh, yeah, great. That's a concrete idea." I don't know if it's a good idea or a bad idea, but it's concrete. At least we could, you know, very efficiently figure out, do customers want this? Is it technically feasible? And get going. So I find that at AI Fund, um, when we're trying to decide what to build, we screen a long list of ideas to try to select, you know, a small number that we want to go forward on. Um, we, we, we don't like looking at ideas that are not concrete.

Sarah Guo 34:03 What do you think

Sarah Guo 34:04 investing firms or incubation studios like yours will not do two years from now? Like, not do manually, sorry.

Andrew Ng 34:10 I think there's a lot could be automated, but the question is, what are the tasks we should be automating? So for example, you know, we don't make follow-on decisions that often, right, 'cause our portfolio of some dozens of companies. So do we need to fully automate that? Probably not, because we've already looked at it. Very hard to automate. Um, I feel like doing deep research on individual companies and competitive research, that seems ripe for automation. Uh, so I, I sh- might, I don't know, I, I personally use whether OpenAI's Deep Researcher and other deep researcher types of tools a lot to just do at least the cursory market research things. Um, LP reporting, that is a massive amounts of paperwork that maybe we could simplify.

Sarah Guo 34:52 Yeah. Mm-hmm. I'm taking the strategy of general avoidance (laughs) .

Andrew Ng 34:57 (laughs)

Sarah Guo 34:57 Besides, you know, basic compliance. You know, one of my partners, uh, Bella, she worked at Bridgewater before, uh, where they had, like, an internal effort to take a chunk of capital and then try to disrupt what Bridgewater was doing with AI, um, and it's like, you know, macro investing. It's a very different style, but I think, uh, but I think it probably gives us some indications where the human judgment piece for a business, I think, is not obvious, like, does an entrepreneur have the qualities that we're looking for-

Andrew Ng 35:26 Mm-hmm.

Sarah Guo 35:26 ... when, you know, your resume on paper or your GitHub or, you know, what minor work history you have when you're a new grad, it's not very indicative. And so people have other ideas of doing this. Like, I know investors that are, like, you know, looking at recordings of meetings with entrepreneurs and seeing if they can get some signal out of, like, uh, communication style, for example. But I think that part is very hard. I do think you can be, like, programmatic about looking at materials, for example, and it's like ranking, you know, um, quality of, of teams overall.

Andrew Ng 36:00 There's actually one thing. I, I, I feel like, um, our AI models are getting really intelligent, but there's still some places where humans still have a huge advantage of AI as often if the human has his a- has additional context that for whatever reason the AI model can't get at. It could be things like meeting the founder and sussing out their, you know, just how they are as a person and their leadership qualities, their communication or, or whatever. Um, and those things, maybe reviewing video, maybe eventually we can get that context in AI model, but I find that all these things, like, as humans, you know, we do a...... background reference check and someone makes an offhand comment that we catch that, that affects the decision, then how does the AI model get this information? Especially when, you know, a friend will talk to me but they don't really talk to my AI model. So I find that there are a lot of these tasks where human have a huge information advantage still because they've not figured out the plumbing or whatever's needed to get information to the AI model.

Sarah Guo 36:57 The other thing I think is, like, very durable is, um, things that rely on a, like a relationship advantage. Right? If I'm convincing somebody to work at one of my companies and they worked at a previous company and they trust me because of it or whatever reason, like, you know, all the information in the world about why this is a good opportunity isn't the same thing as me being like, "Sally, you got to do this. It's going to work." It remains to be seen whether or not company building is actually that correlated with investment returns, but I do think that that side of it feels harder to, um, fully automate.

Andrew Ng 37:26 Yeah, yeah, yeah. No, yeah. Uh, yeah. I think, I think, like, trust because, um, people know and, you know, people do trust you. I trust you, right? Because you can only say so many things and it's very easy to lose trust, you know. So that, that makes sense.

Sarah Guo 37:39 (laughs) Yeah.

Andrew Ng 37:39 Actually one thing I'm curious of your take on is, um, you know, we increasingly see, um, highly technical people try to be, uh, first time founders. You know, set up the processes to, to, to set up first time founders, to learn all the hard lessons and all the craziness needed, right, to be a successful founder. I spent a lot of time thinking through that, how to set up founders for success when they have, you know, 80% of the skills needed to be really great but there's another, just a little bit that we can help them with.

Sarah Guo 38:08 That's a very manual process. (laughs)

Elad Gil 38:10 I don't sweat it.

Sarah Guo 38:11 You don't sweat it?

Elad Gil 38:12 I just view it as, like, a mix of peer groups. Like can you surround people with other people who are either similar or one or two steps ahead of them on the founding journey? And then the second thing is, um, complementary hire. I think in general one of my big learnings is, um, I feel like early in careers people try to complement or try to build up the skills that they don't have. In late in careers, they lean into what they're really good at and then they hire people to do the rest. And so if the company's working, I think you just hire people. Like Bill Gates would notoriously talk about his COO was always the person he learned the most off of, and then once he hit a certain level of skill, he'd hire his next COO.

Andrew Ng 38:45 I see, yeah.

Elad Gil 38:45 And so I almost view it through that lens for founders.

Andrew Ng 38:48 Yeah. Complementary hires makes sense.

Elad Gil 38:49 But I think the best way to learn something is to do it. And so that, therefore just go, you know, you'll screw it up, it's fine. As, as long as it's not existential to the business, who cares? (laughs)

Sarah Guo 38:58 I probably-

Elad Gil 38:58 So I tend to be very lackadaisical.

Sarah Guo 39:00 I probably think too many things are existential for companies.

Elad Gil 39:03 Yeah, it's something... It's like do you have customers and are you building product?

Andrew Ng 39:06 Most imp- Yeah. A- a- are you building a product that users love, right? And then of course go-to-market is important and all that is important, but just solve for the product first, then usually, sometimes you can figure out the rest too.

Elad Gil 39:17 I agree with that most of the time but not always. Yeah. I think there's lots of... There's some counterexamples, but yeah, I generally agree with you.

Andrew Ng 39:22 Yeah. Some- sometimes you can build a sucky product-

Elad Gil 39:25 Yeah.

Andrew Ng 39:25 ... and have a, you know-

Elad Gil 39:26 It's

Narrator 39:26 Just out of the gate.

Andrew Ng 39:26 ... sales channel you can force it through, but I'd rather not... That's not my default model of operation.

Elad Gil 39:29 Yeah, I don't love that either. I'm just saying right now, it doesn't work. (laughs)

Andrew Ng 39:32 (laughs)

Narrator 39:32 (laughs)

Elad Gil 39:32 There's a lot of really bad technology that-

Andrew Ng 39:34 Yeah. (laughs)

Elad Gil 39:35 ... is big companies right now.

Andrew Ng 39:36 Yeah. (laughs)

Elad Gil 39:37 (laughs)

Sarah Guo 39:37 Okay, if you have these, uh, you know, first time, very technical founders with gaps in their knowledge or skill set, uh, being like the core profile of folks you're backing again, like do you augment them somehow? Like what's, what helps them when they begin?

Andrew Ng 39:56 I think a lot of things. That's actually one thing I realized, that, you know, at, at venture firms, venture studios, we do so many reps that we just see a lot that even repeat founders have only done, like, twice in their life or even once or twice in their life. So, I find that, um, when my firm sits alongside the founders and shares our instincts on, you know, when do we get customer feedback faster? Are you really on top of the latest technology trends? How do you just speed things up? Uh, how do you fundraise? You know, mo- most people don't fundraise that much in their lives, right? Most founders just do it a handful of times. That helps even very good founders with things that because of what we do, we've had more reps at. And then I think, um, hiring others around them, peer group, I know these are things that you guys do, uh, I think there's a lot we could do. It, it turns out even the best founders need help. Um, so hopefully, you know, VCs, venture studios who can provide that to great founders.

Sarah Guo 40:50 Elot's wiser about this than I am. Uh, I mean, I, I can't help myself but, like, want to specifically try to upskill founders on a few things they have to be able to do, like recruiting, right? But I, uh, I would agree that the higher leverage path is absolutely like you can put people around yourself to do this and to learn it on the

Sarah Guo 41:08 job. Last question for you, what do you, what do you, uh, believe about broad impact of AI over the next five years you think most people don't?

Andrew Ng 41:15 I think many people will be much more empowered and much more capable in a few years than they are today. And the capability of individuals is probably, of those that embrace AI, will probably be far greater than most people realize. Um, two years ago, who would have realized that software engineers would be as productive as they are today when they embrace AI? I think in the future people will also do job functions and also for personal tasks. I think people who embrace this will just be so much more powerful and so much more capable than they probably can imagine.

Sarah Guo 41:49 Awesome.

Elad Gil 41:50 Very exciting, yeah.

Sarah Guo 41:51 Thanks, Andrew.

Elad Gil 41:51 Thanks for joining us.

Andrew Ng 41:51 Thanks. Thanks, thanks a lot, Lisa.

Narrator 41:53 (instrumental music)

Sarah Guo 41:54 Find us on Twitter @nopriorspod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.