The Ultimate AI Roundtable: What Happens Now in AI, Why Google are Vulnerable | E1085

1. 0:00 – 0:38

   Format setup: a “roundtable” built from contrarian clips across top AI leaders

   1. HSHarry Stebbings0:00

      Welcome to 20VC with me, Harry Stebbings. And for this Thanksgiving special, I wanted to bring some of the best minds in AI together for an amazing panel. The only thing is, this panel never really happened. What you're about to hear is the leading minds in AI, from the head of AI at Meta, the founder of Intercom, Stability, Runway, leading AI investor Tom Tunguz, all debate some of the core questions in AI. I pulled together some of those best moments and the most contrarian elements from their different episodes. Let me know what you think of this different style and format. I think it's really special and cool. You can do that on Twitter @HarryStebbings.

   2. RSRichard Socher0:36

      You have now arrived at your destination.

2. 0:38 – 1:08

   Will foundation models commoditize—and how many winners will exist?

   1. HSHarry Stebbings0:38

      So I want to start at the very core, the foundational model layer. And I want to ask, how do we see this layer in terms of the foundational model providers playing out? And will we see the commoditization of LLMs? Emad, I know that you have some strong opinions here. So Emad at Stability, handing the mic over to you.

   2. EMEmad Mostaque0:55

      I think that there's only going to be five or six foundation model companies in the world in three years, five years. I think it's going to be us, NVIDIA, Google, Microsoft, OpenAI, and, uh, Meta and Apple probably are the ones that train these models.

3. 1:08 – 1:59

   Do LLMs already feel interchangeable? Intercom’s real-world torture tests

   1. HSHarry Stebbings1:08

      What about you, Des? You're the founder of Intercom. Do you think we'll see the commoditization of LLMs?

   2. DTDes Traynor1:13

      I, I don't know if it's actually happened yet, if I'm clear, right? Like, we actually torture test all of the LLMs. It's not yet the case that they're all equal. We-

   3. HSHarry Stebbings1:19

      And when you compared OpenAI to the alternative providers-

   4. DTDes Traynor1:22

      Yeah.

   5. HSHarry Stebbings1:22

      ... what did the test show you?

   6. DTDes Traynor1:24

      It's basically the quality of conversation, and, like, does it fail any of our, like, hallucination tests? Does it fail any of our trustworthiness tests? Can it infer its own confidence?

   7. HSHarry Stebbings1:33

      Was it close though? Was there a wide-

   8. DTDes Traynor1:34

      Yeah, uh, it... Close and narrowing and, and, like, it's also a work in progress. All of these things are moving targets, right? So, like, we haven't even gotten around to maybe testing the latest and greatest of all the providers, which are increasing in number. You mentioned Mistral, there's also GLaMa, there's Anthropic, there's, like, Cohere, like, there's, like, a whole chunk of them. And it's a bit of work to go around and constantly be trying to find out, has anyone got... We only really care about better, right? Right now, we're not in cost optimization mode, we're just like, "Who's got the best?"

4. 1:59 – 2:41

   Open-source pressure and the inevitability argument for commoditization

   1. HSHarry Stebbings1:59

      Jeff Sibert, you're the founder of Digits. So you sit on top of these foundational LLMs and then fine-tune them with your own data. What do you think in terms of this commoditization of the foundational model layer?

   2. JSJeff Seibert2:11

      I certainly think we will. And this may not be a popular position. Obviously, OpenAI is charging ahead, sort of leading the way right now. I think the market forces at work mean there's just immense energy to have an open source equivalent. Meta appears to be highly motivated to open source its work. Many folks want to run these themselves and tune them themselves and so on. That is hard and expensive today, but I can't think of another thing in time in history where something hard and expensive in tech has lasted all that long. It's going to be commoditized.

5. 2:41 – 3:49

   Model size, lifespan, and rapid obsolescence: ‘none of today’s models in a year’

   1. HSHarry Stebbings2:41

      Okay, so if we go one layer deeper from just the commoditization of these models to actually how important is the size of these models, and then how do we think about the lifespan and longevity of these models? Emad, I know that you have quite a few thoughts on this. I'll start with you.

   2. EMEmad Mostaque2:55

      The reality is no models that are out today will be used in a year. So again, you see the order of magnitude improvement. PaLM last year was 540 billion parameters, then Chinchilla 67, and now 14. 540 to 14 is a big step. You see the quality of GPT-3 versus GPT-4.

   3. HSHarry Stebbings3:09

      Is there any extent to how low it can go?

   4. EMEmad Mostaque3:11

      We have no idea. You would have said this is impossible two years ago. You were like, "No way." You have a single file that's maybe a few hundred gigabytes that can pass every exam apart from English lit. There is no such thing as an unbiased model. DALLE-2, when OpenAI had that and they introduced a bias filter, any non-gendered word that had a random gender and a random ethnicity. So you typed in sumo wrestler and you get Indian female sumo wrestler. This is why you need national data sets, you need cultural data sets, you need personal data sets that can interact with these base models and customize to you and your stories, 'cause you and I both have our stories that make up our psyche and understand that context is so important to have AIs that can work for us, not on us.

6. 3:49 – 4:48

   Do we need giant models? Efficiency, local inference, and new system designs

   1. HSHarry Stebbings3:49

      Yann, help me out here. You run AI at Meta. We heard there cultural data sets, we heard national data sets. Just start, Yann, how important is the size of the model first?

   2. YLYann LeCun4:01

      You don't need those models to be very large to, to work really well. And I think it caused a bit of a epiphany for a lot of people realizing, oh, okay, maybe you need 1,000 GPUs running for a couple weeks to train it, the base system. In fact, that number is going down too (laughs) because people are figuring out how to do this more efficiently. But once it's pre-trained, you can use it for all kinds of stuff and you can fine-tune it really easily. And then at the end, you can run it on your laptop, right? That's kind of amazing. Or maybe on a desktop machine with a GPU in it or a couple GPUs. So I think it opened the minds of people to the fact that there is, like, enormous opportunities that really weren't thought to be possible before. And I think it's going to make even more progress because if we go towards the design of AI systems perhaps along the lines of what I described with objectives and planning, I think those systems could actually be even smaller to some extent.

7. 4:48 – 6:29

   Counterpoint: why scale still matters + ‘models aren’t the moat’ debate

   1. HSHarry Stebbings4:48

      That's so interesting, Yann. So you said there about it becoming less important to have larger and larger models. I, I'm really intrigued, Richard Socha, the founder at You, I know that you have a different take on this. So do you think that it is important in terms of the size of the models themselves?

   2. RSRichard Socher5:04

      It is super important. You just cannot train a single model for all of these different tasks with a small model. That's exactly why and how it would have always failed in the past.

   3. HSHarry Stebbings5:14

      Hmm. Some different opinions there. Chris, a founder at Runway, what are your thoughts on bluntly the defensibility in terms of the size of the models being used, and how do you think about that?

   4. CCChris (Runway co‑founder)5:25

      I find myself hearing a lot about models as the mode, and mode has been something that Silicon Valley has been discussing for, for some months now. I think models are not a moat. Models eventually don't matter. What matters most is the people building those models and how fast can you change and learn from those models. And so I don't think... That's why I go back, there's no one singular model that's going to rule them all.

   5. HSHarry Stebbings5:48

      So models are not the moat. Now, Jeff, founder at Digits, who we heard from earlier, I know that you maybe disagree with this slightly. So do you disagree with this in terms of models not being the moat? And how do you think about that data size and quality also being a moat?

   6. JSJeff Seibert6:03

      At the base LLM layer, the data size so far has been very correlated with performance. And so, right, the bigger the models, the more data, the more parameters, et cetera, the better they do. Now there's starting to be this counterpush of like, okay, can we compress them? Can we pull that back? Like, how do we maintain the performance improvements without the size? So I think that's a super interesting part of R&D. What I'm talking about is sort of the next tier of how do you fine-tune the models, and that's where actually I think the quality of data is most important.

8. 6:29 – 8:23

   Open vs closed: ‘recruit the world’s intelligence’ vs scale-and-usage moats

   1. HSHarry Stebbings6:29

      I mean, I think before we go deeper in terms of data quality and data size, I just want to ask in terms of the models themselves, there's a core challenge today in terms of two opposing ideologies, which is open versus closed. Yann, you know, you run all things AI at Facebook, or Meta. How do you feel about the open versus closed discussion? I know you've got some very strong opinions. Why does the future have to be open, not closed?

   2. YLYann LeCun6:53

      It's very simple. It's because no outfit, as powerful as they may be, has a monopoly on good ideas. If you do it in the open, you recruit the entire world's intelligence to contribute to things and having ideas, and ideas that you may not have thought about, which an outfit with 400 people has no chance of thinking about, or even a large company with 50,000 employees may not want to devote any resources to because they may not think it's useful in the long term or they have more urgency to take care of. So you give it away and then you have tons and tons of people, some of whom are undergraduate students or people, you know, in their parents' basement, so coming up with amazing ideas that you would never have thought about or willing to spend the time to crunch down the 7 billion weight LLAMA so that it runs on a Mac, on a laptop. I think that's why open source projects succeed, particularly when they concern basic infrastructure.

   3. HSHarry Stebbings7:44

      Now, I'm really intrigued. Duy, you're the founder of Contextual, which is essentially a contextual foundational model.How do you feel about this? 'Cause I know that you do have some opinions in terms of the open source versus closed.

   4. DFDuy (Contextual founder)7:56

      So I'm a big fan of open source, right? I would like it to be true that with open source we could just keep up with all of that. But I think that's just incredibly naive. OpenAI has this very deep understanding of how people want to use language models basically nobody else has, and they have this giant economy of scale where they can serve up language models very cheaply because they get so many requests coming in at the same time. So they have a giant moat.

9. 8:23 – 10:32

   Will open source catch GPT-4? Universities, ecosystems, and ‘good enough’ models

   1. HSHarry Stebbings8:23

      Richard, founder at you, you were chief data scientist at Salesforce before, I'm really intrigued, how do you feel about this lead position that OpenAI has that Due just illustrated there, and what that means then for the potential for an open source competitor to rise?

   2. RSRichard Socher8:37

      I mean, certainly you can't deny that OpenAI is ahead by a lot. I predicted that we'll have a GPT-4-equivalent model before the end of the year that's open source. Of course, GPT-4 keeps getting better and better, so my prediction was for the version we had, like, a few months ago, but I actually think that with models like LLaMA 2 from Facebook and everyone there, I do think open source will take over a lot of use cases, right? It's already getting close to GPT-3.5. When there's this much excitement and so many careers depending on understanding these models, imagine all the researchers in all these universities, right? They're all of a sudden kind of out of a job unless they have an LM that works really, really well and they can do useful things with it. They're not gonna just say, "Oh, let's just from now on run our entire research agenda on some closed API that we cannot analyze and understand and improve and publish papers on." So they need to have a model to exist, and those are all very, very smart people. Now they don't have as many resources usually. They can't train a single model for, like, $20 or $50 million, uh, 'cause they're in universities, but they're finding ways, they're collaborating, and, and they're probably going to work on foundational models that are fully open source. Uh, and we now see this with, like, surprisingly Facebook being at the very forefront of it. People will layer on top of that, make it better, and then there will be open source versions you can run on your phone, and those will get better and better over time. And, and so I'm, I'm quite bullish on the LMs in particular getting more and more commoditized. And yes, there will be a few foundational companies, you know, there's Cohere and Anthropic also, uh, behind OpenAI, working, um, very hard to catch up with them. Uh, and they did raise a lot of money too, and it's good to have some competition in that space, but my hunch is a lot of people will be okay with an open source model too.

10. 10:32 – 13:18

    Where does value accrue: infrastructure concentration vs application diversity

    1. HSHarry Stebbings10:32

       Okay, so some different thoughts there in terms of open versus closed. I think the next big debate is where does the actual value accrue? Is it in the infrastructure layer or is it in the application layer? Now, Tom Tunguz, you know, you founded Theory Ventures, which is one of the leading AI investors. When you think about infrastructure versus application layer in terms of where value accrues, how do you think about that?

    2. JSJeff Seibert10:53

       I ran this analysis. So in Web2, if you take the top three clouds and you look at their market cap, so AWS, GCP, and Azure, it's about a $2.1 trillion market cap just for the cloud businesses. And then if you take the top 100 publicly traded cloud companies, both on B2C and B2B sides, so Netflix and ServiceNow, they have equivalent market cap, about 2.1 trillion for both. So one's at the infrastructure layer, one's at the application layer. Market cap is basically equivalent. The difference is, at the infrastructure layer there are three businesses, and at the application layer there are 100. If the analogy holds, as an investor, the odds of success (laughs) are gonna be significantly higher at the application layer because the diversity of needs there is greater.

    3. HSHarry Stebbings11:28

       That's so interesting to hear about the concentration of value going to those three core providers in the infrastructure layer. Des, obviously with Intercom, you sit at the application layer. How do you think about this question of where value accrues infrastructure versus application layer?

    4. DTDes Traynor11:42

       I think right now a lot of value is going straight into the infra, like, as in we're handing it all out the back door to OpenAI in, in this case, right? In your first question about commoditization, if that happens, then the value starts to reduce there, right? Like, 'cause if there's more competitors offering the same thing, that's the beginning of commoditization, or at least you go from a monopoly, to an oligopoly, to ultimately perfect competition where, you know, your price goes down at each step. Th- that's how I expect that will play out. Unless OpenAI can continue to find mass market differentiation, which is always a hard thing to find, right? Like, like, as in the differentiation that all the customers care about, not just specifics, right? So let's assume over time, I believe that, like, just, you know, the amount of people investing in this space means that I think the infrastructure layer will gradually get a lot more AWS-like, right? Like, just, like, razor-thin margins. Well, sorry, not razor-thin margins. That's the wrong way to frame it. Price mixed with strength of, uh, product as a competitive, uh, weapon, if you know what I mean, right? Like, Amazon make good money off Intercom, you know, but they're also, like, we're all in on them and we're very... you, you know, we're very committed to them. So that's what the competitive battleground. There'll probably be three or four big providers, is my guess. Probably GCP, AWS, maybe OpenAI directly, and I'm sure Azure. That's h- how I think the, that layer will play out. And they'll end up in direct price competition with each other. Where does the value accrue beyond that? I think just differentiation. Value just generally follows differentiation around a stack. Whoever has the stuff you can't get anywhere else can charge the margins that no one else can charge. That sometimes means quality of software product. Sometimes it means quality of network or social network. Sometimes it means, like, we've got the most distributors in our marketplace. But, like, whoever has the differentiation that no one else can get is the person who can actually, like, basically charge the highest price.

11. 13:18 – 16:34

    Pricing and business models: from seats to ‘selling the work’ and outcome SLAs

    1. HSHarry Stebbings13:18

       That's so interesting. Des, you said there about price, you said about margin. I think another big question is, bluntly, what does the pricing model and the business model look like for the next generation of AI? I wanna bring in Miles Grimshaw at Benchmark, general partner at Benchmark, for this one. So Miles, what do you think in terms of the next generation of both pricing and business model for AI?

    2. DTDes Traynor13:37

       If AI is to be the force that it can be, I think you will get a new architecture, a new business model emerging from it versus what we only see right now, which is kind of a sustaining architecture, which is just that...... have a CoPilot, which fits on top.

    3. HSHarry Stebbings13:52

       What is the new architecture and what is-

    4. DTDes Traynor13:53

       Yeah.

    5. HSHarry Stebbings13:53

       ... the new business model?

    6. DTDes Traynor13:54

       I think the way to encapsulate would be this idea of selling the work, not the software. And that we'll move from a paradigm where you might think of it as moving from what we see right now is a CoPilot, and moving to what I think about as a control center, where we'll sell an SLA on work, not an SLA on uptime. And so we'll move from a world where we all, as users of software, kind of are like monkeys doing data entry usually. Like, well, that's most software. It's a, it's a database with a form on top of it for users to manage information, put information in, get information out. And-

    7. HSHarry Stebbings14:26

       And so an example is you set your objectives on your marketing spend and what you want in terms of CAC and LTVs, and then actually a s- marketing efficiency engine will go across channels, spend, and deliver you back results.

    8. DTDes Traynor14:39

       That might be an example, right? And so we'll move from a world where the users are doing all this work to a world where the application is doing a lot more of the work, right, where the AI, the notion of agents inside of it, et cetera, is doing it. And where right now, like, you go to any SLA for any software, you get uptime, you get support SLAs for questions and things like that. I think there's a world where we move to in the future where, like, an SLA maybe almost looks more like a BPO would in some sense. An SLA will be you wanted a efficiency of X on your market, we delivered that. You wanted this many leads from an SDR team, like, we do that. You wanted this sort of accounting and books closed by two days at the end of a quarter, we'll give you an SLA on that, not on the software is up. You'll go from CoPilot to, to control center. It'll be a UX for worker, which is dominant, to a UX for managers. You'll go from a seat add-on to software and labor, and you'll go from SLA on, like, reliability to SLA on outcomes, on work performance. That's what can be offered up by this. It's th- you see very little of it so far. I think it- AI is offering the potential for that architecture shift, and if we get that, th- the whole seat model, the whole the worker does it, and the product paradigm, the distribution in terms of who you can reach changes because ACVs change. And in that way, uh, I think it will be very different to mobile, which was mostly another UX, but the same architecture, same business models.

    9. HSHarry Stebbings16:01

       Des, as we know, you're the founder of Intercom, you know, you're at the forefront of customer service. How do you feel about the business model that will be prominent in this next wave of AI, and how you think about that today?

    10. DTDes Traynor16:12

        I think a lot of work is gonna get handed over to LLMs over the next five years. We're gonna start trying to, like, price against the work that's being done, not price against the seats or the employees, but just say, "How much is it worth for you to have all of your digital assets created dynamically?" Or, "How much is it worth for you to have, like, your customers get sub-second replies to common questions?" Like, that's the actual right way to think about pricing in the future.

12. 16:34 – 18:05

    Consumption vs seat-based pricing: demographic pressure, but AI may stay ‘a tool’

    1. HSHarry Stebbings16:34

       I want to bring in a new voice here. Christian Lange, you're the former founder of Tradeshift, now the president or co-founder of Beyond Work. I, I have to ask, how do you think about this consumption model pricing versus seat-based pricing question in the future?

    2. UCUnspecified female guest commentator16:48

       Well, I do think the world is changing. I think we're gonna absolutely move to consumption-based pricing. It's the only way. I think it's gonna be very hard to hold the moat on, on recurring revenue as it is today, because customers will wanna see more value, and they'll wanna see a more soft ramp-up to that value. But I also think on the work piece, look at the demographic problem. Like, most of the people I talk to, even if they wanted to replace the workers they have in their shared service centers today one-to-one, they can't, 'cause that generation of young people, they don't wanna go in and sit in front of a computer and type formulas into Workday every day. So, so we gotta completely change the work experience in next 5 to 10 years, or we're gonna run out of people to do the work.

    3. HSHarry Stebbings17:21

       So everyone seems in line that we're moving to consumption-based pricing, away from seat-based pricing. But Jeff Seibert at Digits, we have one who thinks otherwise. So Jeff, I wanna move to you. This is great. Why do you think that maybe we'll stay in the realm that we're in today?

    4. JSJeff Seibert17:36

       So this may be just me, but I, I very much see AI as a tool, not a product. And so it's a, it's a technology, it's like your database, it's like memcache back in the day. And so because of that, I don't think it'll change how people price in specific industries. Like, if your market does per-seat pricing, that'll probably stay. If your business and product does consumption pricing, that'll probably stay, and you'll have to work that into how you use the AI. I think it'll be commoditized and seen as technology within a couple years.

13. 18:05 – 21:59

    Copilots vs ‘pilot’ agents: incumbent distribution vs a new interaction paradigm

    1. HSHarry Stebbings18:05

       I do have to ask, we see CoPilots everywhere. Miles, you mentioned CoPilots earlier. How do we feel about the rise of CoPilots, are they an incumbent strategy? How useful are they? Christian, I, I wanna start with you. Christian Lange, over to you. How useful are CoPilots, and are they an incumbent strategy?

    2. UCUnspecified female guest commentator18:24

       I mean, who wants a CoPilot? I, I wanna be a pilot (laughs) . Like, and I wanna have a pilot. Like, I don't wanna... (laughs) ... have a co-pilot. I don't want to be inside an application. I don't care how many co-pilots you have. The problem is not to have an AI help you navigate an application that's shit. The better solution is to remove the application that's shit and just talk straight with AI. And I think the co-pilot metaphor, right, I mean, what happens when we have 10,000 Clippys all just talking to each other? And for you to work, you have to go and work in all of these Clippys, and you're going to have to tell Clippy A to go talk to Clippy B about that thing you have over in Clippy C.

    3. HSHarry Stebbings18:56

       (gasps) .

    4. UCUnspecified female guest commentator18:56

       I mean, it's going to be worse than it is now.

    5. HSHarry Stebbings18:58

       Myles Grimshaw at Benchmark, you were the first in this discussion to bring up co-pilots. Are co-pilots an incumbent's strategy?

    6. DTDes Traynor19:05

       I think co-pilot is an incumbent's strategy. Incumbents own distribution, they own data, they own the UX, and they own a business model that all aligns to a co-pilot. Co-pilot as GitHub Co-pilot, right, like inline suggestions, think of it like how most, go to, go to any Microsoft product right now. Every Microsoft product now has a co-pilot experience in sort of a sidebar, an autofill, things like that, right? Where the UX, the, the core product is a layer on top of it, right? It's immediately added in, which is also totally incumbent strategy. And it's still about sort of supercharging that worker, but still where every user has a seat and every user's doing most of the work. And it works probably, you know, if you think about the evolution here, the models, most of what's rolled out might not be good enough for some of this yet, right? But that's what will come around the corner. You know, if you think back to Salesforce disrupting C- Siebel, Salesforce launched, like, five years after Netscape launched. Like, it might take a moment for that to happen, but the co-pilot, this idea of, "I'm still the pilot, I'm still the user controlling everything and it's sort of, like, giving me assistive suggestions," like GitHub Co-pilot, fits into the UX of incumbents, it fits into the business model of incumbents, and they already control all that distribution. The opportunity offered up to a startup, being a co-pilot for something else, like, probably won't be that amazing. And there might be pockets of it where it can really work, but the opportunity to disrupt is to be orthogonal to the incumbents.

    7. HSHarry Stebbings20:25

       Yann, we said there about it not being the best position to maybe be in for startups. Christian mentioned, "We don't want a co-pilot, we want a pilot." How do you feel about co-pilots and intelligent assistants, and is it brighter than maybe we think in terms of what we have coming?

    8. YLYann LeCun20:40

       Let's imagine a future where everyone can talk to their intelligent assistant. That, that system will have pretty close to human level intelligence, probably more accumulated knowledge than most humans. You know, they could translate in any language and give you a quick summary of yesterday's newspaper and things like that, right? Explain mathematical concepts to you, things like that. So, people are probably going to use this almost exclusively in the future for their interaction with the digital world. You're not gonna go to Google or Wikipedia, you're just gonna talk to your assistant. The only way to do this properly is for the base infrastructure for those assistants. They will be so pervasive, so much will ride on those systems, that I don't think anyone will accept that those assistants be behind a event horizon in a private company. They will insist that the infrastructure is open. They will insist also that the vetting process by which those systems are trained be something maybe like Wikipedia. We tend to trust Wikipedia, sometimes with a grain of salt, but we tend to trust Wikipedia because there is a vetting process so that whenever a, an article is modified, some editor kind of check on it, and then the changes are accepted or not, things like that. So you can imagine that the sort of common repository of all human knowledge that will be our assistants will be constructed through some sort of crowdsourcing process, perhaps similar to Wikipedia, where you're gonna have a bunch of people training those systems and fine-tuning them so that whatever they... and so they produce, uh, correct.

14. 21:59 – 29:49

    Incumbent outlooks: Apple’s on-device privacy advantage; Google’s innovator’s dilemma; Amazon’s moves

    1. HSHarry Stebbings21:59

       Yann, you mentioned Wikipedia. I do want to move to the company level element and discuss which companies or incumbents are best positioned. Now, I want to start with Apple. And Des, I know you have some strong thoughts here. So, what are your thoughts on how Apple are positioned for this next wave in the next three to five years of AI?

    2. DTDes Traynor22:16

       I think Apple will make massive, massive strides forward, but I'm kind of disappointed how long it's taken them. But I think-

    3. HSHarry Stebbings22:21

       And this is my point. What makes you say that? Because so far, we're kind of left searching. (laughs)

    4. DTDes Traynor22:25

       Yeah, yeah, for sure. You have to assume Apple's a really well-run company, and you have to assume that there's a head of AI in there, and you have to assume that they're training LLMs and they're looking for LLMs that can possibly run on their hardware natively, and not even have to talk to the cloud. And Apple are very privacy-focused, so they're gonna get all that shit correct. And you have to assume it's all gonna work with your AirPods, your watch, and your phone, and all that sort of stuff. That that's, like... I would be shocked if that wasn't the case. So then, what will they win, is the question. I think what they'll win is this idea of Siri might finally become useful. Siri is currently not useful because it doesn't really have enough smarts. But I think when Siri can be as conversational as ChatGPT and can take actions on the device, it'll change the entire interaction model across desktop and iOS as well, in huge ways. So I think Apple will win there.

    5. HSHarry Stebbings23:10

       Emad at Stability AI, are you with Des in saying that Apple will win there? Or do you have a different perspective? Apple's a black box, right? And so they could surprise us all. But let's face it, Siri's crap. But they have all the ingredients in place, the identity architecture, the secure enclave, other things. Neural engine. Stable diffusion was the first model ever optimized on the neural engine, et cetera. But let's see that one. Geoff at Digits, Apple's a black box. We get it, agree. How do you feel about how they're positioned for the next few years?

    6. JSJeff Seibert23:36

       So Apple, of course, is super focused on privacy. They don't want your data to leave the device. The only way to do that with AI is if you can fit a machine learning model on the device and keep all the data there. I bet Apple can and will. And so if you project forward five years, if they get to the point where they can run a sufficiently large LLM on your iPhone, then OpenAI's out of the picture. Don't even need to hit their servers. It's just on your phone.

    7. HSHarry Stebbings24:00

       Okay, so Apple's in a very strong position looking forwards. Tom Tunguz, they're your former employer. How do you see Google playing out over the next few years?

    8. JSJeff Seibert24:08

       I didn't believe that chat would replace search, but I think it... for many use cases, it will. And I think Google had a rude awakening where, I don't know, for 20, 25 years, they were uncontested, and now all of a sudden there's this disruptive technology, to some extent they developed in-house but ignored. So it's the classic innovator's dilemma. And so this technology went to other places and now is, uh, challenging the hegemony, the monopoly power, and that is so exciting if you think about, like, the ads ecosystem, like, the B2C ecosystem has been relatively quiet over the last 10 years because of that dominance of Facebook and Google. And now all of a sudden you have a technology and a re-platforming where all that market share is conceivably up for grabs. You could create a new travel agency, you could create a new shopping experience, you could create a new Stack Overflow, you could create a new social experience based on chat. And so it's wide open. And when you have a golden goose, when you have an incredible business model, you're always faced with a choice of disrupting yourself and destabilizing the ship or waiting until somebody destabilizes it for you. And it's... I think as a leadership team, it is so difficult.... to have the discipline to say, "We are going to destabilize this ourselves." That's what happened.

    9. HSHarry Stebbings25:12

       Des, what are your thoughts on decisions from leadership team at Google and how they stand today looking forward?

    10. DTDes Traynor25:19

        I feel like Bard unfortunately felt like we had to release this because ChatGPT was getting a lot of traction. It didn't feel like, "We've actually cracked search again. We've reinvented ourselves all over again." You know? They need to have that sort of a, um, Jay-Z like, "Allow me to reintroduce myself," moment, right? Where they come back and they say like, "Google 2.0 is here."

    11. JSJeff Seibert25:36

        Yeah.

    12. DTDes Traynor25:36

        Now you're pulling into the real potential problem, which is are they willing to risk it all to win it all, right? Like are they willing to disrupt themselves or are they happy to take the like long slow decline into obsolescence or irrelevance or whatever, right?

    13. HSHarry Stebbings25:48

        What would you do?

    14. DTDes Traynor25:48

        You, yeah.

    15. HSHarry Stebbings25:49

        Genuinely. I mean, it's easy to say here but if you were CEO of Google and you have shareholders on Wall Street.

    16. DTDes Traynor25:55

        I kind of hate what I'm about to say but I'm going to say it because I think it's what I, it's what I-

    17. HSHarry Stebbings25:58

        (laughs) .

    18. DTDes Traynor25:58

        ... would feel compelled to do. I'd be scrambling to find ways that companies can sponsor injections into the LLM. So if I said like, "Who is the best footballer in the world?" And the answer is clearly, like, Lionel Messi. But you could say something like, "According to transfer market, the answer is Lionel Messi," right? Like, and that's with a sponsored injection, right? And then, like, you could augment the Bard style answer with such injections so that it's presenting you facts that it, it is kind of disowning, right? Like it's kind of saying, "Hey, like, this isn't the LLM deducing this, this is just what we think. This is what we've been paid to say whenever we talk about this type of thing." That's kind of the attack factor I'd go on. And I'd try and float that with all my big ad buyers and sort of say, "Hey look, let's be honest. The world is going to go to LLMs. Even if this doesn't work, we have to give it a lash." And then I'd try and like get some sort of traction going for that type of ad model. Then I'd explain to the investors, "We have to move to this because the alternative is the company basically starts to be on a ticking clock," right? I can totally imagine Google doing a thing where they give away effectively free Android phones powered by the fact that they now control the intent layer, which is when you say, "Okay Google, call me a taxi," Google goes and they get you whatever taxi which comes from whichever of the highest paid provider.

    19. HSHarry Stebbings27:05

        I mean, wow, I love that idea of giving away phones for free and what that enables for Google. Jeff, I think that you're quite pessimistic about Google's future in terms of the next wave of AI. Jeff Seibert obviously from Digits, how do you think about the next few years for Google in this respect?

    20. JSJeff Seibert27:19

        I think Google is by far the most vulnerable because again their business model is pretty binary, right? Search is all their revenue and so if that gets damaged they're in a huge problem and they've been slow to react. They combined two different MLAI teams, they've just punted Gemini into Q1 which tells me it's not doing very well, uh, so I would be nervous. They need to go all in on it. I don't think they have a choice. I, I agree with you, I think it's existential for them because if AI replaces search, their golden goose has been killed. It is way more effective to kill your own golden goose than let, than watch someone else do it. And again, I mean, going back to Apple, it reminds me of the iPod Nano. Apple killed their most popular product because they knew there was better tech coming. I think Google needs to get bold and do the same.

    21. HSHarry Stebbings28:01

        Richard Socha, I'm fascinated to hear your thoughts on this. Obviously with you.com you compete in many ways with Google on the search side. Richard, how do you think about this and Google's next steps?

    22. RSRichard Socher28:11

        We don't see Google change and become a chat first search engine. They have some features somewhere else but the main Google experience is the same. That big change will be hard for Google too because they make $500 million a day with privacy invading advertisements on that page. And so you don't just willy nilly change most of that page and you get rid of the five, six ads that are on top of that page followed by a bunch of SEO'd microsites that are not as good as the ads so people click more on the ads. Like, you don't just replace all of that with a chat, right?' Because you would just lose hundreds of millions of dollars a day. And so there is still some innovator's dilemma that will not make them change their main experience overnight.

    23. HSHarry Stebbings28:52

        Okay, so it's pretty universal that it's existential for Google and they have to change and innovate, otherwise their cool golden goose is in trouble. If we switch tacks and just look at Amazon, how do we think about the next few years for Amazon and how they've done so far? If we start with you, Emad at Stability, I'd love to hear your thoughts.

    24. EMEmad Mostaque29:08

        Amazon have moved faster than I think they've moved before. Amazon's interesting because they're an engineering organization so they have self-driving cars, they have satellite internet, because once they've got it and they can take it from research to engineering it's there. And one of the struggles they've had is that it's not moved from the research side yet, it's still evolving on research so they're like, "What do we do now?" But they are inclusive. Jeff Bezos said for his first 100 billion in revenue he envisioned half of it being proprietary and half of it being marketplace and they're having the same approach with Bedrock and things.

    25. HSHarry Stebbings29:33

        Des Traynor, I'd love to hear your thoughts. How do you think about what will be a strategic next move for Amazon and whether we should be wary or not?

    26. DTDes Traynor29:41

        I'd be wary of Amazon so like I could, I could see Amazon just like flat out like buying Anthropic and being like, "Let's just make this part of the EC2 cluster."

15. 29:49 – 33:23

    Society, jobs, and regulation: LeCun’s optimistic ‘renaissance’ view

    1. HSHarry Stebbings29:49

       Now I want to finish with an eye to the future and I want to finish with Yann LeCun. A lot of people suggest that we should be concerned about AI, its impact on jobs, its impact on society. How do you feel about this, the concern, and how we should think about the next few years of AI and its role in society moving forwards?

    2. YLYann LeCun30:07

       People are kind of extrapolating if we let those systems do whatever, we connect them to internet and they can do whatever they want, they're going to do crazy things and stupid things and perhaps dangerous things and we're not going to be able to control them and it's, they're going to escape our control and they're going to become intelligent just because they're bigger. And that's nonsense. No economist believes this. No economist believes we're going to run out of job because no economist believes that we're going to run out of problems to solve or requirement for human creativity and human communication and stuff like that. This is going to create as many jobs as it makes disappear and, and those jobs, by the way, are going to be more productive. So overall, technology makes people more productive. In other words, for the same amount of hours worked, you produce more wealth, okay? But every technological revolution, unless it's accompanied by political changes and social changes, generally profit a small number of people, at least temporarily, right? That happened in the industrial revolution in the late 19th century where few people became extremely rich and a lot of people were exploited and then society changed and there were like social programs and income tax and high tax for richer people and stuff like that which the US has backpedaled on this but not Europe. So there is a question of how you distribute the wealth, if you want, okay? How do you organize society so everyone profits from it? But that's a political question, it's not a technology que- question and it's not new, it's not caused by AI, it's just caused by technological evolution, right? It's not a recent phenomenon. AI is going to bring a new renaissance for humanity, a new form of enlightenment if you want, because AI is going to amplify everybody's intelligence. Every one of us will have a staff of people who are smarter than us and know most things about most things, so it's going to empower every one of us. It's going to make us more creative because we're going to be able to produce text, art, music, videos without necessarily having all the technical skills that are currently required for doing those things, and so exercise our creative juices. So that's the positive side. There are risks, there's no question, but it's not like those risks... Don't believe the people who tell you that those risks are inevitable or that they will inevitably lead to catastrophe. That's just not true. Place yourself in 1920, who would have thought that a mere 50 years later you could cross the Atlantic in a few hours in complete safety, you know, at near the speed of sound? And would people seriously want to ban aviation or call for regulation of jet engines before jet engines existed? I mean, that's kind of insane. So I'm not against regulation. The, there should be regulation of AI products, particularly the ones that involve making critical decisions for people. But regulating or slowing down research is complete nonsense. (music)

    3. HSHarry Stebbings32:39

       I mean, I absolutely love doing that show. For me, it was so interesting bringing all the different thoughts and opinions together. I would love to hear your thoughts. Did you enjoy the show? You can say if you didn't like it. Let me know on Twitter @harrystebbings. We have so many more of these that we can do, whether it's on price sensitivity for venture deals, reserve decision-making, best and worst investments. There are so many where we could bring awesome, awesome opinions together in a really cohesive episode like this. So let me know what you think, and I can make more or less of them. I love your thoughts. I do this for you, and what you want always comes first. So let me know. And oh my God, we have an amazing show coming on Monday with Keith Rabois and Mike at Traba. And so stay tuned for that, 'cause that is a fantastic show.

Episode duration: 33:23