AI Eats the World: Benedict Evans on the Next Platform Shift

1. 0:00 – 1:07

   Intro

   1. BEBenedict Evans0:00

      ChatGPT has got eight or nine hundred million weekly active users. And if you're the kind of person who is using this for hours every day, ask yourself why five times more people look at it, get it, know what it is, have an account, know how to use it, and can't think of anything to do with it this week or next week. The term AI is a little bit like the term technology. When something's been around any-for a while, it's not AI anymore. Is machine learning still AI? I don't know. In actual general usage, AI seems to mean new stuff.

   2. ETErik Torenberg0:27

      And AGI seems [chuckles] new scary stuff.

   3. BEBenedict Evans0:30

      AGI seems to be a bit, a little bit like this. Like either it's already here and it's just more software, or it's five years away and will always be five years away. We don't know the physical limits of this technology, and so we don't know how much better it can get. You've got Sam Altman saying, "We've got PhD level researchers right now," and Demis Hassabis says, "No we don't. Shut up." Very new. Very, very big. Very, very exciting. World changing things tend to lead to bubbles. So yeah, if we're not in a bubble now, we will be.

   4. ETErik Torenberg0:56

      [upbeat music] Benedict, welcome back to the a16z Podcast.

   5. BEBenedict Evans1:02

      Good to be back.

   6. ETErik Torenberg1:03

      We're here to discuss your latest presentation, AI Eats the World. So for

2. 1:07 – 3:12

   AI's Impact, Platform Shifts and Historical Comparisons

   1. ETErik Torenberg1:07

      those who haven't read it yet, may-maybe we can go share the high level thesis and, and maybe contextualize it in light of recent AI presentations. I'm curious how, how your thinking ha-ha-ha-has evolved.

   2. BEBenedict Evans1:19

      Yeah, it's funny, one of the slides in the deck references a conversation where I had with a big company CMO who said, "We've all had lots of AI presentations now."

   3. ETErik Torenberg1:26

      [laughs]

   4. BEBenedict Evans1:26

      "Like we've had the Google one and the... We've had the Google one and the Microsoft one. We've had the Bain one and the BCG one. We've had the one from, from, from Accenture and the one from our ad agency. Um, so now what?"

   5. ETErik Torenberg1:37

      [laughs]

   6. BEBenedict Evans1:37

      So, um, what's... It, it, it's sort of a ninety-odd slide. So there's kind of this, th-th-there's a bunch of different things I'm, I'm trying to get at. One of them is, I think, just to say, well, if this is a platform shift or more than a platform shift, how do platform shifts tend to work? What are the things that we tend to see in it? And how many of those patterns can we see being repeated now? And of course, those, some of the patterns that come out of that are things like bubbles, but another, others are that lots of stuff changes inside the tech industry. And, you know, there are winners and losers, and people who were dominant end up becoming irrelevant. And then there are new billion, trillion dollar companies created. But then there's also, what does this mean outside the tech industry? Because if we think back over the last waves of platform shifts, there were some industries where this changed everything and created and uncreated industries. But there are others where this was just kind of a useful tool. Like, so, you know, if you're in the newspaper business, that had a very different impact. The last t-thirty years looked very different to if you were in the cement business, where, you know, the internet was just kind of useful, but didn't really change the nature of your industry very much. And so what I tried to do is give people a sense of, well, what is it that's going on in tech? How much money are we spending? What are we trying to do? What are the unanswered questions? What might or might not happen, um, within the tech industry? But then outside technology, how does this tend to play out? What seems to be happening at the moment? How is this manifesting into tools and deployment and new use cases and new behaviors? And

3. 3:12 – 6:12

   Generative AI: Potential and Challenges

   1. BEBenedict Evans3:12

      then as we kind of step back from all of this, how many times have we ge-- Again, how many times have we gone through all of this before? You know, the... It's funny, I went on a podcast this summer, and I, the sort of opening line, I said something like, "Well, you know, I'm a centrist. I think this is as big a deal as the internet or smartphones, but only as big a deal as the internet or smartphones." And there's like two hundred YouTube commenters underneath saying, "You know, this moron, he doesn't understand how big this is." And I think, well-

   2. ETErik Torenberg3:35

      Those are pretty big

   3. BEBenedict Evans3:35

      ... the internet was kind of a big deal.

   4. ETErik Torenberg3:37

      [laughs]

   5. BEBenedict Evans3:37

      It was kind of a big deal. Um, and, you know, I sort of finish the deck by, by looking at elevators 'cause I, I live in an apartment building in Manhattan, and we have an attended elevator, which means it's, there's a hand... There's no buttons. There's an accelerator and a brake, and the, the doorman gets in and drives you to your floor in a streetcar. And in the fifties, Otis deployed automatic elevators, and then you get in and you press a button, and they marketed it by saying, "Ah, it's called electronic politeness."

   6. ETErik Torenberg4:04

      [laughs]

   7. BEBenedict Evans4:04

      Your... Which means the infrared beam.

   8. ETErik Torenberg4:06

      [laughs]

   9. BEBenedict Evans4:07

      And today when you get into an elevator, you don't say, "Ah, I'm using an electronic elevator." It's automatic. It's just a lift, which is what happened with databases and with the web and with smartphones. And I kind of think now, this is funny, I did... I've done a couple of polls on this in LinkedIn and Threads of like, is machine learning still AI? 'Cause AI is kind of, and AI is, the word AI, the term AI is a little bit like the term technology or automation. It see- It only kind of applies when something's new. When something's been around any-for a while, it's not AI, AI anymore. So like databases certainly aren't AI. Is machine learning still AI? Uh, I don't know. I mean... And there's obviously, there's like an academic definition where people say, "This guy's an idiot, and of course, I'm going to explain the definition of AI." But then the, in, in actual general usage, AI seems to mean new stuff.

   10. ETErik Torenberg4:58

       Yeah. And AGI seems, [chuckles] you know, like new scary stuff. Um-

   11. BEBenedict Evans5:04

       Yeah, it's funny. There's, I was thinking about this. There's, there's an old theologian's joke that, um, the problem for Jews is that you wait and wait and wait for the Messiah and he never comes, and the problem for Christians is that he came and nothing happened.

   12. ETErik Torenberg5:13

       [laughs]

   13. BEBenedict Evans5:14

       Like, you know, the world didn't change, like there was still sin, you know, like the, like the... For all practical purposes, nothing happened. And AGI seems to be a bit, a little bit like this, like either it's already here, and so you've got Sam Altman saying, "We've got PhD level researchers right now," and Demis Hassabis says, "What? No we don't. Shut up."

   14. ETErik Torenberg5:31

       [laughs]

   15. BEBenedict Evans5:32

       A-And so either it's already here and it's just more software, or it's five years away and will always be five years away.

   16. ETErik Torenberg5:40

       Yeah. Yeah. It's, um, it's a journey. L-L-Let's compare back to previous platform shifts because some people, you know, look at, you know, something in the internet and say, "Hey, there were net new trillion dollar companies, you know, F-Face-Facebook and Google, um, that, that were created from it," and just sort of all sorts of new em-emerging winners. Whereas they look at something like mobile and say, "Hey, you know, there were big companies like Uber and Snap and, and, uh, Instagram and WhatsApp, but these were, you know, these were billion dollar outcomes or, or tens of billion dollar outcomes, but really the big winners werewere

4. 6:12 – 8:28

   AI's Market Dynamics and Investment

   1. ETErik Torenberg6:12

      in fact Facebook and Google. Um, and, and so in some sense, mobile perhaps was sustaining. Um, you feel free to quibble with the definition of, you know, sustaining disruptive, but sustaining in the sense that maybe more of the value went to incumbents or, or, or companies that existed prior to the, to the, to the shift. I'm, I'm, I'm curious how you think about AI in, in, in light of that in terms of is it enabling-- is more of the gains coming from, you know, net new-- going to come to net new companies like, like OpenAI and Anthropic and others that, that follow? Or, um, you know, are more of the gains going to be captured by, you know, Microsoft and, and, and, and Google and Facebook and, and Meta and, you know, companies that existed prior?

   2. BEBenedict Evans6:53

      So I think it's... Well, there's several answers to this. One of them is like you kind of have to be careful about like framings and structures and things because you end up arguing about the framing and the definition rather than arguing about what's gonna happen. And, you know, they're all useful, but they've all, they've all got holes in them. And, you know, what, what, what mobile did was it kind of it, it, it shifted us. You know, there's a bunch of things that it changed fundamentally. It shifted us from the web to apps, for example, and it gave everybody in the world a ca-- a phone. Uh, it gave everybody in the world a pocket computer. So even today, there's less than a billion consumer PCs on Earth, and there's something between five and six billion smartphones. And, um, it made possible things that would not have been possible without it, whether that's TikTok or arguably, I think things like online dating. And, you know, the, the-- you can map those against dollar value. You can also map those against kind of structural change in, in consumer behavior and access to information and things. And I think you could certainly argue that Meta would be a much smaller company if it wasn't for mobile, for example. So, you know, you can kind of argue the puts and calls on, on, on this stuff a lot. Um, there's certainly, you know, not all platform shifts are the same. And you know, you can do the sort of standard sort of teleology of say, well, there were mainframes and then PCs and then the web and then smartphones. But you kind of want to put SaaS in there somewhere, and you kind of want to put open source in there, and maybe you want to put databases. And so, you know, these are kind of useful framings, but like they're not predictive. They don't tell you what's gonna happen. They just kind of give you one way of understanding what seemed some of the patterns that, that, that we have here.

5. 8:28 – 10:22

   AI Deployment and Use Cases

   1. BEBenedict Evans8:28

      Um, and of course, the big debate around generative AI is this is just another platform shift or is it something more than that? And of course, the problem is we don't know, and we don't have any way of knowing other than waiting to see. So this may be as big as PCs or the web or SaaS or, or open source or something, or it may be as big as computing. And then you've got the very overexcited people living in group houses in Berkeley who think, you know, this is as big as fire or something. Well, well, well, great. Um, but, but, but does this create new companies? I mean, you go back to the mobile. You know, there was a time when people thought that blogs were going to be a dif-different to the web, which seems weird now. Like Google needed like a separate blog search. This was seriously, this was a thing. Um, there was a time when it was really not clear, and I think you kind of generalized this point. You go back to the internet in the mid-'90s. You know, we kind of knew this was gonna be a big thing. We didn't really know it was gonna be the web. So before that, we didn't know it was gonna be the internet. We knew there were gonna be networks. We weren't-- didn't know. It wasn't clear it was gonna be the internet. Then it wasn't clear it was gonna be the web. Then it wasn't really clear how the web was gonna work. And, you know, when, when Netscape launched, like Mark Zuckerberg was in junior high or, you know, elementary school or something. And, you know, Larry and Sergey were students and like Amazon were the bookstore. So you can know it but not know it. And you could make the same point about smartphones. Like it was, we knew everyone was gonna have an Internet-connected thing in their pocket, but it was not clear it was basically going to be a PC from this has-been PC company from the '80s and a search engine company. It was not clear it wasn't gonna be Nokia or Microsoft. See, I think you have to be super careful in like predict-- making, making kind of deterministic predictions about this. What you can do is say, "Well, when this stuff happens, everything changes," and that's happened five or ten times before.

   2. ETErik Torenberg10:13

      I'm curious how you got conviction in th-this idea or wh-what's the prediction that, hey, AI is gonna be as big as the internet, which of course is pretty big, but I'm not yet,

6. 10:22 – 19:33

   AI's Future and Speculations

   1. ETErik Torenberg10:22

      I, Benedict, I'm not yet at the conviction that it's gonna be any bigger. I'm curious what, what sort of in-inspires that sort of, uh, you know, sort of stat-statement, and then also what might change your mind either way, you know, that it might not be as big as the internet, because of course, the internet was obviously very big, uh, but also that, hey, perhaps it might be bigger.

   2. BEBenedict Evans10:40

      Well, so I think, you know, I don't wanna... I remember I made a diagram of kind of S curves kind of going up slightly, and someone said, "Well, what's the axis on this diagram?" I, you know, I don't wanna kind of get into, you know, is this, is this five percent bigger than, than Internet or is it twenty percent bigger? I think the question is more like, is it another of these industry cycles, or is it a much more fundamental change in, in what technology can be? Is it more like computing or electricity as a sort of structural change rather than here's a whole bunch more stuff we can do with computers? I think that's sort of the, the, the question. And there's a funny sort of disconnect, I think, in, in looking at debates about this within tech, because, you know, I watched this, this, this, this one of the, um, OpenAI livestreams a couple of weeks ago, and they spend the first twenty minutes talking about how they're gonna have like human-level, PhD-level AI researchers like next year. And then the second half of the stream is, "Oh, and here's our API stack that's going to enable hundreds and thousands of new software developers, just like Windows," and in fact, literally quote Bill Gates. And you think, well, those can't kind of both be true. Like, either I've got a thing which is a PhD-level AI researcher, which by implication is like a PhD-level CPA.

   3. ETErik Torenberg11:47

      Yeah.

   4. BEBenedict Evans11:48

      Or I've got a new piece of software that does my taxes for me, and well, which is it? Either this thing is going to be like human level and some-- and that's a very, very challenging, problematic, complicated statement. Or this is going to let us make more software that can do more things the software couldn't be. And I think there's a real like schizophrenia in conversations around this because like scaling laws and it's gonna scale all the way. And meanwhile I'm going, "Here look how good it is at writing code." And again like, well, is it writing code or do we not need software anymore?'Cause in principle, if the models keep scaling, nobody's gonna write code anymore. You'll just all say to the model, like, "Hey, can you do this thing for me?"

   5. ETErik Torenberg12:26

      Yeah. Is it a little bit of a hedge or like a sequencing thing or?

   6. BEBenedict Evans12:29

      Well, it's a, it's some of it's a sequencing thing.

   7. ETErik Torenberg12:33

      Yeah.

   8. BEBenedict Evans12:33

      But, you know, in principle, if you think this stuff is gonna keep scaling, like why are you investing in a software company?

   9. ETErik Torenberg12:38

      Yeah. [laughs]

   10. BEBenedict Evans12:38

       Like, [laughs] 'cause, you know, we'll just have this like gold in a box that can do everything.

   11. ETErik Torenberg12:43

       Right.

   12. BEBenedict Evans12:43

       And, and, and I think this is, this is the, the, the kind of the funny kind of challenge, and this is, I think, really the, the fundamental way that this is different from previous platform shifts, is that with the internet or with mobile or indeed with moba- mainframes, like you didn't know what was gonna happen in the next couple of years. You didn't know that Ama- what Amazon would become, and you didn't know how Netscape was gonna work out, and you didn't know what next year's iPhone was gonna be, and ten years ago when we cared about that. But you kind of knew the physical limits. Like you knew in nineteen ninety-five, you knew that telcos were not gonna give everybody gigabit fiber next year. And you knew that the iPhone wasn't gonna like have a year's battery life and unroll and have a projector and fly or whatever. But we don't know the physical limits of this technology because we don't really have a good theoretical understanding of why it works so well, nor indeed do we have a good theoretical understanding of what human intelligence is. And so we don't know how much better it can get. So you can do, you could do a chart and you could say, "Well, you know, this is the roadmap for modems and this is the roadmap for DSL, and this is how fast DSL will be." And then you can make some guesses about how quickly telcos will deploy DSL, and then you can say, "Well, clearly we're not gonna be able to replace broadcast TV with streaming in nineteen ninety-eight." But we don't have an equivalent way of modeling this stuff to know what is the fundamental capability of it going to look like in three years. Um, which gets you to these kind of slightly vibes-based forecasting where no one really knows. So, you know, Geoff Hinton says, "Well, I feel like," and Demis Hassabis says, "Well, I feel like," but no one knows.

   13. ETErik Torenberg14:13

       And then Karpathy goes on Dwarkesh's podcast and says, "I feel like, you know, it's a decade out." [laughs]

   14. BEBenedict Evans14:17

       Yeah, I know. Well, I saw this, this meme of, um, of what's his name, Ilya Sutskever, but like where he says, like, "The answer will reveal itself."

   15. ETErik Torenberg14:24

       [laughs]

   16. BEBenedict Evans14:24

       And somebody like memed, I w- I'm gonna say photoshopped-

   17. ETErik Torenberg14:27

       Yeah

   18. BEBenedict Evans14:27

       ... but of course it wouldn't have been photoshopped, turned him into a Buddhist monk wearing like an orange, like an orange outfit.

   19. ETErik Torenberg14:32

       [laughs]

   20. BEBenedict Evans14:32

       "The future will reveal itself."

   21. ETErik Torenberg14:34

       [laughs]

   22. BEBenedict Evans14:34

       Well, [laughs] but this is the problem. We don't know. We don't have a way of mo- of modeling this.

   23. ETErik Torenberg14:39

       Yeah. And so let's connect this to sort of the, you know, the upfront investment that some of these companies are making. Um, because we don't know, you know, is there a risk of over-investment leading to some, you know, potential, uh, you know, bubble-like mechanics? Or h-h-how do you think about that, that question?

   24. BEBenedict Evans14:56

       Well, deterministically very new, very, very big, very, very exciting worlds changing things tend to lead to bubbles.

   25. ETErik Torenberg15:03

       Yeah.

   26. BEBenedict Evans15:04

       And you... I don't think anybody would dispute that you can see some bubbly behavior now and, you know, you can argue about what kind of bubble, but again, like, that doesn't have very much predictive power. And, you know, one of the, the features of bubbles is that when everything's going, you know, everything goes up all at once and everyone looks like a genius and everyone leverages and cross-leverages and does circular revenue, and that's great until it's not. Um, and then you get a kind of a ratchet effect as it goes back down again. Um, so yeah, if we're not in a bubble now, we will be. I remember Mark Andreessen saying, you know, "Nineteen ninety-seven was not a bubble, ninety-eight was not a bubble, ninety-nine was a bubble." Um, are we in ninety-seven now or ninety-eight or ninety-nine? I, you know, if we could predict that, you know, we'd live in a parallel universe. Um, I think, you know, b- to the-- there's I suppose maybe kind of two more specific, more, more, more tangible answers to this. The first of them is we don't really know what the compute requirements of this stuff are going to be. And forecasting that, except like more, and forecasting that feels a lot like trying to forecast like bandwidth use in the late nineties. Now imagine if you're trying to do the algebra on that, and you say, "Well, this many users, you know, how much bandwidth does a web page use? How will that change? How will that change if bandwidth gets faster? What happens with video? What kind of video? What bandwidth? What, what bitrate of video? How long do people watch a video? How much video?" And then you'd like, you'd, you could build the spreadsheet and it would tell you what bitrate would, what global bandwidth consumption would be in ten years, and then you could try and use that to back-calculate how many routers is this gonna, gonna sell. And you could get a number, but it wouldn't be the number. You know, there'd be a, you know, a hundredfold range of possible outcomes from that. And you could, you know, you could make the same point about algebra of, of consumption now. So, you know, right now we have a bunch of rational actors saying, "Well, this stuff is transformative and a huge threat, and we can't keep up with demand for it now, and as far as we know, the demand is going to keep going up." And, you know, we've had a variety of quotes from all of the hyperscalers basically saying the downside of not investing is bigger than the downside of over-investing. Um, that, um, or that kind of thing always works well until it doesn't.

   27. ETErik Torenberg17:22

       Yeah.

   28. BEBenedict Evans17:22

       Um, and I saw a slightly strange quote from Mark Zuckerberg saying, "Well, if it turns out that we've over-invested, we can just resell, resell the capacity." And I thought, let me just, like-

   29. ETErik Torenberg17:32

       [laughs]

   30. BEBenedict Evans17:32

       ... stop you there, Mark. 'Cause if it turns out that you can't use your capacity-
7. 19:33 – 29:27

   Generative AI in Practice

   1. BEBenedict Evans19:33

      this question. The first of them is, I think we've had this sort of a, a bifurcation of what all the questions are. So there are now very, very detailed conversations about chips and then very, very detailed conversations about data centers and about funding for data centers, and then about what is a, a new enterprise SaaS company built on AI, what margins will it have, and how much money does it need to raise? And so there are venture capital conversations, and so there are many different conversations w-within which like, I don't know anything about chips. You know, I can spell ultraviolet, but, like, I don't know what, like, an ultraviolet process is. Um, it's like it's more, it's more, more violet. So I don't know. Um, [laughs] and so you've got this, you know, it's like the, the Milton Friedman line, "No one know-knows how to build a pencil." You've got the right, you know, we've got this-

   2. ETErik Torenberg20:18

      Yeah

   3. BEBenedict Evans20:18

      ... you know, it's turned into deployment. I think a, a, a second answer might be, I think there's two kinds of AI deployment, Generative AI deployment. One of them is there are places where it's very easy and obvious right now to see what you would do with this, which is basically software development, marketing, um, point solutions for many very boring, very specific enterprise use cases. And also basically people like us, which are people who have kind of very open, very free form, very flexible jobs with many different things, and people who are always looking for ways to optimize that.

   4. ETErik Torenberg20:58

      Yeah.

   5. BEBenedict Evans20:58

      And so you get people in Silicon Valley who are like, "You know, I spend all my day-day time in ChatGPT. I don't use Google anymore. You know, I've replaced my CRM with this." Um, and you kind of... And then you obviously people who write, if you're writing code, this works really well if you're in marketing, you know, all these stories of big companies where, you know, they're making three hundred assets where they would have made thirty. Um, and then Accenture and Bain and McKinsey and Infosys and so on sitting and solving very specific problems inside big companies. Then there's a whole bunch of other people who look at it and they're like, "It's okay." And you go and look at the usage data and you see, okay, ChatGPT has got eight or nine hundred million weekly active users. Five percent of people are paying. And then you go and look at all the survey data and, you know, it's very fragmented and inconsistent, but it all sort of points to like something like ten or fifteen percent of people in the developed world are using this every day. Another twenty or thirty percent of people are using it every week. And if you're the kind of person who is using this for hours every day, ask yourself why five times more people look at it, get it, know what it is, have an account, know how to use it, and can't think of anything to do with it this week or next week.

   6. ETErik Torenberg22:15

      Hmm.

   7. BEBenedict Evans22:17

      Why is that?

   8. ETErik Torenberg22:18

      Yeah.

   9. BEBenedict Evans22:19

      Is it because it's early? And it's not like a young people thing either, incidentally. So is that just because it's early? Is it because of the error rates? Is it because you have to map it against what you do every day? And one of the, the analogy I always used to use, which isn't in the current presentation, I've used in previous presentations, is imagine you're an accountant and you see software spreadsheets for the first time. This thing can do a month's of work in ten minutes, almost literally.

   10. ETErik Torenberg22:47

       Yeah.

   11. BEBenedict Evans22:49

       You wanna change-- You wanna recalculate that DCF, that ten-year DCF with a different discount rate. I've done it before you finished asking me to, and that would have been like a day or two days or three days of work to recalculate all those numbers. Great. Now imagine you're a lawyer and you see it, and you think, "Well, that's great. My accountant should see it. Maybe I'll use it next week when I'm making a table of my billable hours, but that's not what I do all day." And Excel is doesn't use, do things that a lawyer can do every day. And I think those, there's this other class of person that's like, "I'm not sure what to do with this." And some of that is habit, some of that is like realizing, "No, instead of doing it that way, I could do it this way." But that's also what products are. Like, every entrepreneur who comes into a16z when I was there from two thousand and fourteen to two thousand and nineteen, and I'm sure now, like, you could look at any company that comes in and say that's basically a database.

   12. ETErik Torenberg23:48

       Yeah.

   13. BEBenedict Evans23:48

       That's basically a CRM. That's basically Oracle or Google Docs.

   14. ETErik Torenberg23:54

       Yeah.

   15. BEBenedict Evans23:55

       Except that they've realized there's this problem or this workflow inside this industry and worked out how to use a database or a CRM or basically concepts from five, ten, twenty years ago and solve that problem for people in that industry and go in and sell it to them and work out how they can get it to use it. And so this is why, you know, you look, look, look at data on this, that, you know, depending on how you count it, a typical big company today has four to five hundred SaaS apps in the US. Four to five hundred SaaS applications, and they're all basically doing something you could do in Oracle or Excel or email.

   16. ETErik Torenberg24:31

       Yeah.

   17. BEBenedict Evans24:33

       A-and that's the other side... I'm monologuing, I'm afraid, but, like, this is the other side of what is-- what do you do with these things? Do you just go to the bot and ask it to do a thing for you? Or does an enterprise salesperson come to your boss and sell you a thing that means now you press a button and it analyzes this process that you needed, that you never realized you were even doing?

   18. ETErik Torenberg24:57

       Yeah.

   19. BEBenedict Evans24:58

       And I feel like that's, I mean, that's why there are AI software companies.

   20. ETErik Torenberg25:04

       That's right.

   21. BEBenedict Evans25:05

       Really. [chuckles] And isn't that what they're doing? They're unbundling ChatGPT, just as the enterprise software company of ten years ago was unbundling Oracle or Google or Excel.

   22. ETErik Torenberg25:13

       Do you have the view that, you know, what Excel did for, for, for accountants, um, you know, we're, we're, uh, sort of AI is now doing for, for coders, um, a-and developers, but hasn't quite figured out that sort of, you know, daily critical workflow for, for other job positions and so it's unclear for people who aren't developers, you know, why I should be using this for many, many hours a day, or...?

   23. BEBenedict Evans25:35

       I think there's a lot of people who don't have tasks that work very well with this.

   24. ETErik Torenberg25:40

       Yeah.

   25. BEBenedict Evans25:41

       And then there's a lot of people who need it to be wrapped in a product and a workflow and tooling and UX and someone to come and say, "Hey, have you realized you could do it with this?" Um, I had this conversation with, um, in the summer with, with, with, with Balaji, who's another a-- former a16z person, and he was making this point about validation, that can you... Because these things still get stuff wrong, and people in the Valley often kind of hand wave this away. But, you know, there are questions that have specific answers where it needs to be the right answer or one of a limited set of right answers. Can you validate that mechanistically? Um, if not, is it efficient to validate it with people? So, you know, with a marketing use case, it's a lot more efficient to get a machine to make you two hundred pictures and then have a person look at them and pick ten that are good than to have, um, people make ten good images or a hundred, you know, even more. If you're gonna make five hundred images and pick a hundred that are good, that's a lot more efficient than having a person make a hundred images. Um, but on the other hand, if you're doing something like data entry, and this-- I wrote something about this, about, um, about Open- OpenAI launched Deep Research. OpenAI launched Deep Research. Their whole marketing case is it go- goes off and collects data about the mobile market. I used to be a mobile analyst. The numbers are all wrong. The, the, their use case of, "Look how useful this is," their numbers are wrong. And in some cases, they're wrong because they've tran-- literally transcribed the number incorrectly from the source. In other cases, it's wrong because they've used a source that they shouldn't have used. But, like, if I'd have asked an intern to do it for me, that intern would probably have picked that. And to my-- the point about, you know, verification, if you're gonna do data entry, if I'm gonna ask a machine to copy two hundred numbers out of two hundred PDFs, and then I'm gonna have to check all two hundred of those numbers, I might as well just do it myself.

   26. ETErik Torenberg27:27

       Yeah.

   27. BEBenedict Evans27:28

       So you've got, like, a whole swirling matrix of how do you map this against existing problems? But the other side of it is how do you map this against new things that you couldn't have done before? And this comes back to my, my point about platform shifts because, you know, you know, I see people looking at ChatGPT or looking at, at generative AI and saying, "Well, this is, this is useless 'cause it makes mistakes." And I think that's kind of like looking at, like, an Apple II in the late '70s and saying, "Could you use these to run banks?" To which the answer is no. But that's kind of the wrong question.

   28. ETErik Torenberg28:04

       Right.

   29. BEBenedict Evans28:05

       Like, could you build video edi- professional video editing inside Netscape? No. But that's the wrong question.

   30. ETErik Torenberg28:12

       Right.
8. 29:27 – 31:29

   New Behaviors and Market Opportunities

   1. ETErik Torenberg29:27

      Right. And, and on, on mobile s- you know, some of the new use cases, you know, were g- you know, getting in strangers' cars, you know, we mentioned Lyft and Uber or sort of, you know, dating people you met via an app or, um, sort of, um, you know, lending your h- spare bedroom out, um, you know, e-et cetera. And, and those were net new companies that, that, that, you know, were built around those behaviors. And I think for AI, there's still the questions of, you know, what are those net new behaviors? We're, we're starting to see some in terms of, you know, people en-engaging and talking with, you know, chatbots in-instead of humans or, or, um, or in addition. Um, and then there's the question of, A, are these done by the, uh, model providers that, that currently exist, or are these done by, you know, net new companies both on, you know, sort of enterprise and, and consumer?

   2. BEBenedict Evans30:09

      Well, this is always the question, is how far up the stack does the new thing go?

   3. ETErik Torenberg30:12

      Yeah.

   4. BEBenedict Evans30:12

      Um, and you know, I was, I was, I was talking about this with another former, former a16z person who pointed out that, like, in the, the, the mid-'90s, um, people kind of argued that, well, you know, the operating system does all of it, and the a- Windows apps are basically just kind of thin Win32 wrappers.

   5. ETErik Torenberg30:27

      Yeah.

   6. BEBenedict Evans30:28

      And, you know, Office is basically just, you know, a thin Win32 wrapper, like all the important stuff is being done by the OS, whether it's, you know, the document management and printing and storage and display, which all stuff that used to be done by apps like, you know, on DOS, the apps had to do printing, the apps had to manage the display. You move to Windows, like ninety percent of the stuff that the app used to do is now being done by Windows.

   7. ETErik Torenberg30:46

      Yeah.

   8. BEBenedict Evans30:46

      And so Office is just like a thin Win32 wrapper, and all the hard stuff is being, being done by the OS. And it turns out, well, that was again, it's like frameworks are useful, but that's not maybe, maybe not a useful way of thinking about what's going on. And the same thing now, like how much does this need single dedicated understanding of how that market it works or what that market is and what you would do with that? Um, I mean, I remember when we were at a16z, there was an in-investment in a company called Everlaw, which is cloud, um, cloud-- legal discovery in the cloud.

   9. ETErik Torenberg31:18

      Yeah.

   10. BEBenedict Evans31:18

       And so machine learning happens, and so now they can do translation. Are they worried that lawyers are gonna say, "Well, we don't need you guys anymore. We're just gonna go and get a translate app and a sentiment analysis app from AWS."

9. 31:29 – 32:05

   Understanding Law Firms' Needs

   1. BEBenedict Evans31:29

      Like, no, that's not how law firms work. Law firms wanna buy a thing that solves-- They wanna buy legal discovery, software management. You know, they don't wanna, you know, go out and write their own, buy, buy-- do API calls. I mean, very, very big law firms might, but you know, typical law firm isn't gonna do that. People buy solutions, they don't buy technologies. And the same thing here, like how far up the stack do these models go? Um, how much can you turn things into, um, a widget? How much can you turn things into an LLM request? And how much, no, does it turn out that you need that dedicated

10. 32:05 – 33:40

    The Role of User Interfaces

    1. BEBenedict Evans32:05

       UI? The funny thing is you can see this around Google, 'cause Google had this whole idea that everything would just be a Google query, and Google would work out what the query was. And guess what? You know, now you want me-- There's Google Flights is not a Google query. You know, they use certain point... And, and one of the, one of the interesting things about this, and I think it's interesting to think about w-what a GUI is doing, that some of what a GUI is doing, and the obvious thing that a GUI is doing is that it enables Office to have five hundred application, uh, five hundred features, and you can find them all. Or at least it's po-- You don't have to memorize keyboard commands. You can now have effectively infinite features, and you can just keep adding menus and dialog boxes, and eventually, you know, you run out of screen space for dialog boxes. But, like, you can have hundreds of features without people needing to memorize keyboard commands. But the other side of it is you're in that dialog box, or you're in that screen, in that workflow in Workday or Salesforce or whatever the enterprise software is, whatever your software or, or, or the airline website or, or Airbnb or whatever it is, and there aren't six hundred buttons on the screen. There's seven buttons on the screen because a bunch of people at that company have sat down and thought, "What is it that the user should, uh, be asked here? What questions should we give them? What choices should there be at this point in the flow?" Um, and that reflects a lot of institutional knowledge and a lot of learning and a lot of testing, a lot of really careful thought about how this should work. And then you give somebody a raw prompt, and you just say, "Okay," you just tell the thing how to do the thing, and you're like, but you've kind of got to shut your eyes, screw your eyes up, and think from first principles, how does this all of this work?

11. 33:40 – 35:26

    Machine Learning and Interns

    1. BEBenedict Evans33:40

       It's kind of like I always used to talk about machine learning as giving you infinite interns.

    2. ETErik Torenberg33:44

       Yeah.

    3. BEBenedict Evans33:44

       So, you know, imagine you've got a task and you've got an intern, and the intern doesn't know what venture capital is.

    4. ETErik Torenberg33:53

       How helpful are they gonna be?

    5. BEBenedict Evans33:54

       And like they... And they don't know that companies publish quarterly reports and that we've got a Bloomberg account that lets us look up multiples and that then you should probably use, um, PitchBook for this data and rather than using Google. This is my point about deep research, like, no, you should use this source and not that source. Um, do you want to have to work that out from scratch, or do you want a bunch of people who know a lot about this stuff to have spent five years working out what the choices should be on the screen for you to click on it?

    6. ETErik Torenberg34:30

       Yeah.

    7. BEBenedict Evans34:30

       I mean, it's the old user interface saying the computer should never ask you a question that you should have to work out, that it should know by itself. You go to a blank raw chatbot screen, it's asking you literally everything. It's not just asking you one question. It's asking you absolutely everything about what is it is that you want and how you're gonna work out what to-- how to do it.

    8. ETErik Torenberg34:51

       The... And so, you know, you're mentioning Chat... You know, you wrote about, uh, ChatGPT isn't sort of a product as much as this chatbot dis-is disguised as a, as a product. I, I'm curious, you know, wh-when we sort of look back at this sort of, you know, platform shift, do you think that there will be another sort of iPhone m-- sort of S pro-- or Excel-esque product that kind of defines the, the, the feature, the sort of platform shift in a way that ChatGPT won't or, or, or is it sort of that the world has to catch up to how to use Cha-ChatGPT or, or something like ChatGPT?

    9. BEBenedict Evans35:25

       So both

12. 35:26 – 39:43

    The Evolution of Tech Products

    1. BEBenedict Evans35:26

       of these can, both of these can be true because there was a lot of like it took time to realize how you would use Google Maps and what you could do with Google and how you could use Instagram, and all of these products have evolved a huge amount over time. So some of it is like you grow towards realizing what you could do with this. Like you realize that's just a Google query now. You realize that you could just do it like that, and you realize I spent, you know, hours doing this, and I just realized, oh, I could actually just make a pivot table.

    2. ETErik Torenberg35:50

       Yeah.

    3. BEBenedict Evans35:50

       Um, the other side of it is then, but you're still then expecting people to work it out themselves from first principles and, you know, it's kind of useful to have somebody really a hundre-a hundre- a thousand, ten thousand really clever people sitting and trying to work out what those things are and then showing it to you as a pr- as a product. I think another side to this is like, you know, there are always these precursors. So like there were lots of other things before Instagram.

    4. ETErik Torenberg36:19

       Yeah.

    5. BEBenedict Evans36:19

       You know, YouTube didn't start as YouTube. It started as video dating, I think. Um, there were lots of, of attempts to do online dating that all kind of worked until Tinder kind of pulled the whole thing inside out. And so there were always lots of things, what's the phrase? Local maxima. In fact, this is where we were, particularly with the iPhone, um, before, 'cause I was working in mobile for the previous decade. Um-It didn't feel like we were waiting for a thing. It felt like it was kind of working. Like every year the networks got faster and the phones got better, and you got a little bit better every year. And we had apps, and we had app stores, and we had 3G, and we had cameras, and stuff seemed to be, you know, every year it was a bit better. And then the iPhone arrives and it just, you know, just, you know, blow the chart, kind of, you know, you've got this line doing this, and then there's a line that does that. Although remember also the iPhone took like two years before it worked because, you know, the price was wrong and the feature set was wrong, and the distribution model didn't quite work. Um, and so, yeah, you, you know, you can think you're, you know, you can think everything is going well and then something comes along and you realize, "No. Oh, no, no, no, that's..." Which is the same for Google. You know, like search was a thing before Google, it just wasn't very good. Um, so there were lots of, so there was lots of social stuff before Facebook and, you know, that was the thing that, that catalyzed it. So, you know, I just think deterministically this whole thing is so early that it feels like of course there are going to be, you know, dozens, hundreds of, of new things. Otherwise a16z should just kind of shut down and give the money back to the LPs because-

    6. ETErik Torenberg37:49

       Right

    7. BEBenedict Evans37:49

       ... the founding, the founding models will just do the whole thing and like I don't think you're gonna do that, at least I hope not.

    8. ETErik Torenberg37:53

       No, no, no. If we have any regrets from the last few years, it's, it's, it's not going bigger. I, I think we didn't fully appreciate how much specialization there would be a-a-across, uh, sort of, you know, whether it's voice or image generation or, or take any sort of subsector that there would be, um, you know, net new companies created that would be better than the, than the, the, the model providers that, that there would be even multiple model providers at the, that, or that in every category, um, you know, one, one thing we've always in the web two era, we always bet on the category winner, right? And, and the category winner would take mo-most of the market, but th-these markets are so big, um, and the, the, the-- there's so much expertise and specialization that in, that there... One, there can be winners in, in every category. It's not just sort of the, the model providers taking everything, but that even in every category, including the model providers, there can be multiple winners and increasing, you know, s-s-specialization and, and the, the markets are just big enough to, to contain m-m-multiple winners.

    9. BEBenedict Evans38:49

       I think that's right, and I think, you know, the categories themselves aren't clear.

    10. ETErik Torenberg38:55

        Right.

    11. BEBenedict Evans38:55

        And, you know, many, you know... Things you think this is a category and it turns out, no, it's actually that whole other thing. And the categories kind of get unbundled and bundled and recombined in different ways. I mean, I remember I was a student in nineteen ninety-five and, um, though I think I had like four or five different web browsers on my PC, web, web servers on my PC. 'Cause I mean, Tim Berners-Lee's original web browser had a web editor in it 'cause he thought this was kind of like a network drive, and it was a sharing system and didn't realize, not, not really a publishing system. So you would have your web pages on your PC, and you'd leave your PC turned on, and that would be how your colleagues would look at your Word documents or your web pages. And so again, like we just don't know how... And, and I just kind of keep coming back to this point. I feel like most of the questions we're asking at the moment are probably the wrong question. I'm picking up on, on a, on a strand within what you just

13. 39:43 – 43:17

    The Competitive Landscape of AI

    1. BEBenedict Evans39:43

       said, though. The interesting... One of the things I'm sort of thinking about a lot is looking at, looking at OpenAI, um, because, you know, I'm, I'm, I'm sort of fascinated by disconnections and we've got this interesting disconnect now, which is that, you know, if you look at the benchmark scores, so you've got these general purpose benchmarks where the models are basically all the same. And if you're... Yes, if you're spending hours a day and then, and you've got this opinion about, "Oh, I like Claude's tone of voice more than I like GPT, and I like GPT 5.1 more than GPT 4.9," or whatever the hell it's called. If you're using this once a week, you really don't notice this stuff. And the benchmark scores are all roughly the same and, but the usage isn't. It's basically the, the only con-- Claude has basically no consumer usage, even though on the benchmark score it's the same. And then it's ChatGPT, and then halfway down the chart it's, um, Meta and Google. And the funny thing is, you know, that you read all the AI newsletters again, then like Meta's lost, they're out of the game, they're dead. Mark Zuckerberg is spending a billion dollars a researcher to get back in the game. But from the consumer side, well, it's, it's distribution. And the interesting thing here is that you've got... What I'm kind of circling around is if the model for a casual consumer user certainly is a commodity and there's no network effects or winner-takes-all effects yet, they may, those may emerge, but we don't have them yet. And things like memory aren't network effects, they're stickiness, but they can be copied. Um, how is it that you compete? Do you just compete on being the recognized brand and adding more features and services and capabilities and people just don't switch away? Which is kind of what happened with Chrome, for example. There's not a network effect for Chrome, but it... and it's not actually any better much, but maybe it's a bit better than Safari. But, you know, you use Chrome because you use Chrome. Or is it that you get left behind on distribution or network effects that emerge somewhere else, and meanwhile you don't have your own infrastructure? So I suppose what I'm, what I'm getting at is like you've got these eight or nine hundred million weekly active users, but you don't have-- but that feels very fragile because all you've really got is the power of the default and the brand. You don't have a network effect, you don't really have feature lock-in, you don't have a broader ecosystem. You also don't have your own infrastructure, so you don't control your cost base, you don't have a cost advantage. You get a bill every month from Satya. Um, so you've kind of got to scramble as fast as you can in both of those directions to, on the one side, build product and build stuff that on top of the model, which is our earlier conversation. Is it just the model?

    2. ETErik Torenberg42:29

       Yeah.

    3. BEBenedict Evans42:29

       That you've got to build stuff on top of the model in every direction. It's a browser, it's a social video app, it's an app platform, it's this, it's that. It's like, you know the meme of the guy with the map with all the strings on it.

    4. ETErik Torenberg42:41

       Yeah.

    5. BEBenedict Evans42:41

       You know? Um, it's all of these things. We're gonna build all of them yesterday. And then in parallel it's infrastructure.Like, you know, we, we do-- we've got to deal with OpenAI. We-- So a deal with, with Nvidia, with, with, with Broadcom, with AMD, with Nvidia, with Oracle, and with, with petrodollars, um, because you're kind of scrambling to get from this amazing technical breakthrough and these eight hundred, nine hundred million wows to something that has like really sticky, defensible, sustainable business value and product value.

    6. ETErik Torenberg43:16

       Yeah.

14. 43:17 – 45:27

    The Future of AI Models

    1. ETErik Torenberg43:17

       And, and so as you're evaluating the, the competitive landscape among the, the hyperscalers, what are the, the questions that you're asking... That you think are gonna be most important in determining, um, you know, who, who's gonna gain, you know, durable competitive advantages or, or how this competitive is going to-- competition is gonna play out?

    2. BEBenedict Evans43:38

       Well, this kind of comes back to your point about sustaining advantage, and we, we talked about Google. Like if we think about the shift to, particularly shift to mobile, for Meta, this turned out to be transformative, like it made the product way more useful.

    3. ETErik Torenberg43:51

       Yeah.

    4. BEBenedict Evans43:51

       Um, for Google, it turned out mobile search is just search.

    5. ETErik Torenberg43:55

       [chuckles]

    6. BEBenedict Evans43:58

       And Maps changed probably, and YouTube changed a bit. But basically for Google Search, Google Search is search, and the web s-web search is just mean, means more people doing more search more, more of the time.

    7. ETErik Torenberg44:07

       Yeah.

    8. BEBenedict Evans44:08

       Um, and the default view now would seem to be, well, Gemini is as good as anybody else. Next week, like the new model, I haven't looked at the benchmarks for GPT 5.1, which is out today. Is it better than Gemini? Probably. Will it still be better next month? No. So that's a given. Like you've got a frontier model, fine. What does that cost? It costs you, pick a number, two hundred and fifty billion dollars a year, a hundred billion dollars a year.

    9. ETErik Torenberg44:37

       Yeah.

    10. BEBenedict Evans44:37

        What's this, what's this earlier conversation about CapEx? Okay. So Google can pay that and-- because they've got the money, they've got, they've got the cash rate from everything else. And so you do that and your existing products get... You optimize search, you optimize your ad business. You build, you, you build new experiences. Maybe you invent the new-- the iPhone of AI. Maybe there is no iPh-iPhone of AI. Maybe someone else does it, and you do an Android and just copy it. Um, so fine, it's a new mobile. We'll just carry on. Search is search. AI is AI. We'll do the new thing. We'll make it a feature. We'll just carry on doing it. Um, for Meta, it feels like there are bigger questions on what this means for search, um, or what it means for content and social and experience and recommendation, which makes it all that more imperative that they have their own models, just as it is for Google. Um, for Amazon, okay,

15. 45:27 – 46:49

    Impact on Various Industries

    1. BEBenedict Evans45:27

       well, on the one side it's commodity infra, and we'll sell it as commodity infra. And on the other side... I mean, maybe, maybe, maybe stepping back, if you're not a hyperscaler, if you're a web publisher, a marketer, a brand, an advertiser, a media company, you could make a list of questions, but like you don't even know what the questions are right now.

    2. ETErik Torenberg45:48

       Yeah.

    3. BEBenedict Evans45:49

       What is this? What happens if I ask a chatbot a thing instead of asking Google, even if it's Google? From, from Google's point of view, why not ask Google's chatbot? It's fine. But as a marketer, what does that mean? What happens? If I ask for a recipe and the LLM just gives me the answer, what does that mean if my business is having recipes?

    4. ETErik Torenberg46:08

       Yeah.

    5. BEBenedict Evans46:08

       Do you have a kind of split between... And this is also an Amazon question. How does the purchasing decision happen? How does this decision to buy a thing that I didn't know existed before happen? What happens if I wave my phone at my living room and say, "What should I buy?" Where does that take me in ways that it wouldn't have taken me in the past? So there's a lot of questions further downstream, and that goes upstream to Meta and to some extent for Google. It's a much bigger question in the long term for Amazon. Do, do LLMs mean that Amazon can finally do really good at scale recommendation and discovery and suggestion in ways that it couldn't really do in the past, um, because of this kind of pure commodity retailing model that it has?

16. 46:49 – 50:08

    Apple's Unique Position

    1. BEBenedict Evans46:49

       Um, Apple, Apple's sort of off on one side. You know, interestingly, they produced this incredibly compelling vision of what Siri should be two years ago. It just turned out that they couldn't make it. Interestingly, nobody else could have made it either. You go back and watch the Siri demo that they gave, and you think, okay, so we've got multimodal, instantaneous, on-device tool using agentic multi-platform e-commerce in real time with no prompt injection problems and zero error rates. Well, that sounds good. [chuckles] I mean, has anyone got that working? Like, no. OpenAI, Open-- Google and OpenAI don't have that working. Google-- I don't think Google or OpenAI could deliver the Siri demo that Apple gave two years ago. I mean, they could probably do the demo, but they couldn't like consistently, reliably make it work. I mean, that, that demo, that product isn't in Android today. Um, and Apple, I mean, Apple to me has the most kind of intellectually interesting question, which is, um... So I saw Craig, Craig Federoigi make this point, which is like, we don't have our own chatbot. Fine. We also don't have YouTube or Uber. [chuckles] What, what ex-explain why those are different, which is a harder question to answer than it sounds like. Um, and of course, the answer is if this actually fundamentally changes the nature of computing, then it's a problem. If it's just a service that you use, like Google, then that's not a problem, um, which is kind of the point about, about, you know, where does Siri go? But the interesting counter example here would be to think about what happened to Microsoft in the 2000s, which is the entire dev environment gets away from them, and no one builds Windows apps after like two thousand and one or something. But you need to use the internet. To use the internet, you need a PC. And what PC are you gonna buy? Well, like Apple's like not really a player at that time and of just getting back into the game. Linux is obviously not an option for any normal person. Um, so you buy a Windows PC. So basically, Microsoft loses the platform war and sells an order of magnitude more PCs, like, well, not selling them, but in order of mag-- there are an order of magnitude more Windows PCs as a result of this thing that Microsoft lost. Um, and then it takes until mobile that like then they lose the device as well as the development, development environment. So here's this kind of question is, if all the new stuff is built on AI and I'm accessing it in an app that I download from the App Store, to what extent is this a problem for Apple?

    2. ETErik Torenberg49:04

       Sure.

    3. BEBenedict Evans49:06

       And what would have to... You, you would need a much more fundamental shift in what it was that w- that was happening for that to be a problem for Apple. And even if you take like the, you know, not the like the full like the rapture arrives, and we all just kind of go and live, sleep in pods like the guys in Up, um, not Up, um, yes. What is it? The one with the robot that's capturing the trash. Which one is that?

    4. ETErik Torenberg49:25

       WALL-E.

    5. BEBenedict Evans49:26

       WALL-E. WALL-E, yeah. You know the guys in the pods in that movie. Maybe we'll all be the people... Maybe we'll all be like that, in which case, fine. Um, but like there's a sort of a mid case, which is like the whole nature of software changes, and there are no apps anymore, and you just go and ask the LLM a thing. Fine. What is the device on which you ask the LLM a thing? Well, it's probably gonna have a nice big color screen, and it's probably gonna have like a one-day battery life. Probably use a microphone, probably a good camera.

    6. ETErik Torenberg49:50

       Yeah.

    7. BEBenedict Evans49:50

       [chuckles] Kind of sounds like an iPhone.

    8. ETErik Torenberg49:55

       Yeah.

    9. BEBenedict Evans49:57

       Am I going to buy the one that's a tenth of the price and just use the LLM on it? No, because I'll still want the good camera and this good screen and the

17. 50:08 – 58:44

    Strategic Questions for Tech Giants

    1. BEBenedict Evans50:08

       good battery life. So it's not... There's a bunch of kind of interesting strategic questions when you start poking away. Well, what does this mean for Amazon? Those are completely different questions to what does it mean for Google? Or what does it mean for Apple? What does it mean to Facebook? Or what does it mean to Salesforce? What does it mean to, you know, Uber? And then right back to what we were saying at the beginning of this conversation, you know, what does this mean for Uber? Well, their efficiency get... Operations get X percent more efficient, and now the fraud detection works and, you know, okay, maybe they're autonomous cars, different conversation. But presume no autonomous cars. That's a whole other conversation. Otherwise, as Uber, what does this change? Well, not a huge amount.

    2. ETErik Torenberg50:50

       I wanna sort of zoom out a little bit th-this whole framing. The, um... So you've been doing these presentations for a while now. You've, you know, you even bumped them up f- to two times, um, because there's so much is changing.

    3. BEBenedict Evans51:00

       Mm.

    4. ETErik Torenberg51:00

       Um, and, and one of the things you do in each presentation is, is you're famous for asking, you know, really great questions and chronicling what, what are the important questions to, to be asking.

    5. BEBenedict Evans51:07

       Mm.

    6. ETErik Torenberg51:08

       I'm, I'm curious, as you reflect, you know, maybe post, uh, you know, ChatGPT in 2022 or GPT-3 rather, um, the questions you were asking then and you reflect on to now, uh, to what extent, uh, do we have some direction on some of those questions or to what extent are they the same questions or, or new and, and different questions or what, what is sort of your... You know, if I woke up on a, in a coma, uh, after reading your, you know, your ori-original presentation, let's say, you know, the one after GPT-t uh, three launch, uh, came out, um, and then seeing this one now, what, what were the sort of most surprising things or things that we, we, we learned that updated those questions?

    7. BEBenedict Evans51:48

       So I think we have a lot of new questions this year. So I feel like, you know, you could make a list of, as it might be, half a dozen questions in spring of '23, like open source China, NVIDIA, s- does scaling continue? Um, what happens to images? Um, does O- how, how long does OpenAI's lead remain? And those questions didn't really change in '23 and '24, and most of those questions are kind of still there. Like, the NVIDIA question hasn't really changed. You know, the, like the answer on Chi... The answer on, you know, will ev- will... How many models will there be? The answer is, okay, there's gonna be t- anybody who can spend a couple of hundred, you know, can, can, can spend a couple of billion dollars can have a frontier model.

    8. ETErik Torenberg52:31

       Yep.

    9. BEBenedict Evans52:31

       And that was ac- pretty obvious in early '23. Um, but it took a while for everyone to understand that. And big models and small models, will we have small models running on devices? No, because the small models, the, the capabilities keep mo- are moving too fast for the small models to get... To shrink the small model onto the device. But those questions kind of didn't change for two, two and a half years. I think we now have, I think, a bunch of more product strategy questions as you see real consumer adoption and OpenAI and Google building stuff in different directions, Amazon going in different directions, Apple trying and obviously failing and then, then trying again to do stuff. There's some sense of like, there is something more going on in the industry than just, well, let's just build another model and spend more money.

    10. ETErik Torenberg53:14

        Yeah.

    11. BEBenedict Evans53:15

        There's more questions and more decisions now. There's also more questions outside of tech in certainly on like the retail media side of, um, how do you start thinking about what you would do with this? And again, you know, classic framing in my deck is like step one is you make it a feature and you absorb it and you do the obvious stuff. Step two is you do new stuff. Step three is maybe someone will come and pull the whole industry inside out and completely re-redefine the question. And so you could kind of do like an imagine if here of like step one is, um, you know, your ma- you're, you're a manager at a Walmart in the Bay Area or DC or in like whatever it is. Step one is find me that metric. Step two is build me a dashboard. Step three is it's Black Friday and I'm running... managing a Walmart outside of DC, what should I be worried about? Like, and that might be the wrong one, but it's like, you know, step one for Amazon is you bought light bulbs, so here's... So you bought bubble wrap, so here's some packing tape. But what Amazon should actually be doing is saying, "Hmm, looks like this person's moving home. We'll show them a home insurance ad," which is something that Amazon's correlation system wouldn't get because they wouldn't have that in their purchasing data. And we're still very much at the like... We're still starting to... We're, we're, we're still on the step one of that, but thinking much more what would the step two, step three be? What would new revenue be for this other than just like simple dumb automation? What would new things that we would build with this be? Um, where would this actually like might, might actually kind of redefine or change what the market might look like? Um, and that's obviously a big question for anyone in the content business.

    12. ETErik Torenberg55:04

        Yeah.

    13. BEBenedict Evans55:04

        You know, what does it mean if I can just go and ask an LLM this question?What kinds of content were predicated on Google routing that question to you? And what kind of questions, what kind of content isn't really that question? Like, do I want a bolognese recipe or do I want to hear Stanley Tucci talking about cooking in Italy? Like, do I just want the ans-- Do I want that SKU or do I want to work out which product I should buy? Which is Amazon is great at getting you the SKU, terrible at telling you what SKU you want. Um, do I just want the slide deck or do I want to spend a week talking to a bunch of partners from Bain about how I could think about doing this? Do I just want money or do I want to work with a16z's, um, you know, operating groups?

    14. ETErik Torenberg55:56

        Yeah.

    15. BEBenedict Evans55:57

        Like, what is it that I'm doing here? And I think the, the LLM is starting, thing is starting to crystallize that question in lots of different ways.

    16. ETErik Torenberg56:07

        Yeah.

    17. BEBenedict Evans56:08

        Like, what am I actually trying to do here? Do I just want a thing that a computer can now answer for me, or do I want something else that isn't? Because the LLMs can do a bunch of stuff that computers couldn't do before.

    18. ETErik Torenberg56:19

        Right.

    19. BEBenedict Evans56:19

        Is that thing that the computer couldn't do before my business?

    20. ETErik Torenberg56:24

        Yeah.

    21. BEBenedict Evans56:24

        Or am I actually doing something else?

    22. ETErik Torenberg56:26

        We're, we're about to figure out what is the... in a much more granular way, what, what is the true job to be done for, for, for many, many of these, uh,

    23. BEBenedict Evans56:34

        Yeah. And you know, going back to the internet, there was, you know, the, the sort of observation about newspapers is that newspapers looked in the internet and they talked about, you know, expertise and curation and journalism and everything else, and didn't really say, "Well, we're a light manufacturing company and a local distribution and trucking company."

    24. ETErik Torenberg56:49

        Yeah.

    25. BEBenedict Evans56:49

        And that was the bit that was the problem. And until the internet arrived, like, that wasn't a conversation you thought about. And then the internet suddenly makes that clear and suddenly creates an unbundling that didn't exist before. And so there will be those kinds of like, you didn't realize you were that before until an LLM comes along and points to... Someone comes along with an LLM and says, "Oh, I can use this to do this thing," that you didn't really realize was the basis of your defensibility or the basis of your profitability. I mean, it's like the, you know, the, the, the, the joke about, you know, US health insurance that like the basis of US health insurance profitability is making it really, really boring and difficult and time-consuming. That's where the profits come from. Maybe it isn't. I don't know. I don't know that history, but for the sake of argument, say that's, that's your defensibility while an LLM removes boring, time-consuming, mind-numbing tasks.

    26. ETErik Torenberg57:40

        Yeah.

    27. BEBenedict Evans57:40

        So what industries are protected by having that and they didn't realize that? And these, you know, it's like you could have asked these questions about the internet in the mid-'90s or about mobile a decade later, and generally you'd have half of the questions you'd have asked would have been the wrong questions in hindsight. I mean, I remember as a, as a baby analyst in 2000, everyone kept saying, "What's the killer use case for 3G? What's a good use case for 3G?" And it turned out that having the internet in your pocket everywhere was the use case for 3G.

    28. ETErik Torenberg58:09

        Yeah.

    29. BEBenedict Evans58:09

        But that wasn't the question that people were asking, and I'm sure that will be the thing now, is there's so much that we will, that will happen and get built where you go and you realize, "Oh, that's how you would do this. You can turn it into that."

    30. ETErik Torenberg58:27

        Yeah.
18. 58:44 – 1:02:06

    Reflecting on AI's Potential

    1. ETErik Torenberg58:44

       percent. My, my, my last question that'll get you out of here is, um, if, if we're talking two or three years from now or, you know, you're doing a presentation, you say, "Oh, this is actually bigger than the internet," or, or may-maybe this is like, like computing, um, what would need to be true? What, what would need to happen? What, what, what would, uh, would evolve our thinking?

    2. BEBenedict Evans59:03

       I mean, I, I kind of, you know, sort of come back to my point about, you know, Jews and Christians. The Messiah came, nothing happened. Um, we forget... I mean, there's maybe two, two ways, very brief ways to think about this. One of them is I think we forget how enormous the iPhone was and how enormous the internet was, and you can still find people in tech who claim that smartphones aren't a big deal.

    3. ETErik Torenberg59:30

       Yeah.

    4. BEBenedict Evans59:31

       And this was the basis of people complaining about me, like this idiot who thinks like Generative AI is big as those silly phone things. Like, come on. I think a-another answer would be like, I don't want to get into the argument about, you know, what is the growth rating capability and benchmarks and, and all... You know, you can see lots of five-hour long podcasts of people talking about this stuff. But the stuff we have now is not a replacement for an actual person outside of some very narrow and very tightly constrained guardrails, which is why, you know, Demis's point that ni- it's absurd to say that we have PhD level capabilities now. Um, w-what... We would have to be seeing something that would really shift our perception of the capability of this stuff-

    5. ETErik Torenberg1:00:27

       Yeah

    6. BEBenedict Evans1:00:27

       ... so that it's actually a person as opposed to it can kind of do these people like things really well sometimes but not other times. And it's, you know, it's a very tough conceptual kind of thing to think about because, you know, I'm, I'm deliberate... I'm, I'm conscious I'm not giving you a falsifiable answer, but I'm not sure what a falsifiable answer would be to that. When would you know whether this was AGI? You know, it's the Larry Tesla line, "AI is whatever doesn't work yet." As people, as soon as people say it works, people say, "Well, that's just not AI, that's just software." It's a, you know, it's a, it's an in, in... And it becomes like a kind of a slightly drunk philosoph-philosophy grad student kind of conversation as much as it is a technology conversation. Like, what would it... Have you ever considered, Erik, that maybe we're not conscious either? Ooh, there's a thought.

    7. ETErik Torenberg1:01:24

       Sure.

    8. BEBenedict Evans1:01:24

       It's, it... I, I... All I can say to an- give a tangible answer to this question is what we have right now isn't that. Will it grow to that? We don't know. You may believe it will. I can't tell you that you're wrong. We'll just have to find out.

    9. ETErik Torenberg1:01:43

       I think that's a good place to, to, to wrap. The, the presentation is AI Eats the World. We'll, we'll, we'll link to it. It's fantastic. Benedict, thanks so much for coming on the podcast to discuss it.

    10. BEBenedict Evans1:01:51

        Sure. Thanks a lot. [outro music]

Episode duration: 1:02:06