EVERY SPOKEN WORD
70 min read · 13,873 words- 0:00 – 1:05
Intro
- BEBenedict Evans
Mobile didn't need to wait for the internet. The internet didn't need to wait for PCs, and PCs didn't need to wait for consumer electronics and semiconductors and so on. So you've always got this accelerating adoption.
- ETErik Torenberg
Benedict Evans is a tech analyst known for his presentation, AI Eats the World. He sees AI differently than the world, spotting patterns others miss, and dives into how people really use AI.
- BEBenedict Evans
They built this amazing piece, incredibly sophisticated, very expensive global infrastructure with enormous growth in use all the time, and it changed all of our lives, and we all pay for it, and they didn't make any money from it because all the value moved up stack. The place that's got product market fit right now is coding. And Swap, it's gone from whatever it was, nine billion run rate at the end of last year to forty-seven billion dollars run rate now. That's all software, isn't it? So what happens when someone else in some other field gets something working? One of the characteristics of tech is that the moment that you understand something and you know what's gonna happen is the moment you should move on to something else.
- ETErik Torenberg
You know, Google said that the risk of underinvesting is riskier than overinvesting.
- BEBenedict Evans
The investors are kind of looking at all these companies and saying-
- ETErik Torenberg
Benedict, welcome back to the a16z
- 1:05 – 6:00
AI Adoption Accelerates
- ETErik Torenberg
podcast.
- BEBenedict Evans
Thank you.
- ETErik Torenberg
Last time you were here, we were discussing the first iteration of your presentation, AI Eats the World. Uh, you know, you've since wrote it almost a, a, you know, a year and a half ago. At, at this point, we're gonna-- You always begin your presentation with your, you know, what are the big questions. But I'm curious this time, first, before getting to the what are questions going forward, I want you to reflect on what have we learned since you originally, uh, made, made the presentation. What, what's played out? Um, and let, let's reflect back on it.
- BEBenedict Evans
What's changed in the last year? So I think we have much more of a sense of diverging product strategy. We have much more a sense of a sense of kind of competitive tension that goes beyond just make a bigger model faster with more comp-- more, more compute. Um, we've had several iterations of OpenAI strategy, in particular from sort of everything all at once yesterday to, oops, no, maybe we should double down on coding. Um, clearly agentic coding started working, and so all the focus in tech has kind of narrowed in massively onto that as something that has absolute product market fit in the sense that like the customers are pulling it out of your hands. Um, and, um, and of course, that comes with a supply crunch around capacity and price imbalance, imbalance of supply, demand, capacity, Capex pricing that we see at the moment. Um, so that's kind of the big shift. Like we had a moment of like this is kind of sort of working and kind of exciting, but we're not quite sure what we're gonna do with it to like, right, it works for coding. Um, will it work for anything else? Like, yes, almost certainly, but that's what's working right now, and so that's become... We've got this kind of much narrower focus. Um, otherwise, um, you know, the Chartmor numbers keep coming up. The models keep getting bigger. The Capex keeps growing. The usage keeps growing, people using this more. But most of the sort of fundamental questions you might have had two or three years ago didn't really have answers. Like, we don't know if there'll be a winner in the models. We don't know if they can capture value up the stack. We don't know how much the models can do. Um, we don't see a way that consumers will use this daily rather than weekly with the technology we have right now. So all, all of those sort of questions are still open.
- ETErik Torenberg
Yeah. And just on, on the, on the coding, how could-- could we have figured-- could we have foreseen that that would have been the, the, the u- use case that really would have taken off? Or what, what's your reflection on that?
- BEBenedict Evans
Well, um, every... Deterministically, you could have said, "Well, look, who's messing about with this stuff? Software developers. Well, a software developer's gonna try and make work software development." Um, [chuckles] so, you know, at a very kind of simplistic, naive level, well, yeah, the stuff that should work is software developer firstly, software development, just as like kind of... I, I, I often compare this moment to like the internet in like ninety-seven, ninety-eight, but it's also like the PCs in the early eighties or the late seventies. It's incredibly exciting, but it's not quite cl-clear what it's for, and it doesn't quite work yet. And clearly, the first thing that people did with PCs was make computers. Um, and the first thing that people are doing with LLMs, in a sense, LLMs are computers, is to make more compute. Um, and so that's not terribly surprising. I think the shift has been at the beginning of this year, clearly, that, uh, agentic coding went from being kind of useful to really changing everything. And I'm not sure you could have... You-- Clearly, there were people who would say, "Well, this is gonna be able to do absolutely anything." So they will say, "Well, yes, look, I told you." Um, but I don't think anyone kind of, kind of could have deterministically predicted exactly when that was gonna happen and that it was gonna be coding that would work first.
- ETErik Torenberg
And, and what have we learned about sort of, uh, you know... Say more about what this means for engineers, junior engineers, senior engineers, sort of the, the jobs discussion, how, how teams are organized, uh, et cetera. What have we learned so far?
- BEBenedict Evans
I don't think we've learned anything. I mean, you know, this, this didn't, this didn't w-- this didn't work six months ago.
- ETErik Torenberg
Yeah.
- BEBenedict Evans
And everyone is scrambling around trying to work out what it means. And, you know, you can get very, very into the noise and the detail on what did somebody say at a party yesterday. Say, "Oh my God, that's how it's all gonna work." Um, you know, it's gonna take a couple of years for this all to settle down. You know, if nothing else, be-because of the pricing. You know, you've got this enormous crunch between the demand and supply, and hence the pricing. Um, so we don't know what, you know, what a team's going to look like. I think people are asking new questions around, you know, the sort of the obvious one of, you know, do you hire junior people? And if so, what are they doing? And why were you hiring junior people in the past? And were you actually hiring to do the thing that they did, or were you hiring them to do something else? And so if you automate away a class of stuff that used to get done by people, then what will happen? And that sort of becomes much more real now in software development because you actually are automating a bunch of stuff that used to be done by people. So those questions are kind of now rather than theoretical. But I don't think anybody can possibly say they kind of know what the market structure is going to look like or what the career of a software engineer is going to be in three years' time. I think it would be-- you'd be insane to think that you could know that yet.
- ETErik Torenberg
Yeah.
- 6:00 – 9:27
OpenAI Strategy And Usage Gap
- ETErik Torenberg
The-Talk about, uh, OpenAI. Uh, ta-talk about what's most, uh, s-surprised you or w-w-how have you kind of made sense of their sort of strategy development and, and the questions that they have going forward?
- BEBenedict Evans
Well, you know, it's always been such a, uh, such a, a tranquil drama-free environment.
- ETErik Torenberg
[laughs]
- BEBenedict Evans
So, you know, it's... [sighs] And, you know, obviously they've had the, the issue with, with, with Fidji Simo having to take a medical leave, um, which kind of shuffled things up a bit. Look, clearly the, uh, the second half last quarter of last year, this-- their question was, right, well, the models are the models, but what else and how do we get people to do other stuff with this? So we'll do ads, we'll do e-commerce, we'll do shopping carts, we'll do payments, we'll do a browser, we'll do a social video app. We'll, you know, everything. You know, ask ChatGPT, GPT for fifteen ideas for what we could do to build value on top of infrastructure, and then we'll do all of them. It's almost literally what, what, what it looked like. And then, um, um, Anthropic, with having less capital raised, said, "No, we're gonna focus on coding," and they got coding working. Um, whether that was like a deliberate strategy or kind of they stumbled into it is, you know, for other people to say. But, like, clearly that worked. And then so OpenAI kind of swing round and like, okay, well, clearly that's the thing. Um, but the question kind of still remains, like the stuff that's working right now is software development and some things in some other fields. And then there's a lot of people who are kind of excited about using this around the edges and using it for some things, but it's very unclear, um, how it is that this is instantiated as product and taken to, you know, the other ninety percent of people. Um, you know, we still see in the data that sort of ten percent of people are daily active users and thirty, forty percent of people are weekly active users. And if you're only using this once a week, then you haven't, like, achieved nirvana yet. And there's clearly this kind of very wide spread between people in the Valley who bought, you know, a, a cluster of Mac Studios and are running OpenClaw all day versus, um, you know, those other forty percent of people who say, "Yeah, it's kind of useful. Um, I used it last week for something."
- ETErik Torenberg
[laughs]
- BEBenedict Evans
And I'm like: How do you br-how do you bridge that? And I don't think that question-- You know, softwa-soft, software is a place where that's really, really bridged, jumped over that bridge. And I don't think th-th-- And then there's a lot of other places where people are kind of scratching their heads and using it up to a point. And then there's a lot of places where corporations are using it to automate some, like, specific back office process where you're not asking the user to work out what they do with the new tool. Instead, you're saying, "Okay, here is a problem that we can solve." And, you know, I go and talk to, you know, companies outside America and, and outside of tech and talk to consultants and, um, you know, investors. They're looking at those one-at-a-time point solutions. Um, so like I'm speak-- so a couple of days ago to a commodities company, and they want to use LLMs to get better predictions on their cash flow because they deal with all sorts of small producers, and they don't necessarily know when their invoices are gonna get paid. And it's a very li-low margin business, so that's a big deal. And so they want to use LLMs to get better cash flow forecasting. That's a very different thing from kind of going to ChatGPT or Claude and saying, "Hey, you know, give me a summary of my meetings this week."
- ETErik Torenberg
Yeah.
- 9:27 – 30:43
Platform Shifts And Value Capture
- ETErik Torenberg
H-Can you share how, uh, how did this compare with mobile, um, or other sort of platform shifts in terms of, you know, us-user, early user adoption on, on sort of the, you know, week-weekly or daily user?
- BEBenedict Evans
So I think there's, there's, there's, there's, there's a bunch of different ways to answer this. One of them is like we're always standing on the shoulders of giants, and the growth is always compounding. So mobile didn't need to wait for, um, the internet or cellular networks. Like mobile data, mobile internet didn't need to wait for ce-- It kind of needed to wait for cellular data, but it didn't need to wait for like the internet to happen. And the internet didn't need to wait for PCs, and PCs didn't need to wait for consumer electronics and semiconductors and so on. So you've always got this accelerating adoption. And you know, when, when, when your boss, my old boss, Marc Andreessen, was working on Netscape, there were like double-digit millions of PCs on the entire planet. So like, no, you couldn't have nine hundred million weekly active users 'cause there weren't nine hundred million PCs. So there's always that acceleration. So that's one point. I think the second point here is like at the early stage of any of these shifts, it's not really clear how it's going to work, and nothing works. So, you know, like I'm just about old enough to remember this. I'm not sure how, how, how old you are, but like, you know, anyone in their thirties doesn't really remember a time when it was completely normal that you'd be working and then everything on the screen would just freeze, and you'd just have to crawl under your desk and unplug the computer.
- ETErik Torenberg
[laughs]
- BEBenedict Evans
And then pray that like some of what you've done in the last hour might still be there. That just doesn't happen anymore. Go back to, you know, the eighties and like you bought a sound card. Well, that's three hundred dollars. You want to have sound on your computer. Okay, that's three hundred dollars, and it's like that's the weekend to make that work. I mean, I remember this, you know, trying to get this stuff to work. And the same thing with the internet. Like, you know, you've got to get a floppy disk that has TCP/IP on. And you know, it's slow, and none of the stuff that you need to do existed. And the same with mobile. Now we're kind of at that stage. And of course, it's not clear which of these things are gonna work. And that's the same thing now, like are browsers gonna work? Is it gonna be this? Is it gonna be that? How is this all gonna fit together? And there's a gap between what's incredibly exciting and the small number of people who are willing to put the work in to get something to work and just turning that into a thing where you can just press a button and it all happens. I think the, the third poi-point here is, and, you know, there's a much more tangible observation, is that the, the pricing crunch that we've already mentioned looks to me a lot like what happened with mobile data in sort of two thousand nine, '10, where, um, suddenly people got bills for like five, ten thousand dollars of data.On the one side, and on the other hand, if you had flat rate data, which is kind of what happened in the US with the iPhone. The AT&T launches-- AT&T Cingular launched the iPhone with flat rate data, and then unfortunately everybody buys iPhones and starts wa-... And then they get 3G, and people start watching YouTube, and the whole network goes down because they just don't have capacity to do that. It's funny, there are still people in tech who don't understand that cellular networks have, have marginal cost. And like they have to add more capacity, and that costs more money. And so the networks kind of had to scramble to get like the cost curve aligned with the infrastructure pricing system, aligned with the underlying cost, and aligned with perceived value, which they kind of did with, with, with capped bundles and fair use and throttling and so on. But the other side of that com-- and that's exactly, you know, what you, what you see now. It's like on the one hand you're paying twenty dollars a month and you get ten grand worth of tokens. And on the other hand, like, you know, you messed about for a couple of days and you get a bill for ten grand, and you're like, "What the hell is this?" Um, that's exactly what... You know, you see, you literally see these stories now, um, which is exactly what happened in kind of two thousand nine, eight, nine, ten. Also what happened in like two thousand one and two and three with GOS. Um, but the, I think the other interesting part of that analogy is-- or that comparison is that since then, mobile data traffic has risen by something like one and a half to two thousand times. And the mobile inter-networks collectively have revenue of about a trillion dollars, and they spend about two hundred billion dollars a year on Capex. And the sorts have been flat for twenty years, and all the cool stuff got built by somebody else. And they kind of all thought that they were gonna build all the cool stuff. Like, I worked for a phone company that had a banking license because they thought like they would, they would do, they would do mobile banking, um, which now seems absolutely insane. Um, but that's kind of the point is they built this amazing piece of global, incredibly sophisticated, very expensive global infrastructure with enormous growth in use all the time. And it changed all of our lives, and we all pay for it, and they didn't make any money from it because all the value moved up stack. And this is, of course, absolutely, as I said earlier, this is one of the absolutely central question for, for LLMs is can the model do the whole thing or do you have to have three hundred apps built on top of it? Can you just go to the model and say, "Do my taxes for me," or do you need to have a tax thing that uses it, that might use some AI in ten, ten different ways inside it? Um, and if not, then what is it to be a foundation model provider? Is this just commodity infrastructure that gets sold at marginal cost? Which is-- somehow seems to be a very difficult concept for people to grasp right now because you can sell all the tokens you can make so you can price it at, at ROI. But over the next couple of years, we've got like a trillion, two trillion dollars of Capex coming down the pipe, and the models get hundred X, two hundred X if it's more efficient every year. And then there's new models, and will the models use more tokens or less tokens? But where we will, like we'll get to a different equilibrium. And why would that equilibrium be one where the model companies have pricing power when the models are all kind of the same, doing kind of the same thing with the same chips? Why would they have pricing power? And I think that's... So it's a long answer to your question, but you know, you go back and look over time, like chip companies didn't capture the value. Um, ISPs didn't capture the value. Mobile network operators didn't capture the value. Windows and iOS did, but they were doing something else. They were-- They had all these levers to go up the stack. Um, and of course, they had network effects which models don't have. So that's sort of the, the question is: Do they end up like the infrastructure layers or do they end up like the operating system layers and capture value, um, and actually get to decide what gets built? Or do they end up... I mean, the irony of that, of course, is Netscape, where, um, you know, Marc Andreessen famously said that he was going to turn Windows into a set of badly debugged device drivers. And, um, then Microsoft kind of crowbarred their way into the market, but it turned out that web browsers weren't the point because all the value was somewhere else. And so I think that's kind of the more-- the, the, the, these kind of swirling mass of questions around how this settles out, which comes back right through to all, all of my kind of answer to your question is like, you know some of this stuff, but you don't know how it's gonna work.
- ETErik Torenberg
Yeah. Yeah. It's unclear whether it looks more like the, you know, i-internet or sort of, um, you know, software where, where a lot of the value or, or just better margins happen at the application layer or sort of the cloud where it seems the, you know, where sort of it, uh, existed at the hardware layer. And right now, so far, it seems like the NVIDIA and going, you know, up, it, it seems like, uh, they have better margins and are accruing a, a lot of the value, but it's unclear if that will, you know, remain the same or if, uh, there will be sort of applications, uh, you know, if it will look more like the internet. How, how would you even begin to predict so-- you know, the answer to such?
- BEBenedict Evans
Well, so, so two answers to this. I mean, there's all these sorts of quotes about how history works and, you know, my favorite one is that history teaches us nothing except that something will happen. And, you know, you can always ex post facto say, "Well, of course it worked out like that." But it was-- generally wasn't obvious at the time. And, you know, in particular, I remember, you know, like I said, fifteen years ago, a lot of really, really clever people in tech looked at the iPhone and Android and said, you know, "This is open versus closed again, and Android is gonna crush the iPhone." Which of course isn't what happened. And then they can go and explain why. But, you know, all of these, all of these comparisons are useful. None of them are predictive. Um, and, you know, it's always obvious in hindsight. I-- You know, it's funny, I've, I've, I've done a couple of podcasts recently and I published this presentation, and there's like a, there's like a, a class of comment on this stuff, which is to say, you know, "Benedict, you're not doing your job. You're supposed to tell us what's gonna happen. You're supposed to make predictions, and all you seem to do is say, 'Well, we don't know.'" And there's kind of two problems with that. One of them is there's about a class of places where I actually do say, like, "I don't think this is gonna work. I think it's gonna work like that. Like, I don't think foundation models are a product. I don't think a chatbot is a product. I think the value will be further up." But the other side of this is like when you're at this stage in the cycle, you-- there's, there's, there's many pathsAnd you don't know which of those paths it's going to be. And to try and say, "Well, I think it's going to be that one," is, you know, you might be right. But you do have to kind of be conscious of, of like how uncertain this is and how many different paths it could take. Um, that's the nature of this part of the cycle is all the bets are open. You know, we get to the point where the S-curve kind of curves up and it narrows in. And you know, there was a moment when, you know, Windows Phone might have worked. In hindsight, no, it probably wasn't gonna work. But you know, there was a moment when it wasn't clear how mobile was gonna work, and then there was a moment where it was clear, right? This is what's happening. Now we move on to next ques- the next question.
- ETErik Torenberg
Yeah.
- BEBenedict Evans
And I think kind of the, the... I'm sorry I'm, I'm kind of monologuing a bit. But like one of the characteristics of tech is that the moment that you understand something and you know how it works and what's gonna happen is the moment you should move on to something else. You should always be looking for the qui- places where we don't know what the answers are. Because, you know, I haven't updated my Apple spreadsheet in like five years-
- ETErik Torenberg
Mm-hmm
- BEBenedict Evans
... 'cause we know what happened. They won. Like, I don't care what the next year's i- what, what, what, you know, this year's iPhone looks like. I don't pay attention to their market share in China. Like it happened. Next question.
- ETErik Torenberg
Yeah. The, um... B- b- but just to flesh it out, y- you mentioned the prediction of you don't think foundation models are, are the product you think will move up. E- e- explain that, that, that, the re- reasoning there, there a bit and what they could look like.
- BEBenedict Evans
So I think there's like three or four like building blocks you can put on the table. One of them is that, um, it's not clear how you could build a model that was fundamentally better than everybody else's model in some sort of sustainable, differentiated way. There isn't-- doesn't seem to be a network effect. There doesn't seem to be sort of levers you can pull and a strategy play, a position you can get into w- that where Instagram is or YouTube is, or Google Search is. And we don't see an equivalent of that for LLMs. Now, you have different emphasis. You know, this, maybe this one's better than that. Maybe you like this one more than that. Um, but there doesn't seem to be a sort of fundamental differentiation, fundamental competitive difference between the models, except your willingness to spend money. Um, second problem is the chatbot itself is like a kind of a weird limited V1 UI. And there's some things and some people and some kind of task where it works really well. But there are most of the others, you need a bunch of other stuff. You need tooling, and it needs to be set up right, it needs to have the right data, and it needs to be configured and controlled and have the right user interface, and people need to have kind of sat down and thought about how this should work. Because generally, people who are good at using the tool and doing the job that needs the tool are not the same people who are good at deciding what the tool should be. So, you know, people who are really, really good at, you know, designing print publications are not the people who should create InDesign. That's a different set of skills. And you know, people who are really, really good at doing financial advice are not the right people to design TurboTax. These are different people with different skills. So, um, and you have kind of groping around the middle of this, so you now have, you know, Claude for this, Claude for that, and you have skills and so on. To me, this is kind of like... Well, one question is: well, who builds the skill? Another question is like: well, you know, that seems to be a bit like what you get if you do file new in Excel, like user templates, and they'll take you so far. But at a certain point, you know, people outgrow the templates. Um, there's a slide in my presentation, which is a quote from, that somebody said to me on Twitter years ago. You know, so they said they were a consultant, and half of their jobs were telling people who used Excel to use a database, and the other half were telling people who used a database to use Excel. So there's this kind of fuzzy, swirly place of like, do you need dedicated software? Do you need horizontal software? Do you need vertical software? But you know what? Just do everything in Excel. There's alw- you know, you, you know, we've all like seen the department that runs along on a 10-meg Excel file. And I run my business in numbers, but on a spreadsheet. But like there's a certain point where like you outgrow that. Um, and so f- following that on, well, can the model labs build all of that? Well, of course not, no more than like Microsoft or Apple could build every Windows app or every iPhone app. So then do the model labs have leverage? Are they, are they, are they Windows? Are they iOS? And again, well, is there a network effect? Like if you're a law firm right now and you buy a piece of software, like, you know, do the cust- all pieces of enterprise software that a16z has invested in, um, how often does like the law firm or the manufacturing company or the bank say, "Oh, well, does this use Claude or does it use AP-- or O- OpenAI? 'Cause we, we standardize on Claude." Well, no, that's not how it works any more than it worked like that for cloud. Like you didn't say, "Well, we, you know, our company standardize on AWS." Like you don't even know what company, what, what cloud that, that, that SaaS product com- runs on. That's the whole point. It's abstracted away. It's not your problem. And so the foundation models seem to look more like that. They seem to more lo- lo- look more like hyperscalers in that sense, in that they don't have the... You know, they might have competitive advantages. Um, but further up the stack, you don't have leverage, you don't have a network effect, you don't have control. Um, that sort of prompts me to, incidentally, to, to say, well, maybe the right comparison here is with semiconductors, where with each generation it just gets more expensive, and so you have fewer players. Um, so all of that kind of taken together, well, the models are kind of diff- commodities, and the chatbot isn't the right UI or the right product, and the companies aren't gonna be able to build all of that stuff themselves, so therefore, they're low-level infrastructure. And so then, well, do they have pricing power? Well, you're gonna have, pick a number, three to six companies making a frontier model, spending, no one knows, no one almost knows, like something between $200 billion and $2 trillion a year on building these models. Plus, there'll be a bunch of edge and a bunch of open source. I know. Go, go, go and get-- go and ask me to-- ask Martin Casado what he thinks. I don't know. I mean, he doesn't know either. He probably has a better way of saying he doesn't know than I do. Um, but like we don't know. So where, so where is this gonna settle down? You're going to have, as it might be, half a dozen companies that are all competing to sell this stuff. And so where is the price discipline gonna come from? Particularly when some of them have got like whole other business models as well, like, you know, um, Google selling ads. So, you know, they've got a different attitude to pricing to, to OpenAI. Um-And so, like, I think the c- the, the challenge here is, like, there's a difference between where we are right now and where this should end up, which is kind of a first-year economics student kind of conversation. Like, right now, we're in this period of extreme disequi-equilibrium of supply and demand and price and Capex and capacity. But just because demand for tokens is infinite, that doesn't mean that you can't get to a different price equilibrium. Because, of course, that's what happened with mobile data. Like, demand for this is infinite. It's grown fifteen hundred, two thousand X in the last fifteen years. But you still got to a supply and demand price equilibrium, and you still got a murderous price war between telcos in most parts of the world. Because fundamentally, you're selling kind of a commodity, um, to people who will swap back and forth, and of course, developers will also swap back and forth. Now, this is... You know, I'm happy to say that this might be completely wrong. It may be that we, we, we get to a world in which there's only two companies that can make an LLM, and they have pricing power, or we get to a world in which most of what we do gets subsumed into the model, or they have leverage further up the stack. Um, and, you know, it's kind of my point about iOS versus Android. Just 'cause you can say, "Well, it worked like that the last three times," that doesn't prove that it's g- what, tell, prove what's gonna happen this time. But it doesn't mean at least you should sort of ask the questions, and you should certainly, I'll just say as a sort of a primary observation, like, this situation right now is transitory. You know, we're in this extreme scarcity, and then we have a pricing system, and we have a free market, and we have a surge of Capex and, like, a trillion dollars of Capex. So, like, those multiples are gonna move around, and then what?
- ETErik Torenberg
Yeah. I mean, g- going back to your, your... It's a good segue to your point you made earlier of, like, hey, you know, we know, you know, Apple's, y- Apple won. Next question. W- d- to, as a segue, what are some of the next questions that you're most f-f-focused on or that, you know, we should be paying the most attention to?
- BEBenedict Evans
So, I think one way you want, one... The, so, so there's some of the questions I, uh, we've already talked about.
- ETErik Torenberg
Yeah.
- BEBenedict Evans
Like, well, how far up the stack do the models go-
- ETErik Torenberg
Yeah
- BEBenedict Evans
... can the models differentiate? And so I think another is obviously is, is at what point are there, do we see more and more classes of use case where the models are good enough where we don't need the most expensive, fastest, biggest, heaviest model in the cloud, and you can use an older model, you can use an open source model, you can have a model running on-device? Obviously, this is what Apple's gonna be talking about in a couple of weeks. Um, you know, how much can you push onto the device where the compute is free or free to you anyway? It doesn't have marginal cost for the developer. A-another classical question is, it's almost like the questions move out of technology. So if you're looking at a law firm or a cons-consultancy, um, or an investment bank, or basically anyone in professional services where you traditionally have this pyramid structure, and you can automate a great chunk of what the people at the bottom of the pyramid are doing, what happens? And the only thing you can say there is, if you have never worked at a law firm or never worked at Bain, BCG, McKinsey, probably not gonna have a good idea of how this works. 'Cause you probably don't really know what it is that all those associates are doing, and you also don't really know what it is that the client's paying for. And how do those things kind of get reconfigured? Um, and so, like, well, what does AI mean for finance? What does this mean for finance, both for h- that, that internal, like, hiring structure and the kind of products you can create and, like, the margin structure? What does it mean for consultants? What does it mean for the Big Four, for the Big Three, for Accenture, um, for big law firms, um, and for advertising? And you some, probably can know some of those questions. Um, but if you're not kind of in that industry, you don't really know the answers. The thing this reminds me a lot of is, is something I wrote when I was at a16z, uh, which I called Content Isn't King. And I also wrote something that said Netflix isn't a tech company. And the point I was kind of getting at is that if you looked at Netflix, this whole thing is enabled by stuff that the tech industry has built. But all the questions for Netflix are, are TV, are LA questions. Like, what shows? How many shows? What kind of shows? What should you pay the talent? Should you aim for awards? Should you do movies? Should you buy sports? What kind of sports? These are all Los Angeles questions. These are not San Francisco questions. Like, no one in San Francisco even knows what the right questions are. They're media industry questions, and this is kind of, was kind of my point, that, that, that, that, that all the questions that matter for Nex- Netflix are the company are media industry questions. You... This is obviously the great tension point about Tesla. Is it a car company, or is it a technology company? Um, and so what I'm kind of getting at is, is that, you know, what does this stuff mean for law is kind of a question for lawyers as much as it is, or people who understand a lot about law firms and how they actually work and what they're actually doing and what the client's actually buying from them. Same thing for, like, what does generative video mean for Hollywood? Like, you know, Ben Affleck probably knows a lot more about this than I do. Like, he built a company and sold it for, like, a hundred billion, hundred million dollars, so obviously he does. But, like, you know. That, so that's kind of a second question, which is the questions move outside of AI, and they become sort of half AI questions, half something else kind of questions. Um, and then the third level, which is, I think, and I probably should've said it this earlier, w- the way that all of this is sort of fundamentally different from previous platform shifts, is that with, you know, 3G or the iPhone or b- b- with the web or whatever it was, you didn't know what was gonna happen next, but you knew the physical limits. Like, you know, nineteen ninety-five, you knew that telcos weren't gonna give everybody in the world broadband next week, and you knew that everyone in the world wasn't gonna go out and buy a PC 'cause a PC cost, like, three thousand dollars. So you kind of knew the basic physical limits of what, what could and couldn't happen. And with generative AI, obviously we don't. Like, we might, like, like, look at our phones when we get off this recording, and there's a push notification that says that, like, OpenAI's new model is out, and it's, like, two percent of the price 'cause they worked something out. I mean, I don't think it's very likely at this stage, but, like, we don't know those kinds of questions. So how much bigger will the models get? How much better? How much faster? How much cheaper? How much, you know, p-pick up, you know... In what ways will the characteristics of models change? We don't know, and that is different to previous platform shifts where you did know the sort of fundamental constraints. Um, and so that will, that will kind of spin off new questions. And in a sense, this is something I pointed to earlier. I said, like, the place that's got product market fit right now is coding. Nothing else has equivalent product market fit right now. I don't... You know. I'm think I'm pretty safe in saying that, you know, when Shopify's gone from whatever it was, nine billion run rate at the end of last year to forty-seven billion dollars run rate now, that's all software, isn't it?So what happens when someone else in some other field gets something working?
- ETErik Torenberg
Yeah. The, I mean, if you had-
- BEBenedict Evans
And which field? Like law, or like law bank, I don't know where, but something.
- ETErik Torenberg
If you had to guess, what are the use cases out- outside of coding that could potentially yield, you know, daily activity?
- BEBenedict Evans
Hmm. So the, the, I should say, the sort of presentation that I published a couple of weeks ago, there's sort of three sections, and one of them is talking about capital and Capex and infrastructure and foundation models and differentiation, which is
- 30:43 – 33:27
Automation And Jevons
- BEBenedict Evans
the stuff we talked about. And the second is, well, how would you build software with this, and what does this do for the software industry, and w-what would your software look like if you... And what, what are the margins? What happens to the margins of the companies and everything else? And the third section I called Change, which is kind of getting to this point. Um, and I opened it with, again, what appears to upset a certain category of person, where I said, you know, the Yogi Berra quote that, you know, predictions are hard, especially about the future. Um, and I think this is sort of a back test point, which is imagine asking these kind of questions about the internet in nineteen ninety-seven. What would you have got? What would you have not got? But I think one way you can look at this is to say, well, this is automation, that this makes a class of thing that people used to do, but that couldn't be automated. Now you can automate that. And so then, well, what does that mean? And I propose, like, three or four ways, the, the buttons to press. First one is, is this just price elasticity, which is really what the Jevons Paradox is. Like, if you make it cheaper to do stuff, do you do the same amount of stuff for less money? Or do you do m- the same more for the same money? Or do you more, do more for more money, um, because it becomes so much cheaper? Um, do you... Was there something that you couldn't do before that now becomes cheap? Was there something that was expensive and you were doing as a barrier to entry, like owning a printing press as a newspaper? Like, is there something that now was, was a barrier to entry and a cost base at a barrier to entry that now goes away? Is there something that gets unlocked in your business model or in your competitive space because this thing became cheap? And then, like, the sort of the final question would be, like, what stuff was just completely impossible, like, cost, computationally cost prohibitive that so that nobody even, even thought about it, and now that's within reach? And the example I used to give here was like, well, steam engines make trains possible, and it wouldn't matter how many horses you buy, you couldn't have a train or like an express train. Um, much more contemporary example would be to point to something like YouTube or indeed point to Spotify. Like, you know, Spotify says, you know, step one of the, you know, look at the last twenty-five years of the music business. The first half is what happens if you don't have to buy a fifty dollar CD to get that track, but then the second half is, what if fifty dollars a month gets you all the music that there is? Which is something that was just completely impossible. Um, this is all... The problem, the problem with these kind of making predictions like this is like on the one hand, you're gonna say stuff that's kind of clever and obvious, but, but, like, you don't actually know what it's gonna mean industry by industry. Um, so, like, if we'd been back in the late nineties and we'd said, "Well, you know, internet will destroy the value of physical distribution," it turned out that meant completely different things for newspapers and movie studios. Like, newspapers got completely screwed by this, and movie studios have kind of not really changed very much. So again, it depends. Like, sorry for the people that, that that annoys. Um, but the other kind of part of this is, you know, to tr- you try-- you can... There are some places where I think you kind of can, like, ask more, more useful questions.
- 33:27 – 39:41
Ads And Shopping Agents
- BEBenedict Evans
And the one that sort of intrigues me is to say, well, how does this change advertising in e-commerce and brands and marketing and everything that we buy? Which is, you know, advertising is a trillion dollars, and retail is twenty-five trillion dollars. So, you know, this is a reasonable sized tab. Um, and um, so the, the thing that I've always used to think about was that Google and Meta and Amazon don't really know what that product is. They know it's a SKU. They know what the publisher typed in the metadata field. And they know that people who bought this also bought that. But they don't know why, and they don't really know what those things are, which is why you get these jokes about, you know, "Hey, Amazon, I bought a toilet seat cover. I'm not collecting toilet seats." 'Cause Amazon doesn't really know what a toilet seat is and doesn't know that people don't buy two. Actually, they should know that. That should be frequency analysis. But they don't. Um, and with, um, with an LLM, like in principle, you would kind of know what those things are and why people buy them and what other things people buy. And obviously, know is like a, a difficult, tricky term to use. What do you mean when you say know? But at a minimum, like a much, a very different level of statistical correlation of, of what, um, an AI system would be able to do. Which is of course why you see the ad numbers and the, you know, the conversion rates shooting up in the every quarter from, from, from Google and, and fa- and, um, and Facebook. 'Cause they're rolling all of this into their ad systems and their, um, recommendation engines and their, um, uh, prediction algorithms. And like the... You get shown more stuff that you would like, and the ads that you're shown are more likely to be things you'd like to buy. And so they have these enormous, this sudden acceleration in their ad revenue. And all of which is, is to say, like, you kind of look at, um, how these systems work, and right now they say, "Well, people who bought that could buy this." And you should now be able to say, like the slide I had in the presentation was like, "Here's a picture of a coat. What is it? Where can I buy that?" And like five years ago, that really wouldn't have... Ten years ago, that certainly wouldn't have worked. Five years ago, probably wouldn't work. Now that should work. And then you can say, "Okay, suggest ten other coats like that with different prices and tell me where I can buy them, and suggest the pros and cons of each one." And you'll kind of get that too. And then you can push one step further and say, um, "Look at my Instagram and suggest a winter coat I should buy that will change my look, but not too much." And again, like three years ago, that would've been total science fiction, and now you think, yeah, you could probably build something like that. That would kind of work. And those kinds of shifts in, like, what the computer knows, what it can automate, what suggestions it can make. The-- Going back right to the beginning, like whenever you get a new technology, you start by doing the old thing, but more. You need more spreadsheets, like more PowerPoints, more email, better email. Um-But the important stuff is not doing the old thing, but more it's doing something new that you couldn't have done with the old thing. I mean, this is a pretty banal observation, but we kind of le-lose sight of it. And so what are the new things that you can only do with this as opposed to automating the old stuff? Um, I mean, I think the, you know, the enterprise version of this would be, you know, you've got all our Zoom calls with clients recorded, and you've got all the flows of emails in and out of Salesforce, and you can see all of the telemetry and the metrics and the analytics of how people use our product. So how should we change our prices to improve our churn? And again, that's something that an LLM might be able to do, which is very different to saying, you know, "Do sentiment analysis on calls into the call center and tell me which customers are angry." You know, you get kind of multiple shifts in the layer of abstraction around what analysis you can do. Um, and of course, that then creates new companies and destroys old companies and creates new businesses and everything else. But again, we're in nineteen ninety-seven and I'm trying to predict Uber and Airbnb. And, um, if I could actually do that, there's a sort of general point here, which is if we could actually predict what was gonna happen, we live in a parallel universe. You know, VCs would have, you know, it wouldn't be a one-in-ten hit rate. It would be a ten out of ten hit rate.
- ETErik Torenberg
It, it, it seems like, yeah, o-one of the questions w-we're now asking is what... or, or sort of if I was, is, um, [lip smack] sort of what was un-unreasonably expensive to do before that now is, is possible? And may- you know, is it so, I don't know, something crazy like rebuilding YouTube from scratch or rewriting Linux from scratch or...
- BEBenedict Evans
Mm. Yeah, it's funny. I mean, the, you know, the other, the, the, the paired fallacy, of course, is the new thing comes along and says, "Well, we're gonna build a new thing with the-- the old thing with the new thing." Um, of course, we're gonna, we're gonna build Office with open source. We're gonna rebuild it on the web. And, you know, it turns out like, you know, guess what? Look at Google Docs. It's got like twenty percent of the market 'cause that's actually not the point. What's interesting is to do something else, is to do something new. Um, and it's to shift that level of abstraction and, and, and to kind of spot problems that have never existed. I mean, I, you know, the, it's the experience you get sitting in pitches all day at a venture firm is there's some stuff where you think, "Well, that sounds kind, sounds kind of useful." And there's stuff where you think, "I'm not quite sure that why that would work." But there are some things that like kind of fill a hole in the universe. And like as soon as somebody explains it to you, you think, "Wow, why did never, nobody do that before? Why did no one see that that thing existed?" And that's, you know, where, you know, part of the fun part of looking at startups is, and that's what people will do with this. People will suddenly work out a way that you could turn... that that will work out that that problem existed. And no one, including the people who have that problem, no one realized that problem existed. And then they'll go out and make a thing to solve it. This is also incidentally going back to an earlier point. This is why I don't see the mo-- I think this is the problem with the idea the model will do the whole thing. And if you kind of, you know, kind of go back and think about all the pitches you've seen since you joined a16z, how many of them were things where people in the industry knew that was a problem? Like quite often the answer is actually no. Actually, no one in the industry thought that was a problem. And it was actually took like two years to explain to them and persuade them that that problem actually existed at all, and that this new thing would fix that for them. And that's kind of the problem with the idea that, you know, you know, middle manager in finance is going to use this tool to solve this big global industry problem. Like, no, 'cause no one knew that industry problem was there, let alone could, could work out the right way to build a tool to solve it.
- 39:41 – 49:57
Enterprise Stack Rewired
- ETErik Torenberg
Does this imply a less consolidated SaaS environment than, than before AI? Maybe less bundling or single bi- behemoths like the Microsoft Enterprise?
- BEBenedict Evans
Gosh, way to bring me back down to earth. Like-
- ETErik Torenberg
[laughs]
- BEBenedict Evans
... is the SaaS industry gonna be less consolidated, Benedict? That's all great, but like tell us about the stocks.
- ETErik Torenberg
[laughs]
- BEBenedict Evans
Um, what are the kind of building blocks that we can put down here? So obviously it's gonna be way cheaper and quicker to build software. Obviously, there's gonna be a bunch of stuff you could do with software that you just couldn't do before at all. Um, and so there will be more competition. There will... And of course, this comes with a new margin structure, but as per our conversation earlier, we don't really know what that margin structure is gonna look like. Um, are you going to go to, you know, outcome-based pricing? It's really hard to like tie each button press in a piece of enterprise software to P&L. Like sometimes you can, in Salesforce or something. There's an awful lot of software. It would be really hard to say, well, you know, the, the, the work I did today did this to the EPS, therefore this is what we should pay for it. Um, this is what we should pay for that piece of software. I don't think that makes sense. Um, long... Anyway, but what, what does the pricing structure look like over time versus now? Um, there will be more competition. It will be easier to build stuff and quicker to build stuff. The way that I sort of thought about... I suppose this is, there's maybe kind of two framings to think about this that are kind of useful. One of them is to say that if you think about the sort of enterprise software fleet today, you've got like three buckets. You've got like your big iron horizontal systems, so SAP and Workday and your CRM and your capital management software and your payroll management software and so on. And then you've got, um, vertical software, and typical big US company has like three to four hundred SaaS apps and then like another thousand apps that they've built, bought or built themselves internally running on-prem. And then in the middle, you've got this kind of fuzzy improvised space of Excel and email and shared file system. And so, and stuff kind of moves back and forth between those. And like in principle, every SaaS app is doing something that you could have done in SAP or you could have done in Excel. Like you could have like managed your graduate recruiting in Workday. But at a certain point, like if you're... I was having this conversation the other day. Like if you're PwC, um, and you hire however many thousand graduates every year to train to be accountants, um, you probably got a piece of dedicated software that you built for yourself or maybe you hired Accenture to build and you probably hate it. But anyway, you've got this piece of dedicated hiring software or you bought something. If you are a company that hires five graduates a year, you're doing that in email and a shared Google sheet, like 'cause why would you buy software for that? And then like, then there's a space in the middle. Do you do it in Workday? Do you do it in Excel? Do you do it in a dedicated app? And now you add ChatGPT to thatDo you do that in an LLM? Is there an LLM tool that means you can do that in Salesforce where you couldn't do it before, or you can do it in your vertical app that you couldn't do before? Um, do you use the LLM to build yourself a tool for that, just as you might have a company department that runs on a 10 mega- 10 mega Excel spreadsheet that someone built 15 years ago and no one knows how it works- no one knows how it works, but they're still using that. So it kind of-- it's, it arrives within this kind of broad, fragmented, complicated landscape, and it's another set of options for how you would do that task. So this is kind of one framing to think about, to think about it. I think the other framing to think about this is, does the LLM go at the top of the stack or the bottom of the stack?
- ETErik Torenberg
Right.
- BEBenedict Evans
So on one hand, bottom of the stack, it's a feature inside Salesforce. So you're in Salesforce, look at the history with this customer, look at the context of every other sales call we've done, look at our business objectives and suggest an email or suggest me what I should do here, what I should say on the call to the customer. So it's a feature. It's a button that's controlled and has tooling and guardrails and everything else that are driven by that particular use case. But the other way to look at it is the example I gave earlier, which is, you know, go look at Salesforce and Workday and all of our email and Google Analytics and, and, and, and then synthesize something that you couldn't have done before. So the, the, the tension in both cases is where do you put the probabilistic software that can make mistakes, and where do you put the deterministic system software that can't answer these kind of questions? So where do you put the database and where do you put the LLM? And is it like a which is at the top and which is the bottom? The answer is probably both, depending on, on, on what you're, what you're do- what you're doing and where it goes. Um, all of which is a long way of saying, like, what does this do to software? The answer is more software. Like, way more software. [chuckles] I mean, all software companies exist to solve problems created by other software companies. Um, that was the joke in soft- in security. Like, all security software is-- exists to solve problems created by other security software. And like, clearly that's, that's what we went through with SaaS. Like, SaaS gave us an order of magnitude, two orders of magnitude more software. Um, and we should probably expect that with this. Um, what that gets to with the SaaS apocalypse is all the investors are kind of looking at all these companies and saying, "Well, we don't really know which of these companies are gonna get screwed by all of this. Some of them must be. Like, obviously, there must be some, you know... Go through the end of this and, you know, X percent of all the SaaS companies that are out there are gonna get wiped out by this, but you don't know which ones." So you probably shouldn't derate the whole thing by 50%, but clearly you're gonna, like, go, "Mm, I'm not sure I'm gonna be long software at the moment until I have some idea of what the hell is going on."
- ETErik Torenberg
Yeah. You, you, you said in your talk with Ben Thompson that software is someone sat down and designed a workflow and said, "This is the right way of doing this from now on." But you also said that a process grows out of the way just a business runs. Does that just take time, or do you think we need more experimentation, iteration from these vertical AI startups to get just the right shape of software for the future?
- BEBenedict Evans
Well, in a sense. I mean, maybe kind of a, an interesting turn on this is, this is both what, what strategy consultants do and software companies do, is they kind of look at what's going on inside a company and say, "Well, this is a crap way of doing it. This would be a better way of doing it that would achieve your objectives better." And a software company kind of encodes that in software and, you know, a strategy consultancy kind of encodes that in, you know, workflows and org charts and processes and training and, you know, objectives and, you know, maybe tells them to buy some software to do that thing. Or maybe now increasingly maybe builds them that software as well. Um, I think, um, another w- thing to talk about here is how much of what's done inside an organization is implicit and not documented and not in the training data and not something that anybody in that company could actually kind of sit down and draw you a neat flow chart of and explain to you. Um, that's what... That's a big chunk of the value of being BCG and McKinsey, is that they have license to come into a company and talk to everyone and talk to the people you're not allowed to talk to that are in a different org and not get fired. And to go and work out how this actually works as opposed to how it's supposed to work and why it is that people aren't doing the strategy because actually, guess what? Their bonus targets depend on them not doing the strategy. And work all of that out and be a team that's ready to come in from the outside and give you the answer, and then you can blame them or have that kind of that, that pre-baked solution. Um, that's not, you know, you... That's, that, that's-- tho- these are sort of problems in organizational management and how people function and how people can explain what they do that are very hard to write down and very hard to kind of bake into a Claude skill and say, "There you are. Like, make me a PowerPoint." Um, and so there's a sort of broader how does this always work challenge here of how do you get people to use these technologies? Um, how do pe- people adopt new tools? How do you work out how to help people adopt new tools and work out what new things you would do with them? Which is also what happened with cloud and web and mobile and the internet and PCs and spreadsheets and so on.
- ETErik Torenberg
To that end, do, do you think there's some kind of co-evolution between AI native software and new types of interfaces? For, for example, new customer service AI platforms that might not have had as much human-facing UI or system of record software being built without a front end at all because its primary user will be AI agents querying it directly?
- BEBenedict Evans
So I think these are kind of interesting ideas. They're things I struggle to have a strong opinion on because, you know, they're not kind of, uh, d- not deep, deep into the weeds of, of how enterprise infrastructure gets bought. I, I wonder how new some of these questions are. Um, I may remember, um, Chris Dixon saying like 10, 15 years ago that, you know, APIs are the new BD and software wouldn't need... Kind of software companies could just, you know, open up your APIs and like, well, like what's old is new. Um, you know, you don't need an API anymore. You just have an MCP server, and like people will just plug... The agent will just plug into that. Um, I don't know. I think the challenge with a lot of this stuff is that all the, the decisions are really exception handling. Like with the question is always what can you not automate? What requires someone to make a decision, um, and some judgment and have an opinion about it because maybe that hasn't been written down or that didn't happen before or it doesn't look quite the way it happened before. Um-I think there's a sort of, you know, there are various ways of kind of think about separating out what gets automated and what doesn't. Um, the way that I used in the deck was to talk about what's a task versus what's a sk-- what's a job. Very often, the tasks that are used to accomplish the job might change without the job itself changing very much or without the thing that the job is selling to the client changing very much. Like, if you think about what accountants did fifty years ago and what accountants do today, um, they do spend almost none of their time doing the same things. But, like, to the client, it's kind of the same thing. Um, it just gets done in a completely different way with a whole bunch of different tasks. Um, and one of the sort of the more sort of either profound or kind of abstract ways to think about this is where is it that you want the ab-- the average? Where is it that what you want is the way that everybody will do this? That's the way everyone would do it. That's what anyone would say. That's what anyone would make. That's what any associate would make. That's what anybody would give me. That's the answer anyone would give. Versus where is that not what you want? Where is it that you want the answer to a new question or a different answer or a different idea? Um, 'cause LLMs are be-- are going to be very good at anything where you can describe how people do it and where what you want is the way anybody would do that, and not so good at where you can't really explain why you did it like that and where you're doing it differently to the way people would normally do it.
- ETErik Torenberg
Yeah. Yeah, that, that's helpful framing.
- 49:57 – 1:02:13
Capex Commodities And Magic
- ETErik Torenberg
I, I w-- I wanna zoom back out from the weeds gearing towards closing here, and a, a couple last questions. O-one is, various people, including, you know, uh, CEO of Google, said that the risk of underinvesting is riskier than overinvesting. Is there any level of-
- BEBenedict Evans
Mm
- ETErik Torenberg
... Capex where that stops being true, and are we getting there now?
- BEBenedict Evans
Well, there's a financial gravity problem in that, um, Microsoft, Meta and Google are all on... in line to spend a fifty percent of revenue on Capex this year. And, you know, we think of telecoms as being capital intensive. Telecoms spend sort of fifteen to twenty percent of revenue on Capex. Um, and so, you know, seven hundred billion dollars is the guidance from the big four companies this year. Well, you know, telecoms is three hundred, mobile is two hundred. Total telecoms is three hundred. Oil and gas, depending on which definition and which bits of it you're counting, is anything from seven hundred billion to a trillion dollars, I think, from memory. I think it depends which... exactly who you ask. Um, so seven hundred billion dollars a year is an impossibly large amount of money. It's what big global infrastructure costs. It's just a lot of money. Clearly, like, those companies could not spend one and a half trillion next year, or if they did, they'd have to borrow it, and they certainly couldn't sustain that level of spending, um, for any length of time. Um, and so there's a certain point at which, like, that growth has to slow down 'cause, like, there isn't any more money. Um, now, clearly, you can talk about ROI and your ability to produce returns from that investment. Um, and, you know, clearly the capital markets are willing to fund that up to a point, but, like, pick a number at random. Like, we can't spend ten trillion dollars a year on AI infras-- AI infrastructure 'cause there isn't ten trillion dollars a year there to spend on it. So there's a finan-- there are kind of, like, laws of physics caps on the amount of money, um, that's available. I'd hesitate to say something more tangible than that at the moment. I mean, I kind of go... almost go back to what I said at the beginning, that, like, we've got a bunch of mult-- we've got a bunch of multiples. So, um, there's far more demand than supply. On the other hand, the efficiency is increasing massively. Um, we don't know what the next model will be. We don't know where edge or open source come in yet... when edge and open source come in yet. And meanwhile, you are always chasing the next model. And so this is kind of the line that runs across all of it is if the model is only relevant for three to six months, six to nine months, whatever you want to say, and the model costs how many billion dollars, um, and how much infrastructure do you need to do that? Um, I don't think that math is really shaken out yet. I mean, obviously you can... You know, there's a bunch of very clever, savvy tech and an-analysts who spend lots of time trying to put numbers on this. It is kind of like trying to put numbers on bandwidth, internet bandwidth in the late nineties. Like, you kind of know what the rows in the spreadsheet are, but you don't really know where the values are. All you can really say is, "Well, look, it can't be." You know, there's, there's clearly physical limits on this. Um, I think, you know, another way to answer the question is, like, if you're Google or Meta or Microsoft, um, to some extent Amazon, some extent Apple, this is sort of an existential problem. And you have a sort of, you know, a FOMO problem in that, um... So on the one hand, your returns on the investment at the moment are hugely positive. Um, on the other, you can't let other people get away with this without you participating because then your company's gone and you are... You don't want to end up like Microsoft in the 2000s or IBM in the '90s or indeed Meta in the 2010s, where they are kind of continually getting shafted by Apple. Um, so if this is the future of compute, then you need to be participating in it. Um, but obviously at the same time, the CFO is sitting there saying, "Well, yeah, that's great, but, um, how much participation are we talking about here?" And I don't think we've-- You know, it's clearly at a certain point that curve is going to have to taper off 'cause, like, like, there's nowhere else it can go.
- ETErik Torenberg
Do you think there's gonna be a reckoning around token maxing? Uh, is, is it possible that companies have been overshooting AI usage, and when they do proper ROI studies, they'll pull back?
- BEBenedict Evans
Well, obviously, you know, you've had people, like, using the most expensive model to dick around on the internet. Um, which is kind of what happened with mobile, you know, in 2010. Like, you know, you, you got a ten thousand dollar bill and you went and said, "Wait, wait, I thought this was a flat rate bundle. Like, what happened?" Um, so there's like, you've got a, you've got a... You-- obviously you've got a bunch of, of, like, silly/painful stories. Um-I think there's also a p- a point of like... I think what, what, maybe what's slightly more interesting as a question is, um, and clearly there's going to be a point in which, as I've said s-several times, we're at a po- a moment of kind of massive disequilibrium, and the pricing has got to get back into alignment with the cost, and the usage has got to get into alignment with the pricing and the ROI. The challenge is it's a bit tricky. At this early stage, it's quite hard to know what the ROI is. It's, it's rather like giving everybody the internet in the late nineties and saying, "Okay, go off, be more productive." And if you look at, uh, like there's a survey from Deloitte, there's also a survey from the Fed that's in my presentation, where if you go and ask CFOs, where have you seen tech-- seen the benefits? And most of the benefits so far have been stuff that's pretty hard to measure. So like better analytics, better customer support, um, more productivity. You could make more slides more quickly. You could do the analysis more quickly. It's kind of tough to put a financial value on that. It has a financial value, but it's not the same as saying, "Well, we made this new thing with AI and it had this revenue," or, "It saved us this much money." Those things... Just obviously, those things take longer. It's, it's harder to build a new revenue line than to give this to everybody and have them use it to make spreadsheets more quickly. So there's a little bit of like, "Well, well, how long does this take?" I think the other answer to, of course, the problem here, of course, is consumer surplus, which is to say that, um, it's kind of what happened with Excel in that, you know, guess what? You know, if a, if a DCF takes you a week, then you probably only do one or two DCFs. And if a DCF takes you ten seconds, then you do fifty DCFs. But you really can't charge any more money for that. Um, so some of what happens is that, um, uh, these things become competitive necessities and everybody has to buy it and use it. Um, but the cost saving or the productivity gain that you get from it just kind of gets competed away, so you don't get to charge more for it. You know, I mean, if you're, you know, if you're McKinsey and, you know, doing that or, or Bain or BCG, and doing that piece of analysis used to take a week and now it takes a day, um, you probably do five times more analysis and charge your customer the same. And like your cost base hasn't changed either. So like, you know, it's... Which is exactly the way to think about, you know, what happened with, with, with, with, with investment banks and, and financial analysis. You just would-- did way more analysis with probably fewer people and charged customers the same amount of money.
- ETErik Torenberg
Part of your, your... A big part of your thesis is this idea that models are gonna end up as commodities, and yet the, you know, the layer that's raising the most money, uh, you know, i-in the, in the fastest time in history is these foundation mo-model companies. Um, so given that, what advice might you have for them, either, either collectively or we can pick on someone i-individually, um, in order to, in order to adapt?
- BEBenedict Evans
It's not that I know that they're gonna become commodities. My position is more, well, more now. Well, like, "Hey, here is a, here is a chain of argument that says that deterministically it looks like these things will be commodities, and explain to me why they won't be." Um, and that's as far as I would commit to that. Um, I think the, you know, the, the raising all this money, I kind of go back to my point about mobile, which again, has no predictive value, but it's a worthwhile observation, is that the mobile industry is very big and spends a lot of money on infrastructure and isn't very profitable, and all the cool stuff is done by somebody else. And then you can do, you know, well, what's the return on capital? And the answer is, well, it depends on which mark, whether you're in America or Europe or, or India or, or China. Um, but meanwhile, like that was a worthwhile thing to do, and it produced a return for somebody, but then it didn't-- ended up not controlling the whole thing, and other people ended up getting more value from that, um, than they did. Um, you know, I don't have the number in my head. What was Google's net income last year? Was what, fifty billion dollars or something? What's the net income for, you know, the total telecoms industry? I should really subscribe to Bloomberg, then I could just answer these questions instantly. Um, but like a pretty safe bet that Google, Meta, Amazon, um, Microsoft, Apple produce more profits than the entire telecoms industry. Um, so this is a, um... It's a puzzle is you're build-- you're driving the frontier forward. You're kind of caught in this trap that you have to keep competing because otherwise they'll do it and you'll fall behind. You've also got this thing that we haven't talked about at all, which is, you know, hey, aren't we just building AGI? Like we're gonna build God in a box. Which, you know, some people they do believe, although it's kind of hard to, it's hard to analyze, but maybe. Um, so, you know, carry-- we're going to carry on building this stuff. But the practical question is, well, how do you get things that people want to use that aren't software, that aren't, that aren't software development? I mean, that's a good business. Um, is that the only business? If there's, you know, if, you know, take a number of how many hundreds of billions of dollars it is to make the software industry more productive, great. Then what? And that's, you know, that's worth a trillion dollars maybe. But, but then what? Like, how do you expand this into the rest of the economy, into everybody else? Which is why you get these conversations about, you know, partnering private equity, partnering with consultancies, where, you know, exactly as we've been discussing, guess what? It's actually quite hard to work out what to do with this stuff if you're actually running a real company. Um, so you go to Bain, BCG, McKinsey or Infosys and Cognizant and IBM and Ap- Accenture or private equity shareholders. So there is this sort of sense of like on the one hand, you're... Sorry, I'm sort of trying to work out the answer as I speak. But like on the one hand, you're building this big-- these bigger and bigger models, and you kind of feel like you've got to keep doing it. But on the other hand, yes, but what are people doing with it?
- ETErik Torenberg
Yeah.
- BEBenedict Evans
Why do most people look at ChatGPT and not really think of anything to do with it today?
- ETErik Torenberg
Yeah. Uh, l-last question. Is there anything I forgot to ask you or anything else from the presentation that you want to make sure, uh, listeners leave with?
- BEBenedict Evans
Is there? I don't know. It's a seventy, eighty slide presentation, and many of them could be kind of twenty-minute conversations. Um, the thing that I, I used last year and I, I used again is an IBM ad I found from the early fifties, which has got a picture of a sea of, of engineers all holding up slide rules. And it's an IBM ad, and it says, you know, "An el- IBM electronic calculator gives you one hundred and fifty extra engineers." And that's like, how many pitches have you seen at a16z where that was the pitch? Um, and we kind of remember, like we go through these waves of, of these, these fundamental technology changes every ten or fifteen or twenty years. And they're all amazing and change everything in a completely unlike anything that's happened before. And so AI is amazing and transformative and completely unlike anything that's happened before. Mobile was quite a big deal too, and so was the internet, and so were PCs, and so was computing. Those were all also very big deals where it was hard to tell what was going to happen. And so we should sort of presume as a base case, okay, well, we're going to go through that again. And, you know, that will produce a bunch of things that were in people's lives, and it will put a bunch of people out of work. Um, and, you know, there'll be a bunch of stuff that we're not very happy about. Um, and there'll be a bunch of stuff that we all think is great. And then in twenty years' time, we'll kind of forget that there was a world when computers couldn't do that. Um, I mean, here we are, we've been on this call for an hour, and our computers didn't crash, and we're streaming HD video to each other, and it's like, well, of course that worked. In fact, I'm also doing it with my iPhone. So my iPhone is streaming to my Mac over Wi-Fi, streaming video here. And it like, it just works. Like it's magic, and we don't notice it anymore. And I think that's really my kind of one-line description of how all of this is going to end up. It's going to be magic, and in twenty years' time, we'll just say, "Well, of course that's how it is. Computer's always done that."
- ETErik Torenberg
Yeah. That's a great place to, great place to wrap. The presentation is called AI Eats the World. It's on Benedict Evans' website. It is excellent. Uh, g- uh, there's a lot more that we didn't get to, so definitely go check it out. Benedict, this has been a great conversation. Thanks so much for coming to the podcast.
- BEBenedict Evans
Great. Thanks. Great to chat.
Episode duration: 1:02:28
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode gCOKfK9jE7I
