EVERY SPOKEN WORD
45 min read · 9,055 words- 0:00 – 1:00
Intro
- SPSpeaker
I think we don't yet know the shape and form of the ultimate products. Just one just obvious historical analogy is, you know, the personal computer from sort of invention in nineteen seventy-five through to, you know, basically nineteen ninety-two was a text prompt system. Seventeen years in, you know, the whole industry took a left turn into GUIs and never looked back. And then by the way, you know, five years after that, the industry took a left turn into web, web browsers and never looked back, right? And, you know, look, I'm, I'm sure there will be chatbots twenty years from now, but I'm pretty confident that both the current chatbot companies and many new companies are gonna figure out many kinds of user experiences that are radically different that we don't, we don't even know yet. Please join me in welcoming Marc Andreessen and Ben Horowitz with General Partner Erik Torenberg.
- BHBen Horowitz
Follow me into a solo. Get in the flow. And you can picture like a photo. Music makes mellow maintains to make melodies. For MCs, motivates the pranks. I'm everlasting, I do-
- BHBen Horowitz
Thank you for the-
- SPSpeaker
Ben-
- BHBen Horowitz
... Rakim, who did that. [laughs]
- SPSpeaker
Ben, Ben picks the mu-- Ben picked the music.
- BHBen Horowitz
[laughs]
- ETErik Torenberg
Marc, there's been a lot of talk lately about the limitations
- 1:00 – 3:32
Can AI Truly Create? Intelligence vs. Invention
- ETErik Torenberg
of LLMs, that they, they can't do true invention of, say, new science, that they can't do true creative genius, that it's just combining or packaging. You have thoughts here. What say you?
- SPSpeaker
Yeah. So, so for me, it's, yeah, so you get all these questions, and yeah, they usually come in either if, if, you, you know, sort of are, are, are language models intelligent in the sense of like, can they actually, um, you know, can they actually process information and have sort of conceptual breakthroughs the way that people can? And then there's, are, are language models or, or video models creative? You know, can they, can they create new art, um, actually have genuine creative breakthroughs? And of course, my, my, my answer to both of those is, well, can people do those things?
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, and, um, I think there's two, two, two questions there, which is like, okay, even if some people are, quote, unquote, "intelligent" as in having original, uh, conceptual breakthroughs and not just, let's just say, regurgitating the training set, um, uh, or following scripts, um, uh, how, you know, what percentage of people can actually do that? And I'd say I've, I've only met a few. Some of them are here in the room. Um, but, uh, you know, not that many. Most people never do. And then creativity, I mean, how many people are actually genuinely creative, right? And so, you, you kind of point to a Beethoven or a, you know, Van Gogh or something like that, and you're like, "Okay, that's creativity." And yeah, that's creativity, and then how many Beethovens and Van Goghs are there? Obviously, not very many. So, so one is just like, okay, like, you know, if it's, if it, if, if, if these things clear the bar of, you know, ninety-nine point nine nine percent of humanity, you know, then that's pretty interesting just in and of itself. But then I-- you dig into it further, and you're like, "Okay, like, how many actual real conceptual breakthroughs have there ever been actually ever in human history a-as compared to sort of remixing, remixing ideas?" Um, and, you know, like, if you look at the history of technology, it's almost always the case that the big breakthroughs are the result of, you know, usually at least forty years of sort of work ahead of time, you know, four decades. Right. In fact, language models themselves are the culmination of eight decades, right, of, of previous work. And so there's remixing. And then in the arts, it's the exact same thing. You know, novels and, and music and everything, like, you know, there are clearly creative leaps, but, you know, there's tr- just tremendous amounts of influence that come in from, from people who came before. And even if you think about, like, somebody with the creativity of a Beethoven, like, there was a lot of Beethoven in Mozart and Haydn and in the composers that came before. And so there's just tremendous amounts of, of, of remixing and combination. And so it's, it's, it's a little bit of an angel's dancing on the head of a pin question, which is like, if you can get, if you can get, you know, within, you know, I don't know, one, you know, point oh one percent of, of kind of world-beating, uh, you know, generational creativity intelligence, like you're, you're, you're probably all the way there. So I-- so emotionally, I wanna, like, hold out hope that there is, you know, still something special a-about human creativity, and I, I certainly believe that, and
- 3:32 – 6:20
Remix, Originality, and the Nature of Human Creativity
- SPSpeaker
I, and I, and I very much wanna believe that. But, um, I don't know. When I use these things, I'm like, "Wow, they seem to be awfully smart and awfully creative." So I, I'm, I'm, I'm pretty convinced that they're gonna clear the bar.
- ETErik Torenberg
Yeah. The-- I think that seems to be a common theme in your analysis when, when people talk about the limitations of LLMs of, you know, can they do transfer learning or, or just learning in general? You seem to ask, can people do, do that? [laughs]
- SPSpeaker
Yes. Can people do these things? Well, it's like lateral thinking, right? So yeah. So it's like reasoning in or out of distribution, right? And so it's like, okay, I know a lot of people who are very good at reasoning inside distribution. How many people do I actually know who are good at reasoning outside of distribution and doing transfer learning? And, and the answer is, like, I know a handful. Like, I, I know a few, I know a few people, um, where whenever you ask them a question, you get an extremely original answer, and usually, that answer involves bringing in some idea from some adjacent space and basically being able to bridge domains. Um, and so, you know, you'll ask them a question about, I don't know, you know, finance, and they'll, they'll bring you an answer from psychology, or you ask them a question about psychology, and they'll bring you an answer from biology, right, or whatever it is. And so I, I know, you know, I know, I don't know, sitting here today, probably three. I probably know three people who can do that reliably-
- ETErik Torenberg
[laughs]
- SPSpeaker
... out of the, you know... I've, I've got, you know, I've got ten thousand in my address book, um, and so three out of ten thousand-
- ETErik Torenberg
Yeah
- SPSpeaker
... is not, is not that high a percentage. By, by the way, I find this very encouraging. Like, I, I... Yeah, immediately the mood in the room has gone completely to hell.
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, I find this very encouraging. I find this very encouraging because look at what humanity has been able to build, right, despite all of our limitations, right? And, and look at all the creativity that we've been able to exhibit, and all the amazing art and all the amazing movies and all the amazing novels and all the amazing technical inventions and scientific breakthroughs. And so we, you know, we've been able to do, you know, everything we've been able to do with the limitations that we have. Um, and so, you know, I think that, you know, like, you know, do you need to get to the thing where you are one hundred percent positive it's actually doing, you know, original thinking? I don't think so. Uh, I, I, I think it'd be great if you did, and I think ultimately we'll probably conclude that that's what's happening. Um, but, like, I... It's, it's not necess- it's not even necessary for, like, just tremendous amounts of improvement.
- ETErik Torenberg
Ben, we were just celebrating some-
- BHBen Horowitz
Yeah
- ETErik Torenberg
... some hip-hop legends at your Paid in Full, uh, event last week, and so you think a lot about creative genius. H-how do you think about this question?
- BHBen Horowitz
Yeah, I mean, I think that, uh, I, I agree with Marc that it's... Whatever it is, it's very useful, um, even if it isn't all the way that level. Uh, I think that y- you know, there's something about the actual, like, real-time human experience that humans are very into, um, at least in art, whereYou know, with the current state of the technology, kind of the, the pre-training doesn't have quite the, the right data to, to, to get to, um, what you really wanna see. Uh, but it, you know, it's pretty good. [chuckles]
- 6:20 – 9:10
Ben on Hip-Hop, Innovation, and Creative Genius
- BHBen Horowitz
It is.
- ETErik Torenberg
Uh, we're-
- BHBen Horowitz
It's pretty amazing.
- SPSpeaker
How many-- H- so how many true conceptual in-- So Ben, Ben, Ben's been-
- BHBen Horowitz
Yeah.
- SPSpeaker
One of Ben's nonprofit activities is something-
- BHBen Horowitz
Yeah
- SPSpeaker
...called the Paid in Full Foundation, which is h-honoring and actually providing essentially a pension, a pension for, uh, you know, for sort of, uh, you know, the great innovators in, in, in rap and hip-hop. Um, uh, and, and so he has-- he knows and has many of m-
- BHBen Horowitz
Yeah
- SPSpeaker
...we were just at the event, and he, you know, has many of the k- kind of leading lights of that field for the last fifty years, you know, perform, and, and it's really fun to meet them and talk to them. Um, but, like, how many people in that entire field over the course of the last fifty years would you classify as, like, a true conceptual innovator?
- BHBen Horowitz
Yeah, well, y-y-you know, it's interesting. It, well, it de- it depends how broadly you define it, but, you know, th- there were several [chuckles] of them there last, you know, on Saturday. So Rakim, I-
- ETErik Torenberg
Dr. Dre, Nas
- BHBen Horowitz
...I think we, uh, Rakim, you'd certainly put in that category. Dr. Dre, you'd certainly put in that category. George Clinton, you'd certainly put in that category. Um, you know, in a narrower sense, like, Kool G Rap certainly had a new idea. Um, but, y-you know, it de- it depends. Like, a fundamental kind of musical breakthrough, you'd probably just say, like, Rakim and George Clinton. Um, but-
- ETErik Torenberg
Are, are they excited?
- SPSpeaker
So, so, so two out of...
- BHBen Horowitz
Uh, uh, uh, well, I mean, those are-
- SPSpeaker
Yeah
- BHBen Horowitz
...of the guys who were there. [chuckles]
- SPSpeaker
Oh, yeah, yeah. Okay.
- BHBen Horowitz
Yeah. But, uh, yeah, it's a tiny percentage.
- SPSpeaker
Yeah.
- BHBen Horowitz
Tiny, tiny, tiny, tiny, tiny.
- ETErik Torenberg
We had, uh, Jared, at the Fireside last night with Jared Leto. He was talking about how many people in Hollywood are, are really scared or against this, um, what, what's happening here. Is, is it-- What do you see in, you know, when you talk to the Dr. Dres, the Nas, the Kanyes? Are they excited? Are they using it? Are they-
- BHBen Horowitz
Yeah, no, I, the... So everybody, uh, who I speak to, uh, there are definitely people who are scared in music, but, like, there are a lot of people who are very, very interested in it. And particularly the hip-hop guys are interested because, um, it, it's almost like a replay, uh, of what they did, right? The-- They just took other music [chuckles] and they kind of built new music out of it, and I think that, you know, AI is, uh, a fantastic creative tool for them. It, like, way opens up the palette. And then for, y-y-you know, a, a lot of what hip-hop is, is it's, um, kind of telling a very specific story of a specific time and place, which, um, having intimate knowledge and being trained just on that thing [chuckles] is, is actually an advantage, as opposed to being a, like, a generally smart, uh, music model.
- ETErik Torenberg
Um, people al-also use the same logic of, "Hey, whatever is more intelligent will rule whatever is le-less intelligent." And, and Marc, you recently, uh-
- BHBen Horowitz
[laughs]
- ETErik Torenberg
[laughs]
- BHBen Horowitz
Not, not, not said by anybody who owns a cat.
- ETErik Torenberg
Yeah, exactly. [laughs]
- SPSpeaker
[laughs]
- 9:10 – 12:20
Intelligence, Power, and Who Really Leads
- ETErik Torenberg
shape rotators." And, and also-
- BHBen Horowitz
Mm.
- ETErik Torenberg
[chuckles] Someone's clapping. [chuckles] "And also high IQ experts work for mid IQ generalists. What means?"
- SPSpeaker
Yeah. What, what means? Uh, yeah, so, and so, so the PhDs all work for MBAs, right?
- ETErik Torenberg
[laughs]
- SPSpeaker
So it's like, you know.
- ETErik Torenberg
[laughs]
- SPSpeaker
Okay. So yeah, like, so yeah. Well, I just, you know, just take it up a level. It's just like when you look at the world today, do you think we're being ruled by the smart ones?
- ETErik Torenberg
[laughs]
- SPSpeaker
Right? Like, is that, is that your big conclusion from, like, current events, current affairs? Uh, right? Like, okay, we put the geniuses in charge. Like-
- ETErik Torenberg
You mean Kamala and Trump aren't the best, the-
- SPSpeaker
Well, uh, let's not even be specific towards the US.
- ETErik Torenberg
Yeah.
- SPSpeaker
Let's just look all over the world. Um, uh, you know, uh, yeah. And so, like, it's just like, I, it, it, there, there's this thing. Uh, so y- I think two, two things are true. One is we, we probably all kind of underrate the importance of intelligence. Um, and actually there, there's a whole kind of backstory here of, like, intelligence actually turns out to be this, like, incredibly inflammatory, you know, kind of topic for lots of reasons o-over the last hundred years, um, uh, which, which we could, we could talk about in great detail. But, like, it, it, and, you know, and it's, and the, e-even the just very idea that, like, some people are smarter than other people, you know, just, like, really freaks people out, and peop-people don't like to talk about it. It's, we really struggle with that as a society. And so, like, and, and then it is true that intelligence is like, in humans, intelligence is correlated to almost every kind of positive life outcome, right? And so, um, intelligence, uh, generally in the social sciences, what they'll tell you is what they call fluid intelligence or, or the G factor or IQ is sort of, it's sort of zero point four correlated to basically everything. Um, and so it's, it has zero point four correlation to, like, educational outcomes and, like, ec- you know, professional outcomes and, and, you know, income, and by the way, also, like, life satisfaction and by the way, non-violence, you know, being able to solve problems without physical violence and so forth. And so, like, on the one hand, like, we probably all underrate intelligence. Um, on the other hand, the people who are in the fields that involve intelligence probably overrate intelligence. Um, and you, you might even, you might even coin a term like maybe like intelligence supremacist or something like that, where it's just like, oh, like, intelligence is very important, and so therefore, maybe it's, like, the most important thing or the only thing. But, but then you look at reality and you're like, "Okay, that's clearly not the case."
- BHBen Horowitz
Yeah, it's still zero, only zero point four, right? Yeah.
- SPSpeaker
Well, so to start with, it's only zero point four, and, and, you know, in the social sciences, zero point four is a giant correlation factor, right? Like, most, most things that where you can correlate, whether it's, you know, genes or observed behavior or whatever, anything in the social sciences, the correlations are much smaller than that. So zero point four is, is tiny, but it's still only zero point four. So even if you're like a full on-If you-- Even if you're, like, a full-on genetic determinist, and you're just like, you know, g-genetic IQ just, like, drives all these outcomes, like, it, it still doesn't explain, you know, point six o-of the correlation, and so that leaves it. But, but, but that's just on the individual level. Then you just look at the collective level. Well, you just look at the collective level, and it's, it's like a famous, famous observation is you take a bu-- you take a bunch of s-- you, you take any group of people, you put them in a mob, and the mob is dumber [laughs]
- ETErik Torenberg
[laughs]
- SPSpeaker
...right, than the average. A-and, and you put a bunch of smart people in a mob, and they definitely turn dumber. Like, and, and you see that all the time, right? Um, uh, and so y-you put people in groups, and they, they, they behave very differently. And then you, you create, and then you create these, you know, questions around, like, who's in charge, whether who's in charge at a, at a company or who's in charge of a, of a country. And, like, it, it's-- Whatever the filtration process, it's clearly not-- it's not, it's, it's not, it's certainly
- 12:20 – 16:40
Beyond IQ: Leadership, Emotion, and Theory of Mind
- SPSpeaker
not only on IQ, and it may not even be primarily on IQ. And so, so therefore, it's just like this assumption that you kind of hear in some of the AI circles, uh, which is like inevitably the smart, you know, kind of thing is gonna govern the dumb thing. Like, I, I just think that's, like, very easily, uh, it's just sort of very easily and obviously falsified. Like, intelligence isn't sufficient. And then you just, you just, you just convey it. You know, we're all in this room lucky enough to know a lot of smart people, and you, you just kind of observe smart people. And, like, some smart people, you know, really figure out how to have their stuff together and become very successful, and a lot of smart people never do. Um, and so there, there's, there, there must be... There obviously are, and there, and there, in fact, must be many other factors that have to do with success, um, and have to do with, like, who's in charge, uh, than just raw intelligence.
- ETErik Torenberg
I-it begs the q- the follow-up question of what are, what are some examples of what that might be, you know, skills sort of outside of intelligence, and more particularly, specifically, why couldn't AI systems, you know, l-learn them?
- SPSpeaker
Yeah. So Ben, like, what, what, what-- Other than intelligence, what, what in your experience determines, for example, success in leadership or in entrepreneurship or in, in, uh, solving complex problems or organizing people?
- BHBen Horowitz
[laughs] Yeah. Well, there, there, there are many things. Um, you know, like, a lot of it is, uh, being able to have a confrontation in the correct way. And there-- like, there's some intelligence in that, but a lot of it is just under- really understanding who you're talking to, you know, being able to interpret everything about how they're thinking about it and just kinda generally seeing decisions through the eyes of the people working in the company, not through your eyes, is a skill that y-you know, you develop by talking to people all the time, understanding what they're saying, so forth, these kinds of things. And it's just, um, y-you know, it's certainly not an IQ thing and not that... L-like, I, I could imagine [laughs] an AI training on any individual and, like, figuring it all out and knowing what to say and so forth. Um, but then you also need that integrated with, uh, you know, like, whatever the business ought to be doing. So you're not, you're not trying to do what's popular. You're trying to get people to do what's correct even if they don't like it, and, you know, that's a lot of management. So, uh, it's not a problem anybody's working on [laughs] AI currently, but maybe they will.
- SPSpeaker
It's some co-- Right.
- BHBen Horowitz
Yeah.
- SPSpeaker
Some combination of, like, courage-
- BHBen Horowitz
Yeah
- SPSpeaker
...some combination of motivation, some combination of, um, of emotional, uh, understanding, theory of mind.
- BHBen Horowitz
Yeah. What, you know, what do people want, like, you know, married to, you know, what needs to be done? And then, like, how talented are they? Like, which ones can you afford? Like, if they jump out the window, it's fine. You know, which one's not fine? You know, this kind of thing. It's a... Th-there's a lot of, like, weird subtleties to it, uh, and it's very situational. I think the hardest thing about it, uh, and why management books are so bad is because it's situational. Um, you know, like your company, your product, your people, your org chart is very, very different than, you know, you know, here are the five steps to building a strategy. It's like, well, that's the most useless fucking thing I ever read because i-it has n-nothing to do with you.
- SPSpeaker
So one of the interesting things o-on this, like, on... This is... Right. The co- the concept of theory of mind is really important, right? So theory of mind is can you in your head model what's happening in the other person's head, right? And, and, and you would think that maybe that, you know, maybe obviously people who are smarter should be better at that. It turns out that that may not be true, and I'll... Th-the reason to believe that that's not true, which, which is as follows. So the, so the, the, the U.S. military is- was the early adopter and has continued to be sort of the leading adopter in, in, in U.S. society of, of act- of actually IQ testing, and they, uh, they, they, they basically launder it through something called the ASVAB, which is their, like, a vocational aptitude battery test, but it's basically an IQ... It's, it's essentially an IQ test. Um, and, uh, so they, they, they, they still use basically explicit IQ tests, and they, they slot people, uh, into different specialties and, and roles, uh, you know, in, in part w-with, uh, according to IQ, um, i-including into leadership roles. Um, and, and so they, they know what everybody's IQ is, and they, they kind of organize, organize around that. And one of the things that they found over the years is if the leader is more than one standard deviation of IQ away from the followers, it's a real problem.
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, and, and, and that's true in both directions, right?
- ETErik Torenberg
Yeah.
- SPSpeaker
Um, if, if the leader is not smart enough to be able to, right, manage the, you know, is, is... To, to be able to mo- You know, for somebody who is less smart to model the mental behavior of somebody who's more smart is inher-
- 16:40 – 20:14
Embodied Intelligence – The Mind-Body Question
- SPSpeaker
of course, inherently very challenging and maybe impossible. But it turns out the reverse is also true, which is if the leader is two standard deviations above the norm of the organization that he's running, he also loses theory of mind, right? It, it, it's, it's v- actually very hard for very smart people to, um, to, to model, uh, the internal thought processes of even moderately smart people. Um, and so th-there, there's actually li- there's actually a real, there's actually a real need to have a, a level of connection there that's not just... Right? A-and therefore, by inference, if you had a person or a, or a machine that had a, you know, a thousand IQ or something like it, it may just be... It would be so alien. Its understanding of reality would be so alien to the people or the things that it was managing that it wouldn't, it wouldn't even be able to connect in any sort of, any sort of realistic way. So again, this is a very good argument that, like, it... Yeah, this is... The world is gonna be far from organized by IQ for, yeah, [laughs] for centuries to come.
- BHBen Horowitz
Yeah, and Zuckerberg had a great line, which is, "Intelligence is not life." [laughs] And life, l-life has a lot of dimensionality to it that is independent of intelligence. I think that, you know, if you spend all your time working on intelligence, you lose track of that.
- ETErik Torenberg
We, we sometimes say [clears throat] about, uh, some specific people that they're too smart to properly model or, or, or, you know, um, too-- they sort of, sort of assume too much rationality on other people, or they just o-overthink things or over-rationalize them. Um, yeah, just to, to your point that it's, it's not everything.
- BHBen Horowitz
Yeah. Yeah. Yeah. [chuckles] People often... People seldom do what's in their best interest, [chuckles] I should say.
- ETErik Torenberg
Yeah.
- BHBen Horowitz
Yeah, yeah.
- SPSpeaker
You know, I also suspect, this kind of gets more into the biology side of things. I, you know, there's more and more scientific evidence that basically also that, like, human, human cognition, h-human cognition or human, I don't, whatever you wanna call it, self-awareness, information processing, decision-making sort of experience, um, uh, is, is not purely a brain. Like, the basically d- the do- the sort of mind, famous mind-body dualism is just not correct. Like, a-and again, this is an argument against sort of IQ supremacism or intelligence supremacism is it's not... You know, we, we-- Human beings didn't experience existence just through the rational thought, uh, and, and, and specifically not through just the rational thought of the brain, but rather it, it's a whole body experience, right? And there's, there's, there's aspects of our nervous system, and there's aspects of everything from our gut biome to, you know, to, to, you know, to, to smells, you know, to the olfactory senses and, and, um, you know, and hormones and, like, all, all kinds of, like, biochemical kind of aspects to life. Um, like, I su- If you just are gonna track the research, I suspect we're gonna find is human cognition is a full body experience, uh, much, much more than, much more than people thought. Um, and, and so therefore to actually... To-- And this, you know, this is like a... And this is, you know, one of the kind of big fundamental challenges in the AI f- AI field right now, which is, you know, the form of AI that we have working is, is the f- is the fully mind-body dual version of it, which is it's just a disembodied, you know, like a disembodied brain. You know, the robotics revolution for sure is coming when that happens, when we put AI in, in physical objects that move around the world. You know, you're, you're gonna be able to get closer to having that kind of, you know, inte-integrated intellectual, physical, you know, experience. You're gonna have sensors in the robots. They're gonna be able to, you know, gather a lot more, you know, real world data. And so maybe you can start to actually think about, you know, synthesizing-
- ETErik Torenberg
Interesting
- SPSpeaker
... you know, a more advanced model of cognition. And, and, you know, maybe we're gonna actually discover more both about how the human version of that works and also how the machine version of that works. But it-it-it's just, to me at least reading the research, like, that all those ideas feel very nascent, and we have a lot of work to do to try to figure that out.
- ETErik Torenberg
Do you have a sense for h-h-how they are, LLMs are at theory of mind today? Um, or do you have a sense where the limitations are? You, you like to talk to them a lot. Are there any particular things that are particularly surprising to you as you do?
- SPSpeaker
Yeah. I would say generally they're really good. Um, yeah. And so, like, one of the, one, one of the more... I, I find one of the more fascinating ways, you know, to, to, to work with language models is actually have them create personas, um, and, uh, and then, you know, basically have... Well,
- 20:14 – 23:02
How Advanced AI Really Is at “Theory of Mind”
- SPSpeaker
actually, so the way... I like, I like, you know, I like basically, I like Socratic dialogues. I like when things are argued out in, like, a Socratic dialogue. And so you, you know, tell a, tell any advanced LLM today to create a Soc-Socratic dialogue, and it'll either make up the personas. You can tell what it is. It does a good job. It has this very, very annoying property, which is it wants everybody to be happy.
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, and so it wants all of its personas to agree. Um, and so by default, it will have a, uh, it will have a briefly interesting discussion, and then it will sort of figure out-
- ETErik Torenberg
[laughs]
- SPSpeaker
... you know, basically, it... Like you're watching, I don't know, a PBS special or something. It'll, it'll kind of figure out how to bring everybody in agreement.
- ETErik Torenberg
Mm-hmm.
- SPSpeaker
Everybody's happy at the end of the discussion. And of course, I fucking hate that.
- ETErik Torenberg
[laughs]
- SPSpeaker
Like, it drives me nuts. I don't want that.
- ETErik Torenberg
[laughs]
- SPSpeaker
So instead, I, I tell it. I'm like, "Make the conversation more tense," right?
- ETErik Torenberg
[laughs]
- SPSpeaker
"And, like, fraught with, like, anger and, like, you know, people sh- you know, gonna get, like, increasingly upset-
- ETErik Torenberg
[laughs]
- SPSpeaker
... throughout the conversation." And then it starts to get really interesting. Um, and then I, and then I tell it, you know, "Bring it, you know, use-- introduce a lot more cursing."
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, you know, really have them go at it.
- ETErik Torenberg
[laughs]
- SPSpeaker
Like, all the gloves come off. They're going for full, full, you know, you know, reputational destruction of each other.
- ETErik Torenberg
And you do a lot of these skits. You're like-
- SPSpeaker
Yeah, skits. And then I, I get carried away, and then I'm like, it turns out they're all, like, secret ninjas, and then they all start fighting.
- ETErik Torenberg
[laughs]
- SPSpeaker
And you've got Einstein, you know, you know, you know, hi-hitting, you know, Niels Bohr with nunch-nunchucks. And it, it...
- ETErik Torenberg
[laughs]
- SPSpeaker
And by the way, it's happy to do that too.
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, so you do have to, you have to, you have to control yourself. But it is, it is very good at theor-at theory of mind. And then I'll, I'll give you another example. There's a, there's a startup actually in the UK, uh, in, uh, in, in the world of politics. And, and what, what they've found is that, um, they found that language models now are, are good enough, so specifically for, for politics, which is sort of a sub-subcategory where th-this, this idea matters. Um, so, you know, in politics, people do focus group. You do focus groups of voters all the time. And, and by the way, many businesses also do that. Um, you know, so you get a bunch of people together from different backgrounds in a room, and you kind of guide them through a discussion and try to get their, their points of view on things. And, and focus groups are often surprising. Like, politicians who... If you talk to politicians who do focus groups, they're often surprising. They're often surprised by the things that they thought voters cared about is actually not the things that voters care about, and so you can actually learn a lot by doing this. But focus groups are very expensive to run, and then there's a long lag time 'cause they have to be actually physically organized, and you have to recruit people and vet people and, and, and so forth. Um, and so it turns out that the, the, the, the state-of-the-art l-models now are, are good enough at this that they can actually... They can, they can correctly, um, accurately reproduce a focus group of real people, um, in-inside the model. Um, so, so, so they're good enough to clear that bar. In other words, you, you can basically have a focus group actually happening in the model where you create personas in the model, and then it actually accurately represents, you know, a college student from, you know, Kentucky as contrasted to a housewife from Tennessee as contrasted to a, you know, whatever, whatever. You, you just, like, specify this. And so, you know, they're good enough to clear, they're good enough to clear that bar, and, you know, we'll, we'll see how far they get.
- ETErik Torenberg
I wanna segue to the bubble conversation. Uh, Amin and G2, Jensen and Matt spoke about the enormous scale of physical infrastructure being built out. AI CapEx is
- 23:02 – 27:58
Are We in an AI Bubble? Fundamentals vs. Hype
- ETErik Torenberg
1% of GDP. H-how should we understand and think about this bubble question?
- BHBen Horowitz
[chuckles] Well, I think the fact that it's a question means we're not in a bubble. That's the first thing to understand. I mean, bu- a bubble is a psychological phenomenon, um, as much as anything, and in order to get to a bubble, everybody has to believe it's not a bubble. That, that's sort of the, the, the core mechanic of it. And they, you know, we call that capitulation. Everybody just gives up, like, "Okay, I'm not gonna short these stocks anymore. I'm tired of losing all my money. I'm gonna go long." Uh, and we saw that actually in y-you know, and a little bit of question, like, really, what was the tech bubble? Um, but in the kind of dot-com era-Right as the prices went through the roof, Warren Buffett started inviting-- investing in tech. So like... And he swore he would never invest in tech because he didn't understand it. And so if he capitulated, nobody was saying it was a bubble when it became like a quote-unquote bubble. Now, if you look at that phenomenon, um, the internet clearly was not a bubble. Uh, you know, it was a real thing. It was in the short term, there was a kind of price dislocation that happened because, uh, the, the market, um, you know, there were just not enough people on the network to make those products go at the time. Uh, and then the prices kind of outran the market. You know, in AI, it's much harder to see that because there's so much demand in the short term, right [chuckles] ? Like, we don't have a demand problem right now, and like the idea that we're gonna have a demand problem five years from now, to me, seems quite absurd. Uh, you know, could there be like weird bottlenecks that, that appear? You know, like we just-- at some point, we just don't have enough cooling or something like that. You, you know, maybe. But like, like right now, if you look at demand and supply and what's going on and multiples, uh, against growth, it doesn't look like a bubble at all to me. Um, but [chuckles] I don't know. Do you think it's a bubble, Marc?
- SPSpeaker
Yeah. Look, I, I-- I would just say this. Yeah, like nobody know-- so nobody knows in the sense of like the experts. Like if you're talking to anybody at like a hedge fund or a bank or whatever, like they definitely don't know. Um, uh, generally, the CEOs don't know. So it, the, the, these-
- BHBen Horowitz
But by the way, a lot of VCs don't know. They just get upset. Like VCs get like emotionally upset when you guys have higher valuations.
- SPSpeaker
[chuckles]
- BHBen Horowitz
Like it, it makes them like, like angry.
- SPSpeaker
[chuckles]
- BHBen Horowitz
Uh, and, you know, and I, I get it all the time, and I'm like, "What are you mad about?" Like, "The shit is working, man. Be happy. Come on."
- SPSpeaker
[chuckles]
- BHBen Horowitz
But so, so like there's a lot of emotion around like people wanting it to be, be a bubble.
- SPSpeaker
Yeah. No- nothing's worse than passing on a deal than having the company become a great success. Like that's just, I was just saying, it puts you-
- BHBen Horowitz
Yeah. It's like, "That, that valuation's outrageous." [chuckles]
- SPSpeaker
You, you could be furious about that for thirty years in our business.
- BHBen Horowitz
[chuckles]
- SPSpeaker
It's, it's, it's amazing. Um, and you can find... Yeah, you come up with all kinds of reasons to cope and, and explain why it wasn't your mistake. But it's, you know, it's the world, it's the world that's wrong, not me, right? So there, there's a lot of that. Yeah. Uh, yeah. So I, I just-- I would just, I would just say, like I would always say, bring, bring the conversation back to ground truth fundamentals. And the, the two big ground truth fundamentals are, number one, does the technology actually work? Um, you know, can it deliver on its promise? And then number two is, are customers paying for it? Um, and if those, if those two things are true, then it's very hard to, it's very hard to, uh... Like as long as those two things stay grounded, um, you know, gener- generally, generally things are gonna, are I think are gonna be on track.
- ETErik Torenberg
Yeah. When Gavin was up here with DG, he said ChatGPT was a Pearl Harbor moment for Google, the moment when the giant wakes up. When, when we look at history and, and platform shifts, what determine whether the incumbent actually wins the next wave versus, versus new entrants? Or how should we think about that in, in AI?
- BHBen Horowitz
[sighs] Well, you know, reacting to it is important, um, but that doesn't mean... Like, like I-- It's a Pearl Harbor moment, I think. Uh, [chuckles] Google got their head out of their ass. That was the sound of it. [laughs]
- ETErik Torenberg
[laughs]
- SPSpeaker
[laughs]
- BHBen Horowitz
Uh, so you, you know, they're not gonna get completely run over, but nonetheless, like I, I don't think OpenAI is going away, so like they, they definitely let that happen. Um, yeah, some of it is speed, and then just look, it's execution over a long period of time. And, uh, you know, some of these very large companies, to varying degrees, have lost their ability to execute. And so if you're talking about a brand-new platform and you're talking about, you know, kind of building for a long time, it's like, you know, Microsoft got caught with their pants down on Google. Um, Microsoft's still like very strong, but they missed that whole opportunity. They also missed the opportunity... You know, Apple was nothing, and Microsoft fully believed that they were gonna own mobile computing. They completely missed that one. But they were still so big from their Windows
- 27:58 – 31:00
Platform Shifts, Google’s Wake-Up Call, and New UX Paradigms
- BHBen Horowitz
monopoly, they could build into other things. So, you know, I think generally the new companies have won the new markets. Uh, and that doesn't mean the big company-- The biggest companies, the biggest monopolies from the prior generation just last a long time, is the way I would look at it.
- SPSpeaker
Yeah. I, I also think we don't quite know... Like it's all happened so fast. We, we actually don't... I think we don't yet know the shape and form of the ultimate products.
- BHBen Horowitz
Yeah.
- SPSpeaker
Um, right? And so, and so like... 'Cause it's, it's tempting, and this is kind of what, what always happens, it's kinda, it's kinda tempting to look, and I'm not saying what, you know, it's what these guys did on stage, but it's kinda tempting to look... You sometimes you hear the kinda reductive version of this, which is basically it's like, oh, there's either gonna be a chatbot or a search engine, right? The, the competition is between a chatbot and a search engine, and the, the problem Google has i- is the classic problem of disru- you know, disruption. Are you gonna disrupt the ten blue links model and swap in a, a, you know, ad, you know, sort of, uh, AI answers and, you know, potentially disrupt the advertising model? And then the problem OpenAI has is they have the, the full, you know, the full chat product, but, you know, they don't have the advertising yet, and they don't have the distribution, Google-scale distribution. And so, you know, you kinda say, okay, that's a fairly, it's a fairly cla- Like that'd be straight out of a, like, you know, the innovator's dilemma, you know, business textbook. Like this is just a very clear, you know, one, one versus one, you know, kinda dynamic. But that assumes that, you know, or the mistake that you can make in thinking that way is that assumes that the forms of the product in five, ten, fifteen, twenty years that, that, that are gonna be the, the main things that people use are gonna be either a search engine or a chatbot, right? Um, and, and, you know, the, I just... There's, you know, there's just obvious historical analogies. One just obvious historical analogy is, you know, the, the, the personal computer from sort of invention in 1975 through to, you know, basically 1992, you know, was, was a te- was a text prompt system, right? Um, you know, and at the time, by the way, an interactive text prompt was a big advance over the previous generation of like punch card systems, time-sharing systems. Uh, and then, you know, it was, you know, 1992, so it was what? Seven- seventeen years in, you know, the whole industry took a left turn into GUIs and never looked back, you know? And, and then by the way, you know, five years after that, the industry took a left turn into we-web browsers and never looked back, right? And so the very shape and form and nature of the user experience and how it, and how it fits into our lives, uh, you know, is, is, is I think still unformed. And so like, and, and, you know, look, I'm, I'm sure there will be chatbots twenty years from now, but I, I'm, I'm pretty confident that, um, you know, both the current chatbot companies and many new companies are gonna figure out many kinds of user experiences that are radically different that we don't, we don't even know yet.Yeah, and by, and by the way, that's one of the things, of course, that keeps the tech industry fun, which is it, you know, it's, especially on the, especially on the software side, you know, it's, it's not, it's not, it's not obvious what the shape and form of the products are, and there's just... I think there's just tremendous headroom for invention.
- ETErik Torenberg
As, as you're coaching en-entrepreneurs and the entrepreneurs in this room, what, what else feels different a-about this era or, or, or sort of other advice that you find yourself disp- whether it's around, uh, sort of the talent wars that are going on or oth-other aspects that feel unique to this era, what, what other advice do you wanna be leaving our en-entrepreneurs with?
- BHBen Horowitz
That's unique to this era. Well, like I, I, I actually think you said the right thing, which is this is a unique era. And so trying to
- 31:00 – 34:14
Coaching Founders in a Unique AI Era
- BHBen Horowitz
learn the organizational design lessons of the past or trying to learn, um, kind of too much from the last generation is-- can be deceptive because things really are different. Like the way these, you know, the way your companies are getting built is, is quite different in, in many aspects. Uh, and, you know, the types of, [chuckles] you know, what the... Just like our observa-observation on like PhD AI researchers is just very different than like a traditional, um, engineer, full stack engineer or something like that. So, you know, I, I think you do have to think through a lot of things from first principles, uh, because it is different. Uh, and like, you know, ob-observing from the outside, it's really different.
- ETErik Torenberg
Yeah.
- SPSpeaker
Yeah, and I would just offer, look, I, I do think things are gonna change. So I already talked about I think the shape and form of products is gonna change. Um, uh, and so like I think there's still a lot of creativity there. I also think, and I, I, I was supposed to say, I think that, um, like in a, in a world of supply and demand, the thing that creates gluts is shortages, um, right? So like wh-when something becomes too scarce, there becomes a massive economic incentive to figure out how to unlock new supply. And so the, the, the, the current generation of AI companies are really struggling with, uh, particularly shortages of, of the really talented AI, AI researchers and engineers, and then they're really, you know, uh, uh, challenged with a shortage of, of infrastructure capacity chips and, and data centers and power. Um, I, I, I don't wanna call timing on this. There will come a time when both of those things become gluts. Um, a-and so I don't know, I don't know that we can plan for that, um, although I, I would just say the following. Number one, um, the, the, the researcher engineer side of things, it is striking, it is striking the degree to which, um, there are excellent, you know, outstanding models coming out of China now, um, and, you know, and, and, and, and mult- from multiple companies and, you know, specifically, you know, uh, DeepSeek and, and, uh, Qwen and Kimi. Um, it is striking how the teams that are making those are not, you know, the name brand. You know, for the most part, these are not like the name brand people with their names on all the papers. Um, a-a-and so like Ch-China is successfully decoding how to like basically take young people and train them up in the field.
- BHBen Horowitz
Well, and xAI to a large extent too.
- SPSpeaker
Yeah. And, yeah. And so I, I think that, I think there's gonna be... A-a-and look, it makes sense up until... It, it, it makes sense that for a while it's gonna be this super esoteric skill set, and people are gonna pay through the nose for it. But like, you know, there's no question the information is, right, being transferred into the environment. People are learning how to do this. Um, you know, college kids are figuring it out. Um, and so, um, you know, there's, there's... A-and I don't know that there's ever gonna be a talent glut per se, but like I, I think for sure there's gonna, there's gonna be a lot more people in the future who of course know, know how to build these things. Um, and then, and then by the way, also of course, you know, AI building AI, right? So the, the, the, the, the tools themselves are gonna be better, better at, at, at contributing to that. And so, a-a-and I think that, I think this is good 'cause I think that, you know, the current level of, of shortage of, of engineers and researchers is, is, is too constraining. And then, and then on the chip side, I don't, I don't wanna... I'm not a chip guy and I don't wanna call, call it specifically, but like it's never been the case, it's never been the case in the chip industry that there's ever, you know... E-every, every shortage in the chip industry has always resulted in a glut, uh, because the, the, the profit, the profit pool of a shortage, the, the margins get too big. The incentive, uh, for other people to come in and figure out how to commoditize the function get too big. And so, you
- 34:14 – 37:10
Talent, Chips, and the Coming Glut Cycle
- SPSpeaker
know, N-Nvidia has like, you know, the best position probably anybody's ever had in chips, but notwithstanding that, I, I find it hard to believe that there's gonna be this level of pressure on infrastructure in five years.
- BHBen Horowitz
Yeah, and e-even if the bottleneck within the infrastructure moves, so if, if it becomes power, if it becomes cooling or, or, or anything else, then you'll have a chip glut for sure, yeah.
- SPSpeaker
Right. So, so I, I think over the... I w- I would just say this. It's likely the challenges that we, that we all have in five years from now are gonna be different challenges.
- ETErik Torenberg
Yeah.
- BHBen Horowitz
Yeah. Yeah, yeah. Like don't, [chuckles] don't... Definitely, this industry of all industries, don't look at us as static. Like, you know, the positions, uh, could change very, very fast.
- ETErik Torenberg
L-let's actually c-close on, on more of this macro note. Marc, Marc, you mentioned China. Last month we were in DC, and one, one of the big questions the senator has is how should we make sense of sort of the state of the AI race vis-à-vis China? Do you wanna share just the, the high level, um, summary of what, what you shared with them?
- SPSpeaker
Yeah. So my sense of things, and I, and I think the current, I think the current, if you just observe currently, specifically like DeepSeek, Qwen, and Kimi and, and these models coming out of China, I... My, my sense basically is like I would say the U.S.-... ju- specifically in the West generally, but, you know, more and more specifically the U.S., it... Like, the conceptual innovations are, are, you know, have, have been, you know, coming out, coming out of, coming out of the U.S., coming out of the West, you know, kind of the, the, the big kind of conceptual breakthroughs. Um, uh, China i-is extremely good at picking up ideas and implementing them and, and scaling them and, and commoditizing them and, and, you know, that's, the-they do that obviously throughout the manufacturing world. Um, and, and they're doing it, but now very, I think, successfully, uh, sort of i-in AI. Um, and so I would say they're, they're running, they're running the catch-up game, like, really well. Um, you know, and then there's, there's o- sort of always this question of, like, how much of that is, like, being done, let's just say, like, or, or authentically, um, uh, you know, through hard work, um, and smart people, and then how much is being done with maybe a little bit of help. Um, maybe a little USB stick, uh, in the middle of the night, uh, you know, kind of help.
- ETErik Torenberg
[laughs]
- SPSpeaker
Um, so, uh, you know, there's always a little bit of a question. But, like, either, either way, uh, you know, they're, they're, they're doing a great job. Uh, o- obviously they, they aspire to, you know, more than that, um, and there are many very smart and creative people in China. And so, you know, it'll be interesting now to see, you know, the level to which the conceptual breakthroughs start to come from there and whether they, whether they pull ahead. Um, and so s- but, like, I would say, like, w-what we tell people in Washington is, like, look, this, this is a f- this is now a f- this is a full-on race, it's a foot race, it's a game of inches. Like, we're not gonna have a five-year lead. We're gonna have, like, maybe a six-month lead. Like, we have to run fast. We have to win. Like, we, we have to, we have to do this. We, we can't... And then we can't put constraints on our companies that the, the, the Chinese, uh, government isn't putting on their own companies. And so, um, or, you know, we'll just lose. And, you know, do, do you really wa- do you really wanna wake up in the morning and live in a world, you know, really controlled and run by Chinese AI? Most of us would say, no, we, we don't wanna live in that world. Um, and so, um, so, s-so that-- so there's that. And, and I would say I feel moderately good about that, just 'cause I think the... I think we're, we're really good at software. Um, y-you know, the minute this goes into, you know, embodied AI in the form of robotics, I think things get a lot scarier
- 37:10 – 38:52
The U.S.-China AI Race and the Robotics Future
- SPSpeaker
and pro- you know, this is the thing I'm now spending time in DC trying to really educate people on, which is, you know, the Chi- Because the U.S. and the West have chosen to deindustrialize to the extent that we have over the last forty years, um, you know, th- China, China specifically now has this giant industrial ecosystem for building elec- you know, sort of mechanical, electri-electrical, and, um, uh, and, uh, semiconductor and now software, you know, devices of all kinds, including phones and drones and cars, um, a-and robots. Um, and so, uh, you know, there's gonna be a phase two to the A-AI revolution. It's gonna be robotics. It's gonna happen, you know, pretty quickly here, I think. Um, and when it does, like, even if the U.S. stays ahead in software, like, the robot's gotta get built, and that's not an easy thing. And it's not just, like, a company that does that, it's gotta be an entire ecosystem. Um, and it's, you know, it's gonna be... You know, it's gonna... You know, like, I mean, the, you know, the car industry was not three car companies. It was thousands and thousands of, of, of component suppliers building all the parts. And it's been the same thing for airplanes and the same thing for computers and everything else. Um, it's gonna be the same thing for robotics. Um, and, you know, by default, sitting here today, that's all [chuckles] gonna happen in China. And so even if they never quite catch us in software, they might just, like, lap us in hardware, and, and, and, and that'll be that. Um, you know, the, the good news is I, I think there's a growing awareness in... There, there's a growing awareness, I would say, across the politics- political spectrum in the U.S. that, like, deindustrialization went too far, uh, and there's a growing desire to kind of figure out how to reverse that. Um, and, um, you know, I'd say I'm guardedly optimistic that we'll be making progress on that, but I think there's a lot of work to be done.
- ETErik Torenberg
On that call to arms, uh, let's wrap. Uh, thank you, Marc and Ben. To, to wrap up, I'd like to welcome Jennifer-
- SPSpeaker
All right, thank you.
- ETErik Torenberg
Sorry about that.
- SPSpeaker
Thank you, everybody. [audience applauding]
- 38:52 – 39:17
Reindustrialization and What Comes Next
- SPSpeaker
[upbeat music]
Episode duration: 39:17
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode Y7dwbJ0AtUA
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome