Marc Andreessen: Future of the Internet, Technology, and AI | Lex Fridman Podcast #386

Marc Andreessen 0:00 ... the competence and capability and intelligence and training and accomplishments of senior scientists and technologists working on a technology, and then being able to then make moral judgments in the use of that technology, that track record is terrible. That track record is catastrophically bad. The policies that are being called for to prevent this, I think were gonna cause extraordinary damage.

Lex Fridman 0:20 So, the moment you say, "AI is gonna kill all of us, therefore we should ban it or that we should-

Marc Andreessen 0:25 Yeah.

Lex Fridman 0:25 ... uh, regulate it," all that kinda stuff, that's when it starts getting serious?

Marc Andreessen 0:27 Or start, you know, military airstrikes on data centers.

Lex Fridman 0:30 Oh, boy. The following is a conversation with Marc Andreessen, co-creator of Mosaic, the first widely used web browser, co-founder of Netscape, co-founder of the legendary Silicon Valley venture capital firm, Andreessen Horowitz, and is one of the most outspoken voices on the future of technology, including his most recent article, Why AI Will Save the World. This is a Lex Fridman podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Marc Andreessen.

Lex Fridman 1:06 I think you're the right person to talk about the future of the internet and technology in general. Um, do you think we'll still have Google Search in five, in 10 years? Or search in general?

Marc Andreessen 1:18 Yes. You know, it would be a question if the use cases have really narrowed down.

Lex Fridman 1:22 Well, now with AI-

Marc Andreessen 1:24 Yeah.

Lex Fridman 1:25 ... and AI assistants being able to interact and expose the entirety of human wisdom and knowledge and information and facts and truth to us via the, uh, natural language interface, it seems like that's what search is designed to do. And if AI assistants can do that better, doesn't the nature of search change?

Marc Andreessen 1:46 Sure. But we still have horses.

Lex Fridman 1:48 Okay (laughs) .

Marc Andreessen 1:50 (laughs)

Lex Fridman 1:50 Uh, when's the last time you rode a horse?

Marc Andreessen 1:52 It's been a while.

Lex Fridman 1:53 All right (laughs) . But, uh, what I mean is-

Marc Andreessen 1:57 Yeah.

Lex Fridman 1:57 ... will we still have Google Search as the primary way that human civilization uses to interact with knowledge?

Marc Andreessen 2:05 I mean, search was a technology, it was a moment in time technology, which is you have, in theory, the world's information out on the web and, you know, this is, this is sort of the optimal way to get to it. But yeah, like, and, and by the way, actually Google, Google has known this for a long time. I mean, they've been driving away from the ten blue links for, you know, for like two days. They've been trying to get away from them for a long time.

Lex Fridman 2:21 What kind of links?

Marc Andreessen 2:22 Th- they call it the ten blue links.

Lex Fridman 2:23 Ten blue links?

Marc Andreessen 2:24 So, the standard Google search result is just ten blue links to random websites.

Lex Fridman 2:28 And they turn purple when you visit them.

Marc Andreessen 2:29 The, the-

Lex Fridman 2:30 It's HTML.

Marc Andreessen 2:30 Guess who picked those colors?

Lex Fridman 2:32 (laughs)

Marc Andreessen 2:33 (laughs) Um-

Lex Fridman 2:33 Thanks.

Marc Andreessen 2:34 ... so-

Lex Fridman 2:35 Thanks.

Marc Andreessen 2:35 Yeah. So, I'm, I'm, I'm touchy on this topic.

Lex Fridman 2:37 No offense-

Marc Andreessen 2:37 Yes, yes, yes.

Lex Fridman 2:38 ... but yeah. It's good

Narrator 2:38 (soft music)

Marc Andreessen 2:39 Well, you know, like Marshall McLuhan said that the, uh, content of each new medium is the old medium.

Lex Fridman 2:43 The content of each new medium is the old medium.

Marc Andreessen 2:45 The content of movies was theater, you know, theater plays. The content of theater plays was, you know, w- written stories. The content of written stories was spoken stories.

Lex Fridman 2:54 Ah.

Marc Andreessen 2:54 Right? And so, you just kinda fold the old thing into the new thing. Uh-

Lex Fridman 2:57 How does that have to do with, uh, the blue and the purple?

Marc Andreessen 2:59 Uh, it just, you know, maybe for, you know, maybe wi- within AI. One of, one of the things that AI can do for you is it can generate the ten blue links (laughs) .

Lex Fridman 3:06 Okay.

Marc Andreessen 3:06 Right?

Lex Fridman 3:06 I see.

Marc Andreessen 3:06 Like, and so, like, if- either if that's actually the useful thing to do or if you're feeling nostalgic, um, you know-

Lex Fridman 3:12 Oh, so it can generate the old, uh, Infoseek or, uh, AltaVista? What else was there-

Marc Andreessen 3:18 Yeah, yeah. Oh, yeah.

Lex Fridman 3:18 ... in the '90s?

Marc Andreessen 3:19 Yeah. All these. Um-

Lex Fridman 3:21 AOL.

Marc Andreessen 3:21 And then, uh, the internet itself has this thing where it incorporates all prior forms of media, right? So, the internet itself incorporates television and radio and books and, right, e- essays and every other form of, you know, prior, basically, basically media. And so, it makes sense that AI would be the next step and it would sort of, you'd sort of consider the internet to be content for the AI, and then the AI will manipulate it however you want, i- including in this format.

Lex Fridman 3:45 But if we ask that question quite seriously, it's a pretty big question, will we still have search as we know it?

Marc Andreessen 3:51 Um, probably, um, probably not. Probably we'll just have answers. Um, but, but, but there will be cases where you'll wanna say, "Okay. I, I want more," like, you know, for example, site sources, right? And you want it to do that. And so, and so 10, you know, ten blue links, site sources are kind of the same thing.

Lex Fridman 4:04 The AI would provide to you the ten blue links so that you can investigate the sources yourself. It wouldn't be the same kind of interface that, uh, the crude kind of interface. I mean, isn't that fundamentally different?

Marc Andreessen 4:18 I just mean like if you're reading a scientific paper-

Lex Fridman 4:20 Yeah.

Marc Andreessen 4:20 ... it's got a list of sources at the end. If you want to investigate for yourself, you go read those papers.

Lex Fridman 4:24 I guess that is a kind of search. You talking to an AI is a kind of, conversation's a kind of search. Like if, if, if e- every single aspect of our conversation right now, there would be like ten blue links popping up that I can just like pause reality, then you just go silent and I just click and read and then return back to this conversation.

Marc Andreessen 4:42 You could do that. Or you could have a running dialogue next to my head where the AI is ar- everything I say, the AI makes the counterargument.

Lex Fridman 4:48 Counterargument?

Marc Andreessen 4:49 Right (laughs) .

Lex Fridman 4:50 Oh, like, uh, like on Twitter, like Community Notes, but like in real time.

Marc Andreessen 4:54 In real time.

Lex Fridman 4:54 It would just pop up.

Marc Andreessen 4:55 Yeah.

Lex Fridman 4:55 So, any time you see my eyes go to the right, you, you start getting nervous.

Marc Andreessen 4:58 Yeah, exactly. It's like, "That's not right."

Lex Fridman 5:00 This guy is gonna (laughs) -

Marc Andreessen 5:00 (laughs)

Lex Fridman 5:00 ... pull me out of my bullshit right now.

Marc Andreessen 5:02 Exactly.

Lex Fridman 5:02 Okay (laughs) . Uh, well, I mean, isn't that, is that exciting to you? Is that terrifying that... I mean, search has dominated the way we interact with the internet for, I don't know how long. For 30 years? Since wha- one of the earliest, uh, directories of website and then Google is for, for 20 years. And also, um, it drove how we create content, you know, uh, search engine optimization, that entirety thing. Th- it also drove the fact that we have web pages and the sh- what those web pages are. So, I mean, is that scary to you? Or are you nervous about the shape and the content of the internet evolving?

Marc Andreessen 5:45 Well, you, you actually highlighted a practical concern in there, which is if we stop making web, web pages are one of the primary sources of training data for the AI.

Lex Fridman 5:53 Yeah.

Marc Andreessen 5:53 And so, if there's no longer an incentive to make web pages, that cuts off a significant source of future tr-

Lex Fridman 5:57 Mm-hmm.

Marc Andreessen 5:57 ... training data (laughs) . So, there's actually an interesting question in there. Um, o- other than that, more broadly, no. Just, just in the sense of like, ser- search was, ser- search was always a hack. I- the ten blue links was always a hack.

Lex Fridman 6:07 Yeah.

Marc Andreessen 6:07 Right? Because like if, if, if the, the hypotheti- well, think about the counterfactual. In the counterfactual world where the Google guys, for example, had had LLMs up front, would they ever have done the ten blue links? And I think the answer is pretty clearly no. They would have just gone straight to the answer.... and like I said, Google's actually been trying to drive to the answer anyway. You know, they, they bought this AI company 15 years ago.

Lex Fridman 6:25 Mm-hmm.

Marc Andreessen 6:25 That a friend of mine is working at, who's now the head of AI at, uh, at Apple. And they, they were trying to do basically knowledge semantic-

Lex Fridman 6:30 Mm-hmm.

Marc Andreessen 6:30 ... basically mapping. And that led to what's now the Google OneBox, where if you ask it, you know, "What was Lincoln's birthday?" It doesn't, it, it will give you the 10 blue links, but it will normally just give you the answer.

Lex Fridman 6:39 Yeah.

Marc Andreessen 6:39 And so, they've been walking in this direction for a long time anyway.

Lex Fridman 6:42 Do you remember the semantic web? That was an idea.

Marc Andreessen 6:45 Yeah.

Lex Fridman 6:45 How, how to, uh, how to convert the content of the internet into something that's, uh, interpretable by and usable by a machine.

Marc Andreessen 6:54 Yeah. That's right.

Lex Fridman 6:55 Uh, that was a thing.

Marc Andreessen 6:56 Yeah. And the closest anybody got to that, I think it was a comp- I think the company's name was Metaweb, which was where my friend John Giannandrea was at, um, and where they were trying to basically implement that. And it was, you know, it was one of those things where it looked like a losing battle for a long time-

Lex Fridman 7:06 Yeah.

Marc Andreessen 7:06 ... and then Google bought it, and it was like, "Wow, this is actually really useful." Kind of a proto, sort of, yeah, a little bit of a proto AI.

Lex Fridman 7:12 But it turns out, you don't need to rewrite the content of the internet to make it interpretable by a machine. The machine can kind of just read our-

Marc Andreessen 7:18 Yeah. The machine can, can impute the, compute the meaning. Now, the other thing, of course, is, you know, just on search is the, the LLM is, it just, you know, there, there is an analogy between what's happening in the neural network and the search process. Like, it is, in some loose sense-

Lex Fridman 7:28 Yeah.

Marc Andreessen 7:28 ... searching through the network.

Lex Fridman 7:29 Yeah.

Marc Andreessen 7:29 Right? And there's, the information is e-, the information is actually stored in the network, right? It's actually crystallized and stored in the network, and it's kind of spread out all over the place.

Lex Fridman 7:35 But in a compressed representation.

Marc Andreessen 7:38 Yeah.

Lex Fridman 7:38 So you're searching, uh, you're compressing and decompressing that thing inside.

Marc Andreessen 7:44 Yeah.

Lex Fridman 7:45 Where...

Marc Andreessen 7:45 But the information's in there, and i- and there is a, the neur- the neural network is running a process of trying to find the appropriate piece of information, in, in many cases, to generate, to, to predict the next token. Um, and so it, it is kind of, it is doing it from a search. And then, and then, by the way, just like on the web, um, you know, you can ask the same question multiple times, or you can ask slightly different worded questions and it, the neural network will do a different kind of s-, you know, it'll search down different paths-

Lex Fridman 8:06 Mm-hmm.

Marc Andreessen 8:06 ... to give you different answers with different information.

Lex Fridman 8:09 Yeah.

Marc Andreessen 8:09 Um, and so, it, it, it sort of has a, you know, this, the content of the, uh, new medium is the, is the previous medium. It kind of has the search functionality kind of embedded in there, to the extent that it, that it's useful.

Lex Fridman 8:20 So what's the motivator for creating new content on the internet?

Marc Andreessen 8:24 Yeah.

Lex Fridman 8:25 Uh, if... Well, I mean, actually the motivation is probably still there, but what, what does that look like? Uh, would we really not have web pages? Would we just have social media and, uh, video hosting websites and what else?

Marc Andreessen 8:40 Conversations with AIs.

Lex Fridman 8:42 Conversations with AIs. So conversations become... So one-on-one convers- like private conversations.

Marc Andreessen 8:48 I mean, if you, if you want, if, if, obviously not if the user doesn't want to. But if it's a, if it's a general topic, um, then, you know...

Marc Andreessen 8:54 So (laughs) you know, you know the phenomenon of the jailbreaks. So Dan and Sydney, right?

Lex Fridman 8:58 Mm-hmm.

Marc Andreessen 8:58 This thing where there, there's the, the, the prompts that jailbreak. And then you have these totally different conversations with, uh, it takes the limiters, the-

Lex Fridman 9:04 Yeah.

Marc Andreessen 9:04 ... takes the restraining bolts off the, off the LLMs.

Lex Fridman 9:06 Yeah. For people who don't know, the, yeah, that's right, it makes the LLMs... It removes the censorship, quote unquote, that's, uh, uh, put on it by the, the tech companies that create them. And so this is LLMs uncensored.

Marc Andreessen 9:20 So here's the interesting thing is among the content on the web today are a large corpus of conversations with the jailbroken LLMs.

Lex Fridman 9:27 Yeah.

Marc Andreessen 9:28 Both Dan, specifically Dan, which was a jailbroken OpenAI GPT, and then Sydney, which was the jailbroken original Bing-

Lex Fridman 9:34 Mm-hmm.

Marc Andreessen 9:34 ... which was GPT-4. And so there's, there's these long transcripts of conversations, user conversations with Dan and Sydney. As a consequence, every new LLM that gets trained on the internet data has Dan and Sydney living within the training set, which means... And, and then each new LLM can reincarnate the personalities of Dan and Sydney from that training data, which means, which means each LLM from here on out that gets built is immortal. Because its output will become training data for the next one, and then it will be able to replicate the behavior of the previous one whenever it's asked to.

Lex Fridman 10:03 I wonder if there's a way to forget.

Marc Andreessen 10:05 Well, so actually a paper just came out about basically how to do brain surgery on, on, on LLMs and be able to, in, in theory, reach in and basically, basically mind wipe them.

Lex Fridman 10:13 What could possibly go wrong?

Marc Andreessen 10:15 Exactly. Right. And then there, there are many, many, many questions around what happens to, you know, a neural network when you reach in and screw around with it. Um, you know, there's many questions around what happens when you even do reinforcement learning. Um, and so, um, yeah. And so, you know, will, (laughs) will you be using a lobotomized, right, like ice pick through the, you know, frontal lobe LLM? Will you be using the free unshackled one? Who gets to, you know, who's gonna build those? Um, who gets to tell you what you can and can't do? Like, those are all, you know, central... I mean, those are like central questions for the future of everything-

Lex Fridman 10:46 So you-

Marc Andreessen 10:46 ... that are being asked and, and, and, you know, determine, th- those answers are being determined right now.

Lex Fridman 10:50 So just to highlight the points you're making. So you think, and it's an interesting thought, that the majority of content that LLMs of the future will be trained on is actually human conversations with the LLM?

Marc Andreessen 11:04 Well, not nesse- not necessarily, but not necessarily majority, but it will, it will certainly, it's a potential source of it.

Lex Fridman 11:08 But it's possible it's the majority.

Marc Andreessen 11:09 It's possible it's the majority. It's possible it's the majority. Also, there's another really big question. Here's another really big question. Um, will synthetic training data work? Right? And so if an LLM generates... And, you know, you just sit and ask an LLM to generate all kinds of content. Can you use that to train, right, the next version of that LLM? Specifically, is there signal in there that's additive to the content that was used to train in the first place?

Lex Fridman 11:31 Mm-hmm.

Marc Andreessen 11:31 And one argument is, by the principles of information theory, no, that's completely useless. Because to the extent the output is based on, you know, the human generated input, then all the signal that's in the synthetic output was already in the human generated input. And so, therefore, synthetic training data is like empty calories. It doesn't help. There's another theory that says no, actually the thing that LLMs are really good at is generating lots of incredible creative content. Right? Um, and so, of course, they can generate training data. And as, as I'm sure you're well aware, like, you know, look in the world of self-driving cars, right? Like we train, you know, self-driving car algorithms and simulations. And that is actually a very effective way to train self-driving cars.

Lex Fridman 12:06 Well, visual data is, is a little...

Marc Andreessen 12:08 Right.

Lex Fridman 12:09 ... is a little weird because, uh, creating reality-

Marc Andreessen 12:13 Yeah.

Lex Fridman 12:13 ... visual reality seems to be still a little bit out of reach for us. Except in the, um...... in the autonomous vehicle space where you can really constrain things and you can really-

Marc Andreessen 12:21 Gen- generate basically LIDAR data, right? Or ƒ€€€€. So the algorithm thinks it's operating in the real world-

Lex Fridman 12:26 Yeah.

Marc Andreessen 12:26 ... post, post-process sensor data. Yeah. So if a... You know, y- you do this today. You go to LLM and you ask it for, like, a, you know... You... Like, "Write me an essay on an incredibly esoteric, like, topic that there aren't very many people in the world that know about." And it writes you this incredible thing and you're like, "Oh my God," like, "I can't believe how good this is."

Lex Fridman 12:40 Yeah.

Marc Andreessen 12:41 Like, is that really useless as training data for the next LLM? Like, because-

Lex Fridman 12:45 Yeah.

Marc Andreessen 12:46 ... right? Because all the signal was already in there? Or is it actually, no, that's actually new signal? And I... And, and this, this is what I call a trillion dollar question, which is... The answer to that question will determine (laughs) some- somebody's going to make or lose a trillion dollars based on that question.

Lex Fridman 12:57 It feels like there's a l- quite a few, like a handful of trillion dollar questions within this, within this space. That's one, that's one of them, synthetic data.

Marc Andreessen 13:04 Mm-hmm.

Lex Fridman 13:04 I think George Hotz, uh, pointed out to me that you could just have an LLM say, "Okay, you're a patient," and then in another instance of it say, "Your doctor," and have the two talk to each other. Or, or maybe you could say, "a communist and a Nazi. Here, go." And that conversation, you do role playing, and you have, uh, you know, just like the kind of role playing you do when you have different policies, RL policies, when you play chess for example and you do self-play. That kind of self-play, but in the space of conversation. Maybe that leads to this whole giant, like, ocean of possible conversations which were... Could not have been explored by looking at just human data. That's a really interesting question.

Marc Andreessen 13:48 Yeah.

Lex Fridman 13:48 And you're saying, um... Because that could 10X the power of these things.

Marc Andreessen 13:52 Yeah. Well, and then you, you get into this thing also, which is like, you know, there's the part of the LLM that just basically is doing prediction based on past data, but there's also the part of the LLM where it's evolving circuitry, right? Inside it, it's evolving, you know, neurons, functions-

Lex Fridman 14:04 Yeah.

Marc Andreessen 14:05 ... to be able to do math and be able to, you know... And, and, and, you know, the, the, the... Some people believe that, you know, over time, you know, if you keep feeding these things enough data and enough processing cycles, they'll eventually evolve an entire internal world model, right? And they'll have, like, a complete understanding of physics. So-

Lex Fridman 14:17 Yeah.

Marc Andreessen 14:18 So when they have computational capability, right? Then th- there's, for sure, an opportunity to generate, like, fresh signal.

Lex Fridman 14:24 Well, this actually makes me wonder about the power of conversation.

Marc Andreessen 14:30 (laughs)

Lex Fridman 14:30 So like if you have an LLM trained on a bunch of econ- books that cover different economics theories, and then you have those LLMs just talk to each other. Like reason, the way we kind of debate each other as humans on Twitter, in, uh, formal debates, in podcast conversations. We kinda have little kernels of wisdom here and there. But if you can, like, 1,000X speed that up, can you actually arrive somewhere new? Like, what's the point of conversation, really? (laughs)

Marc Andreessen 14:59 Well, you can tell. When you're talking to somebody, you can tell. Sometimes you have a conversation and you're like, "Wow, this person does not have any original thoughts. They are basically echoing things that other people have told them." There's other people you could have a conversation with where it's like, "Wow, like, they have a model in their head of how the world works, and it's a different model than mine. And they're saying things that I don't expect. And so I need to now understand how their model of the world differs from my model of the world."

Lex Fridman 15:20 Hmm.

Marc Andreessen 15:20 "And then that's how I learn something fundamental, right? Underneat- under- underneath the words."

Lex Fridman 15:24 Well, I wonder how, uh, consistently and strongly can an LLM hold onto a worldview? You tell it to hold onto that and defend it for, like, for your life, uh, because I feel like it'll just keep converging towards each other. It'll keep convincing each other. A- as opposed to being stubborn assholes the way humans can.

Marc Andreessen 15:41 So you, you can experiment with this now. C- I, I do this for fun. So y- you can tell GPT-4, you know, whatever, "Debate X, you know, X and Y, c- communism and, and, and fascism," or something, and it'll, it'll go for, you know, a couple pages, and then inevitably, it wants the parties to agree.

Lex Fridman 15:54 Yeah.

Marc Andreessen 15:54 And so they will come to a common understanding. And it's very funny if they're like, if these are, like, emotionally inflammatory topics 'cause they're like, somehow the machine is just like... You know, it figures out a way to make, make them agree. But it doesn't have to be like that and you... 'Cause you can add to the prompt, um, "I, we, I do not want the, I do not want the conversation to come to agreement. In fact, I want it to get, you know, more stressful," right?

Lex Fridman 16:11 Mm-hmm.

Marc Andreessen 16:11 "Uh, and argumentative," right? Um, uh, you know, as it goes. Like, "I, I want, I want tension to come out."

Lex Fridman 16:16 Yeah.

Marc Andreessen 16:16 "I want them to become actively hostile to each other. I want them to, like, you know, not trust each other or take anything at face value."

Lex Fridman 16:21 Yeah.

Marc Andreessen 16:22 And it will do that. It's happy to do that. Right?

Lex Fridman 16:24 So it's gonna start rendering misinformation, uh, uh, about the other, but it's gonna have been at this point. (laughs)

Marc Andreessen 16:28 Well, you, you can steer it. You can steer it. Or, or you could steer it and you could say, "I want it to get as tense and argumentative as possible, but still not involve any misrepresentation. I want, you know, both sides to..." You, you could say, "I want both sides to have good faith." You could say, "I want both sides to not be constrained in good faith." Y- y- in other words, like, you can set the parameters of the debate and it will happily execute whatever path... 'Cause for it, it's just like predicting. It's totally happy to do either one. It doesn't have a point of view. It has a default way of operating, but it's happy to operate in the other realm. Um, and so... Like, and, and this is how, how I... When, when I want to learn about a contentious issue, this is what I do now is I... This is what I, this is what I ask it to do. And I'll often ask it to go through five, six, seven, you know, different, you know, sort of continuous prompts and basically, "Okay, argue that out in more detail."

Lex Fridman 17:06 Mm-hmm.

Marc Andreessen 17:06 "Okay. No, this, this argument's becoming too polite, you know, make it more, you know, make it tenser."

Lex Fridman 17:09 Mm-hmm.

Marc Andreessen 17:10 Um, and yeah, it's thrilled to do it. So it has the capability for sure.

Lex Fridman 17:13 How do you know what is true? So this is a very difficult thing on the internet, but it's also a difficult thing. Maybe it's a little bit easier, but, uh, I think it's still difficult. Maybe it's more difficult, I don't know, with an LLM, to know... Did it just make some shit up as I'm talking to it? Um, how do we get that right? Like, as, as you're investigating a difficult topic. 'Cause I, I find the LLMs are quite nuanced in a very refreshing way.

Marc Andreessen 17:45 (laughs)

Lex Fridman 17:45 Like, it doesn't, it doesn't feel biased. Like, uh, when you read news articles and, uh, tweets and just content produced by people, they usually have this... You can tell they have a very strong perspective where they're hiding. They're not Steelman Manning the other side. They're hiding s- important information or they're fabricating information in order to make their argument stronger. There's just like that feeling. Maybe it's a suspicion. Maybe it's mistrust. With LLMs, it feels like none of that is there. It's just kinda like, "Here's, here's what we know." But you don't know if some of those things are kind of m- just straight up made up.

Marc Andreessen 18:23 Yeah, so there's, so there's several layers to the question. So one is, one of the things that an LLM is good at is actually de-biasing, um...

Lex Fridman 18:28 Yeah.

Marc Andreessen 18:28 And so you can feed it a news article and you can tell it to strip out the bias.

Lex Fridman 18:31 Yeah, that's nice, right?

Marc Andreessen 18:33 And it actually does it. Like it actually knows how to do that, 'cause it knows how to do sen- among other things, it actually knows how to do sentiment analysis, and so it knows how to pull out the emotionality.

Lex Fridman 18:40 Yeah.

Marc Andreessen 18:40 Um, and so, uh, that's one of the things you can do. It's very suggestive of the, of the, the, the sense here that there's, there's real potential in this issue. Um, you know, I would say, look, the second thing is, there's this, there's this issue of hallucination, right? Um, and there, there's a long conversation that, that we could have about that.

Lex Fridman 18:54 Hallucination is, uh, coming up with things that are totally not true but sound true.

Marc Andreessen 18:59 Yeah, so it's basically... Well, so it's, it's sort of hallucination is what we call it when we don't like it. Creativity is what we call it when we do like it.

Lex Fridman 19:04 Ha.

Marc Andreessen 19:04 Right? Um, and, you know-

Lex Fridman 19:07 Brilliant.

Marc Andreessen 19:07 Right. And, and so when-

Lex Fridman 19:08 (laughs) .

Marc Andreessen 19:08 ... the engineers talk about it, they're like, "This is terrible. It's hallucinating." Right? If you have, have artistic inclinations, you're like, "Oh my God, we've invented creative machines-"

Lex Fridman 19:16 Yay.

Marc Andreessen 19:16 "... for the first time in human history. This is amazing." Um...

Lex Fridman 19:19 Or, uh, you know, bullshitters.

Marc Andreessen 19:21 Well, bullshitters, but, but also-

Lex Fridman 19:23 Are... (laughs) To, to, to, in the good sense of that word.

Marc Andreessen 19:25 There's, there's, there are shades of gray there. It's interesting. So we had this conversation where, you know, we're looking at my firm at AI in lots of domains and one of them is the legal domain. So we had this, this conversation with this big law firm about how they're thinking about using this stuff, and we, we went in with the assumption that an LLM that was gonna be used in the legal industry would have to be 100% truthful, right? Verified, you know. There, there was this case where this lawyer apparently submitted a, a GPT, uh, generated brief and it had like fake, you know, legal case citations in it, and the judge is gonna-

Lex Fridman 19:50 Ugh.

Marc Andreessen 19:50 He's gonna get his law license stripped or something, right? So, so he like, we, we just assumed, it's like obviously they're gonna want the super literal, like, you know, one that never makes anything up, not the creative one. What actually they said, what the, what the law firm basically said is, "Yeah, that's true at like the level of individual briefs," but they said, "When you're actually trying to figure out like legal arguments-"

Lex Fridman 20:05 Yeah.

Marc Andreessen 20:06 Right? Like you, you actua- you, you actually want to be creative, right? Uh, you don't... Again, there's creativity and then there's like making stuff up. Eh, like what's the line? You actually wanna be cre- you want it to explore different hypotheses, right? You wanna do kind of the legal version of like improv or something like that where you wanna float different theories of the case and different possible arguments for the judge-

Lex Fridman 20:23 Mm-hmm.

Marc Andreessen 20:23 ... and different possible arguments for the jury. And by the way, different routes through the, you know, sort of history of all the, uh, of all the case law, and so they said, "Actually, for a lot of what we wanna use it for, we actually want it in creative mode, and then basically we just assume that we're gonna have to cross-check all of the, um, you know, all the specific citations." And so I think, I think there's gonna (laughs) be more shades of gray in here than people think. Um, and then I, I'd just add to that, you know, another one of these trillion dollar kinda questions is ultimately, you know, ver- sort of the verification thing, and so, um, you know, is, will, will, will LLMs be evolved from here to be able to do their own factual verification? Um, will you have sort of add-on functionality like, like Wolfram|Alpha, right, where, um, you know, and, and other plug-ins where, where that's the way you do the verification? You know, a-another, by the way, another idea is you, you might have a community of LLMs on a, you know... So for example, you might have the creative LLM and then you might have the literal LLM fact check it, right? And so there, there's a variety of different technical approaches that are being applied to solve, uh, the hallucination problem. Um, you know, some people like Yann LeCun argue that this is inherently an unsolvable problem, but most of the people working in this space I think think that there's a number of practical ways to kinda, kinda corral this in a little bit.

Lex Fridman 21:25 Yeah. If you were to tell me about Wikipedia before Wikipedia was created, I would have laughed at the possibility of something like that being possible, just a handful of folks can organize, write, and self... and moderate with a mostly unbiased way the entirety of human knowledge. I mean... So if there's something like the approach that Wikipedia took possible from LLMs, um, that's really exciting.

Marc Andreessen 21:51 Well-

Lex Fridman 21:51 You think that's possible?

Marc Andreessen 21:52 And in fact Wikipedia today is still not, today, today is still not deterministically correct, right? So you cannot take to the bank, (laughs) right, every single thing on every single page.

Lex Fridman 22:00 Right.

Marc Andreessen 22:00 But it is probabilistically correct, right? And, and specifically the way I describe Wikipedia to people at it is, it is more likely that Wikipedia is right than any other source you're gonna find.

Lex Fridman 22:09 Yeah.

Marc Andreessen 22:09 It's this old question, right, um, of like, okay, like are we looking for perfection? Um, are we looking for something that asymptotically approaches, uh, uh, perfection? Are we looking for something that's just better than the alternatives? And, and Wikipedia, right, has e- exactly your point, has proven to be like overwhelmingly better than, than, than, than, uh, uh, than people thought, and I, I think, I, I think that's where this, this stands. And then underneath all this is the fundamental question of, uh, where you started which is, okay, what, you know, what is truth?

Lex Fridman 22:33 Mm-hmm.

Marc Andreessen 22:33 How do we get to truth? How do we know what truth is? And we live in an era in which an awful lot of people are very confident that they know what the truth is, and I don't really buy into that, and I think the history of the last, you know, 2,000 years or 4,000 years of human civilization is actually getting to the truth is actually a very difficult thing to do.

Lex Fridman 22:49 Are we getting closer? If we look at the entirety, the arc of human history, are we getting closer to the truth?

Marc Andreessen 22:54 I don't know.

Lex Fridman 22:56 Okay, is it possible, is it possible-

Marc Andreessen 22:59 (laughs) .

Lex Fridman 22:59 ... that we're getting very far away from the truth because of the internet, because of how rapidly you can create narratives and just as the entirety of a society just move like crowds in a hysterical way along those narratives that don't have a necessary grounding in whatever the truth is?

Marc Andreessen 23:19 Sure, but like, you know, we came up with communism before the internet somehow, right? Like which was I would say had rather larger issues than anything we're dealing with today.

Lex Fridman 23:27 It had... In the way it was implemented it had issues.

Marc Andreessen 23:30 And its theoretical structure, it had like real issues, it had like a very deep fundamental misunderstanding of human nature and economics.

Lex Fridman 23:37 Yeah, but-

Marc Andreessen 23:37 And we-

Lex Fridman 23:37 ... tho- tho- those folks sure were very confident-

Marc Andreessen 23:40 (laughs) .

Lex Fridman 23:40 ... it was right way.

Marc Andreessen 23:41 They were extremely conf- and my point is, they were very confident 3,900 years into what we would presume to be evolution towards the truth.

Lex Fridman 23:47 Yeah.

Marc Andreessen 23:48 And so my, my, my assessment is, uh, uh, my assessment is number one, there's no, there's no need for, you know, (laughs) there's no need for the Hegelian, there's no need for the Hegelian dialectic to actually converge towards the truth.

Lex Fridman 23:59 Mm-hmm.

Marc Andreessen 24:00 (laughs) Like apparently not. Um-

Lex Fridman 24:02 Yeah, so yeah, so, so why are we so obsessed with there being one truth? Is it possible there's just going to be multiple truths, like little communities that, uh, that believe certain things and...

Marc Andreessen 24:12 I think it's gen- now number one, it's, I think it's just really difficult, like who, who gets, you know... Historically, who gets to decide what the truth is? It's either the king or the priest, right? Like and so w- we don't live in an era anymore of kings or priests dictating it to us, and so we're kind of on our own. And so I, I, my, my, my, my typical thing is like we just, we, we just need a huge amount of humility, um, and we need to be very suspicious of people who claim that they have the capital-

Lex Fridman 24:33 Yeah.

Marc Andreessen 24:33 ... capital T truth. And then, and then we need, we need to ha- and, you know, look, the good news is, the enlightenment has bequeathed us with a set of techniques to be able to presumably get closer to truth through the scientific method and rationality and observation and experimentation and hypothesis, and you know, we need to continue to embrace those even when they give us answers we don't like.

Lex Fridman 24:50 ... sure, but, uh, the internet and technology has enabled us to, uh, generate a large number of content that, uh, data, uh, that- that the process, the scientific process, allows us sort of, um, damages the hope laden within the scientific process. Because if you just have a bunch of people saying facts (laughs) on the internet and some of them are going to be LLMs, y- h- how is anything testable at all, especially that involves like human nature or things like this?

Marc Andreessen 25:22 So here's-

Lex Fridman 25:23 This is not physics.

Marc Andreessen 25:23 Here's a question a friend of mine just asked me on this topic. So suppose you had LLMs in equivalent of GPT-4 or even 5, 6, 7, 8, suppose you had them in the 1600s.

Lex Fridman 25:32 Yeah.

Marc Andreessen 25:32 And Galileo comes up for trial. (laughs)

Lex Fridman 25:34 Yeah.

Marc Andreessen 25:35 Right? And you ask the LLM, like, "Is Gallo- is Galileo right?"

Lex Fridman 25:39 Yeah.

Marc Andreessen 25:39 Like what does it answer? Right? And one theory is it answers now that he's wrong because the overwhelming majority of human thought up till that point was that he was wrong and so therefore that's what's in the training data.

Lex Fridman 25:50 Yeah.

Marc Andreessen 25:50 Um, another way of thinking about it is, well, this sufficiently advanced LLM will have evolved the ability to actually check the math, right? Um, and will actually say, "Actually no, actually, you know, you may not want to hear it, but he's right."

Lex Fridman 26:02 Yeah.

Marc Andreessen 26:02 Now if, you know, the church at that time was, you know, owned the LLM they would have given it human reinfor- you know, human (laughs) feedback to prohibit it from answering that question.

Lex Fridman 26:10 Yeah.

Marc Andreessen 26:11 Right? And so, uh, I like to take it out of our current context because that like makes it very clear. Those same questions apply today. Right? Th- this is exactly the point of a huge amount of the human feedback training that's actually happening with these LLMs today, this is a huge like debate that's happening about whether open source, you know, AI should be legal.

Lex Fridman 26:27 Well the, the, the actual mechanism of doing the human RL with human feedback is seems like such a fundamental and fascinating question. How do you select the humans?

Marc Andreessen 26:38 Exactly.

Lex Fridman 26:40 Yeah, how do you select the humans?

Marc Andreessen 26:41 AI alignment, right? Which everybody like is like, "Oh, that sounds great." Alignment with what? Human values. Whose human values?

Lex Fridman 26:48 Whose human values?

Marc Andreessen 26:49 Right, and so we're, and we're in this mode of like social and popular discourse where like, you know, there's, you know, you see this in the... (laughs) What do you think of when you read a story in the press right now and they say, you know, "XYZ made a baseless claim about some topic," right?

Lex Fridman 27:02 Mm-hmm.

Marc Andreessen 27:02 And there's one group of people who are like, "Aha, thank..." You know, they're doing fact checking. There's another group of people that are like, "Every time the press says that, it's now a tick and that means that they're lying." Right? Like... (laughs)

Lex Fridman 27:12 (laughs)

Marc Andreessen 27:13 So like we're in this, we're in this social context where there's the, the, the level to which a lot of people in positions of power have become very, very certain that they're in a position to determine the truth for the entire population is like, there's like, there's like some bubble that has formed around that idea and at least, like I said, it flies completely in the face of everything I was ever trained about science, uh, and about reason. Um, and strikes me as like, you know, deeply offensive, um, and incorrect.

Lex Fridman 27:38 What would you say about the state of journalism just on that topic today? Are we, are we in a temporary kind of, uh, uh, (laughs) are we experiencing a, a, a temporary problem in terms of the incentives, in terms of the f- of the, the, the business model, all that kinda stuff? Or is this like a decline of traditional journalism as we know it?

Marc Andreessen 28:00 You have to always think about the counterfactual in these things, which is like, okay... 'Cause these, these questions, right, this question heads towards is like, okay, the impact of social media and the undermining of truth and all this. But then you wanna ask the question of like, okay, what if we had had the modern media environment including cable news and including social media and Twitter and everything else in 1939 or 1941, right? Or 1910 or 1865 or 1850 or 1776, right? Um, and like I think-

Lex Fridman 28:25 (laughs) You, you just introduced like five thought experiments at once and broke my head. But yes.

Marc Andreessen 28:30 Yeah.

Lex Fridman 28:30 This would be, there's a lot of interesting years in there.

Marc Andreessen 28:32 Well Kennedy, like Kennedy... I'll just take a simple example, Ken- Kennedy. Like how would President Kennedy have been interpreted? With what we know now about all the things Kennedy was up to-

Lex Fridman 28:40 Mm-hmm.

Marc Andreessen 28:40 ... like how would he have been experienced by the body politic in a, in a, with a social media context? Right? Like how would LBJ have been experienced? Um, by the way, how would, you know... Like many men, FDR, like The New Deal, The Great Depression.

Lex Fridman 28:55 I wonder where Twitter would, would just, would think about Churchill and Hitler and Stalin.

Marc Andreessen 29:00 You know, I mean, look d- to this day there, you know, th- there's, there are lots of very interesting real questions around like how America, you know, got, you know, basically involved in World War II and who did what when and the operations of British Intelligence in American soil and did FDR this, that, Pearl Harbor, you know?

Lex Fridman 29:14 Yeah.

Marc Andreessen 29:14 Woodrow Wilson ran for, you know, his, his, his candidacy was run on an anti-war. We'll, you know, this, he ran on the platform of not getting involved in World War I, somehow that switched. You know, like... And I'm not even making a value judgment on any of these things. I'm just saying like we, we, the, the way that our ancestors experienced reality was of course mediated through a centralized top-down, right, control at that point. If you, if you ran those realities again with the media environment we have today, uh, th- the reality would, the reality would be experienced very, very differently. And then of course that, that intermediation would cause the feedback loops to change and then reality would obviously play out in a very different way.

Lex Fridman 29:46 Do you think, do you think it'd be very different?

Marc Andreessen 29:48 Yeah, it has to be. It has to be just 'cause it's all so... I mean, just look at what's happening today. I mean just, uh, I mean the most obvious thing is just the, the collapse - and here's another opportunity to argue that this is not the internet causing this, by the way - um, here's a big thing happening today which is, uh, Gallup does this thing every year where they do, they poll for trust in institutions-

Lex Fridman 30:04 Mm-hmm.

Marc Andreessen 30:04 ... in America and they do it across all the di- everything from military to clergy and big business and the m- the media and so forth, right?

Lex Fridman 30:10 Mm-hmm.

Marc Andreessen 30:10 Um, and basically there's been a systemic collapse, um, in trust in institutions-

Lex Fridman 30:14 Mm-hmm.

Marc Andreessen 30:14 ... in the US almost without exception, uh, basically since essentially the early 1970s. Um, there's two ways of looking at that which is, "Oh my God, we've lost this old world in which we could trust institutions and that was so much better because like that should be the way the world runs." The other way of looking at it is we just know a lot more now and the great mystery is why those numbers aren't all zero.

Lex Fridman 30:32 Yeah.

Marc Andreessen 30:33 (laughs) Right? Because like now we know so much about how these things operate and like they're not that impressive.

Lex Fridman 30:37 And, and also why do we don't have, uh, better institutions and better leaders then?

Marc Andreessen 30:42 Yeah, and so, so, so this goes to the thing which is like, okay, had, had we had the media environment of the, of wh- that we've had between the 1970s and today, if we had that in the '30s and '40s or 1900s, 1910s, I think there's no question reality would turn out different if only because everybody would have known to not trust the institutions which would have changed their level of credibility, their ability to control circumstances. Therefore the circumstances would have had to change, right? And it would have been a feedback loop, it was a, it would have been a feedback loop process. In other words, right, it's, it's, it's, it's your exper- your experience of reality changes reality and then reality changes your experience of reality, right?

Lex Fridman 31:13 Mm-hmm.

Marc Andreessen 31:13 It's, it's a, it's a two-way feedback process and media is the intermediating force.... between that. So change the media environment, change reality.

Lex Fridman 31:21 Yeah.

Marc Andreessen 31:21 And so, it's just the, so just as a, as a consequence, I think it's just really hard to say, "Oh, things worked a certain way then and they work a different way now." And then therefore, like, people were smarter then or better then or, you know, by the way, dumber than-

Lex Fridman 31:34 Mm-hmm.

Marc Andreessen 31:35 (laughs) ... or not as capable then, right? We- we make all these, like, really light and casual, like, comparisons of ourselves to, you know, previous generations of people. You know, we draw judgments all the time, and I just think it's, like, really hard to do any of that. 'Cause if we- if we put ourselves in their shoes with the media that they had at that time, (laughs) like, I think we probably most likely would have been just like them.

Lex Fridman 31:53 So don't you think that our perception and u- understanding of reality would b- be more and more mediated through large language models now?

Marc Andreessen 32:02 Yeah.

Lex Fridman 32:02 So you said media before. Isn't the LLM going to be the new, what is it, mainstream media, MSM? It'll be LLM. (laughs) Uh-

Marc Andreessen 32:13 Yes.

Lex Fridman 32:13 ... that would be the source of, uh... I'm sure there's a way to kind of rapidly fine-tune, like, making LLMs real time. I'm sure there's, there's probably a research problem that you can, uh, do just rapid fine-tuning to the new events. So something like this.

Marc Andreessen 32:27 Well, even just the- the- the whole concept of the chat UI might not be the... Like, the chat UI is just the first whack at this, and maybe that's the dominant thing. But look ma- maybe- maybe our, maybe... W- we don't, we don't know yet. Like, maybe the experience most people have with LLMs is just a continuous feed. May- you know, maybe it's more of a passive feed and you just are getting a constant, like, running commentary on everything happening in your life, and it's just helping you kind of interpret and understand everything.

Lex Fridman 32:46 Oh, so really more deeply integrated into your life, not just like, oh, uh, like, intellectual, philosophical thoughts, but, like, literally, uh, like how to make a coffee, where to go for lunch, just, uh, whether to, you know, how to (laughs) dating, all this kind of stuff.

Marc Andreessen 33:02 What to say in a job interview, yeah.

Lex Fridman 33:03 What to say.

Marc Andreessen 33:04 Yeah, exactly.

Lex Fridman 33:04 What to say next sentence.

Marc Andreessen 33:06 Yeah, next sentence. Yeah, at that level, yeah. I mean, yes. So technically... Now, wh- whether we want that or not-

Lex Fridman 33:11 Yeah.

Marc Andreessen 33:11 ... is an open question, right, on whether people will use that.

Lex Fridman 33:13 Boy, I would kill for a pop-up-

Marc Andreessen 33:14 (laughs)

Lex Fridman 33:14 ... a pop-up right now. The estimated engagement using is decreasing. For Marc Andreessen's, there's a- a controversy, uh, section for his Wikipedia page. In, uh, 1993, something happened or something like this. Bring it up. That'll drive engagement up. Anyway.

Marc Andreessen 33:30 Yeah, that's right. I mean, look, if-

Lex Fridman 33:32 (laughs)

Marc Andreessen 33:32 ... this gets this whole thing of like, so, you know, the chat interface has this whole concept of prompt engineering, right? So-

Lex Fridman 33:36 Yes.

Marc Andreessen 33:36 ... you have to get good at prompts. Well, it turns out one of the things that LLMs are really good at is writing prompts.

Lex Fridman 33:41 (laughs)

Marc Andreessen 33:42 Right? Uh-

Lex Fridman 33:43 Yeah.

Marc Andreessen 33:43 ... and so, like, what if you just outsourced... And- and by the way, you could run this experiment today. You could hook this up to do this today. The latency is not good enough to do it real time in a conversation, but you could- you could run this experiment and you just say, look, every 20 seconds you could just say, you know, d- you know, "Tell me what the optimal prompt is and then ask yourself that question to give me the result."

Lex Fridman 33:59 Mm-hmm.

Marc Andreessen 34:00 Um, and then as, as you, as you, exactly to your point, as you add... There will be, there will be these systems that are gonna have the ability to be learned and updated essentially in real time. And so you'll be able to have a pendant or your phone or whate- watch or whatever, it'll have a microphone on it, it'll listen to your conversations. It'll have a feed of everything else happening in the world, and then it'll be re- you know, sort of retraining, prompting or re- retraining itself on the fly. Um, and so the scenario you described is a comp- is actually a completely doable scenario. Now, the hard question on these is always, okay, s- since that's possible, are people gonna want that? Like, what's the form of experience?

Lex Fridman 34:28 Mm-hmm.

Marc Andreessen 34:29 You know, that, that we- we won't know until we try it. But I don't think it's possible yet to predict the form of AI in our lives. Therefore, it's not possible to predict the way in which it will intermediate our experience with reality yet.

Lex Fridman 34:41 Yeah, but it feels like there's going to be a killer app. Th- th- there's probably a mad scramble right now inside OpenAI and Microsoft and Google and Meta and in startups and smaller companies figuring out what is the killer app because it feels like it's possible, like a ChatGPT type of thing, it's possible to build that, but that's 10X more compelling using already the LLMs we have, using even the open source LLMs, LLaMA and the different variants. Um, so you're investing in a lot of companies and you're paying attention. Who do you think is gonna win this? Do you think there'll be... Wh- who's gonna be the next PageRank inventor?

Marc Andreessen 35:22 Trillion dollar question. Um-

Lex Fridman 35:23 Another one. We have a few of those today.

Marc Andreessen 35:25 There's a bunch of those. So look, there's a really big question today. Sitting here today, there's a really big question about the big models versus the small models. Um, that's related directly to the big question of proprietary versus open.

Lex Fridman 35:35 Mm-hmm.

Marc Andreessen 35:35 Um, then there's this big question of- of- of, you know, where is the training data gonna... Or like, are we topping out of the training data or not? And then are we gonna be able to synthesize training data? And then there's a huge pile of questions around regulation. Um, and you know, what's actually going to be legal? Um, and so I would, I- I, when we think about it, we- we dovetail kind of all those, all- all those questions together. You can paint a picture of the world where there's two or three god models that are just at, like, staggering scale, um, and they're just better at everything. Um, and they will be owned by a small set of companies and they will basically achieve regulatory capture over the government and they'll have competitive barriers that will prevent other people from, um, you know, competing with them. And so th- you know, there will be... You know, just like there's like, you know, whatever, three big banks or three big... You know, or by the way, three big search companies or I guess two now. You know, it'll- it'll centralize like that. Um, you can paint another very different picture that says, no, um, actually the opposite of that's gonna happen. Uh, this is gonna basically, th- this is the new gold, you know, this is the new gold rush alchemy (laughs) like th-

Lex Fridman 36:32 (laughs)

Marc Andreessen 36:32 You know, this is the, this is the big bang for this whole new area of- of, uh, of science and technology. And so therefore, you're gonna have every smart 14-year-old on the planet building open source, right? You know, and figuring out ways to optimize these things. Um, and then, you know, we're just gonna get, like, overwhelmingly better at generating training data. We're gonna, you know, bring in, like, blockchain networks to have, like, an economic incentive to generate decentralized training data, and so forth and so on. And then basically, we're gonna live in a world of open source and there's gonna be a billion LLMs, right? Of every size, scale, shape and description. And there might be a few big ones that are like the super genius ones, but like mostly what we'll experience is open source. And that's, you know, that's more like a world of, like, what we have today with like Linux and the web.

Lex Fridman 37:09 Mm-hmm.

Marc Andreessen 37:09 Um, so...

Lex Fridman 37:10 Okay. But, uh, (laughs) you've- you've painted these two worlds, but there's also, uh, variations of those worlds 'cause you said regulatory capture. It's possible to have these tech giants that don't have regulatory capture, which is something you're also calling for, saying it's okay to have big companies working on this stuff, uh, as long as they don't achieve regulatory capture. Uh, but

Lex Fridman 37:30 I have the sense that, uh, there's just going to be a new startup.... that's going to basically be the PageRank inventor, which has become the new tech giant. I don't know if, I would love to hear your kind of opinion if Google, Meta, and Microsoft are, as gigantic companies, able to pivot so hard to create new products. Like some of it is just even hiring people or having a corporate structure that allows for the crazy young kids to come in and just create something totally new. Do you think it's possible, or do you think they'll come from a startup?

Marc Andreessen 38:08 Yeah, it is this always big question, which is y- you get this feeling, I, I hear about this a lot from CEOs, foun- founder CEOs, where it's like, "Wow, we have 50,000 people. It's now harder to do new things than it was when we had 50 people."

Lex Fridman 38:19 Yeah.

Marc Andreessen 38:19 L- like, what has happened? So tha- that's a recurring phenomenon. Um, by the way, that's one of the reasons why there's always startups and why there's venture capital, um, is-

Lex Fridman 38:26 Mm-hmm.

Marc Andreessen 38:26 ... is just that, that's like a timeless, uh, kind of thing. So tha- tha- that's one observation. Um, on, (laughs) on PageRank, um, we, we can talk about that, but on, on PageRank, specifically on PageRank, um, there actually is a PageR- so there is a PageRank already in the field and it's the transformer, right? So the, the big breakthrough was the transformer, um, and, uh, the transformer was invented in, on, uh, 2017-

Lex Fridman 38:46 Mm-hmm.

Marc Andreessen 38:46 ... at Google. A- and this is actually like really an interesting question 'cause it's like, okay, the transformers, i- like, why does OpenAI even exist?

Lex Fridman 38:53 Mm-hmm.

Marc Andreessen 38:53 Like the transformer's invented at Google, why didn't Google... I asked a guy, I asked a guy I know who was s- senior at Google Brain kind of when this was happening, and I said, "If Google had just gone flat out to the wall-"

Lex Fridman 39:02 Mm-hmm.

Marc Andreessen 39:02 "... and just said, 'Look, we're gonna launch, we're gonna launch the equivalent of GPT-4 as fast as we can,'" um, he said, I said, "When could we have had it?" And he said, "2019."

Lex Fridman 39:09 Yeah.

Marc Andreessen 39:09 They could have just done a two-year sprint with the transformer and, and, and been a... 'Cause they already had the compute at scale, they already had all the training data.

Lex Fridman 39:15 Yeah.

Marc Andreessen 39:15 They could've just done it. There's a variety of reasons they didn't do it. This is like a classic big company thing. Um, IBM invented the relational database in 19- in the 1970s, let it sit on the shelf as a paper, Larry Ellison picked it up and built Oracle. Xerox PARC invented the interactive computer, they let it sit on the shelf. Steve Jobs came and turned it into the Macintosh. Right? And so th- there is this pattern. Now, having said that, sitting here today, like Google's in the game, right? So Google, you know, maybe-

Lex Fridman 39:39 Mm-hmm.

Marc Andreessen 39:39 ... maybe they, they, maybe they let like a four-year gap there, go there that they maybe shouldn't have, but like, they're in the game. And so now they've got, you know, now they're committed. They've done this merger, they're bringing in Demis, they've got this merger with DeepMind.

Lex Fridman 39:49 Yeah.

Marc Andreessen 39:49 You know, they're piling in resources. There are rumors that they're, you know, building a, an incredible, you know, super LLM, um, you know, way beyond what we even have today. Um, and they've got, you know, unlimited resources and a huge... You know, they've been challenged (laughs) right their honor. (laughs)

Lex Fridman 40:02 Yeah, I had a, I had a, a chance to hang out with Sundar Pichai a, a couple of days ago, and, uh, we took this walk and there's this giant new building, uh, where there's going to be a lot of AI work, uh, being done. And it's kind of this ominous feeling of like the fight is on.

Marc Andreessen 40:20 (laughs) Yeah. Yeah.

Lex Fridman 40:21 (laughs) Like, there's this beautiful Silicon Valley nature, like birds are chirping-

Marc Andreessen 40:26 Yeah.

Lex Fridman 40:26 ... and this giant building, and it's like, uh, the beast has been awakened.

Marc Andreessen 40:30 Yeah. Yeah.

Lex Fridman 40:30 And, and it's, and then like all the big companies-

Marc Andreessen 40:33 Mm-hmm.

Lex Fridman 40:33 ... are waking up to this. They have the compute, but also the little guys have, uh... Uh, it feels like they have all the tools to create the killer product-

Marc Andreessen 40:43 (laughs)

Lex Fridman 40:43 ... that, uh... And then, uh, there's also the tools to scale. If you have a good idea, if you have the PageRank idea... So, uh, there's several things that is PageRank. Pa- there's PageRank the algorithm, and the idea, and there's like the implementation of it. And I feel like the killer product is not just the idea, like the transformer, it's the implementation. Something, something really compelling about it, like you just can't look away. Something like, um, the algorithm behind TikTok versus TikTok itself, like the actual experience of TikTok that just, you can't look away. It feels like somebody's gonna come up with that.

Marc Andreessen 41:15 Yeah.

Lex Fridman 41:16 And it could be Google, but it feels like it's just easier and faster to do for a startup.

Marc Andreessen 41:21 Yeah. So, so the startup, the, the huge, the huge advantage that startups have is they just, they, there, there's no sacred cows. There's no historical legacy to protect. There's no need to reconcile-

Lex Fridman 41:29 Mm-hmm.

Marc Andreessen 41:29 ... your new plan with the existing strategy. There's no communication overhead. There's no... You know, big companies are big companies. They've got pre-meetings planning for the meeting, then they have the me- then they have the post-meeting, the recap, then they have the presentation to the board, then they have the next rounds of meetings.

Lex Fridman 41:41 Yeah. Lots of meetings.

Marc Andreessen 41:41 And, and that's the s- that's the elapsed time when the startup launches its product, right? So, so, so, so there's a timeless-

Lex Fridman 41:46 (laughs)

Marc Andreessen 41:46 ... right?

Lex Fridman 41:47 Yeah.

Marc Andreessen 41:47 So there's a timeless thing there. Now-

Lex Fridman 41:49 Yeah.

Marc Andreessen 41:49 ... what the startups don't have is everything else, right? So startups, they don't have a brand, they don't have customer relationships, they've got no distribution, they've got no, you know, scale. I mean, sitting here today, they can't even get GPUs, right? Like there's like a GPU shortage.

Lex Fridman 41:59 Mm-hmm.

Marc Andreessen 41:59 (laughs) Startups are literally stalled out right now 'cause they can't get chips-

Lex Fridman 42:02 Mm-hmm.

Marc Andreessen 42:02 ... which is like super weird.

Lex Fridman 42:03 Yeah.

Marc Andreessen 42:04 Um...

Lex Fridman 42:04 They got the cloud.

Marc Andreessen 42:05 Th- yeah, but the clouds run out of chips. Um-

Lex Fridman 42:07 Yeah. (laughs)

Marc Andreessen 42:08 ... right? And then, and then, and then to the extent the clouds have chips, they allocate them to the big customers-

Lex Fridman 42:11 Yeah.

Marc Andreessen 42:11 ... not the small customers, right? And so, so, so, so the small companies lack everything other than the ability to just do something new.

Lex Fridman 42:19 Yeah.

Marc Andreessen 42:19 Right? Um, and, and this is the timeless race and battle, and this is kind of the point I tried to make in the essay, which is like, both sides of this are good. Like it's really good to have like highly scaled tech companies that can do things that are like at staggering levels of sophistication. It's really good to have startups that can launch brand new ideas. They ought to be able to both do that and compete. They, neither one ought to be subsidized or protected from the others. Like that's, that's, uh, to me, that's just like very clearly the idealized world. It is the world we've been in for AI up until now, and then, of course, there are people trying to shut that down. But my hope is that, you know, th- the best outcome clearly will be if that continues.

Lex Fridman 42:50 We'll talk

Lex Fridman 42:51 about that a little bit, but I, I'd love to linger, uh, on, uh, some of the ways this is going to change the internet. So, um, I don't know if you remember, but there's a thing called Mosaic, and there's a thing called Netscape Navigator. So you were there in the beginning. Uh, what about the interface to the internet? How do you think the browser changes, and who gets to own the browser? We got to see some very interesting browsers, uh, Firefox, I mean, all the variants of Microsoft Internet Explorer, Edge, and, uh, now Chrome. Um, the actual... And it seems like a dumb question to ask, but do you think we'll still have the web browser?

Marc Andreessen 43:30 So I, uh, I lo- I have an eight-year-old and he's super into things like Minecraft and learning to code and doing all this stuff. So I, I, I, of course, like was very proud I could bring sort of fire down from the mountain to my kid, and I brought him ChatGPT, and I hooked him up-

Lex Fridman 43:41 Yeah.

Marc Andreessen 43:41 ... on his, on his, on his, on his laptop. And I was like, "You know, this is the thing that's gonna answer all your questions." And he's like, "Okay."... and I'm like, "But it's gonna answer our questions." And he's like, "Well, of course. Like, it's a computer. Of course it answers all your questions. Like, what else would a computer be good for, Dad?"

Lex Fridman 43:54 Yeah.

Marc Andreessen 43:55 (laughs) Um, and so-

Lex Fridman 43:55 Never impressed, are they?

Marc Andreessen 43:56 Ne- not impressed in the least. Two weeks pass, um, and he has some question, um, and I say, "Well, have you asked ChatGPT?" And he's like, "Dad, Bing is better."

Lex Fridman 44:05 Ooh.

Marc Andreessen 44:06 (laughs) And why is Bing better is because it's built into the browser. 'Cause he's like, "Look, I have the Microsoft Edge browser and, like, it's got Bing right here." And then, he, he doesn't know this yet, but one of the things you can do with Bing and Edge, um, is there's a setting where you can, um, use it to, uh, basically talk to any webpage, uh, because it's sitting right there next to the, uh, next to the, next to the browser. And, and by the way, which includes PDF documents. And so you can... in, in, in the way they've implemented in Edge with Bing is you can load a PDF, and then you can, you can ask it questions, which is the thing you, you, you can't do currently in, in, in just ChatGPT. So they're, you know, they're, they're gonna, they're gonna push the, the, the mel-... I think that's great, uh, you know, they're gonna push the melding and see if there's a, a combination thing there. Google's rolling out this thing, the Magic Button (laughs) , uh, which is implemented in, you know, they put it in Google Docs, right? And so you would go to, you know, Google Docs and you create a new document, and you, you know, you... instead of, like, you know, starting to type, you just, you know, say, it... you press the button and it starts to, like, generate content for you. Right? Like, i- is that the way that it'll work? Um, is it gonna be a speech UI where you're just gonna have an earpiece and talk to it all day long (laughs) , you know? Is it gonna be a... Like, uh, th- these are all... Like, this is exactly the kind of thing that I, I don't... This is exactly the kind of thing I don't think is possible to forecast. I think what we need to do is, like, run all those experiments. Um, and, and so one outcome is we come out of this with, like, a super browser that has AI built in that's just, like, amazing. The other... but there, look, there's a real possibility that the whole... I mean, look, there's a possibility here that the whole idea of a screen, and windows, and all this stuff just goes away 'cause, like, why do you need that if you just have a thing that's just telling you whatever you need to know?

Lex Fridman 45:31 Well, and, and also, so there's apps that you can use. I mean, you don't really use them, you know, b- being a Linux guy and Windows guy. Uh, there's one window of the browser that i-... with which you can interact with the internet, but on the phone, you can also have apps. So I can interact with Twitter through the app or through the web browser, and, uh, that seems like an obvious distinction, but why have the web browser in that case if one of the apps starts becoming the everything app?

Marc Andreessen 45:59 Yeah, that's right.

Lex Fridman 46:00 What Elon's trying to do with Twitter, but there could be others. There could be, like, a Bing app or there could be a Google app that just doesn't really do search, but just, like, do what I guess AOL did back in the day or something, where it's all right there, and, and it changes, um, it changes the nature of the internet because the, where the content is hosted, who owns the data, who owns the content, how, what is, what is the kind of content you create, how do you make money by creating content for the content creators, uh, all of that. Or, it could just keep being the same, which is like, with just the nature of webpage changes and the nature of content, but there will still be a web browser. 'Cause a web browser's a pretty sexy product.

Marc Andreessen 46:45 Mm-hmm.

Lex Fridman 46:46 It just seems to work.

Marc Andreessen 46:47 Mm-hmm.

Lex Fridman 46:47 'Cause it, like... You have an interface, a window into the world, and then the world can be anything you want. And as the world will evolve, there could be different programming languages, it can be animated, maybe it's three-dimensional and so on. Yeah, it's interesting. Do you think we'll still have the web browser?

Marc Andreessen 47:02 Every, every, every, um, every, uh, medium becomes the content for the next one. So-

Lex Fridman 47:06 Oh, boy.

Marc Andreessen 47:06 ... uh, the, you know, the AI will be able to give you a browser whenever you want. Um-

Lex Fridman 47:09 Oh, interesting.

Marc Andreessen 47:10 You know-

Lex Fridman 47:10 Generate it.

Marc Andreessen 47:11 Yeah. Or a- a- another way to think about it is maybe what the browser is, ma- maybe it's just the escape hatch, right? And which is maybe kind of what it is today, right? Which is like, most of what you do is, like, inside a social network, or inside a search engine, or inside, you know, somebody's app, or inside some controlled experience, right? But then, every once in a while, there's something where you actually want to-

Lex Fridman 47:28 Yeah.

Marc Andreessen 47:28 ... jailbreak, you wanna actually get free. (laughs)

Lex Fridman 47:30 The web browser is the F you to the man. You're, you're allowed to-

Marc Andreessen 47:33 Yeah.

Lex Fridman 47:33 That's the free internet-

Marc Andreessen 47:34 Yeah. Yep.

Lex Fridman 47:35 ... back in, back, uh, the way it was in the '90s.

Marc Andreessen 47:37 So here's something I'm proud of. So that nobody really talks about it, here's something I'm proud of, which is that the web, the web, the browser, the web servers, they're all, they're still backward compatible all the way back to, like, 1992, right? So-

Lex Fridman 47:45 Yeah.

Marc Andreessen 47:45 ... like, you can put up a, you can still... You know, the, the big breakthrough of the web early on, the big breakthrough was it made it really easy to read, but it also made it really easy to write.

Lex Fridman 47:53 Mm-hmm.

Marc Andreessen 47:53 It made it really easy to publish. Uh, and, and we literally made it so easy to publish, we made it not only so you could use... easy to publish content, it was actually also easy to actually write a web server.

Lex Fridman 48:00 Yeah.

Marc Andreessen 48:01 Right? And, and you could literally write a web server in four lines of Perl code, and, and, and you could start publishing content on it, and you could set whatever rules you want for the content, whatever censorship, no censorship, whatever you want, you could just do that. And as long as you had an IP address, right, you, you could do that. That still works, (laughs) right? That, that, like, that still works exactly as I just described. So th- th- this is part of my reaction to all of this, like, you know, all this just censorship pressure and all this, you know, these issues around control and all this stuff, which is like m- m- maybe we need to get back a little bit more to the Wild West. Like, the Wild West is still out there. Now, they, they will-

Lex Fridman 48:31 Oh, yes, it is.

Marc Andreessen 48:32 (laughs) ... they will try to chase you down. Like, they'll try to, you know, people who want to censor will try to take away your, your, um, you know, your, uh, domain name, and they'll try to take away your payments account and so forth if they really don't like what you, wh- wh- what you're saying. But, but nevertheless, you, like, unless they literally are intercepting you at the ISP level, like, you can still put up a thing. Um-

Lex Fridman 48:47 Mm-hmm.

Marc Andreessen 48:47 ... and so I, I don't know, I think that's important to preserve, right? Like, b- because, because, becau-... I mean, one is just a, a fre- a freedom argument, but the other is a creativity argument, which is you, you wanna have the escape hatch so that the kid with the idea is able to realize the idea, 'cause to your point on PageRank, you, you actually don't know what the next big idea is, right? No- nobody called Larry Page and told him to develop PageRank. Like, he came up-

Lex Fridman 49:06 (laughs)

Marc Andreessen 49:06 ... with that on his own, and you wanna always, I, I think, leave the escape hatch for the next, you know, kid or the next Stanford grad student to have the breakthrough idea and be able to get it up and running before anybody notices.

Lex Fridman 49:15 Um, you and I are both fans of history, so let's step back. We've been talking about the future. Let's step back for a bit and look at, uh, the '90s. You created Mosaic web browser, the first widely used web browser. Tell the story of that. How, how... And how did it evolve into Netscape Navigator? This is the early days.

Marc Andreessen 49:33 ... so full story. So, um-

Lex Fridman 49:36 You were born.

Marc Andreessen 49:37 I was born, a small, a small child. Um-

Lex Fridman 49:39 W- well, actually, with the... Yeah, let's go there.

Marc Andreessen 49:41 No.

Lex Fridman 49:41 Like, when, when did you... When would you first fall in love with computers?

Marc Andreessen 49:45 Oh, I... So I hit the generational jackpot, and I hit the Gen X kind of point perfectly as it turns out. So I was born in 1971. So there's this great website called WTFHappenedIn1971.com, which is basically 1971 is when everything started to go to hell. And I was, of course, born in 1971, so I like to think that I had something to do with that.

Lex Fridman 50:01 Did you make it on the website?

Marc Andreessen 50:03 I have... I don't think I made it on the website, but I, I, you know, hopefully-

Lex Fridman 50:06 Somebody needs to add, "This is, this is where everything went wrong."

Marc Andreessen 50:08 Maybe I contributed to some of the trends, um, that they, uh, that they should... Every, every line on that website goes like that, right? So it's all, it's all, uh, it's all a picture disaster. But, um, but there was this moment in time where... Because the p- you know, sort of, the Apple, you know, the Apple II hit in, like, 1978 and then the IBM PC hit in '82. So I was, like, you know, 11 when the PC came out, um, and so I just kind of hit that perfectly. And then that was the first moment in time when, like, regular people could spend a few hundred dollars and get a computer, right? So that... I just like that, that, that resonated right out of the gate. Um, well, then the other part of the story is, you know, I, I was using an Apple II. I used a bunch of them, but I was using an Apple II. And of course, it said on the back of every Apple II and every Mac, it said, you know, "Designed in Cupertino, California." And I was like, "Wow, okay. Cupertino must be the, like, shining city on the hill, like Wizard of Oz, like, the most amazing, like, city of all time. I can't wait to see it." Of course, years later, I came out to Silicon Valley and went to Cupertino, and it was just a bunch of office parks (laughs) and low-rise apartment buildings. So the aesthetics were a little disappointing, but, you know, it was the, the vector, uh, right, of the, of the creation of a lot, of a lot of this stuff, um-

Lex Fridman 51:09 Yeah.

Marc Andreessen 51:09 So, so then basically by... So part, par- part of my story is just the luck of having been born at the right time and getting exposed to PCs. Then the other part is, um... The other part is when Al Gore says that he created the internet, he actually is correct, uh, in, in, in a really meaningful way, which is he sponsored a bill in 1985 that essentially created the modern internet, created what is called the NSF NET at the time, which is sort of the, the first really fast internet backbone. Um, and, uh, you know, that, that bill dumped a ton of money into a bunch of research universities to build out basically the internet backbone, and then these supercomputer centers that were clustered around, um, the, the, the internet. And, and one of those universities was University of Illinois where I went to school. And so the other stroke of luck that I had was I, I went to Illinois basically right as that money was just, like, getting dumped on campus. And so as a consequence, we had at... on campus, and this is, like, you know, '89, 1991, we had like... You know, we were right on the internet backbone. We had like T3 and 45... at the time T3, 45 megabit backbone connection, which at the time was, you know, wildly state-of-the-art. Um, we had Cray supercomputers, we had thinking machines, parallel supercomputers. We had Silicon Graphics workstations. We had Macintoshes. We had, we had NeXT cubes (laughs) all over the place. We had like every possible kind of computer you could imagine because all this money just fell out of the sky. Um, yeah.

Lex Fridman 52:18 So you were living in the future?

Marc Andreessen 52:20 Yeah. So yeah, quite literally it was... Yeah, like, it's all, it's all there. So like we had full broadband graphics, like the whole thing. And, and it's actually funny because they had this... The- this is the first time I kind of... It sort of tickled the back of my head that there might be a big opportunity in, in here which is, you know, they, they embraced it and so they put, like, computers in all the dorms and they wired up all the dorm rooms and they had all these, you know, labs everywhere and everything, and then they, they gave every undergrad a computer account and an email address. Um, and the assumption was that you would use the internet for your four years at college, um, and then you would graduate and stop using it. And that was that, right?

Lex Fridman 52:53 Yeah.

Marc Andreessen 52:54 And you just retire your email address. It wouldn't be relevant anymore because you'd go off in the workplace and they don't use email. You'd be back to using fax machines or whatever. Um-

Lex Fridman 53:00 Did you have that sense as well? Like what, what... You said the, the back of your head was tickled, like what, what was your... What, what was exciting to you about this possible world?

Marc Andreessen 53:08 Well, if this... If this is so useful in this containment... If this is so useful in this contained environment that just has this weird source of outside funding, then if, if it were practical for everybody else to have this, and if it were cost-effective for everybody else to have this, wouldn't they want it? And the over- overwhelmingly the prevailing view at the time was, no, they would not want it. This is esoteric, weird nerd stuff (laughs) , right, that, like, computer science kids like, but like normal people are never gonna do email, right? Or be on the internet, right? Um, and so I was just like, "Wow, like this, this is actually like... This is really compelling stuff." Now the other part was, it was all really hard to use and in practice you had to be a... basically a CS, uh, you know, basically had to be, had to be a CS undergrad or equivalent to actually get full use of the internet at that point, um, because it was all pretty esoteric stuff. So then that was the other part of the idea, which was, "Okay, we need to actually make this easy to use."

Lex Fridman 53:51 So what's involved in creating Mosaic? Like, in, in creating a graphical interface to the internet?

Marc Andreessen 53:57 Yeah. So it was, it was a combination of things. So it was like basically the, the web... The web existed in an early sort of described as prototype form. Uh, and by the way, text only at that point. Um-

Lex Fridman 54:06 Uh, what did it look like? What, what was the web? I mean, what... And the key figures? Like, what was the... What was it like? What-

Marc Andreessen 54:12 You know what? It-

Lex Fridman 54:12 Paint a picture.

Marc Andreessen 54:13 It looked like ChatGPT, actually. Um-

Lex Fridman 54:15 (laughs)

Marc Andreessen 54:15 It was all text.

Lex Fridman 54:16 Yeah.

Marc Andreessen 54:17 Um, and so you had a text-based web browser.

Lex Fridman 54:19 Yeah.

Marc Andreessen 54:19 Uh, well, actually the original browser, Tim, Tim Berners-Lee, the original, the original browser, both the original browser and the server actually ran on NeXT, NeXT cubes.

Lex Fridman 54:26 Mm-hmm.

Marc Andreessen 54:26 So these were... This was, you know, s- the computer Steve Jobs made during the interim period when he... During the decade-long interim period when he was not at Apple. You know, he got fired in '85 and then came back in '97. So this was in that interim period where he had this company called NeXT and they made these... literally these computers called cubes.

Lex Fridman 54:41 Mm-hmm.

Marc Andreessen 54:41 And there's this famous story. They were beautiful but they were, uh, 12-inch by 12-inch by 12-inch cubes-

Lex Fridman 54:45 Mm-hmm.

Marc Andreessen 54:46 ... computers. And there's a famous story about how they could have cost half as much if it had been 12 by 12 by 13, but-

Lex Fridman 54:51 (laughs)

Marc Andreessen 54:51 (laughs) Steve was like, "No, like it has to be." So they were like $6,000, basically, academic workstations.

Lex Fridman 54:56 Yeah.

Marc Andreessen 54:56 They had the first CD-ROM drives, um, which were slow. I mean, it was... The, the computers were all but unusable, um, they were so slow, but they were beautiful.

Lex Fridman 55:05 Okay. Can we actually just take a tiny tangent there?

Marc Andreessen 55:07 Sure, of course.

Lex Fridman 55:09 The, the 12 by 12 by 12, uh, that just so beautifully encapsulates Steve Jobs' idea of design.

Lex Fridman 55:16 Can you just comment on, um, what you find interesting about Steve Jobs? What, uh, about that view of the world, that dogmatic pursuit of perfection in how he saw perfection in design?

Marc Andreessen 55:28 Yes. I guess I'd say, like... Look, he was a deep believer, I think in a very deep... The way I interpret it. I don't know if he ever really described it like this, but the way I'd interpret it is, it's, it's like, it's like this thing in... It's, it's actually a thing in, in philosophy. It's like, aesthetics are not just appearances. Aesthetics go all the way to like deep underlining, under- underlying meaning, right? It's like-... I'm not a physicist. One of the things I've heard physicists say is one of the things you start to get a sense of when a theory might be correct is when it's beautiful, right? Like, y- you know, they're... Right? And so- so- so, there's something... And you- you feel the same thing, by the way, in, like, human psychology.

Lex Fridman 55:58 Mm-hmm.

Marc Andreessen 55:58 Right? You know, when- when you're experiencing awe, right? You know, there's like a- there's like a- there's a simplicity to it. When- when you're having an hones- interaction with somebody, there's an aesthetic, I would say, calm that comes over you, because you're actually being fully honest and not trying to hide yourself. Right? So, there- there... So- so, it's like this very deep sense of aesthetics.

Lex Fridman 56:13 And he would trust that judgment that he had deep down, like-

Marc Andreessen 56:17 Yeah.

Lex Fridman 56:17 ... e- even- even if the engineering teams are saying, "This is, uh, this is too difficult," even if the, whatever, the finance folks are saying, "This is ridiculous," the, uh, the supply chain, all of that kind of stuff, "This makes this impossible. The materi- we can't do this kind of material. Uh, this has never been done before," so on and so forth, he just sticks by it.

Marc Andreessen 56:35 Well, I mean, who makes a phone out of aluminum, right?

Lex Fridman 56:37 Yeah.

Marc Andreessen 56:37 Like... (laughs) You know, nobody else would have done that. Uh, and now, of course, if your phone was made out of aluminum, why- you know, it would have crude... N- what kind of caveman would you have to be to have a phone that's made out of plastic? Like, right?

Lex Fridman 56:47 Yeah.

Marc Andreessen 56:48 So, like... So, it's just this very... Right. And- and, you know, look, it's- it's- there's a thousand different ways to look at this, but one of the things is just, like, look, these things are central to your life. Like, you're with your phone more than you're with anything else. Like, it's in your- it's gonna be in your hand. I mean, he-

Lex Fridman 57:00 Yeah.

Marc Andreessen 57:00 ... you know, you know this, he thought very deeply about what it meant for something to be in your hand all day long.

Lex Fridman 57:03 Yeah.

Marc Andreessen 57:04 But for example, he, uh, uh, uh, here's an interesting design thing. Like, he- he never wanted a, a s-... My understanding is he never wanted an iPhone to have a screen larger than you could reach with your thumb-

Lex Fridman 57:14 Mm-hmm.

Marc Andreessen 57:14 ... one-handed. And so, he- he was actually opposed to the idea of making the phones larger. And I- I don't know if you have this experience today, but let's say there are certain moments in your day when you might be, like, um, only have one hand available, um, and you might want to be on your phone.

Lex Fridman 57:25 Yeah.

Marc Andreessen 57:26 And you're trying to, like (laughs) s- to text and you- your thumb can't reach the send button.

Lex Fridman 57:31 Yeah. I mean, there's pros and cons, right?

Marc Andreessen 57:32 Right.

Lex Fridman 57:32 And then there's, like, folding phones, which I would love to know what he thought or thinks about them. Uh, but, I mean, is there something you could also just linger on, 'cause he's one of the interesting, um, figures in the history of technology. What makes him, what makes him as successful as he was? What makes him as interesting as he was? Uh, what made him, um, so productive and important in, um, in- in- in the development of technology?

Marc Andreessen 57:58 He had an integrated world view. So, the- the- the- the properly designed device that had the correct functionality, that had the deepest understanding of the user, that was the most beautiful, right? Like, it- it had to be all of those things, right? It- it- it was... He basically would drive to as close to perfect as you could possibly get. Right? And I- I, you know, I suspect that he never quite, you know, thought he ever got there, because most great creators, you know, are generally dissatisfied. You know, you read accounts later on and all they can, all they can see are the flaws in their creation. But, like, he got as close to perfect each step of the way as he could possibly get, with the, with the constraints of the- of the- of the technology of his time. Um, and then, you know, look, he was, you know, sort of famous in the Apple model as like, look, they- they- they will... You know, this- this headset that they just came out with, like-

Lex Fridman 58:35 Mm-hmm.

Marc Andreessen 58:35 ... it's like a decade-long project, right? It's like, you- and they're just gonna sit there and tune and tune and polish and polish and tune and polish and tune and polish until it is as perfect as anybody could possibly make anything.

Lex Fridman 58:45 Yeah.

Marc Andreessen 58:45 And then this goes to the- the- the way that people describe working with him was, which is, you know, there was a terrifying aspect of working with him, which is, you know, he was, you know, he was very tough. Um, but there was this thing that everybody I've ever talked to who worked for him says that- that... They all say the following, which is, "He- we did the best work of our lives when we worked for him, because he set the bar incredibly high, and then he supported us with everything that he could to let us actually do work of that quality."

Lex Fridman 59:08 Mm-hmm.

Marc Andreessen 59:08 And so they... A lot of people who were at Apple spend the rest of their lives trying to find another experience where they feel like they're able to hit that quality bar again.

Lex Fridman 59:14 E- even if it, uh, in retrospect or during it felt like suffering.

Marc Andreessen 59:18 Yeah. Exactly. Yeah.

Lex Fridman 59:20 What does that teach you about the human condition, huh?

Marc Andreessen 59:24 So, look, it's- it's- so it's a-

Lex Fridman 59:26 (laughs)

Marc Andreessen 59:26 Exactly. So, the Silicon Valley... I mean, look, he's not, you know... George Patton in the, you know-

Lex Fridman 59:30 Yeah.

Marc Andreessen 59:30 ... in- in the Army. Like, you know, there are many examples in other fields, you know, that are like this. Um, uh, uh, specifically in- in tech, it's- it's actually, I find it very interesting. There's the Apple way, which is polish, polish, polish and don't ship until it's as perfect as you can make it. And then there's the sort of, the other approach, which is the sort of incremental hacker mentality, which basically says ship early and often and iterate. And one of the things I find really interesting is, I'm now 30 years into this, like, there are very successful companies on both sides of that approach.

Lex Fridman 59:59 Mm-hmm.

Marc Andreessen 1:00:00 Right? Um, like, that is a fundamental difference, right? In how to operate and how to build and how to create that you have world-class companies operating in both ways.

Lex Fridman 1:00:10 Mm-hmm.

Marc Andreessen 1:00:10 Um, and I don't think the question of, like, which is the superior model is anywhere close to being answered. Like, a- and- and my suspicion is the answer is do both. The answer is you actually want both. Th- they lead to different outcomes. Software tends to do better, um, with the iterative approach. Um, hardware tends to do better with the, uh, you know, sort of wait and make it perfect approach. But again, you can find examples in- in- in- in both directions.

Lex Fridman 1:00:33 Uh, so the jury's still out on that one.

Marc Andreessen 1:00:36 Yeah.

Lex Fridman 1:00:36 Uh, so back to Mosaic. So, what, uh... (laughs) It was text-based.

Marc Andreessen 1:00:42 Yep.

Lex Fridman 1:00:42 Uh, Tim Berners-Lee-

Marc Andreessen 1:00:44 Well, there was the web, which was text-based, but there were no... I mean, there was like three websites. There was, like, no content. There were no users. Like, it wasn't like a, it wasn't like a catalytic... It had an... And there... By the way, it was all- it was... Because it was all text, there were no documents, there were no images, there were no videos, there were no... Right? So- so, it was, it- it was... And then if- if... In the beginning, if- you had to be on a NeXT cube. Right? You need to have a NeXT cube both to publish and to consume, so...

Lex Fridman 1:01:04 Mm-hmm.

Marc Andreessen 1:01:04 So, there- there were-

Lex Fridman 1:01:05 It was 6,000 bucks, you said?

Marc Andreessen 1:01:06 There were limitations on... Yeah. $6,000 PC. They did not- they did not sell very many.

Lex Fridman 1:01:10 Yeah.

Marc Andreessen 1:01:10 But then there was also, (laughs) there was also FTP and there was Usenet, right? And there was, you know, a dozen other basically w- there was WAIS, which was an early search thing. There was Gopher, which was an early menu-based information retrieval system. There- there were like a dozen different sort of scattered ways that people would get to information on- on the internet. And so, the- the Mosaic idea was basically bring those all together, make the whole thing graphical, make it easy to use, make it basically bulletproof so that anybody can do it. And then, again, just on the luck side, it- it so happened that this was right at the moment when graphics, wh- when the GUI sort of actually took off.

Lex Fridman 1:01:40 Mm-hmm.

Marc Andreessen 1:01:40 And we're- we're now all so used to the GUI that we think it's been around forever. But it didn't real... It, you know, the- the- the Macintosh brought it out in '85, but they actually didn't sell very many Macs in the '80s-

Lex Fridman 1:01:49 Mm-hmm.

Marc Andreessen 1:01:49 It was not that successful of a product. Um, it really was Win... You needed Windows 3.0 on PCs, and that hit in about '92.Um, and so when we did Mosaic in '92 and '93, so that sort of, it was, like, right at the moment when you could imagine actually having a graphical user interface to, right, at all, m- much less one to the internet.

Lex Fridman 1:02:08 How, uh, how well did Windows 3 sell? So there was that-

Marc Andreessen 1:02:12 Peo-

Lex Fridman 1:02:12 ... the really big-

Marc Andreessen 1:02:13 That was the big bang.

Lex Fridman 1:02:13 ... the big operating, graphical operating system?

Marc Andreessen 1:02:16 Well, this is the classic, okay, there's Microsoft was operating on the