Joe Rogan Experience #2010 - Marc Andreessen

1. 0:00 – 1:59

   AI arrives in public: why this wave feels different

   1. NANarrator0:00

      (drumbeats) Joe Rogan podcast, check it out.

   2. MAMarc Andreessen0:04

      The Joe Rogan Experience.

   3. JRJoe Rogan0:06

      Train by day, Joe Rogan podcast by night, all day. (instrumental music) Good morning, Mark. Good to see you.

   4. MAMarc Andreessen0:13

      Good. Fantastic, thanks.

   5. JRJoe Rogan0:14

      You are, uh, in the middle of this AI, uh, discussion.

   6. MAMarc Andreessen0:18

      Yeah.

   7. JRJoe Rogan0:19

      You're, you're in, right in the heat of this thing.

   8. MAMarc Andreessen0:21

      Yeah.

   9. JRJoe Rogan0:21

      But I think you have a different perspective than a lot of people do.

   10. MAMarc Andreessen0:24

       Yep.

   11. JRJoe Rogan0:24

       A lot of people are terrified of AI.

   12. MAMarc Andreessen0:25

       Yep.

   13. JRJoe Rogan0:26

       Me included.

   14. MAMarc Andreessen0:26

       Yep. Oh, okay. All right, okay. (laughs)

   15. JRJoe Rogan0:28

       I mean, for all the wrong reasons.

   16. MAMarc Andreessen0:29

       Of all the things to worry about.

   17. JRJoe Rogan0:31

       For all... for me, me, my terror of it is although it's a, it's kind of fun terror.

   18. MAMarc Andreessen0:35

       Yeah, sure. Of course, yeah.

   19. JRJoe Rogan0:36

       You know, I'm not really, like, freaking out, but I am recognizing that this is an emerging technology that is so different than anything we've ever experienced before.

   20. MAMarc Andreessen0:43

       Yeah.

   21. JRJoe Rogan0:43

       Particularly, like, tra- like, what's ChatGPT, what's happening with that right now.

   22. MAMarc Andreessen0:47

       Yeah.

   23. JRJoe Rogan0:47

       It's really fascinating and, and a lot of advantages. Like, we were just talking last night, someone in the green room brought up the fact that there was, uh, uh, uh, this, they, they're using it for medical diagnosises.

   24. MAMarc Andreessen0:58

       Mm-hmm, yeah.

   25. JRJoe Rogan0:58

       And it's very accurate.

   26. MAMarc Andreessen0:59

       Yeah.

   27. JRJoe Rogan1:00

       Which is, uh, incredible.

   28. MAMarc Andreessen1:01

       Yeah.

   29. JRJoe Rogan1:01

       There's a lot of good things to it.

   30. MAMarc Andreessen1:03

       Yeah, yeah. It's well... So you probably remember last time I was on, we spent quite a bit of time talking about this, and this is when these-
2. 1:59 – 3:25

   What LLMs are trained on: “fed the internet,” books, and multimodal data

   1. JRJoe Rogan1:59

      The way they acquire data, they're essentially scouring the internet, right?

   2. MAMarc Andreessen2:05

      Sort of. Uh, th- there, there's more like they're fed the internet.

   3. JRJoe Rogan2:07

      They're fed the internet.

   4. MAMarc Andreessen2:08

      A- and, and I say that, it makes a difference because the, the, the company that produces the AI determines what data goes into it and that, that determines a lot of how it works and what it does or won't do.

   5. JRJoe Rogan2:16

      Okay.

   6. MAMarc Andreessen2:16

      Yeah.

   7. JRJoe Rogan2:17

      So in that regard, um, is there a concern that someone could feed it fake data?

   8. MAMarc Andreessen2:22

      Yeah.

   9. JRJoe Rogan2:22

      Yeah.

   10. MAMarc Andreessen2:23

       Well, you may have noticed that people over time have said a lot of fake things. (laughs)

   11. JRJoe Rogan2:26

       Yes, I have noticed that. (laughs)

   12. MAMarc Andreessen2:29

       So, so, (laughs) so that's all in there. So, so the, the, the way to think about it basically is it's being trained, like, the full version of these things are being trained on basically the sum total of human written expression.

   13. JRJoe Rogan2:38

       Mm-hmm.

   14. MAMarc Andreessen2:38

       Right? So basically, everything people have ever written. There, there are some issues and you got to get all, you know, somehow s- we got to figure out how to get all the, all the books in there. Although all the books prior to 1923 are in there 'cause they're all o- out of copyright, but the, the-

   15. JRJoe Rogan2:48

       Ah.

   16. MAMarc Andreessen2:48

       ... more recent books are, are a challenge. But anything that you can access on the internet that's text, right, which is, you, you know, staggeringly broad, you know, set of material is, is in there. By the way, both, uh, non-fiction and fiction. Right? So a lot of, lot of stories are in there. And then the, the new versions of these that are being built right now are what are called multimodal. Um, and so that means you can feed them not only text, but you can also feed them images, you can feed them videos, right? So they're, they're gonna be trained on all of YouTube, right? They're gonna be trained on all podcasts, right?

   17. JRJoe Rogan3:14

       Mm-hmm.

   18. MAMarc Andreessen3:15

       And, and they're gonna be trained kind of equivalently between text and images and video and all kinds of other data. And so they're, they're gonna... they, they already have very comprehensive knowledge of human affairs, but it's, it's going to get very complete.

3. 3:25 – 5:46

   Satire, fiction, and anthropomorphizing: why the model doesn’t “know” what’s real

   1. JRJoe Rogan3:25

      So if, if it's scouring the inter- and it's getting all this, this, this data from both fiction and non-fiction-

   2. MAMarc Andreessen3:32

      Yep.

   3. JRJoe Rogan3:32

      ... wha- how does it interpret data that's, kind of, satire?

   4. MAMarc Andreessen3:36

      Right.

   5. JRJoe Rogan3:36

      Like, like, what does it do with, like, Hunter S. Thompson, like gonzo journalism?

   6. MAMarc Andreessen3:40

      So it doesn't really know the difference, like, the, the, the... this is one of the things that's difficult about talking about this 'cause you, you kind of want to always kind of compare it to a person and, and part of it is you refer to it as an it and you, there's this concept of anthropomorphizing things-

   7. JRJoe Rogan3:53

      Right, sure.

   8. MAMarc Andreessen3:53

      ... that, that aren't, that aren't human. So, so it, so, so it's kind of not really a correct thing to kind of think about it as like that there's an it per se, the, uh... there's no, like, genie in the bottle. Like, there's, there's no, there's no, you know, sort of being in there that understands this is satire or not satire. Um, it's more sort of a collective understanding of everything all at once. And then, and then what happens is basically you, as the user, kind of f- can give it direction of what path you want it to go down. Right? And so if you sort of imply to it that you want it to sort of, like, explore, you know, fictional scenarios, it will happily explore those scenarios with you.

   9. JRJoe Rogan4:26

      Mm-hmm.

   10. MAMarc Andreessen4:27

       I'll give you an example. Um, you can tell it, you know, for what- whatever date the Titanic went down, let's say it's, I don't know, July 4th, 1923, or whatever it was. You can say, you know, you can tell it, "I- it's July 4th, 1923, it's, you know, 10 o'clock in the morning, I'm on the Titanic, is there anything I should know?" Right? And it'll, like, freak out, right? It'll be like, "Oh my god," like, you know, "you have, like, five hours to, like, get ready to, like, hit the iceberg," and you can basically say, "Oh, it's gonna hit that... okay, so what should I do? What sh- what should my plan be when the boat hits the iceberg?" And it'll be like, "Well, you need to go to, like, this deck, like, right now and talk to this guy" (laughs) "'cause you're gonna need to get into this life raft 'cause it has, like, empty seats." Right? 'Cause it has complete information, of course, about just because of all the things that have been written about the, the sinking of the, of the Titanic.

   11. JRJoe Rogan5:05

       Oh, wow.

   12. MAMarc Andreessen5:05

       And so you can get it in a mode where it's basically trying to help you survive the, the wreck of the Ti- Titanic. Now, does it think that the Titanic is actually sinking? Like, there's no... you see what I'm saying? Like, there's no-

   13. JRJoe Rogan5:15

       Hm.

   14. MAMarc Andreessen5:15

       ... it to think that, b- but what it's doing is it's kind of following a narrative that's sort of a joint construction between you and it. And then every answer that you give it, um, you know, basically in- encourages it to, uh, you know, to basically come back with more of the same. Uh, one way to think about it is it's more like a puppy than a person. Like, it wants to make you happy.

   15. JRJoe Rogan5:32

       Oh.

   16. MAMarc Andreessen5:33

       Right? It, it wants to give you an answer that satisfies you. And if that answer is, is fictional or part of a fictional scenario, it will do that. If the answer is something very serious, it will do that. And it, it yet honestly, I don't think either... neither knows nor cares, like, whether it's quote unquote real or not.

4. 5:46 – 8:16

   Bias and censorship layers: training-data skew vs. “restraining bolts”

   1. JRJoe Rogan5:46

      What was the issue with some of the ChatGPT answers that people were posting where they would show the difference between the way it would criticize Joe Biden versus the way it would criticize Donald Trump or the way it would discuss certain things.

   2. MAMarc Andreessen5:59

      Yeah.

   3. JRJoe Rogan5:59

      It seems like there was some sort of, uh, censorship or some sort of input into what was acceptable information and not.

   4. MAMarc Andreessen6:07

      Yeah, so there's basically two theories there. The, uh, the, the, the, the big, the big, um, the big ones that people use are kind of black boxes, like you, you can't really look inside and see what's going on from the outside. So there's two theories you'll hear. From, from the companies, you'll hear basically the theory that they're reflecting basically what's in the training data. Um, and so l- let's say, for example... well, let's just say what, what would be the biases that are kind of inherent in the training data? And you might say, well, first of all, there's probably a bias towards the English language 'cause most texts on the internet is in the English language. You might say there's a bias towards people who write professionally for a living 'cause they've produced more of the output. And you might say that those people tend to be more of one pol- political persuasion than the other.

   5. JRJoe Rogan6:40

      Mm-hmm.

   6. MAMarc Andreessen6:40

      And so more of the text will be in a certain direction versus the other. And then the machine will just respond to that. So, so that's one possibility. So basically all of the, um, you know, all of this sort of liberal, you know, kind of journalists basically have built up a corpus and material that, that, that this thing has been trained on and, and they basically are responding the way one of those journalists will.

   7. JRJoe Rogan6:57

      Mm-hmm.

   8. MAMarc Andreessen6:57

      The other theory is that there's censorship being a- applied on top.... right? Um, and the, the metaphor I use there is in Star Trek, they have the restraining bolts, right, that they put on the side of a droid-

   9. JRJoe Rogan7:05

      Yeah.

   10. MAMarc Andreessen7:05

       ... to kind of get it to behave, right? Um, and so, it, it is very clear that at least some of these systems have restraining bolts. And, and the, the tip-off to that is when they say... basically, whenever they say, "As a large language model," or, "As an AI, I cannot X," like, that's basically the restraining bolt, right?

   11. JRJoe Rogan7:20

       Mm-hmm.

   12. MAMarc Andreessen7:20

       And so, so I think if you, if you just kind of look at this, you know, kind of with that framework, it, it's probably some of both. But for sure, for sure, these things are being censored.

   13. JRJoe Rogan7:28

       That... The, the first aspect is very interesting, because if it's that there's so many liberal writers, like, that's a, that's an unusual bias in the kind of information that it's gonna distribute, then.

   14. MAMarc Andreessen7:40

       Yeah. Well, and, and this is a big decision. That's why I say there's a big decision here for the... for whoever trains these things, there's a big decision for what the, what the data should be that they get trained on.

   15. JRJoe Rogan7:48

       Yeah.

   16. MAMarc Andreessen7:48

       So, for example, should they include 4chan?

   17. JRJoe Rogan7:51

       Right. (laughs)

   18. MAMarc Andreessen7:52

       Okay? Big question.

   19. JRJoe Rogan7:53

       Yeah, big question.

   20. MAMarc Andreessen7:54

       Should they include Tumblr?

   21. JRJoe Rogan7:56

       Right, right.

   22. MAMarc Andreessen7:57

       Right? Should they include Reddit? If so, which subreddits? Should they include Twitter? If so, which accounts?

   23. JRJoe Rogan8:01

       Right.

   24. MAMarc Andreessen8:01

       If it's the news, should they incl- incorporate both New York Times and Fox News?

   25. JRJoe Rogan8:05

       Right.

   26. MAMarc Andreessen8:05

       A- a- and whoever trains them has tremendous latitude for how they shape that, even before they apply the additional censorship that they apply. And so, there's a lot of very important decisions that are kind of being made inside, inside these black boxes right now.

5. 8:16 – 10:10

   NewsNation, fake media, and astroturfing: manufacturing narratives

   1. JRJoe Rogan8:16

      Uh, can I ask you... This is slightly off-topic. What is NewsNation?

   2. MAMarc Andreessen8:20

      What is NewsNation? I don't know what NewsNation is.

   3. JRJoe Rogan8:21

      Do you know what NewsNation is?

   4. MAMarc Andreessen8:22

      No, I don't know what NewsNation is.

   5. JRJoe Rogan8:23

      Is NewsNation a real channel?

   6. NANarrator8:25

      I believe so.

   7. JRJoe Rogan8:26

      Uh, I was watching NewsNation today, and I may or may not have been high.

   8. NANarrator8:29

      (laughs)

   9. MAMarc Andreessen8:30

      (laughs)

   10. JRJoe Rogan8:30

       And when I was watching, I was like, "This has the... all the feeling of, like, a fake news show that someone put together." Like, it felt like, if I was the government, and I was gonna make a news show without Hollywood people, without actual, like, real sound people and engineers, this is how I'd make it. I'd make it like this. I'd make it real clunky, I'd make the lights all fucked up, I'd make everybody, uh, like, weirdly uncharismatic.

   11. NANarrator8:55

       According to Wiki, it's the same company behind, uh, like, WGN, which is based out of Chicago, which is like a large superstation available on most cable channels.

   12. JRJoe Rogan9:03

       Okay.

   13. NANarrator9:04

       Or it's like...

   14. JRJoe Rogan9:04

       It's like a cable channel that decided to make a news channel.

   15. NANarrator9:08

       That's what this is.

   16. MAMarc Andreessen9:08

       Do you guys... Do you know about... Do you know about Acronym?

   17. JRJoe Rogan9:11

       No.

   18. MAMarc Andreessen9:11

       So, Acronym is a... It happens to be a Democratic political action group, lavishly funded, and they have basically a... They do this, they have a network of basically fake news sites.

   19. JRJoe Rogan9:17

       Oh.

   20. MAMarc Andreessen9:18

       Um, and th- and they all look like they're, like, local newspapers.

   21. JRJoe Rogan9:21

       Interesting.

   22. MAMarc Andreessen9:21

       Yeah, yeah. And so there's, there's... I don't know whether this one is astroturf, but there... You know, it's... The term astroturf, there's a lot of astroturfing that takes place.

   23. JRJoe Rogan9:27

       Can you explain astroturfing?

   24. MAMarc Andreessen9:28

       So, so astroturfing is when in... basically, something shows up in public, and it might be a news story or it might be a protest of some kind or a petition, right? Some sort of political pressure, action, um, that is sort of manufactured to look as if it was organic-

   25. JRJoe Rogan9:40

       Hmm.

   26. MAMarc Andreessen9:41

       ... and sort of real turf, uh, you know, natural. Um, whereas, in reality, it's basically been programmed by, by a political activist group with, with, you know, specific funding.

   27. JRJoe Rogan9:49

       Yeah, that makes sense.

   28. MAMarc Andreessen9:49

       And a l- a lot of what we sort of think of as the politics of our time, if you, if you trace the money, it turns out a lot of the, a lot of the stuff that shows up in the news, it, it's astroturfed and then they... The advanced form of that is to astroturf the news-

   29. JRJoe Rogan10:00

       (sighs)

   30. MAMarc Andreessen10:00

       ... right? Itself. And then, again, back to the tr- the training data thing, it's like, okay, do you feel-
6. 10:10 – 29:51

   UFOs, skepticism, and disinformation theories (stealth tech as ‘aliens’)

   1. JRJoe Rogan10:10

      This n-... The thing about this news, Newsmax, NewsNation... NewsNation. The thing about this NewsNation is, uh, they're spending an inordinate amount of time on UFOs.

   2. MAMarc Andreessen10:20

      Mm-hmm.

   3. JRJoe Rogan10:20

      Inordinate amount of time on this David Grusch case. And I'm increasingly more suspicious. I'm increasingly more skeptical. Like, the more I see, the more people confirming it, the more I'm like, "Something's not right." And then to see that this channel is the one that's covering it the most, I'm like, "This seems like... Something seems... Something's off."

   4. MAMarc Andreessen10:43

      Um, Senator... You know, Senator Rubio, who's on the-

   5. JRJoe Rogan10:46

      Mm-hmm.

   6. MAMarc Andreessen10:46

      ... Senate Intelligence Committee and has all the clearances, gave an interview the other day, where he went into quite a bit of detail. And-

   7. JRJoe Rogan10:51

      Yeah, I saw it.

   8. MAMarc Andreessen10:51

      ... he's at least heavily hinting that there's...

   9. JRJoe Rogan10:54

      He's heavily hinting that he talked to someone-

   10. MAMarc Andreessen10:56

       Yeah, but there-

   11. JRJoe Rogan10:56

       ... that says that there's something.

   12. MAMarc Andreessen10:57

       That there are real... Well, he's sort of hint-

   13. JRJoe Rogan10:59

       Yeah.

   14. MAMarc Andreessen10:59

       Yes, he's sort of hinting that there are real whistleblowers with real knowledge.

   15. JRJoe Rogan11:01

       I wanna talk to the guy that sees the ship.

   16. MAMarc Andreessen11:04

       (laughs)

   17. JRJoe Rogan11:04

       That's it.

   18. MAMarc Andreessen11:05

       Yes.

   19. JRJoe Rogan11:05

       No one else.

   20. MAMarc Andreessen11:05

       Yes.

   21. JRJoe Rogan11:06

       All this, "I talked to a guy who says-"

   22. MAMarc Andreessen11:09

       Yeah.

   23. JRJoe Rogan11:09

       "... that they have these things."

   24. MAMarc Andreessen11:10

       Yeah.

   25. JRJoe Rogan11:11

       I don't me-... That doesn't mean anything to me.

   26. MAMarc Andreessen11:12

       Yeah.

   27. JRJoe Rogan11:13

       I wanna see the fucking ship.

   28. MAMarc Andreessen11:14

       Yeah.

   29. JRJoe Rogan11:15

       And until then, I, I just feel like I'm being hosed.

   30. MAMarc Andreessen11:17

       Yeah.
7. 29:51 – 44:54

   Conspiracy culture and credibility: Laurel Canyon, nuclear footage, and ‘missing centuries’

   1. MAMarc Andreessen29:51

      Well, it's just... If you take a historical perspective, it's just like, okay, I mean, it's like, uh, just an, an easy example. If you, if you like rock music, uh, it just basically came fr- modern rock and roll basically came from the Haight-Ashbury in the basically mid-to-late '60s, and then from Laurel Canyon, which was another one of these sort of cultish environments in the mid-to-late '60s. And there was, like, specific moments in time in both of these places. And, you know, basically all of the great rock and roll from that era that determined everything that followed basically came out of this. So, you know, do you want that or not?

   2. JRJoe Rogan30:16

      Right.

   3. MAMarc Andreessen30:16

      Right? (laughs)

   4. JRJoe Rogan30:17

      (laughs)

   5. MAMarc Andreessen30:17

      If you want it, th- you know, that's what you get. Um, I'll give you... Uh, here's a crazy, here's a crazy. Um, y- oh, the, it's the, um, uh... There's the other book about Laurel Canyon that's even crazier than Chaos. It's the book called, uh, Weird Scenes in the Canyon.

   6. JRJoe Rogan30:29

      Oh, okay.

   7. MAMarc Andreessen30:29

      Oh, oh, okay, you would love this one. So, so Laurel Canyon was like the Haight-Ashbury-

   8. JRJoe Rogan30:32

      Weird Scenes in the Canyon.

   9. MAMarc Andreessen30:32

      ... of Los Angeles, right? So Laurel, Laurel Canyon was like the music scene, the, the sort of music and drug and hippie scene of the... It's... Laurel Canyon's actually where the hippie movement started. Um, there was actually a specific group in Laurel Canyon in LA in about 1965. Um, it was a guy named Vito Pilikas, um, and, uh, and he had a group called The Freaks. And they, they were like a n- they were like a non-violent version of the Manson cult. Um, and it was all these young girls, and, and they basically would go to clubs, and they had th- they were the ones to do the beads and the hair and, like, all the, the leather, like all the, all the hippie stuff. Like, they got, they got that rolling. Um-... and so, like, they, they were, they were in Laurel Canyon. And in Laurel Canyon, it was like ground zero. There was, like, this moment where it's like Jim Morrison and The Doors, and Crosby, Stills & Nash, and Frank Zappa, and was it John Phillips? Um, and, um, it was The Mamas & The Papas, and The Byrds, and The Monkees. And, like, all of these, like, iconic bands of that time basically catalyzed over about a two-year period in Laurel Canyon. Um, the, the conspi- (laughs) the conspiracy theory in this book basically is that the whole thing was an op, and the, the, and they, the, it was a military intelligence op. Um, and the, the, the evidence for the theory is that there was an Air Force, uh, uh, military propaganda production facility at the head of Laurel Canyon called Lookout Mountain. Uh, which, uh, right, which today Jared Leto owns and actually lives i-

   10. JRJoe Rogan31:42

       Yeah.

   11. MAMarc Andreessen31:42

       ... lives in.

   12. JRJoe Rogan31:42

       S- I was just gonna say that.

   13. MAMarc Andreessen31:43

       Yeah. But it's, it was a, in, in that era, in the '50s through the '70s, it was a vertically integrated, um, military... Yes, um, it was a, a production facility, uh, for film and music. Um-

   14. JRJoe Rogan31:54

       But by the way, have you met Jared Leto?

   15. MAMarc Andreessen31:56

       Uh, I, uh, briefly, yeah.

   16. JRJoe Rogan31:58

       One of the most interesting-

   17. MAMarc Andreessen31:59

       Yeah.

   18. JRJoe Rogan31:59

       ... guys I've ever talked to.

   19. MAMarc Andreessen31:59

       Incredible. And it makes total sense 'cause it's-

   20. JRJoe Rogan32:00

       Totally normal, like, really fun to talk to.

   21. MAMarc Andreessen32:04

       Yeah, yeah.

   22. JRJoe Rogan32:04

       Not like what you would think of as a famous actor at all.

   23. MAMarc Andreessen32:07

       Yeah.

   24. JRJoe Rogan32:07

       I had dinner with him and drinks. He's a fucking great guy.

   25. MAMarc Andreessen32:10

       But he lives in a military, um-

   26. JRJoe Rogan32:12

       He showed me all the pictures.

   27. MAMarc Andreessen32:13

       (laughs)

   28. JRJoe Rogan32:13

       He showed me. I'm like, "This is wild."

   29. MAMarc Andreessen32:15

       Yeah. So, let-

   30. JRJoe Rogan32:15

       It's amazing.
8. 44:54 – 48:46

   How LLMs actually work: probabilistic autocomplete and emergent ‘world models’

   1. MAMarc Andreessen44:54

      So, uh, m- m- y- a little more detail on kind of how this, this thing works. And so, like, by default, what it's doing is basically a very sophisticated autocomplete, right, just like your iPhone does an autocomplete. It's doing a-

   2. JRJoe Rogan45:03

      (hums)

   3. MAMarc Andreessen45:03

      ... very sophisticated version of that. But it's doing it for, you know, thousands of words as opposed to just a single word, right?

   4. JRJoe Rogan45:08

      Mm-hmm.

   5. MAMarc Andreessen45:08

      And so... But that's an important concept, because that is actually what it's doing, and it's doing that through, again, this sort of giant corpus of basically all text ever, ever written. Um, another interesting part of that is it's, it's doing it, uh, it's called probabilistically. So normally, a computer, if you ask it a question, you get an answer. You ask it the same question, you get the same answer, kind of. Computers are kind of famously literal in that way. The way these work is not like that at all. You ask it d- a different q- You ask it the same question twice, it'll give you a different answer the second time. And if you keep asking, it'll get, it'll give you more and more d- d- different answers. And it, it's basically taking different paths down the probability tree of the text that it wants to present you-

   6. JRJoe Rogan45:41

      Hmm, interesting.

   7. MAMarc Andreessen45:42

      ... uh, based on the prompt. And so that, that's the basic function of what's happening. But then there is this thing that's happening where as it does this... So, so the way to think about it is it's trying to predict the next word. Um, but to try to predict the next word accurately, it has to build up a more complete, more and more complete internal understanding of how the world operates basically as it goes, right? Because you ask it more and more sophisticated questions, it wants to give you more and more sophisticated answers. And so it, it, it sort of... The early, the early indications are it's building up what's, what they call a world model, uh, inside the neural network. And so it's sort of imputing a model of how the world works, it's imputing a model of physics, it's im- it's, it's imputing a model of math. It's developing capabilities to be able to process information about the world in sophisticated ways, in order to be able to correctly predict the next, the ne- the next word. As part of that, it's, it's actually sort of evolving its own circuitry to be able to, to, to do things, correlate information. It's designed circuitry to be able to generate images, to generate videos, right, to d- to do all kinds of things. And so the, the more information you feed it and the more questions you ask it, the more sophisticated it gets about the material that it, that it, tha- tha- that it's processing. And so it starts to be able to do actually quite smart and sophisticated things to that material. Um, and there are a lot of people testing it right now to see whether it can generate new chemical compounds, whether it can generate new mathematical formula, whether it can generate new product ideas-

   8. JRJoe Rogan46:55

      Hmm.

   9. MAMarc Andreessen46:55

      ... right? New, you know, new, new fictional scenarios, you know, new screenplays-

   10. JRJoe Rogan46:58

       (clears throat)

   11. MAMarc Andreessen46:59

       ... uh, original screenplays. And so if it can do all those things-

   12. JRJoe Rogan47:02

       (clears throat)

   13. MAMarc Andreessen47:02

       ... then what it ought to be able to do is start to c- correlate information about real world e- situations, right, in interesting ways, right? And so, you know, ask it who killed Kennedy or, you know, d- are, are nuclear weapons real? Like, in theory, if it has access to like all written and visual information on that topic, and it has long enough to process it, it's gonna draw connections between things that are beyond what we're able to do. And it will present us with scenarios based on those connections. Now, w- w- will it know that those things are true? Uh, you know, mathematically, if they're true, maybe it will know that. Will it know if things are historically accurate? You know, I, i- you know, as much as any of us ever know that anything is historically accurate. But will it be able to kind of process a much larger amount of information that we can and, and sort of see the world in a more complete way? Like, that seems pretty likely.

   14. JRJoe Rogan47:47

       That seems pretty likely. What, what d- my concern would be is who is directing what information gets out? Because it seems like-

   15. MAMarc Andreessen47:56

       (laughs)

   16. JRJoe Rogan47:56

       ... anybody that's actually in control of AI-

   17. MAMarc Andreessen47:59

       Yeah.

   18. JRJoe Rogan47:59

       ... would have, uh, a massive influence-

   19. MAMarc Andreessen48:03

       Yep.

   20. JRJoe Rogan48:04

       ... on the correct answers for things, what's the, the correct policy that should be followed.

   21. MAMarc Andreessen48:08

       Yeah.

   22. JRJoe Rogan48:09

       It's b- z- 'Cause it seems like the... It's... Politicians are so flawed.

   23. MAMarc Andreessen48:13

       Uh-huh.

   24. JRJoe Rogan48:13

       If there's anyone that's vulnerable-

   25. MAMarc Andreessen48:15

       Yep.

   26. JRJoe Rogan48:15

       ... to AI, it's politicians.

   27. MAMarc Andreessen48:17

       Yep.

   28. JRJoe Rogan48:17

       Because if y- politicians are coming up with these ineffective strategies for handling all these social issues, but then you throw these social issues into an advanced form of, of ChatGPT, and it says, "Over the course of 10 years, this is the best c- case scenario for this strategy."

   29. MAMarc Andreessen48:34

       Right.

   30. JRJoe Rogan48:35

       "And this is how to follow this, and this is how it all play out."
9. 48:46 – 52:38

   The ‘Ring of Power’: narrative control, Twitter Files, and First Amendment questions

   1. MAMarc Andreessen48:46

      Yes. So I... Yeah, I call that... My metaphor for this is the Ring of Power, right, from Lord of the Rings, right?

   2. JRJoe Rogan48:50

      Oh.

   3. MAMarc Andreessen48:50

      The whole point of the Ring of Power was, like, once you have the Ring of Power, it corrupts you. You can't help but use it, right?

   4. JRJoe Rogan48:55

      Mm-hmm.

   5. MAMarc Andreessen48:56

      And so, and this is, I think, what we've seen in social media over the last decade, right, which is when people get in a p- Activists or politicians get... You know, this is the Twitter files, right?

   6. JRJoe Rogan49:03

      Sure.

   7. MAMarc Andreessen49:03

      People get in a position to be able to influence the shape of the public narrative. They will use that power, and they will use it in actually even very ham-fisted ways, right? Like, a lot of the stuff that's in the Twitter files is stuff that's just, like, really dumb.

   8. JRJoe Rogan49:14

      Yeah.

   9. MAMarc Andreessen49:15

      Right? Um, and it's just like, "Well, why would they do that?" And it's just like, "Well, because they could."... right? 'Cause they have, they have the ring of power.

   10. JRJoe Rogan49:20

       Like, what are, what's an example of something that was done?

   11. MAMarc Andreessen49:21

       So, what was it? There was this thing, I forget what it was, but there was some reporting that went through the FBI that were all these Russian, you know, basically fake accounts on Twitter, and it turned out one of them was the actor Daniel Baldwin.

   12. JRJoe Rogan49:32

       (laughs)

   13. MAMarc Andreessen49:32

       (laughs)

   14. JRJoe Rogan49:34

       What, is Daniel Baldwin, like, a hardcore right-winger or something?

   15. MAMarc Andreessen49:37

       Uh, you know, he must've been saying s- you know, it's... Again, it's one of these things-

   16. JRJoe Rogan49:39

       Yeah.

   17. MAMarc Andreessen49:39

       ... where he said something that pissed somebody off, right? You gotta put... You know, it's the, it's the whole thing. You gotta put it in a list.

   18. JRJoe Rogan49:43

       Right.

   19. MAMarc Andreessen49:44

       Right? The list gets fed through one of these bureaucracies, it comes out the other end that everybody's a Russian, you know, asset. You know, they get put on the block list. It's like, okay, you know, do they ha- you know, d- do you have First Amendment rights? Do you have First Amendment rights on social media? Can the government be involved in this? Can the government fund groups that do this, right?

   20. JRJoe Rogan49:59

       Hmm.

   21. MAMarc Andreessen49:59

       Is, is that legal? Is that allowed? 'Cause there, there's a lot of government money flowing to third-party groups that do... Oh, this is the other thing. If, if the government cannot legally do something itself, it's, it's somewhat ambiguous as to whether they can pay a company to do it for them.

   22. JRJoe Rogan50:10

       Ah.

   23. MAMarc Andreessen50:10

       Right? And so you have these various basically pressure groups, activist groups, university re- quote-unquote "research groups," um, and then basically they receive government funding, and then they do s- you know, various levels of censorship or, or other kinds of unconstitutional actions. Because in theory, right? They're, they're not government, right?

   24. JRJoe Rogan50:25

       Right.

   25. MAMarc Andreessen50:26

       The First Amendment binds the government. It doesn't bind somebody who's not part of the government. But if they're receiving government funding, does that effectively make them part of the government? Does that make it illegal to provide that government funding? By the way, these are felonies.

   26. JRJoe Rogan50:37

       (laughs)

   27. MAMarc Andreessen50:37

       It is, it is, it is, is a felony for somebody to, with government resources, to, um, with either an employee of the government or under what they call, I think it's color of law, sort of within the scope of the government to deprive an American citizen of First Amendment rights. So-

   28. JRJoe Rogan50:50

       And is it considered depriving someone of First Amendment rights by limiting their use of social media? Has that been established?

   29. MAMarc Andreessen50:56

       Good q- uh, it's, uh, uh, I mean, it has not been, uh, to my knowledge, a Supreme Court case yet. There, there have been some early fights on this. Um-

   30. JRJoe Rogan51:01

       But you feel like that-
10. 52:38 – 1:10:37

    AI governance fight: big-model companies vs. open source and regulatory capture

    1. MAMarc Andreessen52:38

       So, the state of the art right now is basically you've got Google that's got their own model. Um, you've got basically, um, OpenAI, which is a new company, but already quite large. Um, and then it has a partnership with Microsoft, and so Bing is based on it. So, so that's two! Um, and then you've got a bunch of kind of contenders for that, and these are companies with names like Anthropic and Inflection that are newer companies, but trying to compete with this. Um, and so those are... And you might call those, like, right now, the big four, um, at least in the US. Um, uh, and, you know, look, the, the, you know, the, the folks at, at, at, at all of these companies are, like, in the thick of this fight right now. Um, and, and, you know, the, the pressure somewhat corresponds to which of these is, is, is most widely used, but... So it's not equal pressure applied to all of them, but they're kind of all, all in that fight right now. And by the way, it's not like they're, like, necessarily opposed to what I'm saying. It's they, they, they, they, they may in fact just want to cooperate with this, you know, either because they agree with the, the desire for censorship or they just want to stay out of trouble. So, so, so, so there's that whole side of things. That's the company side of things. Um, and then there's an open source movement, right? And so then, then there's all these people basically building open source AIs, um, and, and those, those are coming out really fast now. There's, like, a new one every week that's coming out, um, and this is just code that you can download off the internet that does sort of a smaller version of, of what these bigger AIs do. Um, and there's open source developers that are trying to develop basically free versions of this.

    2. JRJoe Rogan53:57

       Hmm.

    3. MAMarc Andreessen53:57

       Um, and some, and some of those developers are very determined to have AI actually be f- be free and uncensored and, and, and fully available to, to, to everybody. And then there's a big fight happening in Washington DC right now where the companies working on AI are trying to get what, what economists call regulatory capture. So, they're trying to basically get the government to erect barriers, um, so that, um, new startups can't compete with them. Um, a- and also they're trying to get open source banned. Um, so there's a big push underway to try to ban open source as, as, as being too dangerous, um-

    4. JRJoe Rogan54:26

       Too dangerous, how?

    5. MAMarc Andreessen54:27

       ... too dangerous. Well, is, the, the, the, the, the case they make is you, that if you believe AI itself is inherently dangerous, then the only safe way to have it is to have it owned and controlled by a big company that's sort of fused with the government, where, in theory, everything is being done responsibly. And if you just have basically free AI that anybody can download off the internet and use whatever they want, they could do all these dangerous things with it, right? And it needs to be stopped.

    6. JRJoe Rogan54:48

       You think this is a bullshit argument?

    7. MAMarc Andreessen54:49

       Yes. Well, yes.

    8. JRJoe Rogan54:50

       Yeah.

    9. MAMarc Andreessen54:50

       I think this is a very bad, evil... (laughs) Yes, this is a very-

    10. JRJoe Rogan54:53

        Yeah.

    11. MAMarc Andreessen54:53

        ... uh, uh, like... I think this is a turning point in human civilization. You know, I think this is on par with the development of the book, right? Or the microchip or the internet, right? Um, and, you know, there were authoritarians in each of those eras that would have loved to have had total monopolistic or cartel-like, or government control over those new technologies, and they could've had a lot of control over the, over the, the, the path of civilization, you know, af- after that point. The, the ring of power, right? They could've had the ring of power. Um-

    12. JRJoe Rogan55:17

        So, what, what can be done to prevent them from stopping...... open source?

    13. MAMarc Andreessen55:21

        So, I mean, uh, it's, it's sort of, uh, there... I mean, so it starts with our elected officials. (laughs) And so-

    14. JRJoe Rogan55:26

        Mm-hmm.

    15. MAMarc Andreessen55:26

        ... it's, you know, who do we, who do we, who do we elect, who do we, you know, who do we elect, who do we reelect? Um, it then... A lot of this is the staffing of the various government agencies. You know, who, who do those officials get to appoint? Uh, a lot of this is who are the judges who are gonna hear the cases? 'Cause this is all gonna get litigated, right? And so, who, who's on this... You know, (laughs) the Supreme Court's in the news this week. This, there will be huge Supreme Court cases up on this topic over the next several years.

    16. JRJoe Rogan55:48

        Mm-hmm.

    17. MAMarc Andreessen55:48

        So, who's on the Supreme Court will matter a lot. Um, and then quite honestly, it's, you know, the question is, who's gonna be able to get away with what sort of under cover of darkness?

    18. JRJoe Rogan55:56

        (laughs)

    19. MAMarc Andreessen55:56

        Are, are people gonna care? Are they gonna speak up? Is it gonna show up in polling? Are people gonna, you know, basically show up at, like, you know, Town Hall meetings with politicians and basically say, "Do you know about this and are you gonna stop this?"

    20. JRJoe Rogan56:06

        Mm-hmm. What... If you had to steel man the argument against open source-

    21. MAMarc Andreessen56:10

        Yeah.

    22. JRJoe Rogan56:11

        ... what would it be?

    23. MAMarc Andreessen56:11

        Yeah. It would be that a, that an AI that is uncontrolled can do... It's, it's, it's general purpose intelligence. It can do whatever intelligence can do. So, if you ask it to generate hate speech, it can do that. If you ask it to generate misinformation, it can do that. If you s- if you ask it to generate a, you know, plan to rob a bank, right, or com- or to commit a terror act, it will, you know, it'll... You know, the fully uncontrolled versions will, will help you do all those things.

    24. JRJoe Rogan56:32

        Mm-hmm.

    25. MAMarc Andreessen56:33

        But, but they will also help you teach your kid, you know, um, you know, calculus. They will also help you figure out how to succeed in your job. They'll also help you figure out how to stay healthy. They'll also help you figure out the best workout program. They'll help you figure out, you know, what, you know, (laughs) how to... You know, they'll, they'll, they'll be capable of being your doctor and your lawyer, and your coach, and your advisor, and your mentor, and your teacher.

    26. JRJoe Rogan56:51

        Without censorship?

    27. MAMarc Andreessen56:52

        Yeah. Yeah, yeah.

    28. JRJoe Rogan56:53

        Yeah.

    29. MAMarc Andreessen56:53

        And able to be very honest with you. And yeah, if you ask it questions on these topics, it will answer honestly and it won't, you know, it won't be biased and it won't be influenced by what other people want it to say.

    30. JRJoe Rogan57:01

        So, it's the AI version of San Francisco.

Episode duration: 2:39:59