EVERY SPOKEN WORD
150 min read · 30,019 words- 0:00 – 15:00
(drumming music) Joe Rogan podcast.…
- NANarrator
(drumming music) Joe Rogan podcast. Check it out.
The Joe Rogan Experience.
- JRJoe Rogan
Train by day, Joe Rogan podcast by night. All day. (rock music) What's up, brother? How are you? Good to see you, my friend.
- LFLex Fridman
It's good to see you.
- JRJoe Rogan
Hey, uh, what have your people done? Your, your AI people with this fucking ChatGPT shit? This scares the fuck out of me.
- LFLex Fridman
Your people?
- JRJoe Rogan
It scares me.
- LFLex Fridman
What do you mean?
- JRJoe Rogan
Your AI people.
- LFLex Fridman
Your people? (laughs)
- JRJoe Rogan
Your, your wacky coders. What have you done?
- LFLex Fridman
Yeah, it's super interesting.
- JRJoe Rogan
Fascinating.
- LFLex Fridman
Language models, I don't know if you know what those are, but that's the general, uh, systems that, uh, underlie ChatGPT and GPT. They've been progressing over the past maybe four years aggressively. There's been a lot of development, GPT-1, GPT-2, GPT-3, uh, GPT-3.5. And ChatGPT, there's a lot of interesting technical stuff that maybe we don't wanna get into, but-
- JRJoe Rogan
Sure, let's get into it.
- LFLex Fridman
Well, there was-
- JRJoe Rogan
I'm, I'm fascinated by it.
- LFLex Fridman
So ChatGPT is based on fundamentally on 175 billion, uh, parameter neural network that is GPT-3, and the rest is what data is it trained on and how is it trained. So you already have like a brain-
- JRJoe Rogan
Mm-hmm.
- LFLex Fridman
... a giant neural network, and it's just trained in different ways. So Chat... Uh, GPT-3 came out about two years ago, and it was like impressive but dumb in a lot of ways. It was like you would expect as a human being for it to generate certain kinds of texts, and it was like saying kind of dumb things that were off. And you're like, "All right, this is really impressive, but it's not quite there." You can tell it's not intelligent. And what they did with, uh, GPT-3.5 is they started adding more and different kinds of datasets there. One of them, probably the smartest neural network currently, is Codex, which is fine-tuned for programming. Like, it was, it was, uh, trained on code, on programming code. And when you train on programming code, which Chat, ChatGPT is also, you're teaching it something like reasoning, 'cause it's no longer, uh, information and knowledge from the internet. It's also reasoning. You can like logic. Even though you're looking at code, programming code is- (laughs) You're looking at me like, "What the fuck is he talking about?"
- JRJoe Rogan
(laughs) Oh, Jesus. No, no, no, no, that's not what I'm looking at.
- LFLex Fridman
But, so-
- JRJoe Rogan
I'm looking at you like, "Oh, my God."
- LFLex Fridman
But reasoning is a... In order to be able to stitch together sentences that make sense, you not only need to know the facts that underlie those sentences, you also have to be able to reason.
- JRJoe Rogan
Yeah.
- LFLex Fridman
And, and we think of it, we take it for granted as human beings that we can do some common sense reasoning. Like s- like this war started at this date and ended at this date, therefore it means that, uh, like the start and the end has a meaning. There's a temporal consistency. There's a cause and effect. All of those things are inside programming code. By the way, a lot of stuff I'm saying we still don't understand. We're like intuiting why this works so well.
- JRJoe Rogan
Really?
- LFLex Fridman
But, uh, y- these are the intuitions. Yeah, there's a lot of stuff that are not clear. So J- Chat... Th- so GPT-3.5, which ChatGPT is likely based on, there's no paper yet, so we don't know exactly the, the details, but it was just trained on, on code and more data that's able to give it some reasoning. Then this is really important, it was fine-tuned in a supervised way by human labeling. Small dataset by human labeling of here's what we would like this network to generate. Here's the stuff that makes sense. Here's the kind of dialogue that makes sense. Here's the kind of answers to questions that make sense. It, it's basically pointing this giant titanic of a neural network into the right direction that aligns with the way human beings think and talk. So it's not just using the giant wisdom of, uh, Wikipedia and this... Uh, I can talk about what datasets it's trained on, but just basically the internet. It was pointed in the wrong direction. So this, uh, supervised labeling allows it to point in the right direction to when it says shit, you're like, "Holy shit, that's pretty smart." So that, that's the alignment. And then they did, uh, something really interesting is using reinforcement learning, uh, based on labeling data from humans. That's, that's quite a large dataset. The task is the following. You have this smart GPT-3.5 thing generate a bunch of texts and humans label which one seems the best, so ranking. Like, uh, you ask it a question. Uh, for example, you could do, uh, generate a joke in the style of Joe Rogan, right? And you have a label that has five options, and you've... (laughs) You have a label. Does it mention dick and pussy?
- JRJoe Rogan
(laughs)
- LFLex Fridman
No, no. (laughs) I don't know which, I, I don't know how exactly.
- 15:00 – 30:00
I'm so glad they're…
- LFLex Fridman
able to give you a plan on how to make a lot of money, able to give you a plan on how to manipulate other governments into, um, uh, into... And, uh, uh, any kinda geopolitical resolution that benefits you, all of that. It's able to give you all of that. And you can deploy it, and you can deploy it in a shady way, where it sneaks into, like, TikTok or something like that. Y- it sneaks into everybody's smartphone, uh, pretending to be doing good, but it's actually, whether deliberately or not, is controlling the population. So that, that's a really... The ca- that capability is there. The cool, the great thing is the people at the head of OpenAI currently, uh, Sam Altman and others, really care about this problem. They, they were there in the beginning. They were the ones, like Elon screaming about AI ethics, AI alignment. They're really concerned about superintelligent AI taking over.
- JRJoe Rogan
I'm so glad they're s- they're concerned while they're building it.
- LFLex Fridman
Well, you-
- JRJoe Rogan
(laughs)
- LFLex Fridman
... you'd rather have the people-
- NANarrator
I'm concerned about that stuff.
- LFLex Fridman
(laughs) Yeah, of course.
- JRJoe Rogan
What is going on here, Jamie?
- NANarrator
These aren't real people.
- JRJoe Rogan
What?
- NANarrator
Yeah, so these pictures are going around the inter... They're v- a lot of them look very similar to me, which is kinda weird. I'm sure Lex can explain that part of it, but...
- LFLex Fridman
I am not explaining any of this, no.
- JRJoe Rogan
(laughs)
- NANarrator
Yeah.
- LFLex Fridman
Uh, yeah, so like-
- JRJoe Rogan
So, like these are completely 3D, like-
- NANarrator
People.
- JRJoe Rogan
... CGI-made people?
- LFLex Fridman
Yeah.
- JRJoe Rogan
But...
- LFLex Fridman
Not 3- not 3D, it's 2D.
- NANarrator
Not 3D. So, like photo, very photorealistic, if not photorealistic, but like there are... When you look real close, you can see some weird things going on. Like the background here is a little messed up.
- JRJoe Rogan
Mm.
- NANarrator
This arm is not to the right person. She's sitting on an extra piece of skin here somehow.
- LFLex Fridman
I see you've analyzed this carefully.
- NANarrator
Well, me and my friends have been passing this around 'cause like it's too tricky.
- JRJoe Rogan
No, no, no, listen, you're incorrect. That arm's in the perfect perp-... It's just there's a string from that other girl's bikini on it.
- NANarrator
Uh...
- LFLex Fridman
The analysis continues.
- NANarrator
I'm just saying, so...
- 30:00 – 45:00
Uh, is there results…
- JRJoe Rogan
And, um, they, they found, uh, his Google search. It's horrific. It's like how to dismember a body, how to, how long does it take for a body to dissolve. It's like, ugh. Is it best to cut someone up or move them whole? Like what the... He just Googled the most horri- and he did it for like the entire night into the morning.
- LFLex Fridman
Uh, is there results for that in a Google search?
- JRJoe Rogan
I don't know.... what happens if you put body parts in ammonia, how to clean blood from a wooden floor, dismemberment and the best ways to dispose of a body, can identification be made on partial remains, how long does DNA last. Like, what the fuck, man? How long before a body starts to smell? Can you be charged with murder without a body? This guy is fucking s- it's so sick. So this dude just goes through Google all night long trying to figure out how to get away with murder.
- LFLex Fridman
Well, he might actually get off on just asking the question, right?
- JRJoe Rogan
No. No, because then-
- LFLex Fridman
I don't know.
- JRJoe Rogan
... they found a bloody knife.
- NANarrator
Yeah, like he went into a store and-
- JRJoe Rogan
They found... Yeah, they found-
- LFLex Fridman
No, I'm not- I'm not, like, pushing back. I'm just saying-
- NANarrator
Yeah.
- LFLex Fridman
... he might also get off on...
- JRJoe Rogan
I don't think he's getting off at all.
- NANarrator
At- I don't know.
- JRJoe Rogan
I don't think he has a chance of getting off. They're- they're-
- LFLex Fridman
I- I have a lot of questions about-
- JRJoe Rogan
They found a knife.
- LFLex Fridman
... human nature after, um... Maybe I'm, uh, naive in this, but I watched the- the- The Dahmer, um, documentary.
- JRJoe Rogan
Yeah.
- LFLex Fridman
No, not the documentary.
- JRJoe Rogan
The movie?
- LFLex Fridman
The movie.
- JRJoe Rogan
Yeah.
- LFLex Fridman
And then also the documentary. It's- it's like, it m- it gives you very different perspectives on what, like...
- JRJoe Rogan
Are you a Dahmer sympathizer now, boy?
- LFLex Fridman
No. No.
- NANarrator
(laughs)
- JRJoe Rogan
Okay.
- LFLex Fridman
No.
- NANarrator
(laughs)
- 45:00 – 1:00:00
Mm-hmm. …
- JRJoe Rogan
close to, like, what we were a million years ago. We're real close to, like, very violent, hair-covered, barbaric animals. And now, we have thermonuclear weapons. (laughs) And now, we have satellite imagery and cellphones. And we're close to some new thing. And I think if I was an alien, I would wanna watch.
- LFLex Fridman
Mm-hmm.
- JRJoe Rogan
I would wanna watch this very bizarre transition. Because, like, if you could study, look, think about all the things we go to study that are so boring. I mean, guys dedicate their whole lives to find a new fern, you know? And what, w- what are we, you know? We're the most fascinating fucking thing in the known universe, by far. If we didn't know about people and we f- if we were some logical creature from somewhere else and we found people, we- we would be like, "Holy shit." You know? "Wait til you see these fucking guys."
- LFLex Fridman
What- what- what- what-
- JRJoe Rogan
They- they vote, they have a popularity contest to see who controls the weapons. They're all, like, obviously paid off by these corporations and special interest groups. And everybody's like, "Oh, I don't get it." These politicians (laughs) , they make hundreds of millions of dollars on a job that pays $100,000 a year. And we're like, "What? What the fuck is going on?"
- LFLex Fridman
Well, if from, if they're observing us, do you think humans stand out that much-
- JRJoe Rogan
Yes.
- LFLex Fridman
... uh, from- from the rest of life on Earth?
- JRJoe Rogan
Yes.
- LFLex Fridman
'Cause it could be the same kind of life force that ... You just described some basic stuff, some basic, uh, uh, s- i- uh, dynamics of interactions between species that could be equally as fascinating as the interaction between ants, or-
- JRJoe Rogan
Well, I think those are fascinating too. I don't think anybody would think that ants aren't fascinating. Ants are fascinating to us. I'm sure ants would be fascinating to, uh, someone from another planet that doesn't know what ants are. But ants can't nuke the whole fucking planet 100 times over and be pointing weapons at each other. And like, we- we have a weird ability to change the- the surface of the Earth.
- LFLex Fridman
You don't understand.
- JRJoe Rogan
We've created these structures-
- LFLex Fridman
Yeah.
- JRJoe Rogan
... that, uh, that rise hundreds and hundreds of feet into the sky. They're all made out of glass. Like, we're wild. We're so different than any other animal. I mean, yeah, there's a lot of fascinating other animals. Lions are fascinating, zebras are fascinating. Everything's fascinating. But not like us. No. (laughs) If you came here from another planet, the first thing we'd go is like, "These crazy talking monkeys are out of control." And you would just start rattling off what they do. You'd talk about Real Housewives of Beverly Hills.
- LFLex Fridman
Mm-hmm.
- JRJoe Rogan
You would talk about rock stars. You would talk about the internet. You would talk about TikTok, about phone addictions.
- LFLex Fridman
Or-
- JRJoe Rogan
People would think it's fascinating.
- LFLex Fridman
Or just like ChatGPT and GPT-1, 2, and 3, they see that as a trivial consequence of evolution, that you just increased the po- the computational power of the brain, you're gonna start getting these kinds of interactions, because they know what happens in the next thousands of years. They- they understand-
- JRJoe Rogan
Hmm.
- LFLex Fridman
... the general trajectory is going to be, uh, we don't know that trajectory. It could be AI, uh, th- i- th- the AI, and then there's stages in the development of AI and the kind of system it creates. Maybe it'll be one collective intelligence that, uh, encompasses the whole world towards no longer individual entities. It's one intelligence that's-
- JRJoe Rogan
Hmm.
- LFLex Fridman
... trying to solve nuclear fusion and- and ach- achieve, uh, type I Kardashev scale civilization that's unable to, uh, become a multiplanetary species. They know this whole development is trivial to them. They're gonna yawn. And then, or maybe they know that this is the stage where it's inevitable that these creatures destroy themselves.
- JRJoe Rogan
Hmm.
- LFLex Fridman
Because like, the f- the- that, in order to achieve this level of intelligence, there has to be a fundamental desire for conflict. And the m- the better the weapons get-... the more the conflict will enable them to destroy themselves, if not through nuclear weapons, then through AI, through genetic engineering, through all kinds of stuff.
- JRJoe Rogan
Ma- maybe that's where aliens come in. And maybe what aliens are, is like a caretaker of this process.
- LFLex Fridman
Yeah.
- JRJoe Rogan
Which is why, you know, one of the things about UFO folklore, when they dropped Fat Man and Little Boy, when they dropped those bombs on Hiroshima and Nagasaki, like UFO sightings, there's a, like a pretty big uptick.
- LFLex Fridman
Yeah.
- 1:00:00 – 1:12:54
Yeah. Well, maybe we're…
- JRJoe Rogan
little thing where people stare at you."
- LFLex Fridman
Yeah. Well, maybe we're ... Maybe Earth is a kinda zoo, and that we're in it, and that we're being observed, and maybe all the suffering is a kind, is the kind ... There, there, th- there's probably activist aliens that are saying-
- JRJoe Rogan
(laughs)
- LFLex Fridman
"Why keep the humans, these conscious beings that are capable of so much suffering, why allow them to continue suffering?" I mean, that's the question, the religious question people ask. Why does God allow suffering?
- JRJoe Rogan
Right.
- LFLex Fridman
Why is there evil? Why is there injustice?
- JRJoe Rogan
I think all of these questions are really good questions. But we look at it through the eye of culture. We look at it through the eye of what's meaningful for us, what life means to us. But if you could look at it almost like a computation.
- LFLex Fridman
Huh.
- JRJoe Rogan
If you could step away-... it's impossible for us to do it. But if you just had to pretend, if you could step away and look at it, like, this thing is moving a certain way, like what is it doing? Well, it's making better stuff. That's all it does. All it does is make better stuff. It has a lot of s- things in there, like romance and sound and stories-
- LFLex Fridman
Mm-hmm.
- JRJoe Rogan
... and the, the hero's journey. But what is it really doing?
- LFLex Fridman
Input is energy.
- JRJoe Rogan
It's making stuff. It's making stuff.
- LFLex Fridman
Output is better stuff.
- JRJoe Rogan
It's making better stuff. It's making-
- LFLex Fridman
But it needs energy, so you need the input.
- JRJoe Rogan
And it's recently addicted to stuff. Recently addicted to electronic stuff, where you have to carry around this thing with you.
- LFLex Fridman
Yeah.
- JRJoe Rogan
So this thing has a- got this parasitic relationship with you. And you need a new one every year. You need a better one, because a better one came out.
- LFLex Fridman
Yeah.
- JRJoe Rogan
Oh, what's the better one do? Oh, the better one has a better camera.
- LFLex Fridman
Yeah.
- JRJoe Rogan
Oh, okay. So y- this keeping up with the Joneses, which seems to be a part of, like, just natural human, uh, behavior patterns, like people always want to keep up with their neighbor, right? Well, the thing that fuels this technological, uh, innovation is all materialism. Materialism fuels it, because you have to get the latest, greatest stuff. Like, I- you know, you can have a laptop from four or five years ago, you're not gonna notice.
- LFLex Fridman
Yeah.
- JRJoe Rogan
You know, you're not gonna fucking notice. You can have a phone from, like- I have- one of my iPhones is an iPhone 11. I don't notice. I make calls, take pictures, looks great. Get on the- I- I answer emails, it looks great. But it's a fucking ... I just keep it around just to see how long I'm gonna get mad at it. We- we- I don't c- notice a difference. Right?
- LFLex Fridman
It's, it's, it- it is an open question whether that's a, a permanent state of affairs at this point, this kind of capitalist, materialistic pursuits, or that's a temporary stage. That's what Karl Marx thought, that capitalism was a temporary state. Like, the ultimate place to be is, is a perfectly- is perfect communism, pure communism.
- JRJoe Rogan
Well, I don't think that works with humans, because I think part of what makes us achieve and do these things, and even make it- life better and safer for everybody is we're constantly looking to do better than the people before, because you get rewarded for doing better.
- LFLex Fridman
Yeah, competition.
- JRJoe Rogan
Yeah. It's very im-
- LFLex Fridman
But that's a weird-
Episode duration: 3:10:41
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 9Dkk9-cPpH8
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome