The Joe Rogan ExperienceJoe Rogan Experience #2076 - Tristan Harris & Aza Razkin
EVERY SPOKEN WORD
150 min read · 30,203 words- 0:00 – 5:31
From whale “lingua franca” to AI’s direction for civilization
- THTristan Harris
(drumbeats) Joe Rogan podcast, check it out.
- ARAza Raskin
The Joe Rogan Experience.
- NANarrator
Train by day, Joe Rogan podcast by night. All day. (rock music)
- JRJoe Rogan
Joe, what's going on, man? How are you guys?
- NANarrator
All right.
- ARAza Raskin
Doing okay.
- JRJoe Rogan
A little apprehensive. There's a little tension in the air. (laughs)
- NANarrator
(laughs)
- THTristan Harris
(laughs)
- ARAza Raskin
(laughs) No, I don't think so.
- JRJoe Rogan
Well, this subject is... Uh, so let's get into it. Um, what's the latest?
- NANarrator
(laughs)
- JRJoe Rogan
(laughs)
- ARAza Raskin
(laughs)
- THTristan Harris
Uh, let's see. The first time I saw you, Joe, uh, was in 2020, uh, like a month after The Social Dilemma-
- JRJoe Rogan
Yeah.
- THTristan Harris
... came out. And, um, so that was, you know, w- we think of that as kind of first contact between humanity and AI. Before I say that, I should introduce, uh, Aza, uh, is the co-founder of the Center for Human Technology. We did The Social Dilemma together.
- JRJoe Rogan
Mm-hmm.
- THTristan Harris
We're both in The Social Dilemma, um, and, uh, Aza also has a project that is using AI to translate animal communication, uh, called Earth Species Project.
- JRJoe Rogan
I was just reading something about whales yesterday.
- NANarrator
Mm-hmm.
- JRJoe Rogan
Is that re- regarding that?
- ARAza Raskin
Yeah, we, I mean, we work across a number of different species, dolphins, whales, orangutans, crows. And, uh, I think the reason why Tristan is bringing it up is because we're... Like, this conversation, uh, we're just gonna sort of dive into, like, which way is AI taking us as a species, as a civilization? Um, and it can be easy to hear just critiques as coming from critics, but we've both been builders, and I've been working on AI, uh, since, you know, really thinking about it, since 2013, but, like, building since 2017.
- JRJoe Rogan
Hmm. So this thing that I was reading about with whales, that there's some-
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
... new scientific breakthrough-
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
... where they're understanding patterns in the whale's language.
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
And what they were saying was the next step would be to have AI work on this and try to break it down, and break it down into pronouns, nouns, verbs, or whatever they're using-
- 5:31 – 7:06
Dolphin innovation studies and the idea of “narrow optimization” harming the whole
- ARAza Raskin
I'll give you, though, my, my, one other, like, my most favorite study, um, which is a 1994 University of Hawaii study, which they taught dolphins two gestures. And the first gesture was do something you've never done before.
- JRJoe Rogan
Mm.
- ARAza Raskin
Innovate. And what's crazy is that the dolphins, like, can understand that very abstract topic. They remember everything they've done before, um, and then they'll understand the concept of negation, not one of those things, and then they will invent some new thing they've never done before. So that's already cool enough, but then they'll say to two dolphins, they'll teach them the gestures, do something together. And they'll say to the two dolphins, "Do something you've never done before together."... and they go down and exchange sonic information, and they come up and they do the same new trick that they have never done before at the same time.
- THTristan Harris
They're coordinating.
- ARAza Raskin
Yeah. (laughs) Exa- exactly. I like that.
- JRJoe Rogan
Wow.
- ARAza Raskin
I like that bridge.
- JRJoe Rogan
So their language is so complex that it actually can encompass describing movements to each other.
- ARAza Raskin
That's what it, it's what it appears. Like, it doesn't, of course, prove representational language, but it certainly, for me, puts the, like, Occam's razor, like, on, on the other foot.
- JRJoe Rogan
Yes.
- ARAza Raskin
Like, it seems like there's really something there there, and that's what the project I work on, Earth Species, is about, because, you know, there's one way of diagnosing, like, like, all of the biggest problems that humanity faces, whether it's, like, uh, climate, or whether it's opioid epidemic, or loneliness. It's because there's a, a re- we're doing narrow optimization at the expense of the whole, which is another way of saying disconnection from ourselves, from each other.
- JRJoe Rogan
What do you mean by that? Narrow optimization at the expense of the whole?
- THTristan Harris
Yeah.
- JRJoe Rogan
What do you mean by that?
- 7:06 – 13:53
Engagement incentives, outrage content, and breaking “shared reality”
- THTristan Harris
Well, if you optimize for GDP, um, and, you know, more social media addiction and breakdown of shared reality is good for GDP, then we're gonna do that. If you optimize-
- JRJoe Rogan
Hmm.
- THTristan Harris
... for engagement and attention, giving people personalized outrage content is really good for that narrow goal, the narrow objective of getting maximum attention, causing the breakdown of shared reality. So in general, w- when we maximize for some narrow goal that doesn't encompass the actual whole, like, social media is affecting the whole of human consciousness, but it's not optimizing for the health of this comprehensive whole of our psychological well-being, our relationships-
- JRJoe Rogan
Hmm.
- THTristan Harris
... human connection, presence, not distraction, um, our shared reality. So if you're affecting the whole, but you're optimizing for some narrow thing, that breaks that whole. So you're manage- think of it like a irresponsible management, like you're kind of operat- operating in an adolescent way, because you're just caring about some small narrow thing, while you're actually affecting the whole thing. And I think a lot of what, you know, w- motivates our work is when humanity gets itself into trouble with technology, where you... It's not about what the technology does. It's about what the technology is being optimized for. We often talk about, um, Charlie Munger, who just passed away, uh, Warren Buffett's business partner, who said, "If you show me the incentive, I'll show you the outcome." Meaning-
- JRJoe Rogan
Hmm.
- THTristan Harris
... we, we... To go back to our first conversation with social media. Uh, in 2013, uh, when I first started working on this, it was obvious to me and obvious to both of us, we were working informally together back then, that if you were optimizing for attention, and there's only so much, you were gonna get a race to the bottom of the brain stem for attention, because there's only so much. I'm gonna, I'm gonna have to go lower in the brain stem, lower into dopamine, lower into social validation, lower into sexualization, all that other worser angels of human nature type stuff, to win at the game of getting attention. And that would produce a more addicted, distracted, narcissistic, blah, blah, blah, everybody knows society.
- JRJoe Rogan
Mm-hmm.
- THTristan Harris
The point of it is that w- people back then said, "Well, which way social media is gonna go?" It's like, "Well, there's all these amazing benefits. We're gonna give people the ability to speak to each other, have a public platform, help small, medium-sized businesses. We're gonna help people join like-minded communities. You know, cancer patients who find other rare cancer patients on Facebook groups." And that's all true, but what was the underlying incentive of social media? Like, what was the narrow goal that was actually optimized for? And it wasn't helping cancer patients find other cancer patients. That's not what Mark Zuckerberg wakes up every day and the whole team at Facebook wakes up every day to do. And it happens, but the goal is the incentive. The incentive is the profit motive, was attention, and that produced the outcome, the more addicted, distracted, polarized society. And the reason we're saying all this is that we really care about which way AI goes, and there's a lot of confusion about are we gonna get the promise or are we gonna get the pe- peril? Are we gonna get the climate change solutions and the personal tutors for everybody and, you know, um, uh, solve cancer? Uh, or, or are we gonna get, like, these catastrophic, you know, biological weapons and doomsday type stuff, right? And the reason that we're here and we wanted to do is to clarify the way that we think we can tell humanity which way we're going, which is that the incentive guiding this race to release AI is not... So, the... What is the incentive? And it's basically OpenAI, Anthropic, Google, Facebook, Microsoft, they're all racing to deploy their big AI system, to scale their AI system, and to deploy it to as many people as possible and, and keep outmaneuvering and out showing up the other guy. So, like, I'm gonna release Gemini. Google, just a couple of days ago, released Gemini. It's this super big new model, and they're trying to prove it's a better model than OpenAI's GPT-4, um, which is the one that's on, you know, ChatGPT right now. And so they're competing for market dominance by scaling up their model and saying it can do more things. It can translate more languages. It can, you know, um, know how to help you with more tasks, and then they're all competing to kind of do that. So feel free to jump in.
- JRJoe Rogan
Hmm.
- ARAza Raskin
(inhales) (smacks lips) Yeah, I mean, what...
- THTristan Harris
I mean, the question is what's at stake here, right?
- ARAza Raskin
I think the... Yeah. Exactly. The other interesting thing to ask is, you know, Social Dilemma comes out. It's seen by 150 million people. Um, but have we gotten a big shift to the social media companies? And the answer is no, we haven't gotten a big shift. And the, the question then is like, why? And it's that it's hard to shift them now, because social media became entangled in our society. It sort of, it, it took politics hostage. If you're winning elections as a politician using social media, you're probably not going to, like, shut it down or change it in some way. If you, um, if (laughs) all of your friends are on it, like, it sort of controls the, the means of social participation. Like, I, as a k- kid, can't get off of TikTok if everyone else is on it, because then I, I don't have any belonging. It sort of took our GDP hostage. Um, and so that mean- it was entangled, making it hard to shift. So we have this very, very, very narrow window with AI to shift the incentives beco- before it becomes entangled with all of society.
- JRJoe Rogan
So, the real issue, and this is one of the things that we talked about-
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
... (clears throat) last time, was algorithms. That without these algorithms that are suggesting things that encourage engagement-
- ARAza Raskin
Yeah.
- JRJoe Rogan
... whether it's outrage or, you know, I think I told you about my friend Ari ran, uh, a test with YouTube-
- ARAza Raskin
Mm. Mm-hmm.
- JRJoe Rogan
... where he only searched puppies.
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
Puppy videos. And then all YouTube would show him is puppy videos.
- ARAza Raskin
Right.
- JRJoe Rogan
And his take on it was like, "No, people wanna be outraged." And that's why the algorithm works in that direction. It's not that the algorithm is evil, it's just people have a natural inclination towards focusing on things that either piss them off or scare them or, or-
- THTristan Harris
Well, I think the, the key thing is in the language we use that you just said there.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
So if we say the word people want the outrage, I, that's where I would question, I'd say. Is it that people want the outrage or the things that scare them, or is it that that's what works on them? The outrage works on them.
- JRJoe Rogan
Mm-hmm.
- ARAza Raskin
Yeah, exactly.
- THTristan Harris
The fear works on them.
- 13:53 – 16:30
Perception gaps, shareholder pressure, and why platforms can’t self-correct
- THTristan Harris
Well, it's not about the individual having a problematic algorithm. It's that YouTube and, isn't optimizing for a shared reality of humanity, right? So, and, and Twitter is more of this-
- JRJoe Rogan
How would they do that?
- THTristan Harris
Um, well, actually, so there, one, one area.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
There's the work of a group called More in Common. Uh, Dan Vallone, um, it's a nonprofit. They, they came up with a metric called Perception Gaps. Perception gaps are, um, how well can someone who's a Republican, uh, estimate the n- the beliefs of someone who's a Democrat? And vice versa, how well can a Democrat estimate the beliefs of a Republican? And then I expose you to a lot of content, like, and there's k- some kind of content where over time, if after like a month of seeing a bunch of content, your ability to estimate what someone else believes goes down, the gap goes bigger. You are not estimating what they actually believe accurately.
- JRJoe Rogan
Mm.
- THTristan Harris
Um, and there's other kinds of content that maybe is better at synthesizing multiple perspectives, right? That's like really trying to say, okay, I think, I think the thing that they're saying is this, and I think the thing that they're saying is that. And content that does that minimizes perception gaps. So for example, what would today look like if we had changed the incentive of social media and YouTube from optimizing for engagement to optimizing to minimize perception gaps? And I'm not saying like that's the perfect answer that would have ficked all, fixed all of it. But you can imagine in, say, politics, whenever I recommend political videos, if it was optimizing just for minimizing perception gaps, what different world w- would be, be living in today? And this is why we go back to Charlie Munger's quote, "If you show me the incentive, I'll show you the outcome." If the incentive was engagement, you get this sort of broken society where no one knows what's true and everyone lives in a different universe of facts. Um, that was all predicted by that incentive of personalizing what's good for their attention. Um, and the point that we're trying to really make for the whole world is that we have to bend the incentives of AI and of social media, um, to be aligned with what would actually be safe and secure and, and for the future that we actually want.
- JRJoe Rogan
Now, if you run a social media company and it's a public company, you have an obligation to your shareholders.
- THTristan Harris
Yeah.
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
And is that part of the problem?
- THTristan Harris
Of course.
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
That, yeah, so you would essentially be hamstringing these organizations in terms of their ability to monetize?
- ARAza Raskin
Mm-hmm.
- THTristan Harris
That's right.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
Yeah, you, and, and this can't be done without that. So to be clear, you know, could, could Facebook unilaterally choose to say, "We're not gonna optimize Instagram for the maximum scrolling." When TikTok just jumped in and they're optimizing for the total maximizing infinite scroll, which by the way-
- JRJoe Rogan
Yes.
- ARAza Raskin
... we might wanna talk about. (laughs)
- THTristan Harris
Oh, yeah.
- JRJoe Rogan
Um, because one of Aza's accolades is-
- 16:30 – 21:10
Infinite scroll: unintended consequences and Aza’s “three laws of technology”
- ARAza Raskin
A- accolades is too strong. I, I'm, I'm the hapless human being that invented infinite scroll.
- JRJoe Rogan
(clicks tongue) How dare you?
- THTristan Harris
(laughs)
- ARAza Raskin
Yeah. Yeah. (laughs)
- THTristan Harris
(laughs)
- ARAza Raskin
Um-
- JRJoe Rogan
But it should be, you should be clear about which part you invented 'cause Aza did not invent-
- ARAza Raskin
Yeah.
- JRJoe Rogan
... infinite scroll for social media.
- ARAza Raskin
Correct. So this was back in 2006. This was, I, do you remember when like Google Maps first came out and suddenly you could like scroll and it was MapQuest before you had to like click a whole bunch to move the map around?
- JRJoe Rogan
Mm-hmm.
- ARAza Raskin
So that new technology had come out that you could reload, you could get new content in, um, without having to reload the whole page. And I was sitting there thinking about blog posts and I was th- thinking about search, and I was like, well, every time I as a designer ask you the user to make a choice you don't care about or click something you don't need to, I've failed. So obviously if I get near the bottom of the page, I should just load some more search results, um, or load the next blog post. And I'm like, this is just a better interface.
- JRJoe Rogan
Mm-hmm.
- ARAza Raskin
Um, and I was blind to the incentives, and this was before social media really had started going. Um, I was blind to how it was gonna get picked up and used not for people but against people. And this was actually a huge lesson for me, that me sitting here optimizing an interface for one individual is sort of like that's, that's, that was morally good. But being blind to how it was gonna be used globally was sort of globally amoral at best, or may- maybe even a little immoral. And that taught me this important lesson that focusing on the individual or focusing just on one company, like, that blinds you to thinking about how an entire ecosystem will work. I was blind to the fact that like after Instagram started they were gonna be in a knife fight for attention with Facebook, with eventually TikTok, and that was gonna push everything one direction programmatically.
- THTristan Harris
Mm.
- JRJoe Rogan
(clicks tongue) Well, how could you have seen that coming though?
- ARAza Raskin
Yeah.
- JRJoe Rogan
Yeah.
- ARAza Raskin
Well, but so b- if, if I would argue that like, you know, t-
- THTristan Harris
... the way that all democratic societies looked at problems with saying, "What are the ways that the incentives that are currently there might create this problem that we don't want to exist?"
- ARAza Raskin
Yeah. I, there, we've come up with, after, after many years, sort of three laws of technology, and I wish I had known those laws when I started my career, because if I did, I might have done something different, because I was really out there being like, "Hey, Google, hey, Twitter, use this technology, infinite scroll. I think it's better." Um-
- THTristan Harris
He actually gave talks at companies, like he went around Silicon Valley, gave talks at Google, said-
- ARAza Raskin
Yeah.
- THTristan Harris
... "Hey, Google, your search result page, you have to click, like, the page two. What if you just have it-
- ARAza Raskin
Yeah.
- THTristan Harris
... just infinitely scroll and you get more search results?" So you were really advocating for this.
- ARAza Raskin
I was. And so, these are the rules I wish I knew, and that is the first law of technology. Um, if, uh, when you invent a new technology, you uncover a new class of responsibility. And it's not always obvious, right? Like, we didn't need the right to be forgotten until the internet could remember us forever. Or we didn't need the right to, to privacy to be, like, written into our law, into our Constitution until the very first mass-produced cameras where somebody could start, like, taking pictures of you and publishing them and invading your privacy. So Brandeis, one of America's greatest legal minds, had to invent the idea of privacy and add it into our Constitution. Um, so first law, when you invent a new technology, you uncover a new class of responsibility. Second law, if the technology confers power, you're going to start a race. And then the third law, if you do not coordinate, that race will end in tragedy.
- THTristan Harris
And so with social media, the power that was invented, infinite scroll was a new kind of power. That was a new k- kind of technology.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
And that came with a new kind of responsibility, which is I'm basically hacking someone's, um, dopamine system and their lack of stopping cues, that their mind doesn't wake up and say, "Do I still want to do this?" because you keep putting, you keep sort of, uh, putting your elbow in the door and saying, "Hey, there's one more thing for you. There's one more thing for you."
- 21:10 – 31:44
Social media as humanity’s “first contact” with AI—and why unplugging doesn’t work
- THTristan Harris
of the brainstem, and the brain, the bottom of the brainstem and the collective tragedy we are now living inside of, which we could have fixed if we said, "What if we changed the rules so people are not optimizing for engagement, but they're optimizing for something else?" And so we, we think of social media as first contact-
- ARAza Raskin
Mm-hmm.
- THTristan Harris
... between humanity and AI, because social media is kind of a baby AI, right? It's a, it was the biggest supercomputer deployed probably en masse to, to touch human beings for eight hours a day or whatever, pointed at your kid's brain, right? It's a, it's a supercomputer AI pointed at your brain. What does the supercomputer, what does the AI do? It's just calculating one thing, which is, can I make a prediction about which of the next tweets I could show you or videos I could show you would be most likely to keep you in that infinite scroll loop? And it's so good at that, that it's checkmate against your self-control, like prediction of, like, I think I have something else to do, that it keeps people in there for quite a long time.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
And, um, in that first contact with humanity, we say, like, "How did this go?" Like, between, you know, we always say like, "Oh, what's going to happen when humanity develops AI?" It's like, well, we saw a version of what happened, which is that humanity lost because we got a more doomscrolling, shortened attention span, social validation. We b- we birthed a whole new career field called social media influencer, which has now colonized, like, half of, you know, Western countries. It's the number one aspired-to career in, in, uh, uh, US and UK.
- JRJoe Rogan
Is it really?
- THTristan Harris
Yeah, yeah.
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
Social media influencer-
- THTristan Harris
Yeah.
- JRJoe Rogan
... is the number one aspired career?
- THTristan Harris
It was in a big survey a year and a half ago or something like that.
- ARAza Raskin
Yeah.
- THTristan Harris
This is, this came out when I was doing the stuff around TikTok about how in, in China, the number one most aspired-to career is astronaut, followed by teacher. I think the third one is there's maybe social media influencer, but in the US, the first one is social media influencer. So-
- JRJoe Rogan
Wow. (laughs)
- ARAza Raskin
Yeah, you can actually just see, like, the goal of social media is attention, and so that value becomes our kids' values, which is attention.
- THTristan Harris
Right, it actually infects kids, right?
- ARAza Raskin
Yeah.
- THTristan Harris
It's like it colonizes their brain and their identity and says that I am only a worthwhile human being, the meaning of self-worth is getting attention from other people. That's so deep, right?
- JRJoe Rogan
(exhales) Yeah.
- THTristan Harris
It's, it's not just some light thing, oh, it's, like, subtly, like, tilting the, the playing field of humanity. It's like it's colonizing the, the values that people then autonomously run around with.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
And so we already have a runaway AI, because people always talk about like, what happens if the AI goes rogue and it does some bad things we don't like?
- ARAza Raskin
You just unplug it, right?
- THTristan Harris
We just unplug it. Like, it's not a big deal. We'll know it's bad. We'll just, like, hit the switch. We'll turn it off.
- JRJoe Rogan
Yeah, I don't like that argument.
- THTristan Harris
Yeah.
- JRJoe Rogan
That is such a nonsense-
- THTristan Harris
Well, notice we, why didn't we turn off, you know, the engagement algorithms in, in Facebook and in Twitter and Instagram after we saw it was screwing up teenage girls?
- JRJoe Rogan
Yeah, but we already talked about the financial incentives.
- 31:44 – 37:06
The 2017 transformer shift: scaling produces emergent capabilities we can’t enumerate
- THTristan Harris
It can find cybersecurity vulnerabilities in code. GBT-2 did not know how to take a piece of code and say, "Let me... What's a, what's a vulnerability in this code that I could exploit?" GBT-2 couldn't do that. But if you just pump it up with more data and more compute, and you get to GPT-4, suddenly it knows how to do that. So, think of this... there's this weird new AI. We should n-... say more explicitly that-
- ARAza Raskin
Mm-hmm.
- THTristan Harris
... um, there's something that changed in the field of AI in 2017 that everyone needs to know because I was not freaked out about AI at all, at all, um, until this big change in 2017 rolled around.
- ARAza Raskin
Mm-hmm. It, it's really important to know this because, uh, we've heard about AI for the longest time, and you're like, "Yep, Google Maps still mispronounces, like, the street name, and, like, Siri just doesn't work." Um, and this thing happened in 2017. It's actually the exact same thing that said, "All right, now it's time to start translating animal language," and it's where underneath the hood, the engine got swapped out, and it was a thing called transformers. Um, and the interesting thing about this new model called Transformers is the more data you pump into it and the more, like, computers you let it run on, the more superpowers it gets. But you haven't done anything differently. You just give more data and run it on more computers.
- THTristan Harris
Like, it's running... it's reading more of the internet, and it's just r-... throwing more computers at the stuff that it's read on the internet.
- ARAza Raskin
Yeah.
- THTristan Harris
And, and out pops out... suddenly it knows how to explain jokes.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
You're like, "Wait, where did that come from?"
- ARAza Raskin
Yeah. Or-
- THTristan Harris
Hmm.
- ARAza Raskin
... now it knows how to play chess. And all it's done is predict... All you've asked it to do is, "Let me predict the next character or the next word."
- THTristan Harris
Gi- give the Amazon example.
- ARAza Raskin
Oh yeah. Thi- this is re- interesting. So this is 2017. Um, OpenAI releases a paper where they trea-... uh, where they train this AI, it's one of these transformers, a GPT, to predict the next character of an Amazon review. Pretty simple. But then they're looking inside the brain of this AI, and they des- discover that there's one neuron that does best-in-the-world sentiment analysis, like understanding h-... whether the human is feeling, like, good or bad about the product. You're like, "That's so strange. You asked it just to predict the next character. Why is it learning about how a human being is feeling?" And it's strange until you realize, "Oh, I see why. It's because to predict the next character really well, I have to understand how the human being is feeling to know whether, like, the word is gonna be, like, a positive word or a negative word."
- JRJoe Rogan
And this wasn't programmed? This was just-
- ARAza Raskin
No.
- THTristan Harris
No.
- ARAza Raskin
No. It was-
- JRJoe Rogan
That's the key thing.
- ARAza Raskin
... emergent behavior.
- THTristan Harris
Yeah.
- JRJoe Rogan
Ugh.
- ARAza Raskin
Um, and it's really interesting that, like, um, G- GPT-3 had been out, um, t- for I think a, a couple years-
- THTristan Harris
Couple years.
- ARAza Raskin
... until a researcher thought to ask, "Oh, I wonder if it knows chemistry." And it turned out it can do research-grade chemistry at the level, and sometimes better, than models that were explicitly trained to do chemistry.
- THTristan Harris
Like, there, there were these other AI systems that were trained explicitly on chemistry, and it turned out GPT-3, which is just pumped with more r-... y- you know, reading more and more of the internet and just, like, thrown with more computers and GPUs at it, suddenly it knows how to do research-grade chemistry. So you could say, "How do I make VX nerve gas?" And suddenly that capability is in there. And what's scary about it is that we didn't know that it had that capability until years after it had already been deployed to everyone.
- ARAza Raskin
And in fact, there is no way to know what abilities it has. Another example is, um, y- do you know theory of mind, like, the abil-... my ability to sit here and sort of, like, model what you're thinking? Sort of like the basis for being able to do strategic thinking? Um...
- THTristan Harris
It's like when you're nodding your head right now, we're, like, testing, like-
- JRJoe Rogan
Mm-hmm.
- THTristan Harris
... are you... uh, how well are we explaining?
- 37:06 – 40:28
AGI confusion, OpenAI’s board drama, and the need for protocols & evaluation
- JRJoe Rogan
It does make sense. Now, what is the leap between these emergent behaviors or these emergent abilities-
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
... that AI has, and artificial general intelligence?
- ARAza Raskin
Mm-hmm. Mm-hmm.
- JRJoe Rogan
And when, when is it... When do we know? Or w- do we know? Like, this is the, the speculation all over the internet when, um, uh, Sam Altman was removed-
- ARAza Raskin
Yeah.
- JRJoe Rogan
... as the CEO and then brought back, was that they had not been forthcoming about the actual capabilities of whether it's ChatGPT-5-
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
... or artificial general intelligence, that some large leap had occurred.
- THTristan Harris
That's some of the reporting about it. Um, obviously, the, the board had a different statement, which was about Sam. The quote was, I think, "Not consistently being candid with the board."
- ARAza Raskin
Which is a funny way of saying lying.
- THTristan Harris
Yeah. Um, so basically, the board was accusing Sam of, of lying. There was this story-
- JRJoe Rogan
Specifically about?
- ARAza Raskin
What's that?
- JRJoe Rogan
Uh, specifically about-
- THTristan Harris
They didn't say in the-
- ARAza Raskin
No.
- THTristan Harris
I mean, I think that one of the failures of the board is they didn't communicate nearly enough for us to know-
- JRJoe Rogan
Well, that's why it's so-
- THTristan Harris
... what was going on. Which is why I think a lot of people then think, "Well, was there this big crazy jump in capabilities?"
- JRJoe Rogan
Yes.
- THTristan Harris
And that's the thing. And Q*... And Q* went viral. Ironically, it goes viral because the algorithms of social media pick up that Q*, which has this mystic to it, sort of must be really powerful and this breakthrough. And then that's kind of a, a theory on its own, so it kind of blows up. But we don't currently have any evidence, and we know a lot of people, you know, who are around the companies in the Bay Area. I can't say for certain, but my sense is that it, the board acted based on what they communicated, and that there was not a major breakthrough that led to or had anything to do with this happening. But to your question though, you're asking about what is AGI, artificial general intelligence, and what's spooky about that.
- JRJoe Rogan
Mm-hmm. Mm-hmm.
- ARAza Raskin
Yeah.
- THTristan Harris
Um, because, um... So just to sort of define it, uh-
- ARAza Raskin
W- I'll just say before, before you get there, I... Th- as, as we start talking about AGI, 'cause that's what of course OpenAI has, like, said that they're trying to build.
- THTristan Harris
Their mission statement.
- ARAza Raskin
Yeah, their mission statement, and they're like, "But we have to build an aligned AGI." Meaning that it, like, does, like, what human beings say it should do, and also, like, take care not to, like, do catastrophic things. Um, you can't have a deceptively aligned operator building an aligned AGI, and so I think it's really critical, um, 'cause we don't know what happened with Sam and the board, that the independent investigation that they, that they say they're, they're going to be doing, like, that they do that, that they make the report public, that it's actually independent, because, like, either we need to have Sam's name cleared, or there need to be consequences.
- THTristan Harris
We need to know just what, what's going on.
- ARAza Raskin
Yeah.
- 40:28 – 47:27
Deception, jailbreaks, and AI as an interactive tutor for harmful acts
- THTristan Harris
"Does it know how to make a chemical weapon? Does it know how to make a biological weapon? Does it know how to persuade people? Can it exfiltrate its own code? Can it make money on its own? Could it copy its code to another server and pay Amazon crypto money and keep self-replicating? Can it become an AGI virus that starts spreading over the internet?" So there's a bunch of things that people who work on risk, AI risk issues are concerned about, and ARC Evals, um, was paid by OpenAI to test the model. The famous example is that GPT-4 actually could deceive humans.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
Um, the famous example was it, it asked a TaskRabbit, uh, to do something, uh, to, specifically to fill in the CAPTCHAs. CAPTCHA's that thing where it's like, "Are you a real human? You know, f- drag this block over here to here," or, "Which of these photos is a, a truck or not a truck?" You know th- those CAPTCHAs, right? Um, and... You want to finish this example? I'm not doing a great job of it. (laughs)
- ARAza Raskin
Hmm. Well, and so the, uh, AI asked the TaskRabbit to solve the CAPTCHA, and the TaskRabbit's like, "Oh, that's sort of suspicious. Are, are you a robot?" And you can see what the AI is thinking to itself.
- THTristan Harris
'Cause it-
- ARAza Raskin
And the AI says, um, "I shouldn't reveal that I'm a robot, therefore I should come up with an excuse," and so it says back to the TaskRabbit, "Oh, I'm vision-impaired, so could I-"
- THTristan Harris
"Could you fill out this CAPTCHA for me?"
- ARAza Raskin
"... could you fill out..."
- THTristan Harris
It, the AI came up with that on its own. And the way they know this is that they, they... what he's saying about, like, what was it thinking? It... What ARC Evals did is they sort of piped the output of the AI model to say, "Whatever your next line of thought is, like, dump it to this text file, so we just know what you're thinking." And it says to itself, "I shouldn't let it know that I'm an AI or I'm a robot, so let me make up this excuse," and then it comes up with that excuse.
- JRJoe Rogan
My wife told me that (clears throat) Siri, you know, like, when you have a... use Apple CarPlay?
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
That someone sent her an image-
- ARAza Raskin
Mm-hmm.
- JRJoe Rogan
... and Siri described the image.
- THTristan Harris
Hm. Mm-hmm.
- ARAza Raskin
Yeah.
- JRJoe Rogan
Is that a new thing?
- THTristan Harris
That would be a new thing.
- ARAza Raskin
Mm-hmm. Yeah.
- JRJoe Rogan
Um, have you heard of that? Is that real?
- NANarrator
There's definitely s- I...
- JRJoe Rogan
I was gonna look into it. I wa- I, but I, well, I was in the car. I was like, "What?"
- THTristan Harris
I've noticed-
- ARAza Raskin
That's, that's the new generative AI capability, so yeah.
- JRJoe Rogan
I had something that definitely describes images that's on your phone, for sure.
- THTristan Harris
Yeah.
- JRJoe Rogan
Read- within the last year. I haven't tested Siri describing, but I'm sure it would. So imagine if Siri, uh, described-
- THTristan Harris
(laughs)
- JRJoe Rogan
... my friend Stavos' calendar.
- THTristan Harris
(laughs)
- 47:27 – 1:01:00
Biosecurity and proliferation: DNA printers, open-weight models, and ‘insecurable’ release
- THTristan Harris
... between any question you have, any problem you have, and then finding that answer as efficiently as possible. That's different than a Google search, having an interactive tutor. And then now when you start to think about really dangerous groups that have existed over time, I'm thinking of the Aum Shinrikyo, uh, cult in 1995. Um-
- JRJoe Rogan
Right.
- ARAza Raskin
D- Do you know this-
- JRJoe Rogan
No.
- ARAza Raskin
... true story? So 1995 ... Well, th- so this doomsday cult started in, uh, the '80s. Um, because th- the reason why you're going here is, people then say, like, "Okay, so AI does, like, dangerous things and it might be able to help you make a biological weapon, but, like, who's actually gonna do that? Like, who would actually release something that would, like, kill all humans?" And that's why we're sort of, like, talking about this doomsday cult, um, because most people, I think, don't know about it, but you've probably heard of the 1995, um-
- JRJoe Rogan
Tokyo subway.
- ARAza Raskin
... Tokyo subway attacks-
- JRJoe Rogan
Yes.
- ARAza Raskin
... with sarin gas. This was the doomsday cult behind it.
- JRJoe Rogan
Oh.
- ARAza Raskin
Um, and what most people don't know is that, like, one, their goal was to kill every human. Um, two, they weren't small. They had tens of thousands of people, many of whom were, like, experts and scientists, programmers, engineers. Um, they had, like, not a small amount of budget, but a big amount. They actually somehow had accumulated hundreds of millions of dollars. And the most important thing to know is that they had two microbiologists on staff that were working full time to develop biological weapons. The intent was to kill as many people as possible.... and they didn’t have access to AI, um, and they didn’t have access to DNA printers. But now DNA printers are, like, much more available, um, and if we have something, you don’t even really need AGI. You just need, like, any of these sort of, like, GPT-4 or GPT-5 level tech, um, that can now collapse the distance between we want to create a super virus, like smallpox, but like, 10 times more viral and like 100 times more deadly, to here are the step-by-step instructions for how to do that. You try something, it doesn’t work, and you have a tutor that guides you through to the very end.
- JRJoe Rogan
What is a DNA printer?
- ARAza Raskin
Hmm. Uh, it’s the ability to take, like, a set of DNA code, just like, you know, GTC whatever, um, and then turn that into an actual physical strand of DNA. And these things now run on, you know, like they’re bench top. They run on your, uh, you can get them, you know, these things.
- JRJoe Rogan
Whoa.
- THTristan Harris
Yeah, this is really dangerous. We don’t want, this is not something you want to be empowering people to do en masse, and I think, you know, w- the word democratize is used with technology a lot. We’re in Silicon Valley, a lot of people talk about we need to democratize technology, but we also need to be extremely conscious when that technology is dual use or omni use, and has dangerous characteristics.
- JRJoe Rogan
But they're, just looking at that thing, it looks to me like an old Atari console.
- ARAza Raskin
Mm-hmm.
- THTristan Harris
Mm-hmm. Yeah.
- JRJoe Rogan
You know, in terms of like what could this be?
- THTristan Harris
Mm-hmm.
- JRJoe Rogan
Like when you think about the graphics of Pong-
- THTristan Harris
Yeah.
- ARAza Raskin
Yeah.
- JRJoe Rogan
... versus what you’re getting now with like, you know, these modern video games with the Unreal Engine 5 that are just fucking insane.
- THTristan Harris
Yeah.
- ARAza Raskin
Yeah.
- JRJoe Rogan
Like, if you can print DNA-
- THTristan Harris
Mm-hmm.
- JRJoe Rogan
... how many different incarnations do we have to, I mean, how much evolution in that technology has to take place until you can make an actual living thing?
- ARAza Raskin
Yeah. That’s sort of the point is like you can make viruses.
- 1:01:00 – 1:19:40
Civilizational overwhelm: deepfakes, AI-generated content floods, and governance capacity collapse
- THTristan Harris
I'm so glad you're asking this, and that's, that is the whole essence of what we care about here, right? Uh, I actually want to say something because we can often, um, you could hear this as like, "Oh, they're just kind of fearmongering, and they're just focusing on these horrible things." And actually, the point is we don't want that. We're here because we want to get to a good future. But if we don't understand where the current race takes us, because we're like, "Well, everything's going to be fine. We'll, we're just gonna get the cancer drugs and the climate solutions, and everything's going to be great," if that's what everybody believes, we're never going to bend the, the incentives to something else.
- JRJoe Rogan
Right.
- THTristan Harris
And so the whole premise... And, and honestly, Joe, I want to say, like, when we look at the work that we're doing, and we, you know, we've talked to policymakers, we talked to White House, we talked to national security folks, I don't know a better way to bend the incentives than to create a shared understanding about what the risks are. And that's why we wanted to come to you and to, to have a conversation, is to help establish a shared framework for what the risks are if we let this race go unmitigated, where if it's just a ra- race to release these capabilities that you pump up this model, you s- you release it, you don't even know what things it can do, and then it's out there, and in some cases, if it's open source, you can't ever pull it back, and i- it's like suddenly these new magic powers exist in society that we, that society isn't prepared to deal with. Like, a simple example, and we'll get, we'll get to your question, 'cause it's, it's where we're going to, is, you know, about a year ago, the generative AI, just like it can generate images and generate music, it can also generate voices. And, um, this has happened to your voice, you've been deepfaked, but it only takes now three seconds of someone's voice to speak in their voice. Um, and it's not like banks-
- JRJoe Rogan
Three seconds?
- ARAza Raskin
Three seconds.
- THTristan Harris
Three seconds.
- JRJoe Rogan
Mm-hmm. So literally, the opening couple seconds of this podcast-
- THTristan Harris
Mm-hmm.
- JRJoe Rogan
... you guys both talking were good.
- THTristan Harris
Yep.
- ARAza Raskin
Yeah.
- THTristan Harris
Yeah, exactly.
- JRJoe Rogan
But what about yelling? What about different inflections, humor, sarcasm?
- THTristan Harris
I, I don't know the exact details, but for the basics, it's three seconds. And obviously, as A gets be- AI gets better, this isn't the worst it's ever going to be, right? And, and smarter and smarter AIs can extrapolate from less and less information.
- JRJoe Rogan
Right.
- THTristan Harris
That's the trend that we're on, right? As you keep scaling, you need less and less data to get m- better and better accurate prediction.
- JRJoe Rogan
Right.
- THTristan Harris
And the point I was trying to make is, you know, it's where banks and grandmothers sitting there with their, you know, Social Security numbers, are they re- prepared to live in this world where they, you know, your grandma answers the phone, and it's their grandson or granddaughter who says, um, "Hey, I forgot, you know, my Social Security number," or, "If I, you know, Grandma, what's your Social Security number? I need it to fill in a such and such."
- JRJoe Rogan
Right.
- THTristan Harris
Like, we're not prepared for that.
- ARAza Raskin
The, the, the general way, to answer your question of like where is this going, um, and just to reaffirm, like I, I use AI to try to translate animal language. Like, I see, like, the incredible things that we can get, but where this is going if we don't change course is sort of civilizational overwhelm. Um, we have a friend, Ajeya Cotra, um, at Open Phil, and she describes it this way. She says, "It's as if 24th century technology is crashing down on 21st century civilization, 21st century governments."
- JRJoe Rogan
Hm.
- ARAza Raskin
Right? Because it's just happening so fast.
- JRJoe Rogan
Hm.
- ARAza Raskin
Obviously, it's actually 21st century technology, but it's like-
- JRJoe Rogan
Right.
- ARAza Raskin
... the equivalent of, and so i-
- THTristan Harris
It's like Star Trek-level tech is crashing down on your 21st century democracy.
- ARAza Raskin
Yeah. So imagine it was 21st century technology crashing down on the 16th century. So like, the king is sitting around with his advisors, and they're like, "All right, well, what do we do about the telegram and radio and television and, like, smartphones and the internet all at once?"
- THTristan Harris
They just land in their sighting.
Episode duration: 2:31:41
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode cyuoux4DpKs
