Max Tegmark: The Case for Halting AI Development | Lex Fridman Podcast #371

1. 0:00 – 1:56

   Introduction

   1. MTMax Tegmark0:00

      A lot of people have said for many years that there will come a time when we want to pause a little bit. That time is now.

   2. LFLex Fridman0:10

      The following is a conversation with Max Tegmark, his third time on the podcast. In fact, his first appearance was episode number one of this very podcast. He is a physicist and artificial intelligence researcher at MIT, co-founder of Future Life Institute, and author of Life 3.0: Being Human in the Age of Artificial Intelligence. Most recently, he's a key figure in spearheading the open letter calling for a six-month pause on giant AI experiments, like training GPT-4. The letter reads, "We're calling for a pause on training of models larger than GPT-4 for six months. This does not imply a pause or ban on all AI research and development, or the use of systems that have already been placed in the market. Our call is specific and addresses a very small pool of actors who possesses this capability." The letter has been signed by over 50,000 individuals, including 1,800 CEOs and over 1,500 professors. Signatories include Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak, Yuval Noah Harari, Andrew Yang, and many others. This is a defining moment in the history of human civilization, where the balance of power between human and AI begins to shift. In Max's mind and his voice is one of the most valuable and powerful in a time like this. His support, his wisdom, his friendship has been a gift I'm forever deeply grateful for. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Max Tegmark.

2. 1:56 – 14:20

   Intelligent alien civilizations

   1. LFLex Fridman1:56

      You were the first ever guest on this podcast, episode number one. So first of all, Max, I just have to say, uh, thank you for giving me a chance. Thank you for starting this journey. And it's been an incredible journey. Just thank you for, um, sitting down with me and just acting like I'm somebody who matters, that I'm somebody who's interesting to talk to. And, um, thank you for doing it. That meant a lot.

   2. MTMax Tegmark2:19

      Oh, thanks to you for putting your heart and soul into this. I know when you delve into controversial topics, it's inevitable...

   3. LFLex Fridman2:28

      (laughs)

   4. MTMax Tegmark2:28

      ... to get hit by what, what Hamlet talks about, the slings and arrows and stuff, and I really admire this. It's in an era, you know, where YouTube videos are too long and now it has to be like a 20-minute TikTok, 20-second TikTok clip, it's just so refreshing (laughs) to see you going exactly against all of the advice...

   5. LFLex Fridman2:46

      (laughs)

   6. MTMax Tegmark2:46

      ... and doing these just really long-form things, and the people appreciate it, you know? Reality is nuanced, and, uh, thanks for sharing it that way.

   7. LFLex Fridman2:55

      Uh, so let me ask you again the first question I've ever asked on this podcast, episode number one, talking to you, do you think there's intelligent life out there in the universe? Let's revisit that question. Do you have any updates? What's your view when you look out to the stars?

   8. MTMax Tegmark3:12

      So, when we look out to the stars, if you define our universe the way most astrophysicists do, not as all of space, but the spherical region of space that we can see with our telescopes from which light has had time to reach us since our Big Bang, I'm in the minority. I, I am... estimate that we are the only life in this spherical volume that has, uh, invented internet, radio, has gotten our level of tech. And, um, if that's true, then, uh, it puts a lot of responsibility on us to not mess this one up, because in that c- if it's true, it means that life is, is quite rare, and we are stewards of this one spark of advanced consciousness, which, if- if we nurture it and help it grow, it... eventually life can spread from here, uh, out into much of our universe, and we can have this just amazing future. Whereas if we instead, um, are reckless with the technology we build and just snuff it out through the stupidity or infighting, then may- maybe it m- the rest of cosmic history and our universe is just gonna be a play for empty benches. But I, I do think that we are actually very likely to get visited by aliens, alien intelligence, quite soon. But I think we are gonna be building that alien intelligence.

   9. LFLex Fridman4:36

      So, uh, we're going to give birth to an intelligent alien civilization, unlike anything that human, the evolution here on Earth was able to create in terms of the path, the biological path it took?

   10. MTMax Tegmark4:52

       Yeah, and it's gonna be much more alien than a cat or even the most exotic animal on the planet right now, because it will not have been created through the usual Darwinian competition where it necessarily kill- cares about self-preservation, is afraid of death, um, any of those things. Um, the space of alien minds is, that you can build, is just so much vaster than what evolution will give you. And, um, with that also comes a great responsibility, uh, that, that, for us to make sure that the kind of minds we create are the s- kind of minds that, um, it's good to create, minds that will, uh, share our values and, and be good for humanity and life, and also m- mind th- create minds that don't suffer.

   11. LFLex Fridman5:44

       Do you try to visualize the full space of alien minds that AI could be? Do you try to consider all the different kinds of intelligences, sort of generalizing what humans are able to do to the full spectrum of what intelligent creatures, entities could do?

   12. MTMax Tegmark6:01

       I try, but I would say I fail. I mean, it's...... it's very difficult for human mind th- to f- really grapple with, uh, something s- so completely alien. I mean, even for us, right? S- uh, if we just try to imagine how would it feel if we were completely indifferent towards death-

   13. LFLex Fridman6:23

       Mm-hmm.

   14. MTMax Tegmark6:23

       ... or individuality. Uh, if we, uh, eh, uh, even if you just imagine that for s- for example, you could just copy my knowledge of how to speak Swedish. Boom, now you can speak Swedish. I- uh, and you could copy any of my cool experiences, and then you could delete the ones you didn't like in your own life-

   15. LFLex Fridman6:42

       Mm-hmm.

   16. MTMax Tegmark6:42

       ... just like that. It, it would already change quite a lot about how you feel as a human being, right? Uh, you'd probably spend less effort studying things (laughs) if you could just copy them. And you might be less f- afraid of death because if the plane you're on starts to crash, you'd just be like, "Oh, shucks. I'm gonna... I haven't backed my brain up for four hours."

   17. LFLex Fridman7:03

       (laughs)

   18. MTMax Tegmark7:04

       "So I'm gonna lose this, all this wonderful experiences of this, of this flight." We might also start feeling more, like, compassionate maybe with other people if we can so readily share each other's experiences and our knowledge, and feel more like, uh, a hive mind. It's, it's very hard though. I, I really feel very humble about this, to, to, to grapple with it, the, the, how it might actually feel. The, the, the, the one thing which is s- so obvious though, which I think is just really worth reflecting on is, because the mind space of possible intelligence is so different from ours, it's very dangerous if we assume they're gonna be like us or anything like us.

   19. LFLex Fridman7:49

       Well, there's, uh, the entirety of, uh, human written history has been through poetry, through novels, been trying to describe through philosophy, uh, trying to describe the human condition and what's entailed in it. Like Jesika said, fear of death and all those kinds of things, what is love. And all of that changes-

   20. MTMax Tegmark8:09

       Yeah.

   21. LFLex Fridman8:10

       ... if you have a different kind of intelligence.

   22. MTMax Tegmark8:12

       Yeah.

   23. LFLex Fridman8:12

       Like all of it. The entirety, all those poems that are trying to sneak up to what the hell it means to be human, all of that changes. How AI concerns and, uh, existential crisises that AI experiences, how that clashes with the human existential crisis, the human condition.

   24. MTMax Tegmark8:30

       Yeah.

   25. LFLex Fridman8:31

       That's hard to, hard to fathom, hard to predict.

   26. MTMax Tegmark8:34

       It's hard, but it's fascinating to think about also. Even in the best case scenario where we don't lose control of, over the evermore powerful AI that we're building to other humans whose goals w- we think are horrible, and where we don't lose control to the machines, an- and AI provides the things we want, even then you get into the questions you touched here. You know, maybe it's the struggle that it's actually hard to do things is part of the things that gives us meaning as well, right? So for example, I found it so shocking that this new Microsoft, uh, GPT-4 commercial that they put together has this woman talking about, showing this demo how she's gonna give a, a graduation speech to her beloved daughter.

   27. LFLex Fridman9:23

       Mm-hmm.

   28. MTMax Tegmark9:23

       And she asks GPT-4 to write it. It was freaking 200 words or so. If I realized that my parents couldn't be bothered struggling a little bit to write 200 words and outsource that to the computer, I would feel really offended actually. And so I wa- I'm, I wonder if, um, eliminating too much o- of the struggle from our existence... Do you think that would also take away a little bit of what makes-

   29. LFLex Fridman9:53

       It means to be human, yeah.

   30. MTMax Tegmark9:55

       Yeah.
3. 14:20 – 25:47

   Life 3.0 and superintelligent AI

   1. LFLex Fridman14:20

      uh, allow me to briefly look at the book, which at this point is becoming more and more visionary, that you've written, I- I guess over five years ago, Life 3.0. Uh, so first of all, 3.0, what's 1.0, what's 2.0, what's 3.0? And how has that vision sort of evolved, the vision in the book evolved to today?

   2. MTMax Tegmark14:41

      Life 1.0 is really dumb, like bacteria, in that it can't actually learn anything at all during the lifetime. The learning just comes from this m- genetic process from one generation to the next. Life 2.0 is us and other animals which have a brains, which can learn during their lifetime a great deal.

   3. LFLex Fridman15:03

      Mm-hmm.

   4. MTMax Tegmark15:04

      Right? So ... And, um ... You know, you were born without being able to speak English. And at some point, you decided, "Hey, I wanna upgrade my software. Let's- let's install an en- English-speaking module." So, you did. And, uh, Life 3.0, which does not exist yet, can- can up- replace not only its software the way we can-

   5. LFLex Fridman15:26

      Mm-hmm.

   6. MTMax Tegmark15:27

      ... but also its hardware. And, um, that's where we're hea- heading towards at high speed. We're already maybe 2.1 'cause we can, you know, put in a- an artificial knee, a (clears throat) pacemaker, et cetera, et cetera. And if Neuralink and other companies succeed, we'll be Life 2.2, et cetera. But, uh, what- what the company's trying to build, AGI, or are trying to make, is of course full 3.0. And you can put that intelligence into something that also has no ... biological basis whatsoever.

   7. LFLex Fridman16:02

      So, less constraints and more capabilities.

   8. MTMax Tegmark16:05

      Mm-hmm.

   9. LFLex Fridman16:05

      Just like the leap from 1.0 to 2.0. There is, nevertheless, you speaking so harshly about bacteria, so disrespectfully about bacteria.

   10. MTMax Tegmark16:14

       (laughs)

   11. LFLex Fridman16:14

       There is still the same kind of magic there that permeates Life 2.0 and, uh, and 3.0. It seems like maybe the thing that's truly powerful about life, intelligence, and consciousness was already there in 1.0. Is it possible?

   12. MTMax Tegmark16:34

       I think we should be m- humble and not be so quick to ... make everything binary and say either it's there or it's not. Clearly, there's a, there's a great spectrum. And there is even a controversy about whether some unicellular organisms, like amoebas, can maybe learn a little bit, you know, after all. So, apologies if I offend (laughs) any bacteria here.

   13. LFLex Fridman16:55

       Yeah.

   14. MTMax Tegmark16:56

       It wasn't my intent. It was more that I wanted to talk up how cool it is to actually have a brain-

   15. LFLex Fridman17:01

       Yeah.

   16. MTMax Tegmark17:01

       ... where you can learn dramatically within your lifetime.

   17. LFLex Fridman17:04

       Typical human.

   18. MTMax Tegmark17:05

       And- and the higher up you get from 1.0 to 2.0 to- to 3.0, the more you become the captain of your own desti- of your own ship, the master of your own destiny, and the less you become a slave to whatever evolution gave you, right? By upgrading our software, we can be so different from previous generations and even from our parents, much more so than even a bacterium. You know, no offense to them.

   19. LFLex Fridman17:28

       (laughs)

   20. MTMax Tegmark17:29

       And, uh, if you can also swap out your hardware and take any physical form you want, of course, really the sky is the limit.

   21. LFLex Fridman17:36

       Yeah. So, the ... (sighs) It accelerates the rate at which you can perform the competition that- computation that determines your destiny.

   22. MTMax Tegmark17:45

       Yeah. And I think it's- it's worth commenting a bit on what you means in this context also, if you swap things out a lot, right? Uh, this is controversial, but my ... current understanding is that l- that, you know, life is best thought of, um, not as a bag of meat or even a bag of elementary particles, but rather as an- as, um, a system which can process information and retain its own complexity, even though nature is always trying to mess it up. So, it's all about inf- in- information processing. And-... that makes it a lot like something like a wave in the ocean, which is not, it's, it's water molecules, right? The water molecules bob up and down, but the wave moves forward. It's an information pattern. In the same way, you, Lex, you and I are not the same atoms as during the first-

   23. LFLex Fridman18:43

       ... time we talked, yeah.

   24. MTMax Tegmark18:44

       ... interview you did with me. You've swapped out most of them, but it's still you.

   25. LFLex Fridman18:47

       Yeah.

   26. MTMax Tegmark18:48

       And the, the, the information pattern is still there. And, um, if you, if you could swap out your arms and, like, whatever, you can still have this kind of continuity. It becomes a much more sophisticated sort of wave forward in time, where the information lives on. I, I, I lost both my parents, uh, since, since our last podcast. And, and it actually gives me a lot of solace that this way of thinking about them, they haven't entirely died because a lot of mommy and daddies, um ... Sorry, I'm getting a little emotional here, but a lot of their values and ideas and even jokes and so on, they haven't gone away, right? Some of them live on, I can carry on some of them, and they also live on a lot of other, in a lot of other people. So, in this sense, even with life 2.0, we can, to some extent, already transcend our physical bodies and our death. And particularly if you can share your own information, your own ideas with many others, like you do in your podcast, then, um, you know, that's the closest to immortality we can (laughs) get with our bio-bodies.

   27. LFLex Fridman20:06

       You carry a little bit of them in you, in some sense.

   28. MTMax Tegmark20:10

       Yeah. Yeah.

   29. LFLex Fridman20:11

       Uh, do you miss them? Do you miss your mom and dad?

   30. MTMax Tegmark20:14

       Of course. Of course.
4. 25:47 – 50:54

   Open letter to pause Giant AI Experiments

   1. LFLex Fridman25:48

      Well, you do happen to be working on a thing which seems to have, uh, potentially, uh, some of the greatest impact on human civilization of anything humans have ever created, (laughs) which is artificial intelligence. This is on the both detailed technical level and in a high philosophical level you work on. So, you've mentioned to me that there's an open letter that you're working on.

   2. MTMax Tegmark26:15

      It's actually, uh, going live in a few hours.

   3. LFLex Fridman26:19

      (laughs)

   4. MTMax Tegmark26:19

      So I've been having late nights and early mornings. It's been very exciting, actually. I, in short, I... Have you seen, uh, Don't Look Up, the film?

   5. LFLex Fridman26:31

      Yes, yes.

   6. MTMax Tegmark26:32

      I don't want to be the movie spoiler for anyone watching this-

   7. LFLex Fridman26:35

      Okay.

   8. MTMax Tegmark26:35

      ... who hasn't seen it, but if you're watching this, you haven't seen it, watch it.

   9. LFLex Fridman26:39

      Mm-hmm.

   10. MTMax Tegmark26:40

       Because we are actually acting out, it's, it's life imitating art. Humanity is doing exactly that right now, except it's an asteroid that we are building ourselves.

   11. LFLex Fridman26:51

       Mm-hmm.

   12. MTMax Tegmark26:52

       Almost nobody is talking about it. People are squabbling across the planet about all sorts of things, which seem very minor compared to the asteroid that's about to hit us, right? Uh, most politicians don't even have their radars, this on their radar. They think maybe in 100 years or whatever. Right now, we're at a fork in the road. This is the most important, um, fork humanity has reached in its over 100,000 years on this planet. We're building, eh, effectively a, a new species that's smarter than us. It doesn't look so much like a species yet 'cause it's mostly not embodied in robots, but, um, that's a technicality which will soon be changed. And, and this arrival of, of artificial general intelligence that can do all our jobs as well as us, and probably shortly thereafter, superintelligence, which greatly exceeds our cognitive abilities, it's gonna either be the, the best thing ever to happen to humanity or the worst. I'm really quite confident that there is not that much middle ground there.

   13. LFLex Fridman27:55

       But it would be fundamentally transformative to human civilization.

   14. MTMax Tegmark27:58

       Of course, utterly and totally. You know, again, we, we branded ourselves as homo sapiens 'cause it seemed like the basic thing. We're the king of the castle on this planet. We're the smart ones, if we can control everything else. Uh, this could very easily change. We're, we're certainly not gonna be the smartest on the planet for very long if AI... unless AI progress just halts. And we can talk more about why I, I think that's true 'cause it's, it's controversial. And, and then we can also talk about reasons you might think it's gonna be the best thing ever, and the reason you think it's gonna, going to be the end of humanity, which is, of course, super controversial. But what I think we can... (laughs) Anyone who's working on, uh, advanced AI can agree on is it's, it's much like the film Don't Look Up in that it's just really comical how little serious public debate there is about it, given how huge it is.

   15. LFLex Fridman29:03

       (sighs) So what we're talking about is the development of currently things like GPT-4 and the signs it's showing of a rapid improvement that may, in the near term, lead to development of super intelligent AGI, AI, general AI systems, and what kind of impact that has on society.

   16. MTMax Tegmark29:26

       Exactly.

   17. LFLex Fridman29:26

       When that thing is, achieves general human level intelligence, and then beyond that, general superhuman level intelligence.

   18. MTMax Tegmark29:34

       Mm-hmm.

   19. LFLex Fridman29:34

       (sighs) There's a lot of questions to explore here. So one, you mentioned halt. Is that, uh, the content of the letter is-

   20. MTMax Tegmark29:44

       Yeah.

   21. LFLex Fridman29:45

       ... to suggest that maybe we should pause the development of these systems?

   22. MTMax Tegmark29:49

       Exactly. So this is very controversial. From when, when, um, we talked... The first time, we talked about how I was involved in starting the Future of Life Institute, and we worked very hard on, 2014, 2015, was the mainstream AI safety.

   23. LFLex Fridman30:04

       Mm-hmm.

   24. MTMax Tegmark30:05

       The idea that there even could be risks and that you could do things about them.

   25. LFLex Fridman30:08

       Mm-hmm.

   26. MTMax Tegmark30:09

       Before then, a lot of people thought it was just really kooky to even talk about it. And a lot of AI researchers felt worried that this was too flaky and could be bad for funding, and that the people who talked about it were just not... didn't understand AI. I, I'm very, very happy with th- how that's gone, in that now, you know, just it's completely mainstream. You go to any AI conference and people talk about AI safety, and it's a t- nerdy technical field full of equations and simu- and blah, blah.

   27. LFLex Fridman30:39

       Yes.

   28. MTMax Tegmark30:41

       ... as it should be. Uh, but there is this other thing which has been quite taboo up until now, calling for slowdown. So, what we've constantly been saying, including myself, I've been biting my tongue a lot, you know, is that, you know, we, we don't need to slow down AI development. We just need to win this race, the wisdom race between the growing power of the AI and the growing wisdom with which we manage it. And rather than trying to slow down AI, let's just try to accelerate the wisdom.

   29. LFLex Fridman31:16

       Mm-hmm.

   30. MTMax Tegmark31:17

       Do all this technical work to figure out how you can actually en- ensure that your powerful AI is gonna do what you want it to do, and have society adapt also with, um, incentives and regulations so that these things get put to good use. Um, sadly, that didn't pan out. The, um, progress on technical AI, um, capabilities has gone a lot faster than, than many people thought. Uh, back, uh, when we started this in 2014, turned out to be easier to build really advanced AI than we thought. Um, and, uh, on the other side, it's gone much slower than we hoped with getting, um, policymakers and others to actually put the inpla- incentives in place to, to make, um, steer this in the, in the good directions. We can, uh, maybe we should unpack it and talk a little bit about each. So-
5. 50:54 – 1:19:44

   Maintaining control

   1. LFLex Fridman50:54

      So, there is some rate of development that will lead us as a human species to lose control of this thing, and the hope you have is that there's some lower level of development which will not-

   2. MTMax Tegmark51:07

      Yeah.

   3. LFLex Fridman51:07

      ... which will not l- allow us to lose control. This is an interesting thought you have about losing control. So, what... If you have somebody, if you're somebody like Sundar Pichai or Sam Altman at the head-

   4. MTMax Tegmark51:17

      Yeah.

   5. LFLex Fridman51:17

      ... of a company like this, you're saying if they develop an AGI, they too will lose control of it? So, no one person can maintain control. No group of individuals can maintain control.

   6. MTMax Tegmark51:28

      If it's r- if it's created th- very, very soon and is a big black box that we don't understand, like the large language models, yeah, th- then I'm very confident they're gonna lose control. But this isn't just me saying it. You know, Sam Altman and Demis Hassabis have both said-

   7. LFLex Fridman51:44

      Mm-hmm.

   8. MTMax Tegmark51:44

      ... it themselves, acknowledged that, you know, there's really great risks with this, and they, they wanna slow down once they feel like it's, it's scary. It's, but it's clear that they're stuck in this... Again, Moloch is forcing them to go a little faster than, than they're comfortable with because of pressure from just commercial pressures, right? (clears throat) Uh, it... F- to get a bit optimistic here, of course this is a problem that can be ultimately solved. Uh, it just w- to win this wisdom race, it's clear that what h- we hoped that was gonna happen hasn't happened. The, the capability progress has gone faster than a lot of people thought than, and the po- the progress in, in the public sphere of policymaking and so on has gone slower than we thought. Even the technical AI safety has gone slower. A lot of the technical safety research was kind of banking on that, um, large language models and other poorly understood systems couldn't get us all the way.

   9. LFLex Fridman52:37

      Yeah.

   10. MTMax Tegmark52:37

       That you had to build more of a kind of intelligence that you could understand, maybe it could prove itself safe, you know, things like this. And, um, I'm quite confident that this can be done, um, so we can reap all the benefits. But we cannot do it as quickly as, uh, (laughs) this out of control express train we are on now is gonna get the AGI. That's why we need a little more time, I feel.

   11. LFLex Fridman53:04

       Is there something to be said, what like Sam Altman talked about, which is while we're in the pre-AGI stage, to release often and as transparently as possible to learn a lot? So, as opposed to being extremely cautious, release a lot. Don't, uh, don't invest in a closed development where you focus on AI safety. While it's somewhat "dumb," quote, unquote, uh, release as often as possible. And as you start to see signs of, uh, human-level intelligence and, or superhuman level intelligence, then you put a halt on it.

   12. MTMax Tegmark53:42

       Well, what a lot of safety researchers have been saying for many years is that the most dangerous things you can do with an AI is, first of all, teach it to write code.

   13. LFLex Fridman53:52

       Yeah.

   14. MTMax Tegmark53:52

       'Cause that's the first step towards recursive self-improvement, which can take you from AGI to much higher levels. Okay. O- oops, we've done that. And, uh, another thing, high risk, it's connect it to the internet, let it go to websites, download stuff on its own, uh, talk to people. Oops, we've done that already. You know, Eliezer Yudkowsky, you said you interviewed him recently, right?

   15. LFLex Fridman54:14

       Yes, yep.

   16. MTMax Tegmark54:15

       So, he had this tweet recently which like (laughs) ...

   17. LFLex Fridman54:18

       (laughs)

   18. MTMax Tegmark54:18

       ... gave me one of the best laughs in a while where he was like, "Hey, people used to make fun of me and say, 'You're so stupid, Eliezer,' because you're saying, you're saying, um, 'You have to worry.'" Ob- obviously, developers, once they get to, like, really strong AI, first thing you're gonna do is, like, never connect it to the internet, keep it in, in a box-

   19. LFLex Fridman54:37

       Yeah.

   20. MTMax Tegmark54:37

       ... where h- you know, you can really study it, safe. So, he had written it in the, like in the meme form, so it was like, "Then..."

   21. LFLex Fridman54:44

       Yeah.

   22. MTMax Tegmark54:44

       And then that, and then, "Now."

   23. LFLex Fridman54:48

       (laughs)

   24. MTMax Tegmark54:48

       "Let's m- LOL, let's make a chatbot." (laughs)

   25. LFLex Fridman54:52

       (laughs) Yeah. Yeah. Yeah.

   26. MTMax Tegmark54:53

       And the third thing, S- Stuart Russell-

   27. LFLex Fridman54:55

       Yeah.

   28. MTMax Tegmark54:56

       ... you know, m- amazing AI researcher. He ha- he has argued for a while that...We should never teach AI anything about humans. Above all, we should never let it learn about human psychology and how you manipulate humans. That's the most dangerous kind of knowledge you can give it. Yeah, you can teach it all it needs to know how to... about how to cure cancer and stuff like that. But don't let it read Daniel Kahneman's book about cognitive biases and all that.

   29. LFLex Fridman55:24

       Yeah.

   30. MTMax Tegmark55:25

       And then, oops, LOL, you know, let's invent social media, uh, recommender algorithms, which do exactly that. They, they, uh, get so good at knowing us and pressing our buttons that we're, we're, we're starting to create a world now where we just have evermore pe- hatred. Uh, 'cause they figured out that these algorithms, not for... out of evil, but just to make money on advertising, that the best way to get more engagement, the euphemism, get people glued to their little rectangles, right, is just to make them pissed off.
6. 1:19:44 – 1:30:34

   Regulation

   1. MTMax Tegmark1:19:44

      you know, you didn't, we... Just because cars killed a lot of people, they didn't ban cars, but they got together, a bunch of people, and decided, you know, in order to be allowed to sell a car, it- it has to have a seatbelt in it.

   2. LFLex Fridman1:19:53

      Mm-hmm.

   3. MTMax Tegmark1:19:55

      There... The analogous things that you can start requiring of future AI systems so that they, uh, are safe. And, uh, o- o- once this hav- this heavy lifting, this intellectual work has been done by experts in the field, which can be done quickly, I think it's begin- going to be quite easy to get policymakers to, to see, yeah, this is a good idea. And it's- it's, you know, for the fi- for the companies to fight Moloch, they want, and I- I believe Sam Altman has explicitly called for this, they want the regulators to actually adopt it so that their competition is gonna abide by it too, right? You don't want, uh, you don't wanna be en- enacting all these principles and you abide by them, and then you're, there's this one little company that doesn't sign onto it, and then now they can gradually overtake you.

   4. LFLex Fridman1:20:48

      Mm-hmm.

   5. MTMax Tegmark1:20:49

      Then the companies will get... Be able to sleep s- secure knowing that everybody's playing by the same rules.

   6. LFLex Fridman1:20:56

      So do you think it's possible to develop guardrails that keep the systems from, uh, from basically (laughs) damaging, irreparably, humanity-

   7. MTMax Tegmark1:21:08

      Mm-hmm.

   8. LFLex Fridman1:21:09

      ... while still enabling sort of the capitalist-fueled competition between-

   9. MTMax Tegmark1:21:13

      Yeah.

   10. LFLex Fridman1:21:13

       ... companies as they develop how to best make money with this AI? You think there's a balancing-

   11. MTMax Tegmark1:21:18

       Totally.

   12. LFLex Fridman1:21:18

       ... that's possible?

   13. MTMax Tegmark1:21:19

       Absolutely. I mean, we've seen that in many other sectors where you've had the free market produce quite good things without, uh, causing particular harm. Um, when the guardrails are there and they work, you know, capitalism is a very effect- good way of optimizing for- for just getting the same thing done more efficiently. It was-

   14. LFLex Fridman1:21:39

       It's just-

   15. MTMax Tegmark1:21:39

       But it was good, you know, and like in hindsight, I have never met anyone, even, even on parties way over on the right in- in- in any country, who- who think it was a bad, thinks it was a terrible idea to b- to ban child labor, for example.

   16. LFLex Fridman1:21:55

       Yeah, but it seems like this particular technology has gotten so good so fast, become powerful to a degree where you could see, in the near term, the ability to make a lot of money-

   17. MTMax Tegmark1:22:06

       Yeah.

   18. LFLex Fridman1:22:07

       ... and to put guardrails, to develop guardrails quickly in that kind of context seems to be tricky. It's not, uh, similar to cars or child labor. It seems like-

   19. MTMax Tegmark1:22:17

       Mm-hmm.

   20. LFLex Fridman1:22:17

       ... the opportunity to make a lot of money here very quickly is right here before us.

   21. MTMax Tegmark1:22:22

       Yeah, again, there's this cliff.

   22. LFLex Fridman1:22:24

       Yeah.

   23. MTMax Tegmark1:22:26

       Uh, and supposedly it gets-

   24. LFLex Fridman1:22:26

       (laughs) It gets quite scenic.

   25. MTMax Tegmark1:22:27

       ... uh, the closer to the cliff that you go-

   26. LFLex Fridman1:22:28

       Yeah.

   27. MTMax Tegmark1:22:29

       ... the more, the more-

   28. LFLex Fridman1:22:29

       (laughs)

   29. MTMax Tegmark1:22:30

       ... the g- the more money there is, the more gold ingots that are on the ground you can pick up-

   30. LFLex Fridman1:22:33

       Yeah.

Episode duration: 2:48:12