Max Tegmark: Life 3.0 | Lex Fridman Podcast #1

Lex Fridman 0:00 As part of MIT Course 6.099, Artificial General Intelligence, I've gotten a chance to sit down with Max Tegmark. He is a professor here at MIT. He's a physicist, spent a large part of his career studying the mysteries of our cosmological universe, but he's also studied and delved into the beneficial possibilities and the existential risks of artificial intelligence. Amongst many other things, he's the co-founder of the Future of Life Institute, author of two books, both of which I highly recommend. First, Our Mathematical Universe. Second is Life 3.0. He's truly an out-of-the-box thinker, and a fun personality, so I really enjoyed talking to him. If you'd like to see more of these videos in the future, please subscribe and also click the little bell icon to make sure you don't miss any videos. Also, Twitter, LinkedIn, agi.mit.edu if you wanna watch other lectures or conversations like this one. Better yet, go read Max's book, Life 3.0. Chapter 7 on goals is my favorite. It's really where philosophy and engineering come together, and it opens with a quote by Dostoevsky, "The mystery of human existence lies not in just staying alive, but in finding something to live for." Lastly, I believe that every failure rewards us with an opportunity to learn. In that sense, I've been very fortunate to fail in so many new and exciting ways, and, uh, this conversation was no different. I've learned about something called radio frequency interference, RFI. Look it up. Apparently, music and conversations from local radio stations can bleed into the audio that you're recording in such a way that it almost completely ruins that audio. It's an exceptionally difficult sound source to remove. So, I've gotten the opportunity to learn how to avoid RFI in the future during recording sessions. I've also gotten the opportunity to learn how to use Adobe Audition and iZotope RX 6 to do some noise, some audio repair. Of course, this is an exceptionally difficult noise to remove. I am an engineer. I'm not an audio engineer, neither is anybody else in our group, but we did our best. Nevertheless, I thank you for your patience, and I hope you're still able to enjoy this conversation. Do you think there's intelligent life out there in the universe? Let's open up with an easy question.

Max Tegmark 2:33 I have a minority view here, actually. When I give public lectures, I often ask for a show of hands, who thinks there's intelligent life out there somewhere else, and almost everyone puts their hand up and when I ask why, they'll be like, "Oh, there's so many galaxies out there, there's gotta be." But, I'm a nu- numbers nerd, right? So, when you look more carefully at it, it's not so clear at all. Th- the, when we talk about our universe, first of all, we don't mean all of space. We actually mean, I don't know, you can throw me the universe if you want, it's behind you there.

Lex Fridman 3:06 (laughs)

Max Tegmark 3:07 It's, we simply mean the spherical region of space from which light has had time to reach us so far during the 14.8 billion year, 13.8 billion years since our Big Bang. There's more space here, but this is what we call a universe, because that's all we have access to.

Lex Fridman 3:23 Mm-hmm.

Max Tegmark 3:24 So, is there intelligent life here that's gotten to the point of building telescopes and, and computers? My guess is no, actually. Uh, the probability-

Lex Fridman 3:35 Interesting.

Max Tegmark 3:35 ... of it happening on any given planet is some number, we don't know what it is, and, um, what we do know is that, uh, the number can't be super high, 'cause there's over a billion Earth-like planets in the Milky Way galaxy alone, many, uh, which are billions of years older than Earth. And, um, aside from some, um, UFO believers-

Lex Fridman 4:00 (laughs)

Max Tegmark 4:00 ... you know, there isn't much evidence that any super advanced civilization has come here at all. And so, that's the famous Fermi paradox, right? And, and then if you, if you work the numbers, what you find is that, um, if- if you have no clue what the probability is of getting life on a given planet, so it could be 10 to the -10, 10 to the -20, or 10 to the -2, or any power of 10 is sort of equally likely if you wanna be really open-minded.

Lex Fridman 4:25 Mm-hmm.

Max Tegmark 4:25 That translates into it being equally likely that our nearest neighbor is 10 to the 16 meters away, 10 to the 17 meters away, 10 to the 18. You know, by the time you get much less than, than 10 to the 16 already, we, we pretty much know there is nothing else that close. (clears throat) And when you get beyond-

Lex Fridman 4:47 Because they would've discovered us.

Max Tegmark 4:48 They, yeah, they would've been discovered us long ago or if they're really close, we would've probably noted some engineering projects that they're doing.

Lex Fridman 4:54 Yeah.

Max Tegmark 4:54 And if it's beyond 10 to the 26 meters, that's already outside of here.

Lex Fridman 4:59 Yeah.

Max Tegmark 4:59 So, so my guess is actually that there, we are the only life in here that's gotten to the point of, uh, building advanced tech, which I think is, is very, um, puts a lot of responsibility on our shoulders to not screw up, you know? I, I think-

Lex Fridman 5:15 Absolutely.

Max Tegmark 5:16 ... people who take for granted that it's okay for us to screw up, have an accidental nuclear war or go extinct somehow because there's a sort of Star Trek-like situation out there where some other life forms are gonna come and bail us out, and it doesn't matter, so I, I think they're lulling us into a false sense of security.

Lex Fridman 5:33 Right.

Max Tegmark 5:33 I think it's much more prudent to say, you know, "Let's be really grateful for this amazing opportunity we've had," and, uh, make the best of it, just in case it is down to us.

Lex Fridman 5:44 So from a physics perspective, do you think intelligent life, so it's unique from a sort of statistical view of the size of the universe, but from the basic matter of the universe, how difficult is it for intelligent life to come about, the kind of advanced tech building life?... are, i- is implied in your statement that it's really difficult-

Max Tegmark 6:05 Mm-hmm.

Lex Fridman 6:05 ... to create something like a human species?

Max Tegmark 6:07 Well, I th- I think what we know is that going from no life to having life that can do our k- level of tech, there is some sort of, to going beyond that and actually settling a whole universe with life, there is some road, major roadblock there, which is some Great Filter, as, um, it's sometimes called. Which, whi- which is tough to get through. It's either... That r- that roadblock is either bef- behind us or in front of us.

Lex Fridman 6:38 Right.

Max Tegmark 6:38 I'm hoping very much that it's behind us. I'm, I'm super excited every time we get a new, uh, report from NASA saying they failed to find any life on Mars.

Lex Fridman 6:48 Mm.

Max Tegmark 6:48 My guess, awesome.

Lex Fridman 6:50 (laughs)

Max Tegmark 6:50 Because that suggests that the hard part, maybe wa- maybe it was getting the first ribosome, or, or some-

Lex Fridman 6:54 Yeah.

Max Tegmark 6:54 ... some very low level kind of stepping stone, so that we're home free. 'Cause if that's true, then the future is really only limited by our own imagination. It would be much suckier if it turns out that this level of life is kind of a dime a dozen.

Lex Fridman 7:10 Mm-hmm.

Max Tegmark 7:11 But maybe there is some other problem, like as soon as, uh, civilization gets advanced technology, within a hundred years, they get into some stupid fight with themselves. And poof!

Lex Fridman 7:19 Yep.

Max Tegmark 7:19 You know, th- that would be a bummer.

Lex Fridman 7:21 Yeah. So, you've explored the mysteries of the univ- the cosmological universe, the one that's s- between us today. Uh, I think you've also begun to explore the other universe, which is sort of the mystery, the mysterious universe of the mind, of intelligence, of intelligent life. So, is there a common thread between your interests and the way you think about space and intelligence?

Max Tegmark 7:48 Oh, yeah. Whe- when I was a teenager, I, uh, I was al- already very fascinated by the biggest questions. And I felt that the two biggest que- mysteries of all in science were our universe out there-

Lex Fridman 8:02 Mm-hmm.

Max Tegmark 8:02 ... and our universe in here.

Lex Fridman 8:04 Yeah.

Max Tegmark 8:05 So it's quite natural, after having spent a quarter of a century, uh, s- on my career, thinking a lot about this one, that I'm now indulging in the luxury of, of, uh, doing research on this one. It's just so cool. I feel the time is ripe now, for, uh, trans- directly deepening our understanding of this.

Lex Fridman 8:25 To start exploring this one?

Max Tegmark 8:26 Yeah, 'cause I think, I think a lot of people view intelligence as something mysterious-

Lex Fridman 8:32 Right.

Max Tegmark 8:32 ... that can only exist in biological organisms like us.

Lex Fridman 8:35 Mm-hmm.

Max Tegmark 8:36 And therefore dismiss all talk about artificial general intelligence as, as science fiction.

Lex Fridman 8:40 Mm-hmm.

Max Tegmark 8:41 But from my perspective as a physicist, you know, I am a blob of, of quarks and electrons-

Lex Fridman 8:45 Mm-hmm.

Max Tegmark 8:46 ... moving around in a certain pattern and processing information in certain ways. And this is also a blob of quarks and electrons.

Lex Fridman 8:52 Mm-hmm.

Max Tegmark 8:53 I am not smarter than the water bottle because I'm made of different kind of quarks.

Lex Fridman 8:57 Right.

Max Tegmark 8:58 I'm made of up quarks and down quarks, exact same kind as this.

Lex Fridman 9:01 Mm-hmm.

Max Tegmark 9:02 It's a... There's no secret sauce, I think, in me. It's, it's all about the p- the pattern of the information processing. And this means that there's no law in physics saying that we, that we can't create technology-

Lex Fridman 9:15 Mm-hmm.

Max Tegmark 9:15 ... which can he- help us by being incredibly intelligent, and help us crack mysteries that we couldn't. In other words, I think we've really only seen the tip of, of the intelligence iceberg so far.

Lex Fridman 9:26 Yeah, so the perceptronium.

Max Tegmark 9:30 Yeah.

Lex Fridman 9:31 Uh, so you cur- coined this amazing term, uh, it's a hypothetical state of matter. Sort of thinking from a physics perspective, what is the kind of matter that can help, as you're saying, uh, subjective experience emerge, consciousness emerge. So, how do you think about consciousness from this physics perspective?

Max Tegmark 9:50 Very good question. So, uh, again, um, you know, I think many people have underestimated our ability to make progress on this, by convincing themselves it's hopeless, because somehow we're missing some ingredient that we need, or some new consciousness particle or whatever. I, I happen to think that we're not missing anything, and, (laughs) and that it's al- not... The interesting thing about the consciousness that gives us this amazing subjective experience of colors and sounds and emotions and so on, is rather something at the higher level about the patterns of information processing. And that's why I, that's why I, um, like to think about this idea of perceptronium. W- what does it mean for an arbitrary physical system to be conscious, in, in terms of, uh, what its particles are doing, or, or, or its information is doing? I don't think... I don't, I hate carbon chauvinism, you know-

Lex Fridman 10:49 Right.

Max Tegmark 10:49 ... this attitude you have to be made of carbon atoms to be smart or, or conscious. Um-

Lex Fridman 10:54 So something about the information processing-

Max Tegmark 10:56 Yeah.

Lex Fridman 10:56 ... that this kind of matter performs.

Max Tegmark 10:58 Yeah. And, you know, you can see I have my favorite equations here describing various fundamental aspects of the world. I feel that... I think one day, maybe someone who's watching this will come up with the equations that, that information processing has to satisfy to be conscious. I, I'm quite convinced there is big discovery to be made there, uh, 'cause, let's face it, some infor- we know that some information processing is conscious, 'cause we are-

Lex Fridman 11:24 Yeah.

Max Tegmark 11:24 ... conscious. But we also know that a lot of information processing is not conscious. Like most of the information processing happening in your brain right now-

Lex Fridman 11:31 Mm-hmm.

Max Tegmark 11:32 ... is not conscious. There are like 10 megabytes per second coming in, even just through your visual system, and you're not conscious about your heartbeat regulation, or, or most things.

Lex Fridman 11:41 Right.

Max Tegmark 11:42 Or even, even, and like if, if I just ask you to like read what it says here, you look at it, and then, oh, now you know what it said. But you're not aware of how the computation actually happened.

Lex Fridman 11:50 Right.

Max Tegmark 11:51 You're like the, your consciousness is like the CEO that got an email at the end with the, with the final answer-

Lex Fridman 11:56 Yeah.

Max Tegmark 11:57 So, uh, what is it that makes a difference? I, I think that's a m- both a great science mystery, we're actually studying it a little bit in my lab here at MIT-... uh, but I also think it's just a really urgent question to answer. For starters, I mean, if you're an emergency room doctor and you have an unresponsive patient coming in, wouldn't it be great if in addition to having a CT scanner, you ha- you had a consciousness scanner-

Lex Fridman 12:24 Mm-hmm.

Max Tegmark 12:25 ... that could figure out whether this person is actually having locked-in syndrome-

Lex Fridman 12:30 Right.

Max Tegmark 12:31 ... or is actually comatose? Uh, and in the future, imagine if we build robots or other machine that we can have really good conversations with. It's, I think, it's ve- it's most very likely to happen, right? Wouldn't you wanna know, like, if your home helper robot is actually ex- experiencing anything or just like a zombie? I mean, s- w- would you prefer... What would you prefer? Would you prefer that it's actually unconscious so that you don't have to feel guilty about switching it off or giving it boring chores? Or what wo- what would you prefer?

Lex Fridman 13:02 Well, the... certainly, w- would, we would prefer... I would prefer the appearance of consciousness.

Max Tegmark 13:09 Mm-hmm.

Lex Fridman 13:09 But the question is whether the appearance of consciousness is different than consci- consciousness itself. And, uh, sort of-

Max Tegmark 13:16 Yeah.

Lex Fridman 13:16 ... to a- ask that as a question-

Max Tegmark 13:17 Yeah.

Lex Fridman 13:18 ... do you think we need to, you know, understand what consciousness is, solve the hard problem of consciousness in order to build something, uh, like an, an AGI system?

Max Tegmark 13:28 No.

Lex Fridman 13:29 No.

Max Tegmark 13:29 I don't think that. Uh, I think we, we will, uh, probably be able to build things even if we don't answer that question. But if we wanna make sure that what happens is a good thing, we better solve it first.

Lex Fridman 13:40 Right.

Max Tegmark 13:41 Uh, so it, it's a wonderful controversy you're raising there. There... where you have, um, basically three points of view about the hard problem. So, there are two different points of view that both conclude that the hard problem of consciousness is BS. You have... on one hand, you have some people like Daniel Dennett-

Lex Fridman 13:59 Mm-hmm.

Max Tegmark 13:59 ... who say, uh, that this is, our consciousness is just BS because consciousness is the same thing as intelligence.

Lex Fridman 14:04 Mm-hmm.

Max Tegmark 14:05 There's no difference. So anything which acts conscious is conscious just like, like we are.

Lex Fridman 14:12 (clears throat)

Max Tegmark 14:13 And then there are also a lot of people including many top AI researchers I know who say, "Oh, consciousness is just bullshit," because of course, machines can never be conscious.

Lex Fridman 14:22 Right.

Max Tegmark 14:22 They're always gonna s- gonna be zombies. You never have to feel guilty about how you treat them. And then there's a third group of people, eh, w- including Giulio Tononi, for example, an- and, and another, um, Christof Koch and a number of others, I would put myself also in this middle camp, who say that actually some information processing is conscious and some is not.

Lex Fridman 14:43 Mm-hmm.

Max Tegmark 14:44 So let's find the equation which can be used to determine which it is. And I think we've just been a little bit lazy kind of running away from this problem for a long time, eh. It's been almost taboo to even mention the C word-

Lex Fridman 14:57 Right. (laughs)

Max Tegmark 14:58 ... in a lot of circles because... but, but we should stop making excuses. This is a science question, uh, and we c- and I, uh... there are, there are ways we can even test, test any theory that makes predictions for this. And, uh, coming back to this helper robot, I mean, so you said you'd want your helper robot to certainly act conscious and treat you, like, and, and have conversations with you and stuff?

Lex Fridman 15:21 I think so, yeah.

Max Tegmark 15:21 But wouldn't you... would you feel, would you feel a little bit creeped out if you realize that it was just, like, glossed up tape recorder, you know, the words "Just zombie." And was faking emotion? Would you prefer that it actually had an experience? Or, or, or would you prefer that it's actually not experiencing anything so you, you feel... you don't have to feel guilty about what you do to it? What would you prefer?

Lex Fridman 15:41 It's such a, it, it's such a difficult question because, uh, you know, it's like when you're in a relationship and you say, "Well, I love you," and the other person says, "I love you back." It's like asking, "Well, do they really love you back or are they just saying they love you back?" Uh-

Max Tegmark 15:55 (laughs)

Lex Fridman 15:55 ... do you... don't you really want them to actually love you?

Max Tegmark 15:58 (laughs)

Lex Fridman 15:58 I... it's hard to... it's, it's hard to (laughs) really know the difference between, uh, everything seeming like there's consciousness present, there's intelligence present, there's a- affection, passion, love and, and it, it actually being there. I'm not sure. Do you ha-

Max Tegmark 16:17 But like the mass gen-

Lex Fridman 16:18 Do you, do-

Max Tegmark 16:18 Can I ask you a question about this?

Lex Fridman 16:19 Yes.

Max Tegmark 16:19 Like, to make it a bit more pointed? So Mass General Hospital is right across the river, right?

Lex Fridman 16:22 Yes.

Max Tegmark 16:23 Suppose, suppose you're going in for a medical procedure and they're like, "You know, uh, for, for anesthesia what we're gonna do is we're gonna give you muscle relaxants so you won't be able to move, uh, and you're gonna feel excruciating pain during the whole surgery, but you won't be able to do anything about it. But then we're gonna give you this drug that erases your memory of it."

Lex Fridman 16:40 Mm-hmm.

Max Tegmark 16:42 Would you be cool about that?

Lex Fridman 16:44 Hm.

Max Tegmark 16:45 Or what, what's the difference that you're conscious about it or not if, if, if there's no behavioral change, right?

Lex Fridman 16:51 Right. That's a really in- that's a really clear way to put it. That's... yeah. It feels like in that sense experiencing it is a va- a valuable quality, so actually being able to have subjective experiences, (laughs) at least in that case, is, is valuable.

Max Tegmark 17:09 And I think we humans have a little bit of a bad track record also of making these self-serving arguments that other entities aren't conscious. You know, people-

Lex Fridman 17:18 Right.

Max Tegmark 17:18 ... often say, "Oh, these animals can't feel pain."

Lex Fridman 17:20 Right.

Max Tegmark 17:21 It's okay to boil lobsters because we asked them if it hurt and they didn't say anything, and, and now there was just a paper out saying lobsters did, do feel pain when you boil them-

Lex Fridman 17:29 Yeah.

Max Tegmark 17:29 ... and they're banning it in Switzerland. And, and, uh, and we did this with slaves too often and said, "Oh, they don't mind." Uh. (laughs) And they don't maybe ha- or aren't conscious or women don't have souls or whatever.

Lex Fridman 17:41 Right.

Max Tegmark 17:41 So, I'm a little bit nervous when I (laughs) hear people just take as an axiom that machines can't have experience ever. I think this is just a really fascinating science question is what it is.

Lex Fridman 17:52 Yeah.

Max Tegmark 17:52 Let's research it and try to figure out what it is that makes the difference between unconscious intelligent behavior and, and conscious intelligent behavior.

Lex Fridman 18:01 So in terms of... so if you think of a Boston Dynamics humanoid robot being sort of, uh, with a broom being pushed around-... the, it's starts, it's, it starts pushing on this consciousness question, so let me ask: do you think an AGI system, like a few neuroscientists believe, um, needs to have a physical embodiment? Needs to have a body or something like a body?

Max Tegmark 18:25 No. I don't think so. You mean, to ha- to have a conscious experience?

Lex Fridman 18:30 To have consciousness.

Max Tegmark 18:33 I do think it helps a lot to have a physical embodiment to learn the kind of things about the world that are important to us humans, for sure. But I don't think the physical embodiment is necessary after you've learned it to just have the experience. Think about when you're dreaming, right?

Lex Fridman 18:50 Mm-hmm.

Max Tegmark 18:51 Your eyes are closed, you're not getting any sensory input. You're not behaving or moving in any way, but there's still an experience there, right?

Lex Fridman 18:57 Mm-hmm.

Max Tegmark 18:59 And so clearly the experience that you have when you see something cool in your dreams isn't coming from your eyes. It's just the information processing itself in your brain, which is that experience, right?

Lex Fridman 19:11 But, uh, I, if I put it another way, I'll say because it comes from neuroscience, is: the reason you wanna have a body and a physical, something like a ph- physical, like a, you know, a physical system, is because you want to be able to preserve something. In order to have a self, uh, you could argue, would... you, you'd need to have some kind of embodiment of self to want to preserve.

Max Tegmark 19:38 Well, now we're getting a little bit anthropomorphic, anthro- into anthropomorphizing things ... m- m- maybe-

Lex Fridman 19:45 I see.

Max Tegmark 19:45 ... talking about self-preservation instincts. I mean, we are evolved organisms, right?

Lex Fridman 19:49 Right.

Max Tegmark 19:50 So Darwinian evolution endowed us and other involve- evolved organism with a self-preservation instinct 'cause those that didn't have those (laughs) self-preservation genes are cleaned out of the gene pool.

Lex Fridman 20:01 Right.

Max Tegmark 20:02 Right? Um, but if you build an artificial general intelligence, the mind space that you can design is much, much larger than just a specific subset of, of minds that can evolve, that have-

Lex Fridman 20:13 Mm-hmm.

Max Tegmark 20:14 So they, so an AGI mind doesn't necessarily have to have any self-preservation instinct, and it also doesn't necessarily have to be so individualistic as us. Like, imagine if you could just... first of all, or we're also very afraid of death, you know?

Lex Fridman 20:28 Yeah.

Max Tegmark 20:28 Suppose you could back yourself up every five minutes-

Lex Fridman 20:30 (laughs)

Max Tegmark 20:31 ... and then your airplane is about to crash. You're like, "Shucks, I'm just, I'm t- I'm gonna lose the last five minutes of experiences since my last cloud backup."

Lex Fridman 20:38 You're right.

Max Tegmark 20:39 "Dang!" You know? It's not as big a deal. Where if, if we could just copy experiences between our minds easily like we, which we could easily do if we were silicon-based, right, then, uh, maybe we would feel a little bit more like a hive mind actually. Uh, that maybe it's the... so, so there's a, so we don't think we should take for granted at all that AGI will have to have any of those sort of competitive, I-sa alpha male instincts.

Lex Fridman 21:07 Right.

Max Tegmark 21:07 On the other hand, you know, this is really interesting because I think some people go too far and say, "Of course, we don't have to have any concerns either that advanced AI will have those instincts because we can build anything we want." That there's, there's a very nice set of arguments going back to Steve Omohundro and Nick Bostrom and others just pointing out that when we build machines, we normally build them with some kind of goal, you know?

Lex Fridman 21:34 Mm-hmm.

Max Tegmark 21:34 Win this chess game, drive this car safely, or whatever. And as soon as you put in a goal into machine, eh, especially if it's a kind of open-ended goal and the machine is very intelligent, it'll break that down into a bunch of sub-goals.

Lex Fridman 21:46 Mm-hmm.

Max Tegmark 21:48 And, um, one of those goals will almost always be self-preservation 'cause if it breaks or dies in the process, it's not gonna accomplish the goal, right? Like-

Lex Fridman 21:56 Right.

Max Tegmark 21:56 ... suppose you just build a little, you have a little robot and you tell it to go down to Star Market here and, and, and get you some food, make you, cook you an Italian dinner, you know, and then someone mugs it and tries to break it on the way.

Lex Fridman 22:08 Mm-hmm.

Max Tegmark 22:09 That robot has an incentive to def- to not get destroyed and defend itself or run away because otherwise, it's gonna fail in cooking you dinner. It- it's not afraid of death, but it really wants to complete the dinner cooking goal, so it will have a self-preservation instinct that-

Lex Fridman 22:24 Yeah, continue being a functional agent in the world somehow.

Max Tegmark 22:27 Yeah, e- and, and, uh, and similarly if you give a, any kind of more ambitious goal to an AGI, it's very likely they'll wanna acquire more resources so it can do that better.

Lex Fridman 22:39 Mm-hmm.

Max Tegmark 22:39 And it's exactly th- from those sort of sub-goals that we might not have intended that, that some of the concerns about AGI safety come.

Lex Fridman 22:46 Mm-hmm.

Max Tegmark 22:47 You give it some goal which seems completely harmless and then before you realize it, it's also trying to do these other things which you didn't want it to do, and it's mo- maybe smarter than us. (laughs) So, it's fascinating.

Lex Fridman 23:00 So, and let me pause just because, um, I am, uh, in a very kind of human-centric way see fear of death as a valuable motivator.

Max Tegmark 23:10 Mm-hmm. Uh-huh.

Lex Fridman 23:11 Um, so you don't think... uh, s- do you think that's an artifact of evolution, so that's the kind of mind space evolution created, that we're sort of almost obsessed about self-preservation-

Max Tegmark 23:23 Yeah.

Lex Fridman 23:23 ... on some kind of genetic level? You don't think that's necessary to be r- afraid of death? So not just a, a kind of sub-goal of self-preservation just so you can keep doing the thing, but more, uh, fundamentally sort of have the finite thing, like, "This ends for you at some point."

Max Tegmark 23:43 Interesting. Do, do I think it's necessary for what precisely?

Lex Fridman 23:47 For intelligence, but also for consciousness. So for those, for both. Do you think really, like, a finite death and the fear of it is important?

Max Tegmark 24:01 So, before I can answer wh- w- before we can agree on whether it's necessary for intelligence or for consciousness, we should be clear on how we define those two words 'cause-

Lex Fridman 24:10 Sure.

Max Tegmark 24:10 ... a lot of really smart people define them in very different ways.

Lex Fridman 24:12 Right.

Max Tegmark 24:13 I was in this, on this panel a-... and with AI experts, and they couldn't, they couldn't agree on how to define intelligence even. So, I, I, I define intelligence simply as the ability to accomplish complex goals. I like your broad definition, because again, I don't wanna be a carbon chauvinist.

Lex Fridman 24:29 Right.

Max Tegmark 24:30 And, uh, in that case, no, it cer- cer- certainly doesn't require fear of death. I w- I would say AlphaGo or AlphaZero is quite intelligent.

Lex Fridman 24:40 Right.

Max Tegmark 24:40 I don't think AlphaZero has any fear of being turned off, because it doesn't understand the concept of, of it, even. And, u- and similarly, consciousness, I mean, you, you could certainly imagine, um, ex- v- very simple kind of experience if, if, if, I mean, if certain plants have any kind of experience, I don't think they're very afraid-

Lex Fridman 24:58 (laughs)

Max Tegmark 24:58 ... of dying 'cause there's nothing they can do about it anyway much, so, so there wasn't that much value in... But mo- more seriously, I, I think, uh, if you ask not just about being conscious but maybe having, uh, what you would, we, we would, we might call an exciting life-

Lex Fridman 25:15 Right.

Max Tegmark 25:15 ... where you feel passion and-

Lex Fridman 25:16 That's a good-

Max Tegmark 25:16 ... and, and really appreciate the, the things. Maybe there, somehow, maybe there perhaps it does help having a, having that backdrop that, hey, it's finite, you know? Let- let's, let's make the most of this. Let's live to the fullest.

Lex Fridman 25:31 So-

Max Tegmark 25:31 I mean, if you, if y- if you knew you were gonna just live forever, do you feel or think you would change your...

Lex Fridman 25:37 Yeah, I mean, in some perspective, it, it would be an incredibly boring life, living forever. So, in the sort of loose, subjective terms that you said of something exciting and something in this that other humans would understand, I think, is, yeah, it seems that the f- the finiteness of it is important.

Max Tegmark 25:57 Well, the good news I have for you then is, based on what we understand about cosmology, (laughs) everything is, in our universe, is pro- ultimately probably finite. Although, although-

Lex Fridman 26:08 Big Crunch or bi- uh, or big, uh, what's the expa- the infinite expansion?

Max Tegmark 26:11 Yeah, we could have a Big Chill or a-

Lex Fridman 26:13 Big Chill.

Max Tegmark 26:13 ... Big Crunch or a Big Rip or Death, s- the Big Snap or Death Bubbles.

Lex Fridman 26:18 (laughs)

Max Tegmark 26:18 All of them are more than a billion years away, so it, we, we should, we certainly have vastly more time than our ancestors thought. But, uh, still, still pretty hard to squeeze in an infinite number of compute cycles, even though there are some loophole that just might be possible. But I, I think I... You know, some people like to say that you should live as if you're about to, you're gonna die in five years or so-

Lex Fridman 26:46 Mm-hmm.

Max Tegmark 26:46 ... and that's sort of optimal. Maybe we... It's a good assumption. We should build our civilization as if (laughs) it's all finite to be on the safe side.

Lex Fridman 26:55 Right, exactly. Uh, so y- you mentioned in, in defining intelligence as, uh, the ability to solve complex goals, so where would you draw a line, how would you try to define human-level intelligence and-

Max Tegmark 27:08 Mm.

Lex Fridman 27:09 ... superhuman-level intelligence?

Max Tegmark 27:10 Mm-hmm.

Lex Fridman 27:10 Where... Is consciousness part of that definition?

Max Tegmark 27:13 No. Consciousness does, does not come into this definition. So, so I think of intelligence as, as a spectrum, that there are very many different kinds of goals you can have. You can have a goal to be a good chess player, a good Go player, a good car driver, a good investor, good poet, et cetera. So, intelligence that by, by its very nature isn't something you can measure by just one number as overall goodness. No, no. There are some people who are more, better at this, some people who are better at that. Um, right now, we have machines that are much better than us at some very narrow tasks, like multiplying large numbers fast-

Lex Fridman 27:49 Mm-hmm.

Max Tegmark 27:50 ... memorizing large databases, uh, playing chess, playing Go, and soon driving cars. Um, but there's still no machine that can match a human child i- in general intelligence. But, but artificial general intelligence, AGI, the name of your course-

Lex Fridman 28:06 Mm-hmm.

Max Tegmark 28:07 ... of course, that is by its very definition the, the quest to build a machi- machine that can do everything as well as we can up, up to the old Holy Grail of, of AI from, from back to its inception in the, in the '60s. If that ever happens, of course, I, I think it's gonna be the biggest transition in the history of, of life on Earth. But it, but it doesn't necessarily have to wait, the big impact, right, until machines are better than us at knitting.

Lex Fridman 28:35 Mm-hmm.

Max Tegmark 28:35 The, the really big change doesn't come exactly the moment they're better than us at everything. The really big change comes... Well, first there are big changes when they start becoming better at, as at doing most of the jobs that we do-

Lex Fridman 28:48 Mm-hmm.

Max Tegmark 28:48 ... because that ta- (clears throat) takes away much of the demand for, for human labor. And then the really whopping change comes when they become better than us at AI research.

Lex Fridman 29:00 Right.

Max Tegmark 29:00 Right? Because right now, the timescale of AI research is limited by the human research and development cycle of, of years, typically, you know?

Lex Fridman 29:10 Mm-hmm.

Max Tegmark 29:10 How long does it take from one release of, of some software or iPhone or whatever to the next? But once, once we ha- once Google can replace 40,000 engineers by 40,000 equivalent pieces of software or whatever, by then, th- that doesn't, there's no reason that has to be years. It can be in principle much faster, and, uh, the timescale of future progress in AI and als- all of science and technology will, will be driven by machines, not humans. So, it, it's this point, simple point which lives, gives rise to this incredibly fun controversy about whether there can be intelligence explosion, so-called singularities, Vernor Vinge called it. Now, the idea as articulated by A. J. Good is obviously way back '50s, but you can see Alan Turing and others thought about it even earlier. Uh, so you asked me what exactly would I define human-level at.

Lex Fridman 30:11 Yeah, human-level intelligence. Yeah.

Max Tegmark 30:12 Yeah. So, this, the, the glib answer is just to say something which is better than us at all-

Lex Fridman 30:17 (laughs)

Max Tegmark 30:17 ... cognitive tasks, well, the c-... or better than any human at all cognitive tasks. But the really interesting bar, I think, goes a little bit lower than that, actually. It's when they can, when they're better than us at AI programming and, and, and at general learning, so that they can, can, if they want to, get better than us at anything by just studying up.

Lex Fridman 30:37 So there better is a key word, and better is towards this kind of spectrum of the complexity of goals-

Max Tegmark 30:44 Yeah.

Lex Fridman 30:44 ... it's able to accomplish.

Max Tegmark 30:45 Yeah.

Lex Fridman 30:45 So another way to th- uh, so ano- an- and that's certainly a very, uh, clear definition of human love, so there's, it's almost like a sea that's rising, and you can do more and more and more things.

Max Tegmark 30:56 Yeah.

Lex Fridman 30:56 It's actually a graphic that you show. It's a really nice way to put it. So there's some peaks that, and there's an ocean level elevating, and you solve more and more problems. But, you know, just kind of, uh, to take a pause, and we took a bunch of questions in a lot of social networks, and a bunch of people asked a sort of, a slightly different direction on creativity. And um-

Max Tegmark 31:16 Mm-hmm.

Lex Fridman 31:17 ... and, and things l- that perhaps aren't a peak. Uh, uh, it, it, it's, you know, human beings are flawed, and perhaps better means having, being a, uh, having contradiction, being flawed in some way. So let me sort of-

Max Tegmark 31:31 Yeah.

Lex Fridman 31:31 ... start and s- start easy, first of all. Uh, so you have a lot of cool equations. Let me ask, what's your favorite equation, first of all? You, I know they're all like your children, but like-

Max Tegmark 31:41 That one.

Lex Fridman 31:41 (laughs) Which one is that?

Max Tegmark 31:43 This is the Schrodinger Equation.

Lex Fridman 31:45 Okay, sure.

Max Tegmark 31:45 It's the master key of quantum mechanics of the micro world. So this, with this equation, we can calculate everything to do with atoms and molecules and all the way up to...

Lex Fridman 31:56 (laughs)

Max Tegmark 31:58 ... stuff.

Lex Fridman 31:58 Yeah, so okay. So quantum mechanics is certainly a, a beautiful, mysterious formulation of our world. So I'd like to sort of ask you, uh, just as an example, it perhaps doesn't have the same beauty as physics does, but in mathematics, uh, abstract, the Andrew Wiles who proved the Fermat's Last Theorem-

Max Tegmark 32:18 Yeah.

Lex Fridman 32:19 So uh, he, I just saw this recently, and it, it kinda caught my eye a little bit. This is 358 years after it was c- it was conjectured. So this very simple formulation, everybody tried to prove it. Everybody failed. And so here's this guy, comes along, and eventually, f- uh, it proves it, and then, then fails to prove it, and then proves it again in '94, and he said like, the moment when everything connected into place, uh, in an interview said, "It was so indescribably beautiful," that moment when he finally realized the connecting piece, uh, of two c- conjectures. He said, "It was so indescribably beautiful. It was so simple and so elegant. I couldn't understand how I'd missed it, and I just stared at it in this disbelief for 20 minutes."

Max Tegmark 33:02 (laughs)

Lex Fridman 33:02 "Then, then during the day, I walked around the department, and I'd keeping, keep coming back to my desk, looking to see if it was still there. It was still there. I couldn't contain myself. I was so excited. It was the most important moment of my working life. Nothing I ever do again will mean as much." So that particular moment, and, and it kinda made me think of-

Max Tegmark 33:22 Beautiful.

Lex Fridman 33:23 ... what would it take... And I think you p- we have all been there at small levels. Um, maybe, let me ask, have you had a moment like that in your life, where you just had an idea that's like, "I, wow, yes"?

Max Tegmark 33:40 I wouldn't mention myself in, uh, in the same breath as Andrew Wiles-

Lex Fridman 33:43 (laughs)

Max Tegmark 33:43 ... but I, I've certainly had a number of, of aha moments wh- when I just realize something very cool about physics, just has completely made my head explode. In fact, some of my favorite discoveries I made late- I later realized that they had been discovered earlier-

Lex Fridman 34:00 Mm. Right.

Max Tegmark 34:01 ... by someone who sometimes got quite famous for it. So I, I, and it's too late for me to even publish it, but that doesn't diminish in any way-

Lex Fridman 34:07 The moment.

Max Tegmark 34:07 ... you know, the emotional experience you have when you realize it, like-

Lex Fridman 34:10 Yeah.

Max Tegmark 34:10 ... "Wow."

Lex Fridman 34:11 Yeah. So what would it take in that, that moment, that wow, that was yours in that moment, so what do you think it takes for an intelligent system, an AGI system, an AI system, to have a moment like that? Like-

Max Tegmark 34:25 It's a tricky question, 'cause there're actually two parts to it, right? One of them is, can it accomplish that proof? Can it prove-

Lex Fridman 34:34 Yeah.

Max Tegmark 34:34 ... that you can never write A to the n plus B to the n equals three to the, uh, equals z-

Lex Fridman 34:41 z to the n, yeah.

Max Tegmark 34:42 ... to the n for all integers when et cetera, et cetera, when, when n is bigger than two? That, that's simply in the question about intelligence. Can you build machines that are that intelligent? And I, I think by the time we get a machine that can independently come up with that level of proofs, probably quite close to AGI. Uh, the s- second question is a question about consciousness, of w- when will we, w- will, will and, how likely is it that such a machine would actually have any experience at all, as opposed to just being like a zombie? And would we expect it to have some sort of emotional response to this? Uh, or anything a- at all akin to human emotion, where, you know, when it accomplishes its machine goal, it, it, it, uh, views it as somehow something very positive and-

Lex Fridman 35:32 Right.

Max Tegmark 35:32 ... and, and sublime and, and, and, and deeply meaningful? I would certainly hope that if, in the future, we do create machines that are our peers, or even our descendants and s-

Lex Fridman 35:48 Yeah.

Max Tegmark 35:49 But I would certainly hope that they do have this sort of sublime, sublime appreciation o- of life. In, in a way, my, my absolutely worst nightmare would be that in the, at some point in the future, uh, the distant future, maybe our cosmos is teeming with all this post-biological life doing all this seemingly cool stuff, and maybe the f- last humans, by the time our, our species eventually fizzles out, we'll be like, "Oh, well, that's okay, 'cause we're so proud of our descendants here, and look at what all they..."... my worst nightmare is that the, we haven't solved the consciousness problem-

Lex Fridman 36:30 Mm-hmm.

Max Tegmark 36:30 ... and we haven't realized that these are all the zombies.

Lex Fridman 36:32 (laughs)

Max Tegmark 36:32 They're not aware of anything any more than-

Lex Fridman 36:35 Yeah.

Max Tegmark 36:35 ... a tape recorder that hasn't had any kind of experience, so it, the whole thing has just become a play for empty benches. That would be like (laughs) -

Lex Fridman 36:42 Yeah.

Max Tegmark 36:42 ... the ultimate zombie apocalypse. I mean, I, I, I, I would much rather it in that case that we have these beings which, which can really appreciate how the, the, how amazing it is ?

Narrator 36:57 And in that picture-

Max Tegmark 36:58 To be alive.

Narrator 36:58 ... what would be the role of creativity? We've had a few people ask about creativity.

Max Tegmark 37:03 Yeah.

Narrator 37:04 Do you think when you think about intelligence, I mean, certainly the, uh, the story you told at the beginning of your book involved, you know, creating movies and so on.

Max Tegmark 37:11 Yeah.

Narrator 37:12 Sort of making-

Max Tegmark 37:12 (laughs)

Lex Fridman 37:13 (laughs)

Max Tegmark 37:13 ... (laughs) making money. You know, you can make a lot of money in our modern world with music and movies. So if you are an intelligent system, you may want to get good at that.

Lex Fridman 37:22 Yeah.

Max Tegmark 37:23 But that's not necessarily what, what I mean by creativity. Is it important on that complex goals where the sea is rising for there to be something creat- creative or are, am I being very human-centric and thinking creativity as somehow special, uh, relative to intelligence? My hunch is that, okay, we should think of creativity simply as an aspect of intelligence and, uh, we, we have to be very careful with, with human vanity.

Narrator 37:57 Right.

Max Tegmark 37:57 We, we, we have this tendency to very often want to say... As soon as machines can do something, we try to diminish it and say, "Oh, but that's not like real intelligence," you know?

Narrator 38:05 Right.

Max Tegmark 38:05 Because they're not creative or they're w- or this or that-

Narrator 38:08 Right.

Max Tegmark 38:08 ... or the other thing. Um, maybe if we ask ourselves to write down a definition of what we actually mean by being creative, what we mean by Andrew Wiles, what he did there, for example-

Lex Fridman 38:19 Mm-hmm.

Max Tegmark 38:19 ... don't we often mean that someone takes a very unexpected leap?

Lex Fridman 38:24 Mm-hmm.

Max Tegmark 38:26 It's not like taking f- 573 and multiplying it by 224 by just a step of straightforward cookbook-like rules, right?

Lex Fridman 38:34 Mm-hmm.

Max Tegmark 38:35 It is... You can maybe make a c- s- t- ... You make a connection between two things-

Lex Fridman 38:39 Yeah.

Max Tegmark 38:39 ... that people had never thought was connected-

Lex Fridman 38:41 It's very surprising somehow.

Max Tegmark 38:42 ... or something like that. Uh, I think, I think this is an aspect of intelligence, uh, uh, and, uh, this is some, actually one of the most important aspects of it. Maybe the reason we humans are tend to be better at it than, uh, traditional computers is because it's something that comes more naturally if you're a neural network-

Lex Fridman 39:00 Mm-hmm.

Max Tegmark 39:01 ... than if you're a traditional, um, logic gate-based computer machine. You know, we physically have all those connections and, uh, if you activate here, activate here, activate here, chh, chh, ping! You know, uh, uh, um, I... My, my hunch is that if we ever build a machine, well, you could just give it a task, say, "Hey, uh, I... You, you say, hey, you know, I, I just realized I have... I want to travel around the world instead this month. Can you teach my A- AGI course for me?"

Lex Fridman 39:34 Mm-hmm.

Max Tegmark 39:34 And it's like, "Okay, I'll do it," and it does everything that you would have done and improvises and stuff.

Lex Fridman 39:39 Yeah.

Max Tegmark 39:39 That, that would, in my mind, involve a lot of creativity.

Lex Fridman 39:43 Yeah, so i- it's actually a beautiful way to put it. I think we do try to grab, grasp at the... You know, the, the d- the definition of intelligence is everything we don't understand how, uh, how to build. So like, so we as humans try to find things, well, that we have and machines don't have, and maybe creativity is just one of the things, one of the words we use to describe that. That's a really interesting way to put it.

Max Tegmark 40:07 And I don't think we need to be that defensive. I don't think-

Lex Fridman 40:10 (laughs)

Max Tegmark 40:10 ... anything good comes out of saying, "Oh, we're somehow special," you know?

Lex Fridman 40:13 Right.

Max Tegmark 40:13 It, it, it's, um... Contrariwise, there are many examples in history of where trying to pretend that we're s- somehow superior to all other intelligent beings has led to pretty bad results, right? Uh, in Nazi Germany, they said that they were somehow superior to other people. Uh, today we still, we do a lot of cruelty to animals by saying that we're so superior somehow and th- they can't feel pain. Uh, slavery was justified by the same kind of just really weak, weak arguments and, and, and, uh... I don't think... If, if we actually go ahead and build artificial general intelligence that can do things better than us, I don't think we should try to found our self-worth on some sort of bogus claims of, of superiority in, in terms of our intelligence. I, I think it's, we should instead ju- find our, our calling and, and th- and, and the meaning of life from, from experiences that we have.

Lex Fridman 41:22 Right.

Max Tegmark 41:22 You know? I can have, I can have very meaningful experiences even if there are other people who are smarter than me, you know? When I go to a faculty meeting here and I's a-

Lex Fridman 41:35 (laughs)

Max Tegmark 41:35 ... we're talking about something and then I suddenly realize, "Oh, well, (laughs) he has a Nobel Prize, he has a Nobel Prize, he has-

Lex Fridman 41:39 Yeah.

Max Tegmark 41:39 ... a Nobel Prize. I don't have one." Does that m- make me enjoy life any less or en- enjoy talking to those people less? Of course not.

Lex Fridman 41:48 Right.

Max Tegmark 41:49 A- and contrariwise, I, I, it's a- I feel very pl- honored and privileged to get to interact with, with, uh, other very intelligent beings that are better than me at, at, a lot of stuff. So I don't think there's any reason why we can't have the same approach with, with intelligent machines.

Lex Fridman 42:06 That's a really interesting... So people don't often think about that. They think about when there's going... If, if there's machines that are more intelligent, you naturally think that that's not going to be, um, a beneficial type of intelligence. You don't realize it could be, you know, like peers with Nobel Prizes that, that would be just fun to talk with, and they might be clever about certain topics and, uh, you can have fun having a few drinks with them. So... (laughs)

Max Tegmark 42:32 Well, a- also, you know, w-... another example we sh- we can all relate to i- uh, of why it doesn't have to be a terrible thing to be in pres- the presence of people who are even smarter than us all around is when, when you and I were both two years old, I mean, our parents were much more intelligent than us, right?

Lex Fridman 42:48 Yeah.

Max Tegmark 42:49 Worked out okay.

Lex Fridman 42:50 Yep. (laughs)

Max Tegmark 42:50 S- because their goals were aligned with our goals.

Lex Fridman 42:53 Yeah.

Max Tegmark 42:53 And that, I think, is really the number one key issue we have to solve if we-

Lex Fridman 43:00 Value align.

Max Tegmark 43:00 ... value align, the value alignment problem, exactly, 'cause people who see too many Hollywood movies with, with lousy science fiction, uh, plot lines, they worry about the wrong thing, right? They worry about some machine suddenly turning evil. It's not malice that we sh- that's the iss- the pro- the concern, it's competence, right? By definition, intelligence makes you, makes you very competent. If you have a, a more intelligent Go playing machine, computer playing against a less intelligent one, and, and when we define intelligence as a c- ability to accomplish Go winning, right?

Lex Fridman 43:37 Mm-hmm.

Max Tegmark 43:38 It's gonna be the more intelligent one that wins.

Lex Fridman 43:40 Right.

Max Tegmark 43:40 And if you have a human, and then you have, um, an AGI that's more intelligent in all ways, and they have different goals, guess who's gonna get their way, right? So I was just reading about, I was just reading about this, um, particular rhinoceros-

Lex Fridman 43:56 (laughs)

Max Tegmark 43:56 ... species that was driven extinct just a few years ago.

Lex Fridman 43:59 Yes.

Max Tegmark 43:59 You know, a bummer. I was looking at this cute picture of a mommy rhinoceros with its, its child, you know? And why did we humans drive it to extinction? Wasn't because we were evil rhino haters-

Lex Fridman 44:11 Right.

Max Tegmark 44:12 ... as a whole. It was just because we, our goals weren't aligned with those of the rhinoceros, and it didn't work out so well for the rhinoceros 'cause we were more intelligent, right?

Lex Fridman 44:19 Mm-hmm.

Max Tegmark 44:19 So I, I think it's just so important that if we ever do build AGI, before we unleash anything, we have to make sure that it, it learns to understand our goals, uh, that it adopts our goals, and that it retains those goals. Mm-hmm.

Lex Fridman 44:38 So the cool, interesting problem there is being able... us as human beings trying to formulate our values. So, you know, you can think of the United States Constitution as a, uh, as a way that people sat down, at the time, a bunch of white men, but-

Max Tegmark 44:56 Yeah.

Lex Fridman 44:56 ... which is a good example I should, w- that we should say.

Max Tegmark 44:59 Yeah.

Lex Fridman 44:59 Uh, they formulated the goals for this country, and a lot of people agree that those goals actually held up pretty well. And it's an interesting formulation of values and failed miserably in other ways. So, for the value alignment problem-

Max Tegmark 45:12 Yeah.

Lex Fridman 45:12 ... and the solution to it, we have to be able to put on paper, uh, or in, in, uh, in a program human values.

Max Tegmark 45:20 Yeah.

Lex Fridman 45:20 How difficult do you think that is?

Max Tegmark 45:22 Very. But i- i- it's so important. We really have to give it our best. And it's difficult for two separate reasons. There's the technical value alignment problem of figuring out just how to make machines understand our goals, adopt them, and retain them, and then there's this c- the separate part of it, the philosophical part, whose values anyway? And since we, it's not like we have any great consensus on this planet on values, a- a- how, what mechanism should we create then to aggregate and decide, okay, what's a good compromise?

Lex Fridman 45:55 Right.

Max Tegmark 45:56 Uh, that second discussion can't just be left to tech nerds like myself, right?

Lex Fridman 46:01 That's right.

Max Tegmark 46:01 And, and if we refuse to talk about it, and then AGI gets built, who's gonna be actually making the decision about these values? It's gonna be a bunch of dudes in some tech company, right?

Lex Fridman 46:11 Yeah. Yeah.

Max Tegmark 46:12 And, uh, are, are they necessarily... (laughs) s- so representative of all humankind that we want to just entrust it to them? Are, are they even, uh, uniquely qualified-

Lex Fridman 46:22 Mm-hmm.

Max Tegmark 46:22 ... to speak to future human happiness just 'cause they're good at programming an AI? I'd much rather have this be a really inclusive conversation.

Lex Fridman 46:30 But do you think it's possible, sort of, th- so you create a, a beautiful vision that includes, uh, sort of the diversity, cultural diversity, and f- various perspectives when discussing rights, freedoms, human dignity, but how hard is it to come to that consensus? Do you think, um, it's certainly a really important, uh, thing that we should all try to do, but do you think it's feasible?

Max Tegmark 46:54 I, I think there's no better way to guarantee failure than to-

Lex Fridman 46:59 Not to try.

Max Tegmark 47:00 ... e- either to refuse to talk about it or, or refuse to try.

Lex Fridman 47:02 Yeah.

Max Tegmark 47:02 And, uh, I also think it's a really bad i- strategy to say, "Okay, let's first have a discussion for a long time, and then once we reach complete consensus, then we'll-"

Lex Fridman 47:11 Right.

Max Tegmark 47:11 "... try to load it into some machine." No, uh, we shouldn't let perfect be the enemy of good. Instead, we should start with the kindergarten ethics that pretty much everybody agrees on and put that into our machines now. We're not doing that even.

Lex Fridman 47:25 Mm-hmm.

Max Tegmark 47:26 Look at, uh, you know, uh, anyone who builds a, uh, passenger aircraft wants it to never, under any circumstances, fly into a building-

Lex Fridman 47:34 Right.

Max Tegmark 47:34 ... or a mountain, right? Yet the September 11 hijackers were able to do that. And even more embarrassingly, you know, uh, Andreas Lubitz, this depressed Germanwings pilot-

Lex Fridman 47:43 Mm-hmm.

Max Tegmark 47:44 ... when he flew his passenger jet into the Alps, killing over 100 people, he just told the autopilot to do it. He told the freaking computer-

Lex Fridman 47:52 Yeah.

Max Tegmark 47:52 ... to change the altitude to 100 meters.

Lex Fridman 47:54 Yeah.

Max Tegmark 47:55 And even though it had the GPS maps, everything, the computer was like, "Okay."

Lex Fridman 48:00 (laughs)

Max Tegmark 48:00 You know? So we sh- we should take those very basic values that w- where the problem is not that we don't agree, that... I- it- the problem is just we've been too lazy to try to put it into our machines and make sure that from now on, air- airplanes will just... which all have computers in them, but we'll just never, just refuse to do something like that-

Lex Fridman 48:19 Mm-hmm.

Max Tegmark 48:19 ... go into safe mode, maybe lock the cockpit door going to the nearest airport. And, and there's so much other technology in, in our world as well now, where it's really quite b- you know, coming quite timely to put in some sort of very basic values like this. Even in cars.

Lex Fridman 48:35 Yeah.

Max Tegmark 48:35 We were have- we've had enough, uh, vehicle terrorism attacks by now if you'll have driven trucks-... and vans into pedestrians, that is not at all a crazy idea to just have that hardwired into the car.

Lex Fridman 48:48 Right.

Max Tegmark 48:48 'Cause yeah, there are a lot of... There's always gonna be people who, for some reason, wanna harm others, but most of those people don't have the technical expertise to figure out how to work around-

Lex Fridman 48:58 Mm-hmm.

Max Tegmark 48:58 ... something like that. So, if the car just won't do it, it helps. So let's-

Lex Fridman 49:02 Yeah.

Max Tegmark 49:02 ... let's start there.

Lex Fridman 49:02 So, there's a lotta... That's a g- that's a great point, so not, uh, not chasing perfect. There's a lotta things-

Max Tegmark 49:08 Yeah.

Lex Fridman 49:08 ... that a lot, that most of the world agrees on.

Max Tegmark 49:10 Yeah, let's start there.

Lex Fridman 49:11 Uh, we'll- let's start there.

Max Tegmark 49:12 And then, uh, and then once we start there, we'll also get into the habit of, of having these kind of conversations about, "Okay, what else should we put in here and how... have these discussions?"

Lex Fridman 49:21 Yeah.

Max Tegmark 49:21 This should be a gradual process, then.

Lex Fridman 49:24 Great. So, but, um, that also means describing these things and describing it to a machine, so one thing... We had a few conversations with Stephen Wolfram. I'm not sure if you're familiar with Stephen Wolfram.

Max Tegmark 49:37 Oh, yeah. I know him quite well. Yeah.

Lex Fridman 49:38 So, he is, you know, he play, uh, you know, works with a bunch of things but, you know, cellular automata, these simple computable things, these computation systems. And he kinda mentioned that, you know, we probably have already, within these systems, already something that's AGI, w- uh, meaning like, we just don't know it because we can't talk to it. So, uh, if, if you give me this, uh, chance to try, (laughs) to try to at least form a question outta this, is, I think, uh, it's an interesting idea to th- think that we can have intelligent systems but we don't know how to describe something to them and they can't communicate with us. I know you're doing a little bit of work in explainable AI, trying to get AI to explain itself.

Max Tegmark 50:22 Mm-hmm.

Lex Fridman 50:23 So, what are your thoughts of natural language processing or some kind of other communication? How, how does the AI explain something to us? How do we explain something to it, to machines? Or are you thinking of it differently?

Max Tegmark 50:35 So, there are two separate parts to your question there, the...

Lex Fridman 50:40 (laughs)

Max Tegmark 50:40 One of them has to do with communication, which is super interesting and I'll get-

Lex Fridman 50:43 Right.

Max Tegmark 50:43 ... to that in a sec. The, the, the other is whether we already have AGI but we just haven't noticed it yet.

Lex Fridman 50:49 (laughs) Right.

Max Tegmark 50:49 Um, there I beg to differ. I, I don't think there's anything in any cellular automaton or anything, or the internet itself or whatever, that has artificial general intelligence in that it can really do exactly everything we humans can do better.

Lex Fridman 51:07 Right.

Max Tegmark 51:07 I, I think the day, uh, the day that happens, (laughs) when that happens, we will very soon notice. We'll probably notice even before, uh, because it, in a very, very big way. Uh, but the, for the, for the second part though-

Lex Fridman 51:18 Wait. Can I ask, can, can I, can I-

Max Tegmark 51:20 Yeah.

Lex Fridman 51:20 ... answer this? Sorry. So, uh, the, 'cause you, you have this beautiful way to formulating consciousness as, uh, uh, uh, as, um, you know, as information processing, and you can think of intelligence as information processing, and there's... You can think of the entire universe as these particles and these systems roaming around that have this information processing power. You, you don't, you don't think there is something with the power to process information in the way that we human beings do that's out there that, um, that needs to be sort of connected to? It seems a little bit philosophical perhaps, but there's something compelling to the idea that the power is already there, would you-

Max Tegmark 52:02 Yeah.

Lex Fridman 52:02 ... it's, the, the focus should be more on the ex- on being able to communicate with it.

Max Tegmark 52:06 Hmm. Well, I agree that the... In some, in a certain sense, the hardware processing power is, is already out there 'cause our universe itself can think of it as being a computer already, right? It's constantly computing what water waves, how it's evolved, the water waves in the River Charles and how to move the air molecules around, and-

Lex Fridman 52:28 Right.

Max Tegmark 52:28 ... Seth Lloyd has pointed out, my colleague here, that, uh, you can even in a very rigorous way think of our entire universe as just being a quantum computer.

Lex Fridman 52:35 Mm-hmm.

Max Tegmark 52:35 It's pretty clear that our universe supports this amazing processing power because you can even, it... Within this physics computer that we live in, right, we can even build actual laptops and stuff. So, clearly, the power is there. It's just that most of the compute power that nature has, it's, in my opinion, kinda wasting on boring stuff like simulating yet another ocean wave somewhere where no one is-

Lex Fridman 52:57 Right.

Max Tegmark 52:58 ... even looking, right? So, in a sense, what, what life does, what we are doing when we build computers is we're rechanneling all this compute that nature is doing anyway-

Lex Fridman 53:07 Right.

Max Tegmark 53:07 ... into doing things that are more interesting than just yet another ocean wave, you know?

Lex Fridman 53:11 Yeah.

Max Tegmark 53:11 And let, let's do something cool here. Uh, so the raw hardware power is there, uh, for sure, but, and, and, and even just, like, computing what's gonna happen for the next five seconds in this water ball-

Lex Fridman 53:23 Yes. That. Yeah.

Max Tegmark 53:23 ... you know, it takes a ridiculous amount of, of compute if you do it on a human computer.

Lex Fridman 53:27 Yeah.

Max Tegmark 53:28 This water ball just did it.

Lex Fridman 53:30 (laughs)

Max Tegmark 53:30 But that does not mean that this water ball has AGI and this... Because AGI means it should also be about the... Like, I've written my book, done-

Lex Fridman 53:38 Yeah.

Max Tegmark 53:38 ... this interview.

Lex Fridman 53:39 Yes.

Max Tegmark 53:40 And I don't think-

Lex Fridman 53:40 Okay.

Max Tegmark 53:40 ... it's just communication problems. I don't really-

Lex Fridman 53:42 As far as we know. Yeah.

Max Tegmark 53:43 ... don't think it... (laughs) It can do it and, and, uh-

Lex Fridman 53:46 Although Buddhists say when they watch the water and that, that there is some beauty, that there is some depth and beauty-

Max Tegmark 53:53 Yeah.

Lex Fridman 53:53 ... in nature that they can communicate with.

Max Tegmark 53:54 Communication is also very important though-

Lex Fridman 53:56 Yes.

Max Tegmark 53:56 ... because, I mean, look, um, uh, part of my job is being a teacher. And I know some very intelligent professors even who just have a bit of a hard time communicating.

Lex Fridman 54:09 Mm-hmm.

Max Tegmark 54:09 They-

Lex Fridman 54:10 Absolutely.

Max Tegmark 54:10 ... come up with all these brilliant ideas but, but to communicate with somebody else, you have to also be able to simulate their own mind.

Lex Fridman 54:17 Yes. Empathy and that kind of thing.

Max Tegmark 54:18 And build well enough an under- model of their mind that you can say things that they will understand. And that's quite difficult.

Lex Fridman 54:25 Mm-hmm.

Max Tegmark 54:26 Right? That's why today it's so frustrating if, if you have a computer that makes some cancer diagnosis and you ask it, "Well, why are you saying I should have a surgery if it..." And if it, uh, can only reply, "I was trained on five terabytes of data and this is my-... diagnosis, boop, boop, beep, beep.

Lex Fridman 54:45 Yeah.

Max Tegmark 54:45 Uh, uh, uh, doesn't really instill a lot of confidence, right?

Lex Fridman 54:48 Right.

Max Tegmark 54:49 So, I think we have a lot of work to do on, um, on communication there.

Lex Fridman 54:54 So what kind of s- what kind of, um... I think you're doing a little work in explainable AI-

Max Tegmark 54:58 Yeah.

Lex Fridman 54:59 What do you think are the most promising avenues? Is it mostly about, sort of, the Alexa problem of natural language processing, of being able to actually use human interpretable methods of communication, so being able to talk to a system and it talk back to you, or is there some more fundamental problems to be solved?

Max Tegmark 55:18 I think it's all of the above. Uh, y- h- human, the natural language processing is obviously important, but there are also more nerdy, fundamental problems, like if you, if you take... uh, uh, you play chess?

Lex Fridman 55:31 Mm-hmm. Of course, I'm Russian.

Max Tegmark 55:33 Or you watch?

Lex Fridman 55:33 I have to.

Max Tegmark 55:33 (Russian) .

Lex Fridman 55:34 (Russian) .

Max Tegmark 55:35 (Russian) .

Lex Fridman 55:35 (laughs) When did you learn Russian?

Max Tegmark 55:39 (Russian) .

Lex Fridman 55:39 Wow.

Max Tegmark 55:39 (Russian) . But I would just-

Lex Fridman 55:50 How many languages do you know? Wow. That's really impressive.

Max Tegmark 55:53 (laughs) I don't know, man.

Lex Fridman 55:54 (laughs)

Max Tegmark 55:54 My wife has some count- calculation... But, uh, my point was if you play chess, like, have you looked at the AlphaZero games, the-

Lex Fridman 56:00 Oh, the actual games? No.

Max Tegmark 56:02 Just s- t- check it out. Some of them are just mind-blowing, uh, uh, really beautiful. And, and if you ask, "How did it do that?" You go talk to Demis Hassabis and others from DeepMind, all they'll ultimately be able to give you is, uh, big tables of numbers, matrices-

Lex Fridman 56:23 Yeah.

Max Tegmark 56:24 ... that define the neural network. And you can stare at these num- tables, numbers till your face turn blue-

Lex Fridman 56:29 Right.

Max Tegmark 56:29 ... and, uh, you're not gonna understand much about why it made that move. And, uh, even if you have natural language processing that can tell you in human language about, "Oh, five, seven, point two eight," still not gonna really help. So, so I think, think there's a whole spectrum of, of, of fun challenges there involved in, in taking a computation that does intelligent things and transforming it into something equally k- good, equally intelligent, but that's more understandable. And, and I think that's really valuable, because I think as we put machines in charge of ever more infrastructure in our world, the power grid, the trading on the stock market, weapon systems, and so on, it's absolutely crucial that we can trust these AIs to do all we want. And, and trust really comes from understanding-

Lex Fridman 57:22 Right.

Max Tegmark 57:22 ... in a very fundamental way. And, um, that's why I'm, that's why I'm working on this, because I think the more... if we're gonna have some hope of, of ensuring that machines have adopted our goals and that they're gonna retain them, that kind of trust, I think, needs to be based on things you can actually understand, preferably even make, uh, preferably even prove theorems on. E- even with a self-driving car, right? If someone just tells you it's been trained on tons of data and it never crashed, it's, it's less reassuring than if someone actually has a proof.

Lex Fridman 57:53 Mm-hmm.

Max Tegmark 57:54 Maybe it's a computer-verified proof, but still, it says that under no circumstances is this car just gonna swerve into oncoming traffic, you know?

Lex Fridman 58:02 And, and that kind of information helps buil- build trust, and helps-

Max Tegmark 58:05 Yeah.

Lex Fridman 58:05 ... build the alignment, uh, the alignment of goals, the at, uh, at least awareness that your goals, your values are aligned.

Max Tegmark 58:12 Yeah, and I-

Lex Fridman 58:12 Right.

Max Tegmark 58:12 ... I think even in the very short term, if you look at how, uh, you know, the, today, right, this absolutely pathetic state of, of cybersecurity that we have.

Lex Fridman 58:20 Right.

Max Tegmark 58:20 Where, uh, where is it? Three billion Yahoo accounts were ca- hacked, and almost every American's credit card, uh, and, and so on, you know, it's... Uh, why is this happening? It's ultimately happening because we have software that nobody fully understood how it worked.

Lex Fridman 58:41 Mm-hmm.

Max Tegmark 58:41 That's why the bugs hadn't been found, right?

Lex Fridman 58:44 Right.

Max Tegmark 58:44 And, uh, I think AI can be used very effectively for offense, for hacking, but it can also be used for defense, uh, hopefully automating verifiability and creating ins- uh, systems that are built in different ways so you can actually prove things about them.

Lex Fridman 59:02 Right.

Max Tegmark 59:03 And, uh, it's, it's important.

Lex Fridman 59:05 Yeah. So, speaking of software that nobody understands how it works, of course a bunch of people ask, um, about your paper, about your thoughts of why does deep and cheap learning work so well. That's the paper. But, um, what, what are your thoughts on deep learning, these kind of simplified models of our own brains that have been able to do some successful perception work, pattern recognition work, and now with AlphaZero and so on, do some, some clever things? What are your thoughts about the promise, limitations of this piece?

Max Tegmark 59:35 Great. The, the... I, I think there are a number of, of, of very important insights, uh, very important lessons we can all really draw from these kind of successes. One of them is when you look at the human brain, you see it's very complicated, 10^11 neurons, and there are all these different kinds of neurons, and yada-

Lex Fridman 59:53 Right.

Max Tegmark 59:53 ... yada, and there's been this long debate about whether the fact that we have dozens of different kinds is actually necessary for intelligence. We can now, I think, quite convincingly answer that question with no. It's enough to have just one kind. If you look under the hood of AlphaZero, there's only one kind of neuron, and it's ridiculously simple-

Lex Fridman 1:00:12 Mm-hmm.

Max Tegmark 1:00:13 ... uh, a simple mathematical thing. So it's, it's not the... It's just like in physics, it's not the de- if you have a gas with waves in it, it's not the detailed nature of the molecules that matter, it's the collective behavior somehow. Similarly, (clears throat) it's, it's, it's the higher level of structure of the network that matters, not that you have 20 kinds of neurons. I think (clears throat) our brain is such a complicated mess, because it wasn't evolved just to be intelligent, it was evolved to also be...... self-assembling-

Lex Fridman 1:00:46 Right.

Max Tegmark 1:00:47 ... and self-repairing, right? And, uh, evolutionarily attainable and so on, so-

Lex Fridman 1:00:52 Yeah, it patches and so on, yeah.

Max Tegmark 1:00:53 Yeah. So I, I think it's pretty... My, my hunch is that we're gonna understand how to build AGI before we fully understand how our brains work.

Lex Fridman 1:01:00 Mm-hmm.

Max Tegmark 1:01:00 Just like we, uh, we understood how to build flying machines long before we were able to build a mechanical bird.

Lex Fridman 1:01:07 Yeah, that's right. You've given the ex- (laughs) you've given the, that, uh, the example exactly of mechanical birds and airplanes.

Max Tegmark 1:01:14 Yeah.

Lex Fridman 1:01:14 And airplanes do a pretty good job of flying without really mimicking bird flight.

Max Tegmark 1:01:18 And even now, after 100... It's 100 years later, did you see the TED Talk with, um, this German group-

Lex Fridman 1:01:23 I did not.

Max Tegmark 1:01:23 ... with the mechanical bird?

Lex Fridman 1:01:23 I've heard you mention it. I want to see it.

Max Tegmark 1:01:25 It's... Check it out, it's amazing. But even after that, right, we still don't fly on mechanical birds, because it turned out that the way we came up with was simpler, and it's better for our purposes. And I think it might be the same there. That's one lesson. Uh, and another lesson, which is more what the, what our paper was about... Uh, first w- we, I, I as a physicist thought it was fascinating how there's a very close mathematical relationship actually between our artificial neural networks and a lot of things that we've studied before in physics-

Lex Fridman 1:01:53 Mm-hmm.

Max Tegmark 1:01:54 ... that go by nerdy names like the renormalization group equation and Hamiltonians and yada, yada, yada. And, and, um, when you look a little more closely at this, you have, um... At first I was like, whoa, there's something crazy here that doesn't make sense 'cause we know that if you even wanna build a super simple neural network to tell apart cat pictures and dog pictures, right? Like, we can do that very, very well now.

Lex Fridman 1:02:25 Mm-hmm.

Max Tegmark 1:02:27 But if you think about it a little bit, you convince yourself it must be impossible, because if I have one megapixel, even if each pixel is just black or white, there's two to the power of one million possible images.

Lex Fridman 1:02:38 Mm-hmm.

Max Tegmark 1:02:38 There's way more than there are atoms in our universe, right?

Lex Fridman 1:02:40 Yeah.

Max Tegmark 1:02:40 So in order to s- And then for each one of those, I have to assign a number which is the probability that it's a dog.

Lex Fridman 1:02:48 Right.

Max Tegmark 1:02:49 So an arbitrary function of images is a list of more numbers than there are atoms in our universe, so clearly I can't store that under the hood of my, my GPU or my, my computer, yet it somehow works.

Lex Fridman 1:03:02 Mm-hmm.

Max Tegmark 1:03:03 So what does that mean? Well, it means that the f- out of all of the problems that you could tr- all... Try to solve with a neural network, almost all of them are impossible to solve with a reasonably sized one. But then w- what we sh- showed in our paper was, was that the fractio- the kind of problems, the fraction of all the problems that you could possibly pose that the, that we actually care about, given the laws of physics-

Lex Fridman 1:03:31 Mm-hmm.

Max Tegmark 1:03:31 ... is also an infinitesimally tiny little part.

Lex Fridman 1:03:34 Yeah.

Max Tegmark 1:03:34 And amazingly, they are basically the same part.

Lex Fridman 1:03:37 (laughs) Yeah, it's almost like our world was created for... I mean, they kinda come together.

Max Tegmark 1:03:41 Yeah, well, you could say maybe w- the world created, the world, that the world was created for us. But I have a more modest interpretation, which is that instead evolution endowed us with neural networks-

Lex Fridman 1:03:51 Right.

Max Tegmark 1:03:51 ... precisely for that reason.

Lex Fridman 1:03:52 Right.

Max Tegmark 1:03:53 'Cause this particular-

Lex Fridman 1:03:54 That's-

Max Tegmark 1:03:54 ... architecture, as opposed to the one in your laptop, is very, very well adapted to solving the kind of problems-

Lex Fridman 1:04:02 Mm-hmm.

Max Tegmark 1:04:02 ... that nature kept presenting our ancestors with, right? So it makes sense that why do we have a brain in the first place? It's to be able to make predictions about the future-

Lex Fridman 1:04:11 Mm-hmm.

Max Tegmark 1:04:11 ... and so on. So if we had a sucky system, which could never solve it, (laughs) we wouldn't have evolved.

Lex Fridman 1:04:17 Ooh.

Max Tegmark 1:04:17 So, but it, it... So this is just, this is, uh, a I think a very beautiful fact.

Lex Fridman 1:04:23 Yeah.

Max Tegmark 1:04:24 We also, we also realize that there's a... That there, that the... We... There've been... It's been earlier work on, on why deeper networks are good, but we- we were able to show an additional cool fact there, which is, uh, that, um, even incredibly simple problems, like suppose I give you a, a thousand numbers and ask you to multiply them together, you know?

Lex Fridman 1:04:43 Mm-hmm.

Max Tegmark 1:04:43 You can write (laughs) just a few lines of code, boom, done, trivial. If you're just trying to do that with a neural network that has only one single hidden layer in it, you can do it, but you're gonna need two to the power of thousand neurons-

Lex Fridman 1:04:58 (laughs) Yeah.

Max Tegmark 1:04:59 ... to multiply a thousand numbers, which is again, more neurons than there are atoms in our universe, so-

Lex Fridman 1:05:03 Okay. That's fascinating.

Max Tegmark 1:05:05 But if you allow, if you allow yourself to make it a deep network with many layers, you only need 4,000 neurons.

Lex Fridman 1:05:12 (laughs)

Max Tegmark 1:05:13 It's perfectly feasible. So-

Lex Fridman 1:05:16 That's really interesting, the-

Max Tegmark 1:05:17 Yeah.

Lex Fridman 1:05:18 Yeah. So on another architecture type, I mean, you mentioned Schrodinger's equation, and what are your thoughts about quantum computing and, uh, the role of this kind of comp-, um, computational unit in creating an intelligent system?

Max Tegmark 1:05:34 In some Hollywood movies-

Lex Fridman 1:05:36 (laughs)

Max Tegmark 1:05:38 ... that I will not mention by name because I don't want to spoil them, the way they get AGI is building a quantum computer-

Lex Fridman 1:05:44 Was... Yeah, it's-

Max Tegmark 1:05:45 ... because the word quantum sounds cool and so on.

Lex Fridman 1:05:47 That's right.

Max Tegmark 1:05:48 Mine's, uh, uh, first of all, I think we don't need quantum computers to build AGI. I suspect your brain is not a quantum computer in any profound sense.

Lex Fridman 1:06:01 So you don't-

Max Tegmark 1:06:02 I even wrote a paper about that long, many years ago. I ca- calculated the decoherence, so-called decoherence time that, th- how long it takes until the quantum computerness of, of what your neurons are doing gets erased-

Lex Fridman 1:06:13 Mm-hmm.

Max Tegmark 1:06:15 ... by just random noise from the environment. And it, it's about 10^-21 seconds. So as cool as it would be to have a quantum computer in my head, I don't think that fast, you know?

Lex Fridman 1:06:26 Yeah.

Max Tegmark 1:06:26 Um, uh, on the other hand, there are very cool things you could do with quantum computers, or I think will be able to do soon, when we get big one, bigger ones, that might actually help machine learning do even better than the brain.

Lex Fridman 1:06:42 Mm-hmm.

Max Tegmark 1:06:43 So, for example-... one, this is just a sp- sp- uh, moonshot, but, um, hey-

Lex Fridman 1:06:52 (laughs)

Max Tegmark 1:06:52 ... uh, you know, the, the- Learning is very much s- same thing as search.

Lex Fridman 1:07:00 Mm-hmm.

Max Tegmark 1:07:00 If, if you have a, if you're trying to train a neural network to get really, to learn to do something really well, you have some loss function, you have some... You have a bunch of knobs you can turn-

Lex Fridman 1:07:09 Mm-hmm.

Max Tegmark 1:07:10 ... represented by a bunch of numbers, and you're trying to tweak them, so that it become as good as possible at this thing.

Lex Fridman 1:07:15 Mm-hmm.

Max Tegmark 1:07:15 So, if you think of a landscape w- with some valley where each dimension of the landscape corresponds to some number you can change, you're trying to find the minimum. And it's well-known that if you have a very high dimensional landscape, complicated things, it's super hard to find the minimum, right?

Lex Fridman 1:07:31 That's right.

Max Tegmark 1:07:32 Um, quantum mechanics is amazingly good at this.

Lex Fridman 1:07:36 Right.

Max Tegmark 1:07:36 Like, uh, if I want to know what's the lowest energy state this water can possibly have, incredibly hard to compute, but we, we can d- but nature will happily figure this out for you if you just cool it down.

Lex Fridman 1:07:47 Mm-hmm.

Max Tegmark 1:07:47 Make it very, very cold. If you put a ball somewhere, it'll roll down to its minimum.

Lex Fridman 1:07:53 Mm-hmm.

Max Tegmark 1:07:53 And this happens metaphorically, uh, at the energy landscape too. And, and quantum mechanics even uses some little, some clever tricks, which today's machine learning systems don't. Like, if you're trying to find the minimum and you get stuck in a little local minimum here, in quantum mechanics, you can actually tunnel through the barrier-

Lex Fridman 1:08:10 Mm-hmm.

Max Tegmark 1:08:11 ... and get unstuck again. And, and, um-

Lex Fridman 1:08:14 That's really interesting.

Max Tegmark 1:08:15 Yeah. So, so maybe, for example, we'll one day use quantum computers that- to help train neural networks better.

Lex Fridman 1:08:23 That's really interesting. Okay. So, as a component of kind of the learning process, for example.

Max Tegmark 1:08:28 Yeah.

Lex Fridman 1:08:29 Let me ask, sort of wrapping up here a little bit, let me, let me return to, uh, the questions of our human nature and, and love, as I mentioned. So, do you think, uh, you mentioned sort of a helper robot, but you can think of also personal robots. Y- do you think the way we human beings fall in love and get connected to each other is possible to achieve in an AI system, a human-level AI intelligence system? Do you think we would ever see that kind of connection? Or, uh, you know, in all this discussion about solving complex goals-

Max Tegmark 1:09:08 Yeah.

Lex Fridman 1:09:08 ... is this kind of human social connection, do you think that's one of the goals on, on the peaks and valleys that, or the raising sea levels that we'll be able to achieve, or do you think that's something that's ultimately, uh, or at least in the short term relative to the other goals, is not achievable?

Max Tegmark 1:09:23 I think it's all possible. And I mean, in, in recent... There's a, there's a very wide range of guesses, as you know, among AI researchers when we're gonna get AGI. Some people, you know, like our friend Rodney Brooks says it's gonna be 100, 100s of years at least.

Lex Fridman 1:09:40 (laughs)

Max Tegmark 1:09:41 And then there are many others that think it's gonna happen relatively much sooner. In recent polls, maybe half or so of AI researchers think it's, we're gonna get AGI within decades. Uh, so if, if that happens, of course, I, then I think these things are all possible. But in terms of whether it will happen, I, um, I think we shouldn't spend so much time asking, "What do we think will happen in the future?"

Lex Fridman 1:10:03 Mm-hmm.

Max Tegmark 1:10:04 As if we are just some sort of pathetic, you know, passive bystanders, you know-

Lex Fridman 1:10:07 Yeah.

Max Tegmark 1:10:08 ... waiting for the future to happen to us.

Lex Fridman 1:10:10 Yeah.

Max Tegmark 1:10:10 Hey, we're the ones creating this future, right?

Lex Fridman 1:10:12 Yeah.

Max Tegmark 1:10:12 So, we should be proactive about it and ask ourselves what sort of future we would like to have happen.

Lex Fridman 1:10:19 That's right.

Max Tegmark 1:10:19 Try to make it like that. What, would I prefer it to some sort of incredibly boring zombie-like future where there's always mechanical things happening and there's no passion, no emotion, no experience maybe even? No, I would, of course, much rather prefer if, if all the things that we find, that we value the most about humanity are our subjective experience, passion, inspiration, maybe love, you know. If, if, if, uh, we can create a future when, where those are, where those things do exist, you know. I think ultimately, it's not our universe giving meaning to us. It's us giving meaning to our universe, and if we build more advanced intelligence, let's, let's make sure we build it in such a way that meaning is, is part of it.

Lex Fridman 1:11:08 Uh, when a lot of people that seriously study this problem and think of it from different angles have trouble in... The majority of cases, if they think through, that happen-

Max Tegmark 1:11:19 Yeah.

Lex Fridman 1:11:19 ... are the ones that are not beneficial to humanity.

Max Tegmark 1:11:22 Right.

Lex Fridman 1:11:22 And so, yeah. So what, (laughs) what are your thoughts?

Max Tegmark 1:11:25 (laughs) .

Lex Fridman 1:11:25 What's an int- What's, what should people... You know, I really don't like people to be terrified. Uh-

Max Tegmark 1:11:32 Yeah.

Lex Fridman 1:11:32 ... we should... What, what's a way for people to think about it in a way that, uh, in, in, in a way we can solve it and we can make it better?

Max Tegmark 1:11:39 Yeah. Yeah. No, I, I don't think panicking is gonna help in any way. It's not gonna increase the chances of things going well either. Even if you are in a situation where there is a real threat, does it help if everybody just freaks out?

Lex Fridman 1:11:50 Right.

Max Tegmark 1:11:51 No, of course, (laughs) of course not. Um, I, I think, yeah, there are, of course, ways in which things can go horribly wrong. Uh, uh, first of all, it's important when we think about this thing, this, about the problems and risks that also remember how huge the upsides can be if we get it right, right?

Lex Fridman 1:12:08 Sure.

Max Tegmark 1:12:08 Everything, everything we love about society and, and civilization is a product of intelligence.

Lex Fridman 1:12:13 Right.

Max Tegmark 1:12:13 So, if, if we can amplify our intelligence with machine intelligence and not anymore lose our loved ones to what's, we're told is an uncurable disease and things like this-

Lex Fridman 1:12:22 Right.

Max Tegmark 1:12:22 ... of course, we should aspire to that. So, that can be a motivator, I think. Reminding ourselves that the reason we try to solve problems is not just because we're trying to avoid gloom, but because we're trying to do something great.

Lex Fridman 1:12:35 Right.

Max Tegmark 1:12:35 Uh, but then in, in terms of the risks, I think, um, the int- really important question is to ask, what can we do today that will actually help make the outcome good, right?

Lex Fridman 1:12:47 Yes.

Max Tegmark 1:12:47 And, uh, dismissing the risk is not one of them, you know. It, it-I find it quite funny, often when I'm in... on, on p- discussion panels about these things, how the people who, who are, work for com- for companies will always a- will always be like, "Oh, nothing to worry about. Nothing to worry about. Nothing to worry about."

Lex Fridman 1:13:05 Right.

Max Tegmark 1:13:05 And it, it's always on- it's only academics sometimes express concerns.

Lex Fridman 1:13:09 (laughs)

Max Tegmark 1:13:09 That's not surprising at all if you think about it.

Lex Fridman 1:13:12 Right, right.

Max Tegmark 1:13:13 Upton Sinclair quipped, right, that, uh, "It's hard to make a man believe in something when his income depends on not believing it." And, and-

Lex Fridman 1:13:21 (laughs)

Max Tegmark 1:13:21 ... frankly, we know a lot of these people in companies, that they're just as concerned as anyone else. But if you're the CEO of a company, that's not something you wanna go on record saying when you have silly journalists who are gonna f- uh, put a picture of a Terminator robot when they quote you. So, so the, the, the issues are real, and the, and the way I... uh, the way I think about what the issue is, is basically, you know, the, the, the real choice we have is, first of all, are we gonna just d- dismiss this, the risks, and say, "Well, you know, let's just go ahead and build, um, machines that can do everything we can do better and cheaper," you know?

Lex Fridman 1:13:57 Right.

Max Tegmark 1:13:57 "Let's just make ourselves obsolete as fast as possible. What could possibly go wrong?"

Lex Fridman 1:14:01 Right.

Max Tegmark 1:14:01 That's one attitude. The opposite attitude, I think, is to say, "There is this incredible potential, you know. Let- let's think about what kind of future we're really, really excited about. What are the shared goals that we can really aspire towards? And then let's think really hard on how... about how we can actually get there." Uh, so start with... No, don't start thinking about the risks. Start thinking about the goals.

Lex Fridman 1:14:25 The goals, yeah.

Max Tegmark 1:14:26 And then, when you do that, then you can think about the obstacles you wanna avoid, right?

Lex Fridman 1:14:30 Mm-hmm.

Max Tegmark 1:14:30 I often get students coming in, right here into my office for career advice.

Lex Fridman 1:14:33 Mm-hmm.

Max Tegmark 1:14:34 I always ask them this very question, "Where do you wanna be in the future?"

Lex Fridman 1:14:37 Right.

Max Tegmark 1:14:38 If all she can say is, "Oh, maybe I'll have cancer. Maybe I'll-"

Lex Fridman 1:14:41 Yeah.

Max Tegmark 1:14:41 "... get run over by a truck."

Lex Fridman 1:14:42 Focus on the obstacles instead of the goal, yeah.

Max Tegmark 1:14:44 She's just gonna end up a hypochondriac paranoid, right?

Lex Fridman 1:14:48 Yeah.

Max Tegmark 1:14:48 Whereas, if she comes in with fire in her eyes and is like, "I wanna be there," and then we can talk about the obstacles and see how we can circumvent them.

Lex Fridman 1:14:55 Mm-hmm.

Max Tegmark 1:14:55 That's, I think, a much, much healthier attitude. And, um...

Lex Fridman 1:15:00 That's really well-put.

Max Tegmark 1:15:01 And, and, uh, and, uh, I, I feel it's, it's very challenging to come up with a vision for the f- a future which we're, which we are unequivocally excited about. And I'm not just talking now in the vague terms, like, "Yeah, let's cure cancer." Fine.

Lex Fridman 1:15:14 Right.

Max Tegmark 1:15:14 I'm talking about what kind of society do we wanna create? What do we want it to mean, you know, to be human in the age of AI when we ha- in the age of AGI? So, if we can have this conversation, broad, inclusive conversation, and gradually start converging towards some, some future that... with some direction, at least, that we wanna steer towards, right? Then, then, uh, we'll be much more motivated to constructively take on the obstacles. And I think, uh, if I, uh, if I had to... if, if you make... if I try to wrap this up in a more succinct way, I think, I think we can all agree already now that we should aspire to build AGI that doesn't overpower us, but that empowers us.

Lex Fridman 1:16:05 And think of the many various ways it can do that, whether that's from, uh, my, uh, side of the world of autonomous vehicles. I, I per-... I'm personally actually from the camp that believes there's... human-level intelligence is required to, to achieve something-

Max Tegmark 1:16:19 Mm-hmm.

Lex Fridman 1:16:19 ... like vehicles that would actually be something we would enjoy using-

Max Tegmark 1:16:24 Mm-hmm.

Lex Fridman 1:16:24 ... and being part of. So, that's the one example, and certainly there's a lot of other types of robots in medicine, and, and so on. Uh, so focusing on those, and then, and then coming up with the obstacles, coming up with the ways that that can go wrong, and solving those one at a time.

Max Tegmark 1:16:38 And just because you can build an autonomous vehicle, e- even if you could build one that would drive just fine without you... You know, m- maybe there are some things in life that we would actually wanna do ourselves.

Lex Fridman 1:16:48 That's right.

Max Tegmark 1:16:49 Right? Like, for example in... if you think of our society as a whole, there's some things that w- that we find very meaningful to do, and, uh, that doesn't mean we have to stop doing them just because machines can do them better, you know? I'm not gonna stop playing tennis just the day, the day someone builds a tennis robot-

Lex Fridman 1:17:06 Yeah.

Max Tegmark 1:17:06 ... to beat me.

Lex Fridman 1:17:07 People are still playing chess and even Go.

Max Tegmark 1:17:09 Yeah, and, and, uh, I... in this, the... in the very near term even, some people are advocating basic income-

Lex Fridman 1:17:17 Mm-hmm.

Max Tegmark 1:17:17 ... to replace jobs. But if you... if, if the government is gonna be willing to just hand out cash to people for doing nothing, uh, then one should also seriously consider whether the government should also just hire a lot more teachers, and nurses, and the kind of jobs which people often find great fulfillment in doing, right? I get very tired of hearing politicians saying, "Oh, we can't afford hiring more teachers."

Lex Fridman 1:17:39 Right.

Max Tegmark 1:17:39 "But we're gonna maybe have basic income." If we can have more, more serious research and thought into what gives meaning to our lives, then jobs give so much more than income, right?

Lex Fridman 1:17:49 Mm-hmm.

Max Tegmark 1:17:50 And then think about, in the future, wh- what are the roles that... yeah, what are the, the roles that we wanna have people continually filling empowered by machines?

Lex Fridman 1:18:03 And I think,

Lex Fridman 1:18:04 sort of, um... I come from the... Russia, from the Soviet Union, and, um, I think for a lot of people in the 20th century, going to the moon, going to space was ins- an inspiring thing. I feel like the, the, uh... the, the universe of the mind, so AI, understanding, creating intelligence, is that for the 21st century. Uh, so it's really surprising, and I've heard you mention this. It's really surprising to me, both on the research funding side that it's not funded as greatly as it could be, but most importantly on the pol- politician side, that it's not part of the public discourse except in the killer bots, Terminator kind of view. That people are not yet, I think, perhaps excited by the possible positive future that we can, uh, build together, so-

Max Tegmark 1:18:48 And we should be, because politicians usually just focus on the next election cycle, right?

Lex Fridman 1:18:53 Right.

Max Tegmark 1:18:54 The single most important thing I feel we humans have learned in the entire history of science is that we're the masters of underestimation. We underestimated the size of, of, of our cosmos, again and again, realizing that everything we thought existed was just a small part of something grander, right? Planet, solar system, the galaxy, you know, clusters of galaxies, the universe. And we now know that we- that the future has just so much more potential-

Lex Fridman 1:19:24 Mm-hmm.

Max Tegmark 1:19:25 ... than our ancestors could ever have dreamt of. Uh, this cosmos, I- I- imagine if all of Earth was completely devoid of life except for Cambridge, Massachusetts, right?

Lex Fridman 1:19:38 Mm-hmm.

Max Tegmark 1:19:38 Wouldn't it be kind of lame if all we ever aspired to was to stay in Cambridge, Massachusetts forever, and then go extinct in one week, even though Earth was gonna continue on for longer? That, that, that sort of attitude, I think, we, we have now on the cosmic scale. We can fl- life can flourish on Earth not for, uh, not for four years, but for billions of years.

Lex Fridman 1:20:00 Yes.

Max Tegmark 1:20:00 I can even tell you about how to move it out of harm's way when the sun, sun gets too hot. And, and then we have so much more resources out here, which today, yeah, maybe there are a lot of other planets with bacteria or, or cow-like life on them, but I... Most of this, all this opportunity seems, as far as we can tell, to be largely dead, like the Sahara Desert, and yet we have the opportunity that- to help life flourish-

Lex Fridman 1:20:28 Right. (laughs)

Max Tegmark 1:20:28 ... throughout this-

Lex Fridman 1:20:29 Throughout.

Max Tegmark 1:20:29 ... for billions of years. And so, like, let's quit squabbling about whether some little border should be drawn one-

Lex Fridman 1:20:36 (laughs)

Max Tegmark 1:20:37 ... one mile to the left or right, and-

Lex Fridman 1:20:38 Yeah.

Max Tegmark 1:20:39 ... look up into the skies and realize, hey, you know, we can do such incredible things.

Lex Fridman 1:20:44 Yeah, and that's, I think, why it's really exciting that, uh, uh, you and others are connected with some of the work Elon Musk is doing, because he's literally going out into that space-

Max Tegmark 1:20:54 Yeah.

Lex Fridman 1:20:54 ... really exploring our universe, and it's wonderful.

Max Tegmark 1:20:57 And that is exactly why Elon Musk is so misunderstood, right?

Lex Fridman 1:21:01 Mm-hmm.

Max Tegmark 1:21:02 People misconstrue him as some kind of pessimistic doomsayer. The, the reason he cares so much about AI safety is because he more than almost anyone else appreciates these amazing opportunities that we'll squander if we wipe o- out here on Earth. We're not just gonna wipe out the