Max Tegmark: AI and Physics | Lex Fridman Podcast #155

Lex Fridman 0:00 The following is a conversation with Max Tegmark, his second time on the podcast. In fact, the previous conversation was episode number one of this very podcast. He is a physicist, an artificial intelligence researcher at MIT, co-founder of the Future of Life Institute, and author of Life 3.0: Being Human in the Age of Artificial Intelligence. He's also the head of a bunch of other huge, fascinating projects, and has written a lot of different things that you should definitely check out. He has been one of the key humans who has been outspoken about long-term existential risks of AI, and also its exciting possibilities and solutions to real world problems, most recently at the intersection of AI and physics, and also in re-engineering the algorithms that divide us by controlling the information we see, and thereby creating bubbles and all other kinds of, uh, complex social phenomena that we see today. In general, he's one of the most passionate and brilliant people I have the fortune of knowing. I hope to talk to him many more times on this podcast in the future. Quick mention of our sponsors: The Jordan Harbinger Show, Four Sigmatic Mushroom Coffee, BetterHelp Online Therapy, and ExpressVPN. So the choice is wisdom, caffeine, sanity, or privacy. Choose wisely, my friends. And if you wish, click the sponsor links below to get a discount and to support this podcast. As a side note, let me say that much of the researchers in the machine learning and artificial intelligence communities do not spend much time thinking deeply about existential risks of AI. Because our current algorithms are seen as useful but dumb, it's difficult to imagine how they may become destructive to the fabric of human civilization in the foreseeable future. I understand this mindset, but it's very troublesome. To me, this is both a dangerous and uninspiring perspective, reminiscent of a lobster sitting in a pot of lukewarm water that a minute ago was cold. I feel a kinship with this lobster. I believe that already the algorithms that drive our interaction on social media have an intelligence and power that far outstrip the intelligence and power of any one human being. Now really is the time to think about this, to define the trajectory of the interplay of technology and human beings in our society. I think that the future of human civilization very well may be at stake over this very question of the role of artificial intelligence in our society. If you enjoy this thing, subscribe on YouTube, review it on Apple Podcasts, follow on Spotify, support on Patreon, or connect with me on Twitter @lexfriedman. And now, here's my conversation with Max Tegmark.

Lex Fridman 2:50 So people might not know this, but you were actually episode number one of this podcast just a couple of years ago, and now we're back. And it so happens that a lot of exciting things happened in both physics and artificial intelligence, both fields that you're super passionate about. Can we try to catch up to some of the exciting things happening in artificial intelligence, especially in the context of the way it's cracking open the different problems of s- the sciences?

Max Tegmark 3:21 Yeah, I would love to, especially now, uh, as we start 2021 here. And it's a really fun time to think about w- what were the biggest breakthroughs in AI. Not the ones necessarily the media wrote about, but that-

Lex Fridman 3:33 Yeah.

Max Tegmark 3:33 ... really matter, and- and what do they mean for our ability to do better science, what does it mean for our ability, uh, to help people around the world, and what does it mean for new, um, t- problems that they could cause if we're not smart enough to avoid them. So, you know, what do we learn, basically, from this?

Lex Fridman 3:53 Yes, absolutely. So one of the amazing things you're part of is the AI Institute for Artificial Intelligence & Fundamental Interactions. What's up with this institute? What- what are you working on? What are you thinking about?

Max Tegmark 4:06 The idea is something I'm very on fire with, which is basically AI meets physics, and, um, you know, it's been almost five years now since I shifted my own MIT research from physics to machine learning. And in the beginning, I noticed a lot of my colleagues, even though they were polite about it, were like kind of, "Hm. What is Max doing? What is this weird stuff?" (laughs)

Lex Fridman 4:30 (laughs) He's lost his mind. (laughs)

Max Tegmark 4:32 Then- But then gradually, I, uh, together with some colleagues, were able to persuade more and more of the other professors in the- our physics department to get interested in this, and- and- and, um, now we got this amazing NSF center, so 20 million bucks for- for the next five years, MIT and a bunch of neighboring universities here also. And I notice now those colleagues (laughs) who were looking at me funny have stopped asking-

Lex Fridman 4:58 (laughs)

Max Tegmark 4:59 ... why I'm- what the point is of this because it's becoming more clear. And I ba- really believe that, of course, AI can help physics a lot to do better physics, but physics can also help, uh, AI a lot, both by building better hardware. My colleague Marin Soljačić, for example, is working on an optical chip for much faster machine learning, where the computation is done not by moving electrons around and- but by moving photons around. Dramatically less energy use, faster, better. Um, we all can- we can also help AI a lot, I think, by having a- a different set of tools and a different, maybe more audacious attitude. You know, AI has lo- to a significant extent been an engineering discipline, where you're just trying to make things that work-

Lex Fridman 5:54 Mm-hmm.

Max Tegmark 5:55 ... and being less int- more interested in maybe selling them than in figuring out exactly how they work.... and proving theorems about, that they will always work, right?

Lex Fridman 6:04 Yeah.

Max Tegmark 6:04 Contrast that with physics. You know, when Elon Musk sends a rocket to the International Space Station, they didn't just train with machine learning, "Oh, let's fire it a little bit left, more to the left, a bit more to the right. Oh, that also missed. Let's try here." No, you know, we, uh, figured out Newton's laws of gravitation and other... and got other things, and got a really deep, fundamental understanding. Uh, and that's what gives us such confidence in, in rockets. And my vision is that, in the future all machine learning systems that actually have impact on, on people's lives will be understood at a really, really deep level, right? So we trust them not 'cause some sales rep told us to, but because they've earned our trust. We can... And really safety critical things, even prove that they will always do, you know, what we expect them to do. That's very much the physics mindset. So, it, it's interesting, if, if you look at big breakthroughs that have happened in machine learning this year, you know, from dancing robots, you know... It's pretty fantastic, um, not just because it's cool, but if you just think about, not that many years ago, this YouTube video at this DARPA challenge with the MIT robot, comes out of the car and face plants.

Lex Fridman 7:19 Yeah (laughs) .

Max Tegmark 7:21 (laughs) How far we've come-

Lex Fridman 7:22 Yes.

Max Tegmark 7:22 ... in, in just a few years. Similarly, AlphaFold2, you know, crushing the protein folding problem. We can talk more about implications for medical research-

Lex Fridman 7:34 Yeah.

Max Tegmark 7:34 ... and stuff, but hey, you know, that's huge progress. Uh, you, you can look at the GPT-3 that can spout off English te- texts, which sometimes really, really blows you away.

Lex Fridman 7:49 Yeah.

Max Tegmark 7:49 You can look at the Go- uh, DeepMind's, uh, MuZero, which doesn't just kick our butt in, um, Go and chess and shogi, but also in all these Atari games. And you don't even have to teach it the rules now.

Lex Fridman 8:02 Mm-hmm.

Max Tegmark 8:03 You know, what all of those have in common is, besides being powerful, is we don't fully understand how they work. And that's fine if it's just some dancing robots, and the worst thing that can happen is they face plant, right? Or if they're playing Go, and the worst thing that can happen is that they make a bad move and lose the game, right? It, it's less fine if that's what's controlling your self-driving car or your nuclear power plant. And, uh, we've seen already that even though Hollywood had all these movies where they try to make us worry about the wrong things, like machines turning evil, the actual bad things that have happened with automation have not been machines turning evil.

Lex Fridman 8:45 Mm-hmm.

Max Tegmark 8:45 They've been caused by over-trust in things we didn't understand as well as we thought we did, right?

Lex Fridman 8:51 Yeah.

Max Tegmark 8:51 Even very simple automated systems like what, uh, Boeing put into the 737 MAX, right?

Lex Fridman 8:58 Yes.

Max Tegmark 8:59 Killed a lot of people. Was it that that little simple system was evil? Of course not. But we didn't understand it as well as we should have, right?

Lex Fridman 9:07 And we trusted without understanding.

Max Tegmark 9:10 Exactly. We-

Lex Fridman 9:11 Hence the over-trust.

Max Tegmark 9:12 We didn't even understand that we didn't understand, right?

Lex Fridman 9:15 (laughs)

Max Tegmark 9:15 The... Humility is really at the core of being a scientist. Th- uh, I think step one, if you wanna be a scientist is, don't ever fool yourself into thinking you understand things when you actually don't.

Lex Fridman 9:26 Yes.

Max Tegmark 9:26 Right? Uh-

Lex Fridman 9:27 That's probably good advice for humans in general. (laughs)

Max Tegmark 9:29 I think humility in general can do us good. But in science it's, like, so spectacular. Like, why did we have the wrong theory of gravity ever from A- Aristotle-

Lex Fridman 9:37 Yeah.

Max Tegmark 9:37 ... onward and close, until, like, Galileo's time? Or, like, why would we believe something so dumb as that if I throw this water bottle, it's gonna go up with constant speed-

Lex Fridman 9:46 (laughs)

Max Tegmark 9:47 ... until it realizes that its natural motion is down?

Lex Fridman 9:49 It changes its mind.

Max Tegmark 9:50 You know, because we... People just kind of assumed Aristotle was right, he's an authority, we understand it. Uh, why did we believe things like that the sun is going around the Earth? Why did we believe that time flows at the same rate for everyone until Einstein? Same exact mistake over and over again. We just weren't humble enough to acknowledge that we actually didn't know for sure. We assumed we knew, uh, so we didn't discover the truth, because we assumed there (laughs) was nothing there to be discovered, right? There was something to be discovered about the 737 MAX, and if you had been a bit more suspicious and tested it better, we would have found it. And it's the same thing with most harm that's been done by automation so far, I would say. So I don't know if you... Did you heard- hear of a company called Knight Capital?

Lex Fridman 10:37 No.

Max Tegmark 10:38 So, good. That means you didn't invest (laughs) in them earlier. (laughs)

Lex Fridman 10:42 (laughs)

Max Tegmark 10:42 They deployed this automated trading system.

Lex Fridman 10:44 Yes.

Max Tegmark 10:45 All nice and shiny. They didn't understand it as well as they thought, and it went about losing $10 million bucks per minute-

Lex Fridman 10:52 Yeah.

Max Tegmark 10:53 ... for 44 minutes straight.

Lex Fridman 10:54 Oh, no.

Max Tegmark 10:55 Until someone presumably was like, "Oh no!" (laughs) "Shut this off."

Lex Fridman 10:58 (laughs)

Max Tegmark 10:58 You know. Uh, was it evil? No. It was, again, misplaced trust, something they didn't fully understand, right? And, um, there have been so many, um... Even when people have been killed by robots, which is quite rare still, uh, but in a- factory accidents, it's, in every single case, been not malice, just that the robot didn't understand that, hey, a human is different from an auto part or whatever. And, and we... We, um... So this is where I think there's so, so much opportunity for a physics approach, where you just aim for a higher level of understanding. And if you look at the... All these systems that we talked about, from the... from reinforcement learning systems and dancing robots to all these neural networks that power GPT-3 and, and Go playing software and stuff, they're all basically black boxes, much like... Not so different from if you teach a human something, you have no idea how their brain works, right?

Lex Fridman 11:57 Mm-hmm.

Max Tegmark 11:58 Except the human brain, at least, has been error-corrected (laughs) during many, many centuries of evolution in the way that these... some of these systems have not, right?

Lex Fridman 12:07 Yeah.

Max Tegmark 12:07 And, um, my, my MIT research is entirely focused on demystifying this black box. Intelligible intelligence is my slogan.

Lex Fridman 12:15 That's a good line. Intelligible intelligence.

Max Tegmark 12:18 Yeah, s- and that we shouldn't settle for something that seems intelligent, but we should, it should be intelligible so that we actually trust it because we understand it, right? Like, again, Elon trusts his rockets because he understands Newton's laws and thrust and how everything works. Uh, and let me tell you why... Can I tell you why I'm optimistic about this?

Lex Fridman 12:36 Yes, please.

Max Tegmark 12:37 I think, I think there is, we've made a bit of a mistake, uh, where we... Some people still think that somehow we're never gonna understand neural networks and we're just gonna have to learn to live with this. It's this very powerful black box basically for those, you know, who haven't spent time tr- building their own, it's super simple what happens inside. You send in a long list of numbers, and then you do a bunch of operations on them, multiply by matrices, et cetera, et cetera, and some other numbers come out. That's the output of it, and then there are a bunch of knobs you can tune. And when you change them, you know, it affects the computation-

Lex Fridman 13:16 Mm-hmm.

Max Tegmark 13:16 ... the input-output relation. And then you just give the computer some definition of good, and it keeps optimizing these knobs until it performs as good as possible. And often you go like, "Wow, that's really good."

Lex Fridman 13:26 Mm-hmm.

Max Tegmark 13:27 "This robot can dance."

Lex Fridman 13:28 (laughs)

Max Tegmark 13:28 Or, "This machine is beating me at chess now." Um, and in the end you have something which even though you can look inside it, you have very little idea of how it works. You know, you can print out tables of all the millions of parameters in there. Is it crystal clear now how it's working? And no, of course not, right? S- so many of my colleagues seem willing to settle for that, and I'm like, "No." (laughs) That's like the halfway point. Uh, uh, some have even gone as far as sort of guessing that the myster- the, the inscrutability of this is where the, some of the power comes from and sort of, some sort of mysticism. Uh, I think that's total nonsense. I, I think the real power of neural networks comes not from inscrutability but from differentiability.

Lex Fridman 14:15 Mm-hmm.

Max Tegmark 14:15 And what I mean by that is simply that the output depends... Changes only smoothly if you tweak your knobs.

Lex Fridman 14:23 Mm-hmm.

Max Tegmark 14:23 And then you can use all these powerful methods we have for optimization in science. We can just tweak them a little bit and see did that get better or worse?

Lex Fridman 14:31 Mm-hmm.

Max Tegmark 14:31 That's the fundamental idea of machine learning, that the machine itself can keep optimizing until it gets better. Suppose you wrote this an algorithm instead in Python or some other programming language, and then what, what the knobs did was they just changed random letters in your, in your code.

Lex Fridman 14:48 Mm-hmm. (laughs)

Max Tegmark 14:50 That would just epically fail, right? You change one thing and instead of saying print, it says synt.

Lex Fridman 14:55 Yeah.

Max Tegmark 14:55 Uh, syntax error, you don't even know was that for the better or for the worse, right? This to me is r- re- this is what I believe is the fundamental power of m- of neural networks.

Lex Fridman 15:05 And just to clarify, the changing of the different letters in a program would not be a differentiable process.

Max Tegmark 15:10 It would make it an invalid program-

Lex Fridman 15:12 Yeah.

Max Tegmark 15:13 ... typically, and then you wouldn't even know if, if you changed more letters if it would make it work again, right?

Lex Fridman 15:18 So, that's the magic of, uh, neural networks, uh, the inscrutability. (laughs)

Max Tegmark 15:23 The differentiability, that-

Lex Fridman 15:24 Yeah.

Max Tegmark 15:24 ... that every, every setting of the parameters is a program, and you can tell-

Lex Fridman 15:27 Yeah.

Max Tegmark 15:28 ... is it better or worse, right? And so-

Lex Fridman 15:31 So, you don't like the poetry of the mystery of neural networks as the source of its power?

Max Tegmark 15:35 I generally like-

Lex Fridman 15:35 (laughs)

Max Tegmark 15:36 ... poetry but- (laughs)

Lex Fridman 15:37 Not in this case.

Max Tegmark 15:39 It's mis-

Lex Fridman 15:39 Okay.

Max Tegmark 15:39 It's so misleading and it, uh, above all it, it short-changes us. It fails... It, it makes us underestimate what we can... The good things we can accomplish because... So, what we've been doing in my group is basically step one, train the mysterious neural network to do something well.

Lex Fridman 15:54 Mm-hmm.

Max Tegmark 15:54 And then step two, do some additional AI techniques to see if we can now transform this black box into something equally intelligent that you can actually understand.

Max Tegmark 16:07 So, for example, I'll give you one example, this AI Feynman, um, project-

Lex Fridman 16:10 Yes.

Max Tegmark 16:10 ... that we just published, right? So, we took the 100 most famous or complicated equations from one of my favorite physics textbooks, in fact, the one that got me into physics in the first place-

Lex Fridman 16:23 Mm-hmm.

Max Tegmark 16:23 ... the Feynman Lectures on Physics. And, uh, so you have a formula, you know, maybe it has... What goes into the formula is six different variables and then what comes out is one. So, then you can make like a giant Excel spreadsheet with seven columns.

Lex Fridman 16:39 Mm-hmm.

Max Tegmark 16:39 You put in just random numbers for the six columns for those six input variables and then you calculate with a formula the seventh column-

Lex Fridman 16:46 Mm-hmm.

Max Tegmark 16:46 ... the output. So, maybe it's like the force equals in the last column some function of the other, and now the task is, okay, if I don't tell you what the formula was, can you figure that out from looking at my spreadsheet I gave you?

Lex Fridman 16:59 Yes.

Max Tegmark 17:00 This problem is called symbolic regression.

Lex Fridman 17:04 Mm-hmm.

Max Tegmark 17:04 If I tell you that the formula is what we call a linear formula, so it's just that the output is, uh, some sum of all the things inputted times some constants, that's a famous easy problem we can solve. I, we do it all the time in science and engineering. But the general one if it's more complicated functions with logarithms or cosines or other math, it's a very, very hard one and probably impossible to do fast in general just because the number of formulas with n symbols, you know, just grows exponentially-

Lex Fridman 17:37 Yeah.

Max Tegmark 17:37 ... just like the number of passwords you can make grow dramatically with length. So, but what was... So, we... But we had this idea that if you first have a neural network that can actually approximate the formula, you just trained it, even if you don't understand how it works-

Lex Fridman 17:51 Mm-hmm.

Max Tegmark 17:51 ... that can be the first step-

Lex Fridman 17:54 Mm-hmm.

Max Tegmark 17:54 ... towards actually understanding how it works. So, then what... So, that's what we do first, and then we study that neural network now and put in all sorts of other data that wasn't in the original training data and use that to discover simplifying properties of the formula-... and that lets us break it apart, often into many simpler pieces, in a kind of divide and conquer approach that we... So we were able to solve all of those 100 formulas, discover them automatically-

Lex Fridman 18:21 Mm-hmm.

Max Tegmark 18:21 ... plus a whole bunch of other ones. And, uh, (laughs) it's, uh, it's actually kind of humbling to see that th- this code, which anyone who wants now, listening to this, can type "pip install A.I. Feynman" on their computer-

Lex Fridman 18:33 Mm-hmm.

Max Tegmark 18:33 ... and run it, you know, it can actually do what Johannes Kepler spent four years doing-

Lex Fridman 18:38 (laughs)

Max Tegmark 18:38 ... when he stared at Mars data-

Lex Fridman 18:40 Yeah.

Max Tegmark 18:40 ... until he's like, "Finally! Eureka! This is an ellipse!"

Lex Fridman 18:42 Yeah.

Max Tegmark 18:43 You know? (laughs) This will do it automatically for you in one hour, right? Or Max Planck, he was looking at, uh, how much radiation comes out from, at different wavelengths, from a hot object and discovered the famous blackbody formula. (laughs) This discovers it automatically. Uh, I'm, I'm actually excited about th- seeing if we can discover not just old formulas again, but new formulas that no one has seen before. S-

Lex Fridman 19:12 And do you like this process of using kind of a neural network to find some basic insights, and then dissecting the neural network to then gain the final... So that that's, in that way, you've, um... F- forcing the explainability issue, uh, f- you know, dr- really trying to analyze a neural network, uh, for the things it knows in order to come up with the final, beautiful, simple theory underlying-

Max Tegmark 19:40 Yeah.

Lex Fridman 19:40 ... the whole, uh, the init- the initial system that you were looking at?

Max Tegmark 19:42 I, I love that. And, and the reason I'm so optimistic that it can be generalized to so much more is because that's exactly what we do as-

Lex Fridman 19:52 Right.

Max Tegmark 19:52 ... human scientists. Th- think of Galileo, whom we mentioned, right? I bet when he was a little kid, if his dad threw him an apple, he would catch it.

Lex Fridman 19:59 Mm-hmm.

Max Tegmark 20:01 Why? Because he had a neural network in his brain that he had trained to predict the parabolic orbit of apples that are thrown-

Lex Fridman 20:08 Mm-hmm.

Max Tegmark 20:09 ... under gravity. If you throw a tennis ball to a dog, it also has this same ability of deep learning to figure out how the ball is gonna move and catch it. But Galileo went one step further when he got older. He went back, and he was like, "Wait a minute."

Lex Fridman 20:23 (laughs)

Max Tegmark 20:26 "I can write down a formula for this-"

Lex Fridman 20:28 Yes.

Max Tegmark 20:28 "... Y equals X squared, a parabola," you know? And he h- helped revolutionize physics as we know it, right?

Lex Fridman 20:36 So there was a basic neural network in there from childhood that captured, like, the base, the experiences of observing tr- different kinds of trajectories. And then he was able to go back in with another extra little n- neural network and analyze all those experiences and be like, "Wait a minute. There's a deeper rule here"?

Max Tegmark 20:56 Exactly. He was-

Lex Fridman 20:57 Yeah.

Max Tegmark 20:57 ... able to distill out, in symbolic form, what that complicated black box neural net was doing, right? Not only did he, the formula he get, (laughs) he got, ultimately become more accurate, you know? And, and similarly, this is how we, how Newton got Newton's Laws, which is why Elon can send rockets to the space-

Lex Fridman 21:14 Mm-hmm.

Max Tegmark 21:14 ... station now, right? So it's not only more accurate, but it's also simpler, much simpler. And it's so simple that we can actually describe it to our friends-

Lex Fridman 21:23 Mm-hmm.

Max Tegmark 21:24 ... and each other, right? Uh, we've talked about it just in the context of physics now. But hey, you know, isn't this what we're doing when we're talking to each other also? We go around with our neural networks, just like dogs and cats and chipmunks and blue jays, and we experience things in the world. But then we humans do this additional step on top of that, where we then distill out certain high-level knowledge that we've extracted from this in a way that w- and communicate it to each other in a symbolic form, in English, in this case, right? So if we can do it, and we believe that we are information, information processing entities, then we should be able to make machine learning that does it also.

Lex Fridman 22:07 Well, do you think the entire thing could be learning? Because there, um... This dissection process, like, for A.I. Feynman, the secondary stage feels like something like reasoning, and the initial step feels like more like the, the more basic kind of differentiable learning. Do you think the whole thing could be differentiable learning? Do you think the whole thing could be basically neural networks on top of each other? It's like turtles all the way down. Could it be neural networks all the way down?

Max Tegmark 22:36 I mean, it's, that's a really interesting question. We know that in, in your case, it is neural networks all the way j- down because that's all-

Lex Fridman 22:42 Allegedly.

Max Tegmark 22:42 ... you have in your skull, is a bunch of neurons doing their thing, right?

Lex Fridman 22:45 Yeah.

Max Tegmark 22:46 But, uh, if you ask the question more generally, "What, what algorithms are your brain, is your brain, are, are being used in your brain?"

Lex Fridman 22:53 Yeah.

Max Tegmark 22:54 I think super interesting to compare. I think we've gone a little bit backwards historically, because we humans first discovered good old-fashioned AI, the logic-based AI that we often call GOFAI-

Lex Fridman 23:06 Mm-hmm.

Max Tegmark 23:06 ... for good old-fashioned AI. And then, more recently, we did machine learning because it required bigger computers, so we had to discover it later. So we think of machine learning with neural networks as the modern thing and the logic-based AI as the old-fashioned thing. But if you look at y- evolution on Earth, right? It's actually been the other way around. I would, I would say that-

Lex Fridman 23:31 (laughs)

Max Tegmark 23:32 ... for example, a- an eagle has a better vision system than I have-

Lex Fridman 23:37 Mm-hmm.

Max Tegmark 23:37 ... using... And dogs are just as good at catching tennis balls as, as I am. All this stuff which is done by training a neural network and not interpreting it in words, you know, it's something so many of our animal friends can do, at least as well as us, right?

Lex Fridman 23:53 Mm-hmm.

Max Tegmark 23:53 What is it that we humans can do that the chipmunks and the eagles cannot? It's more to do with this logic-based stuff, right? Where we can extract out information in symbols, in language, and now in s- even with equations if you're a scientist, right?

Lex Fridman 24:11 Mm-hmm.

Max Tegmark 24:12 So basically what happened was, first, we built these computers that could multiply numbers real fast and manipulate symbols.

Lex Fridman 24:18 Mm-hmm.

Max Tegmark 24:18 And we felt they were pretty dumb.... and that, and then we made, uh, neural networks that can see as well as a cat can, and do a lot of the s- inscrutable black box neural networks. What we humans can do also is put the two together in a useful way.

Lex Fridman 24:34 Yes.

Max Tegmark 24:34 So artificially-

Lex Fridman 24:35 In our own brain?

Max Tegmark 24:36 Yes, in our own brain. So if we ever wanna get artificial general intelligence that can do all jobs as well as humans can, right? (clears throat) Then that's what's gonna be required, to be able to combine the neural networks with, with, um, symbolic. Combine the old AI with the new AI in a good way. We do it in our brains,

Max Tegmark 24:57 and there seems to be basically two strategies I see in industry now. One scares the heebie-jeebies out of me, and the other one I find much more encouraging.

Lex Fridman 25:05 Okay. Which one? Uh, can, can we break them apart? Which, which of the two? (laughs)

Max Tegmark 25:09 The one that scares the heebie-jeebies out of me is this attitude that we're just gonna make ever bigger systems that we still don't understand-

Lex Fridman 25:15 Uh-huh.

Max Tegmark 25:15 ... until they can do ... be as smart as humans. I ... What could possibly go wrong? (laughs) Right?

Lex Fridman 25:22 Yeah.

Max Tegmark 25:22 I, I think it's just such a reckless thing to do. And, and unfortunately ... And if we actually succeed as a species to build artificial general intelligence and we still have no clue how it works, I think at least 50% chance we're gonna be extinct before too long, and it's just gonna be an utter epic, uh, own goal, you know?

Lex Fridman 25:40 So it's that 44-minute s- uh, losing money problem or, like, the paperclip problem, like where we don't understand how it works and it just in a matter of seconds runs away in some kinda direction that's going to be very problematic.

Max Tegmark 25:54 Even long before you have to worry about the machines themselves, uh, somehow deciding to do things and ... to us, that we have to worry about people using machines that are short of AI, AGI in power, to do bad things. I mean, just t- take a moment and if, if anyone is not worried particularly about advanced AI, just take 10 seconds and just think about your least favorite leader on the planet right now. Don't tell me who it is.

Lex Fridman 26:24 Mm-hmm.

Max Tegmark 26:25 I wanna keep this apolitical (laughs) . But just see the face in front of you of that person for 10 seconds.

Lex Fridman 26:30 Yes.

Max Tegmark 26:30 Now imagine that that person has this incredibly powerful AI under their control, and can use it to impose their will on the whole planet. How does that make you feel? (laughs)

Lex Fridman 26:43 Yeah. So the, the, the ... Can, can we break that apart just briefly? For the 50% chance that we'll run into trouble with this approach, do you see the bigger worry in that leader or humans using the system to do damage? Or are you more worried ... And I think, um, I'm in this camp, more worried about, like, accidental, unintentional destruction of everything. Sort of like humans trying to do good, and like in a way where everyone agrees it's kinda good. It's just they're trying to do good without understanding, 'cause I think every-

Max Tegmark 27:20 Yeah.

Lex Fridman 27:21 ... evil leader in history thought they're ... to some degree thought they're trying to do good.

Max Tegmark 27:25 Oh yeah, I'm sure Hitler thought he was doing-

Lex Fridman 27:27 Doing good, yeah.

Max Tegmark 27:28 ... committing himself he was trying to do good too.

Lex Fridman 27:29 Stal- I've been reading a lot about Stalin. I'm sure Stalin is from ... He legitimately thought that communism was good for the world and that he was doing good. Yeah.

Max Tegmark 27:37 I think Mao Zedong thought what he was doing-

Lex Fridman 27:39 Yeah.

Max Tegmark 27:39 ... with the Great Leap Forward was good too. Yeah, uh, I th- I'm actually concerned about both of those. Uh, before ... I promised to answer this, uh, in detail, but before we do that I, let me finish answering the first question, 'cause I told you that there were two different routes-

Lex Fridman 27:53 Yes.

Max Tegmark 27:53 ... we could get to artificial general intelligence, and one scares the heebie-jeebies out of me.

Lex Fridman 27:57 (laughs)

Max Tegmark 27:57 Uh, which is this one where we build something, we just say bigger neural networks, ever more hardware and-

Lex Fridman 28:02 Yes.

Max Tegmark 28:02 ... just train the heck out of it with more data, and poof, now it's very powerful. Um, that I think is the w- the most unsafe and reckless approach. The alternative to that is the intelligent, intelligible intelligence approach instead, where we, um, say neural networks is just a tool to disc- like, for the first step to get the intuition. But then we're gonna spend also serious resources, sources on o-other AI techniques for demystifying this black box and figuring out what's it actually doing so we can convert it into something that's equally intelligent, but that we actually understand what it's doing.

Lex Fridman 28:44 Mm-hmm.

Max Tegmark 28:44 Maybe we can even prove theorems about it, that, like, this car here will never be hacked when it's driving, because here is the proof. Uh, there is a whole science of this. It doesn't work for neural networks, they, they're big black boxes, but it works well in more t- certain other kinds of codes, right?

Lex Fridman 29:01 Mm-hmm.

Max Tegmark 29:01 Uh, that approach I think is much more promising. That's exactly why I'm working on it, frankly. Not just because I think it's cool for science, but because I think the more we understand m- these systems, the better the chances that we can make them do the things that are good for us, that are actually intended not unintended. So-

Lex Fridman 29:21 So you think it's possible to prove things about something as complicated as a neural network? That's the hope?

Max Tegmark 29:28 Well, ideally ... I mean, there's no reason it has to be a neural network in the end either, right?

Lex Fridman 29:34 Right.

Max Tegmark 29:34 Like, we discovered Newton's laws of gravity with n- a neural network in Newton's head.

Lex Fridman 29:39 Yes.

Max Tegmark 29:40 But that's not the way it's programmed into the navigation system of Elon Musk's rocket anymore.

Lex Fridman 29:46 Right.

Max Tegmark 29:46 It's written in C++ or I don't know what language he uses exactly.

Lex Fridman 29:51 Yeah.

Max Tegmark 29:51 And then there are software tools called symbolic verification. DARPA and the US, uh, um military has done a lot of really great research on this 'cause they really wanna understand that when they build weapon systems, they don't just go fire at random or malfunction, right? And, um, there is even a whole operating system called Cell3 that's been developed by a DARPA grant where you can actually mathematically prove that this thing can never be hacked. Uh-

Lex Fridman 30:18 Wow.

Max Tegmark 30:19 It ... I wi- o- one day I hope that will be something you can say about the OS that's running on our laptops too. (laughs)

Lex Fridman 30:25 (laughs)

Max Tegmark 30:25 As you know w- we're not there, but I think we should be ambitious.... frankly.

Lex Fridman 30:29 Yeah.

Max Tegmark 30:30 And, and if we can use machine learning to help do the proofs and so on as well, right, then it, it's much easier to verify that a proof is correct than to come up with a proof in the first place. That's really the core idea here.

Lex Fridman 30:45 A- and-

Max Tegmark 30:45 If someone comes on your sh- on your podcast and says they've, they've proved the Riemann hypothesis or some new, sensational new theorem.

Lex Fridman 30:51 Mm-hmm.

Max Tegmark 30:53 It's not ma- o- it's much easier for some- one else take, some smart grad, math grad students to check, "Oh, there's an error here on equat- in equation five."

Lex Fridman 31:02 Mm-hmm.

Max Tegmark 31:02 Or, "This really checks out," than it was to discover the proof.

Lex Fridman 31:07 Yeah, although some of those proofs are pretty complicated. But yes, it's still, nevertheless, much easier to, uh, to verify the proof. I love the optimism. You know, we kinda, e- even with the security of systems, there's a kind of cynicism that pervades people who think about this, which is like, "Oh, it's hopeless." I mean, in the same sense exactly like you're saying with neural networks.

Max Tegmark 31:28 Yeah.

Lex Fridman 31:28 "Oh, it's hopeless to understand what's happening." Uh, with security, people are just like, "Well, it's always going, there's always going to be, um, attack vectors-

Max Tegmark 31:37 Yeah.

Lex Fridman 31:37 ... and like, uh, ways, ways to attack the system." But you're right. We're just very new at these computational systems. We're new with these intelligent systems. And, and it's not out of the realm of possi- just like people that understand the movement of the stars and the planets and so on.

Max Tegmark 31:53 Yeah.

Lex Fridman 31:54 It, it's, it's entirely possible that, like, within, hopefully soon, but it could be within a hundred years-

Max Tegmark 31:59 Yeah.

Lex Fridman 32:00 ... we start to have an obvious, like, laws of gravity about intelligence.

Max Tegmark 32:04 Yeah.

Lex Fridman 32:05 And, uh, uh, God forbid, about consciousness too. That, that one's... (laughs)

Max Tegmark 32:11 Agreed. U- you know, I think, um, of course, if you're selling computers that get hacked a lot, that's in your interest as a company that people think it's impossible to make it safe so nobody's gonna get the idea of suing you. But I wanna really inject optimism here. It, the, there i- it's, it's absolutely possible-

Lex Fridman 32:28 Yeah.

Max Tegmark 32:28 ... to do, to ... much better than we're doing now. And, you know, y- your laptop does so much stuff. You don't need the music player to be super safe in your, in your future t- self-driving car, all right? Um, if someone hacks it and starts playing music you don't like, the world won't end. But what you can do is you can break out and, say, like, the drive computer-

Lex Fridman 32:51 Mm-hmm.

Max Tegmark 32:51 ... that controls your safety must be completely physically decoupled entirely from the entertainment system, and it must physically be such that it can't take on over-the-air updates while you're driving. And it can be, it can have ... It's not that ... It can have, th- ultimately a sort- some operating system on it which is symbolically verified an- and proven, uh, that, that it's always gonna do what it's gonna, what it's supposed to do, right? Uh, we can basically have ... And companies should take that attitude too. They should look at everything they do and say, "What are the few systems in our, in our company that threaten the whole life of the company if they get hacked?" You know, and have the highest standards for them. And then they can save money by going for the el cheapo, poorly understood stuff for the rest, you know? This, this is very feasible, I think. And coming back to the bigger question about, that you, you worried about that, that there'll be unintentional failures, I think there are two quite separate risks here, right? We talked a lot about one of them, which is that the goals are noble, of the human.

Lex Fridman 33:52 Mm-hmm.

Max Tegmark 33:52 The human says, "I want this airplane to not crash," 'cause this is not Mohamed Atta now flying the airplane, right? And now there's this technical challenge of making sure that the, the autopilot is actually gonna behave as, as the pilot wants. Um, if you set that aside, there's also the separate question, how do you make sure that the goals of the pilot are actually aligned with the goals of the passenger? How do you make sure very much more broadly that if we can all agree as a species that we would like things to kind of go well for humanity as a whole, that the goals are aligned here?

Lex Fridman 34:29 Mm-hmm.

Max Tegmark 34:30 The alignment problem. And, um, yeah, there's been a lot of progress in, in the sense that there's suddenly huge amounts of research going on on it, about it. I'm, I'm very grateful to Elon Musk for giving us that money five years ago so we could launch the first research program on technical AI safety and alignment. There's a lot of stuff happening. But I think we need to do more than just make sure little machines do always what their owners do.

Lex Fridman 34:57 Mm-hmm.

Max Tegmark 34:57 Uh, you know, that wouldn't have prevented September 11 if Mohamed Atta said, "Okay, okay, autopilot, please fly into World Trade Center," you know? And it's like, "Okay." (laughs) That even happened in a different situation. There was this depressed pilot named Andreas Lubitz, right? He told his Germanwings passenger jet to fly into the Alps. He just told the computer to change the altitude to 100 meters or something like that. And you know what the computer said?

Lex Fridman 35:24 What?

Max Tegmark 35:25 "Okay."

Lex Fridman 35:26 Okay.

Max Tegmark 35:26 And it had the ph- freaking topographical map of the Alps in there. It had GPS, everything. No one had bothered teaching it even the basic kindergarten ethics of, of like, "No, uh, we never want airplanes to fly into mountains under any circumstances." Uh, and so we have to think beyond just the, (laughs) the technical issues and think about how do we align, in general, incentives on this planet for the greater good? So starting with simple stuff like that. Every airplane that has a computer in it should be taught whatever kindergarten ethics it's smart enough to understand. Like, "No, don't fly into fixed objects. If the pilot ta- tells you to do so, then go on autopilot mode, send an email to the cops, and land at the latest airport, nearest airport." You know, any car with a forward-facing camera should just be programmed by the ve- by the manufacturer so that it will never accelerate into a human, ever.

Lex Fridman 36:25 Mm-hmm.

Max Tegmark 36:26 That would av- avoid things like the Nice attack and many horrible terrorist vehicle attacks where they deliberately did that, right?This was not some sort of thing, oh, you know, US and China, different views. And no, there was not a single car manufacturer on the world, in the world, right, who wanted the cars to do this. They just hadn't thought to do the alignment. And if, if you look at, more broadly, problems that happen on this planet, th- the vast majority have to do with poor alignment. I mean, think about, let's go back really big, 'cause I know y- this is, you're so good at that.

Lex Fridman 37:00 Let's go big, yeah.

Max Tegmark 37:00 Yeah. In the very, so long ago in evolution, we had these genes.

Lex Fridman 37:04 Yes.

Max Tegmark 37:05 And you, they wanted to make copies of themselves, that's really all they cared about. So they, some gene said, "Hey, I'm gonna build a brain on this body I'm in so that I-"

Lex Fridman 37:16 Yeah.

Max Tegmark 37:16 "... I can get better at making copies of myself."

Lex Fridman 37:18 Yes.

Max Tegmark 37:18 And then they decided, for their benefit, to get copied more, to align your brain's incentives with their incentives. So it didn't want you to starve to death, so it gave you an incentive to eat.

Lex Fridman 37:32 Yes.

Max Tegmark 37:33 And it, it wanted you to make copies of, of the genes. So it gave you an incentive to fall in love and do all sorts of naughty things to-

Lex Fridman 37:42 Such as?

Max Tegmark 37:42 To make copies of it, uh, of itself, right?

Lex Fridman 37:44 Yeah.

Max Tegmark 37:44 Uh, so that was successful value alignment done on the genes. They created something more intelligent than themselves, but they made sure to try to align the values. But then something went a little bit wro- against the idea of what the genes wanted, because a lot of humans discovered, "Hey, you know, yeah, we really like this business about sex, uh, that the genes have made us enjoy. But we don't wanna have babies right now."

Lex Fridman 38:10 Yeah.

Max Tegmark 38:10 "So we're gonna hack the genes and use birth control."

Lex Fridman 38:13 Yeah.

Max Tegmark 38:14 "And I really feel like drinking a Coca-Cola right now, but I don't wanna get a pot belly, so I'm gonna drink Diet Coke." You know?

Lex Fridman 38:22 Yeah.

Max Tegmark 38:22 W- we have all these things we've figured out, because we're smarter than the genes, how we can actually subvert their intentions. So s- it's not surprising that the s- we humans now, when we are in the role of these genes, creating other non-human entities with a lot of power, have to face the same exact challenge. How do we make other powerful entities have incentives that are aligned with ours and that, so they won't hack them? Corporations, for example, right? We humans decided to create corporations because can benefit us greatly. Now all of a sudden there's a supermarket, (laughs) I can go buy food there, I don't have to hunt. Awesome. Uh, and then to b- to make sure that this corporation would do things that were good for us and not bad for us, we created institutions to keep them in check.

Lex Fridman 39:08 Mm-hmm.

Max Tegmark 39:08 Like, if the local supermarket sell, sells poisonous food, then those, some, the owners of the supermarket have to spend some years reflecting behind bars, right?

Lex Fridman 39:22 Yes. Yes.

Max Tegmark 39:22 So we, we created in- incentives, get to align them. But of course, just like we were able to see through this thing and you def- develop birth control, if you're a powerful corporation, you also have an incentive to try to hack the institutions that are supposed to govern you.

Lex Fridman 39:36 Yeah.

Max Tegmark 39:36 'Cause you ultimately, as a corporation, have an incentive to maximize your profit, just like you have an incentive to maximize the enjoyment your brain has, not through your genes. So if they can figure out a way of, of bribing regulators, then they're gonna do that. Uh, in the US, we kind of caught on to that and made laws against corruption and bribery. Uh, then, uh, in the late 1800s, uh, Teddy Roosevelt realized that, no, we were still being kind of hacked, 'cause the Massachusetts railroad companies had like a bigger budget than the state of Massachusetts, and they were doing a lot of very corrupt stuff. And so he did the whole trust-busting thing to try to align these other non-human entities, the companies, again, more with the incentives of Americans as a whole.

Lex Fridman 40:23 Mm-hmm.

Max Tegmark 40:23 Um, it's not surprising though that, you know, this is a battle you have to keep fighting. Now we have even larger companies than we ever had before and of course they're gonna try to, uh, again, subvert, uh, e- the institutions. Not because, you know, I think people make the mistake of getting all too, um, bl- thinking about things in terms of good and evil, like arguing about whether corporations are good or evil or whether robots are good or evil. A robot isn't good or evil, it's a tool and you can use it for great things, like robotic surgery, or for bad things. And a corporation also is a tool, of course. And if you have good incentives to the corporation, it'll do great things, like start a hospital or a grocery store. If you have really bad incentives, then it's gonna start maybe marketing addictive drugs to people, and you'll have an opioid epidemic, right? Uh, it's all about... I don't want, we should, we not make the mistake of getting into some sort of fairytale, good/evil thing about corporations or robots. We should focus on putting the right incentives in place. My optimistic vision is that if we can do that, you know, then we can really get good things. We're not doing so great with that right now, either on AI, I think, or on other intelligent non-human entities, like big companies, right? We just have a new, um, Secretary of, of Defense, who's gonna start up now in, in the Biden adminis- administration, who is, was an active member of the board of Raytheon, for example.

Lex Fridman 41:58 Uh, yeah.

Max Tegmark 41:59 So, you know, I have nothing against Raytheon. Uh, I'm al- I'm not a pacifist, but there's a obvious conflict of interest if someone is in the job where they decide wh- who they're gonna contract with. And I think somehow we have, uh, maybe we need another Teddy Roosevelt to come along again and say, "Hey, you know, we want what's good for all Americans, and we need to go do some serious realigning again of the incentives that we're giving to these big companies." And, um...... then we're going to be

Max Tegmark 42:33 better off.

Lex Fridman 42:33 It seems that naturally with human beings, just like you've beautifully described the history of this whole thing, uh, of it all started with the genes, and they're, they're probably pretty upset by all the unintended consequences that happened since.

Max Tegmark 42:45 (laughs)

Lex Fridman 42:45 But the, it seems that it kind of works out. Like, it's in this collective intelligence that emerges at the different levels, it seems to find, sometimes last minute, a way to realign the values or keep the values aligned. It's almost, um, it finds a way, like, uh, different leaders, different humans pop up all over the place that, uh, reset the system. Do, do you, one, I mean, (laughs) do you have an explanation why that is? Or is that just survivor bias?

Max Tegmark 43:17 Uh...

Lex Fridman 43:17 And also, is that different, somehow fundamentally different than with AI systems, where, um, you're no longer dealing with something that was a, a direct, maybe companies are the same, a direct byproduct of the evolutionary process?

Max Tegmark 43:33 I think there is one thing which has changed. That's why I'm not (laughs) all optimistic. That's why I think there's about a 50% chance if we, if we take the dumb route with, um, artificial intelligence, that, that we, human, humanity will be extinct in this century. Uh, f- first, just the big picture, yeah, companies need to have the right incentives. Even governments, right? We used to have governments, usually there were just some king, you know, who was the king because his dad was the king, you know? And, and then there were some benefits of having this powerful kingdom because, or empire of any sort, because then it could prevent a lot of local squabbles.

Lex Fridman 44:18 Right.

Max Tegmark 44:18 So at least everybody in that region would stop warring against each other, and their incentives of different cities in the kingdom became more aligned, right? That's, that was the whole selling point.

Lex Fridman 44:26 (laughs) Yes.

Max Tegmark 44:27 Harari.

Lex Fridman 44:28 Yeah.

Max Tegmark 44:29 Yuval Harari has a beautiful piece on how empires were collaboration enablers, and then we also... Harari says he invented money for that reason, so we could have better alignment, and we could do trade even with people we didn't know. So, this sort of stuff has been playing out since time immemorial, right? What's changed is that it happens on ever-larger scales, right? The technology keeps getting better because science gets better. So now we can communicate over larger distances, transport things fast over larger distances, and the, so the entities get ever bigger, but our planet is not getting bigger anymore. So, in the past, you could have one experiment that just totally screwed up, like Easter Island, where they actually managed to have such poor alignment that when they went extinct, people there, (laughs) there was no one else to come back and replace them, right?

Lex Fridman 45:21 Yes.

Max Tegmark 45:21 If Elon Musk doesn't get us to Mars, and then we go extinct on a global scale, then we're not coming back. That's, that's the fundamental difference, and that's an exp- a mistake I would rather that we don't make for that reason. In the past, of course, history is full of fiascos, right? But it was never the whole planet.

Lex Fridman 45:41 Mm-hmm, right.

Max Tegmark 45:42 And, and then, okay, now there's this nice uninhabited land here, some other people could move in and organize things better. This is different. The second thing, which is also different, is that technology gives us so much more empowerment, right, both to do good things and also to screw up. In the Stone Age, even if you had someone whose goals were really poorly aligned, like maybe he was really pissed up- off because his Stone Age girlfriend dumped him, and he just wanted to... If he wanted to, like, kill as many people as he could-

Lex Fridman 46:12 Yeah.

Max Tegmark 46:12 ... how many could he really take out with a rock and a stick before he was overpowered, right?

Lex Fridman 46:16 Right.

Max Tegmark 46:17 Just a handful, right?

Lex Fridman 46:18 Yeah.

Max Tegmark 46:18 Uh, now... with today's technology, if we have an accidental nuclear war between Russia and the US, which we almost have about a dozen times, and then we have a nuclear winter, it could take out seven billion people, or six billion people, or we don't know. Uh, so the, so the scale of the damage is bigger that we can do, and if, if, um, it's, there's obviously no law of physics that says that technology will never get powerful enough that we could wipe out our species entirely. That would just be fantasy to think that science is somehow doomed not to get more powerful than that, right? And, and it's not at all unfeasible in our lifetime that someone could design a designer pandemic which spreads as easily as COVID, but just basically kills everybody. We already had smallpox. It killed one third of everybody who got it. And, and, um...

Lex Fridman 47:13 What do you think of the... here's an intuition, maybe it's completely naive and this optimistic intuition I have, which it seems, and maybe it's a biased experience that I have, but it seems like the most brilliant people I've met in my life all are g- really, like, fundamentally good human beings. And not like naive good, like they really want to do good for the world in a way that, well, maybe is aligned to my sense of what good means. And so I have a sense that the, uh, the people that will be defining the very cutting edge of technology, there will be much more of the ones that are doing good versus the ones that are doing evil. So the race, I'm optimistic on the us always, like, last minute coming up with a solution. So if, if there's an engineered pan- pandemic that has the capa- uh, capability to destroy most of the human civilization, it, it feels like to me either leading up to that, before, or as it's going on, there will be, uh, we're able to rally the, the collective genius of the human species. (laughs) I could tell by your smile that you're, uh-

Max Tegmark 48:26 (laughs)

Lex Fridman 48:26 ... at least some percentage, uh, um, doubtful. But is there, could that be a fundamental law of human nature, that evolution only create, it creates, uh, like, karma is beneficial, good is beneficial, and therefore we'll be all right?

Max Tegmark 48:44 I (laughs) hope you're right. I would really love it if you're right, if there's some sort of law of nature that says that we always get lucky in the last second-

Lex Fridman 48:52 (laughs)

Max Tegmark 48:52 ... because of karma. But, you know, (laughs) I prefer, I prefer, uh, not playing it so close-

Lex Fridman 49:01 Yes.

Max Tegmark 49:01 ... and, and gambling on that. And I, I think, um, in fact, I think it can be dangerous to have too strong faith in that-

Lex Fridman 49:08 Yes.

Max Tegmark 49:08 ... because it, it, um, makes us complacent. Like, if someone tells you, "You never have to worry about your house burning down," then you're not gonna put in a smoke detector, 'cause why would you need to, right?

Lex Fridman 49:16 Right.

Max Tegmark 49:17 Even if it's sometimes very simple precautions, we don't take them. If you're like, "Oh, the government is gonna take care of everything for us. I can always trust my politicians. I can always..." We abdicate our own responsibility. I think it's a healthier attitude to say, "Yeah, maybe-

Lex Fridman 49:30 Yes.

Max Tegmark 49:30 ... things will work out. Maybe I'm actually gonna have to, myself, step up and take responsibility." Uh, and the stakes are so huge. I mean if we, if we do this right, we can develop all this ever more powerful technology, and cure all diseases, and create a future where humanity is healthy and wealthy for not just the next election cycle, but like billions of years throughout our universe. That's really worth working hard for, and not just, you know, sitting and hoping for-

Lex Fridman 49:59 Hoping.

Max Tegmark 49:59 ... some sort of fairytale karma.

Lex Fridman 50:00 Well, I, I just mean-

Max Tegmark 50:01 Uh-

Lex Fridman 50:01 So, you're, you're absolutely right. From the perspective of the individual, like for me-

Max Tegmark 50:05 Yeah.

Lex Fridman 50:05 ... like the primary thing should be to take responsibility and to build the solutions that your skill set allows to build.

Max Tegmark 50:12 Yeah.

Lex Fridman 50:12 Just-

Max Tegmark 50:12 Which is a lot. I think we underestimate, often, very much how much good we can do.

Lex Fridman 50:16 Right.

Max Tegmark 50:17 If the, uh, if you or anyone listening to this that is completely confident that our government would do a perfect job on handling any future crisis with-

Lex Fridman 50:26 (laughs)

Max Tegmark 50:26 ... engineered pandemics or future AI, I ask you to reflect a bit on-

Lex Fridman 50:31 The, the one or two people out there.

Max Tegmark 50:32 ... on, on, on what actually happened in 2020.

Lex Fridman 50:36 (laughs)

Max Tegmark 50:37 Do you feel that the government, by and large, around the world, has handled this flawlessly?

Lex Fridman 50:43 Uh, that's a really sad and disappointing reality that, uh, ho- hopefully is a wake-up call for everybody. Uh, for the scientists, for the, for the re- for the engineers, for the researchers in AI especially, it was disappointing to see how, uh, inefficient we were as spreading, uh, uh, collecting the right amount of data in a privacy-preserving way, and spreading that data and utilizing that data to make decisions, all that kind of stuff.

Max Tegmark 51:11 Yeah.

Lex Fridman 51:11 Yeah.

Max Tegmark 51:11 I think when something bad happens to me, I made myself, uh, a promise many years ago that I would not be a whiner.

Lex Fridman 51:21 (laughs) Yes.

Max Tegmark 51:22 And so when something bad happens to me, of course, it's, uh, yeah, you have to process the disappointment, but then I try to focus on what did I learn from this that can make me a better person in the future? And there's usually something to be learned when I fail. And I think we should all ask ourselves, what can we learn from the pandemic about how we can do better in the future? And you mentioned there a really good lesson. You know, we were not as resilient as we thought we were, and we were not as prepared, maybe, as we wished we were. You can even see very stark contrasts around the planet. South Korea, right, they have over 50 million people. Do you know how many deaths they have from COVID last time I checked?

Lex Fridman 52:05 No.

Max Tegmark 52:06 About 500.

Lex Fridman 52:08 (laughs)

Max Tegmark 52:08 You know, why is that? Well, the short answer is that they had prepared. They were incredibly quick, incredibly quick to get on it with very rapid testing and contact tracing and so on, which is why they never had more cases than they could contact trace effectively, right?

Lex Fridman 52:29 Mm-hmm.

Max Tegmark 52:30 They never even had to have the kind of big lockdowns we had in the West.

Lex Fridman 52:32 Mm-hmm.

Max Tegmark 52:33 But the deeper answer to... It's, it's not just the Koreans are just somehow better people. The reason I think they were better prepared was because they had already had a pretty bad hit from the SARS pandemic-

Lex Fridman 52:47 Okay.

Max Tegmark 52:47 ... or which never became a pandemic, uh, something like 17 years ago, I think.

Lex Fridman 52:51 Mm-hmm.

Max Tegmark 52:52 So it was kind of a fresh memory that, you know, we need to be prepared for pandemics. So they were, right? And so maybe this is a lesson here for all of us to draw from COVID, that rather than just wait for the next pandemic or the next problem with AI getting out of control or anything else, maybe we should just actually set aside a tiny fraction of, of our GDP to have people very systematically do some horizon scanning and say, "Okay, what are the things that could go wrong? And let's duke it out and see which are the more likely ones and which are the ones that are actually actionable," and then be prepared. Uh...

Lex Fridman 53:31 So one of the observations, as one little ant/human that I am, of disappointment is the political division over information that has been observed, that I observed this year, that it seemed, uh, the discussion was less about, um, sort of, uh, what happened and understanding what happened deeply, and more about there's different truths out there. And it's like a argument, my truth is better than your truth, and it's, it's like red versus blue, or different, like, it, it was like this ridiculous discourse that doesn't seem to get at any kind of notion of the tru- It's not like a, some kind of scientific process. Even science got politicized-

Max Tegmark 54:20 Yeah.

Lex Fridman 54:20 ... in ways that's very-

Max Tegmark 54:22 Yeah.

Lex Fridman 54:22 ... heartbreaking to me. Uh, you have an exciting project on the AI front, uh, of trying to rethink one of the, you mentioned corporations. There's one of the other collective intelligence systems that have emerged through all of this is social networks, and just the spread... The internet is, is the spread of information on the-... uh, the internet, our ability to share that information. There's all different kinds of news sources and so on. And so you said, like, let's, from first principles, let's rethink how we think about the news, how we think about information. Can you talk about this, uh, amazing effort that you're undertaking?

Max Tegmark 55:03 Oh, I'd love to. This has been my big COVID project-

Lex Fridman 55:06 (laughs)

Max Tegmark 55:06 ... to spend nights and weekends on, ever since the lockdown. To segue into this actually, let me come back to what you said earlier, that you had this hope that, in your experience, people who you felt were very talented were often idealistic and wanted to do good. Frankly, I- I feel the same about all people, by and large. There are always exceptions. But I think the vast majority of everybody, regardless of education and whatnot, really are fundamentally good, right? So, how can it be that people still do so much nasty stuff, right?

Lex Fridman 55:38 Mm-hmm.

Max Tegmark 55:39 I think it has everything to do with this, with the information that we're given.

Lex Fridman 55:42 Yes.

Max Tegmark 55:43 You know? If you go into Sweden 500 years ago and you start telling all the farmers that, "Those Danes in Denmark, they're so terrible people," you know, and, "We have to invade them-"

Lex Fridman 55:53 Yes.

Max Tegmark 55:54 ... because they've done all these terrible things, that you can't fact check yourself, eh, a lot of peop- Swedes did that, right? And it, and, um, we've seen, we're seeing so much of this today in the world, both geopolitically, you know, where we are told that, that China is bad and Russia is bad and Venezuela is bad, and people in those countries are often told that we are bad. And we also see it at a micro level, you know, where people are told that, "Oh, those who voted for the other party are bad people." It's not just an intellectual disagreement, but they're bad people. And, um, we're getting ever more divided. And so how do you reconcile this with, with this intrinsic, uh, goodness, I, in people? I, I, I think it's pretty obvious that it has, again, to do with this, with the information that we're fed and given, right? We evolved to live in small groups where you might know 30 people in total, right? So you then had a system that was quite good-

Lex Fridman 56:56 Mm-hmm.

Max Tegmark 56:56 ... for assessing who you could trust and who you could not. And if someone told you that, you know, "Joe there is a jerk," but you had interacted with him yourself and seen him in action, and, and you would pre- quickly realize maybe that that's actually not quite accurate, right? Uh, but now that we ... the most people on the planet are people we've never met, it's very important that we have a way of trusting the information we're given. And so, okay, so where does the news project come in? Well, throughout history, you can go read Machiavelli, you know, from the 1400s, and you'll see how already then, they were busy manipulating people with propaganda and stuff. Propaganda is not new at all.

Lex Fridman 57:35 (laughs)

Max Tegmark 57:35 And the incentives to manipulate people is just not new at all. What is it that's new? What's new is machine learning meets propaganda. That's what's new. That's why this has gotten so much worse. You know, some people like to blame certain individuals. Like, in my liberal university bubble, many people blame Donald Trump and say it was his fault. I see it differently. I think that what ha- ... Donald Trump just had this extreme skill at playing this game in the machine learning algorithm age, a game he couldn't have played, you know, 10 years ago. So what's changed? What's changed is, well, Facebook and Google and other companies, and I- I don't want ... I'm not bad-mouthing them. I have a lot of friends who work for these companies. They're good people. They deployed machine learning algorithms just to increase their profit a little bit, to just maximize the time people spent watching ads. And they had totally underestimated how effective they were gonna be. This was, again, the black box, non-intelligible intelligence.

Lex Fridman 58:38 Mm-hmm.

Max Tegmark 58:39 They just noticed, "Oh, we're getting more ad revenue. Great." It took a long time until they even realized why and how, and how damaging this was for society. 'Cause of course, what the machine learning figured out was that the by far most effective way of gluing you to your little rectangle was to show you things that triggered strong emotions-

Lex Fridman 58:58 Mm-hmm.

Max Tegmark 58:59 ... anger, et cetera, resentment. And, uh, if it was true or not didn't really matter. It was also easier to find stories that weren't true if you weren't limited. That's just a limitation-

Lex Fridman 59:12 Right. (laughs)

Max Tegmark 59:12 ... to show people.

Lex Fridman 59:13 That's a- that's a very limiting factor, yes.

Max Tegmark 59:16 And before long, we've got these amazing filter bubbles on a scale we had never seen before.

Lex Fridman 59:21 Yeah.

Max Tegmark 59:22 Couple this to the fact that also the online news media were so effective that they killed a lot of print journalism. There is on- ... There's on- less than half as many journalists now in America, I believe, as there was, you know, a generation ago. You just couldn't compete with the online advertising. So all of a sudden, most people are not getting ... even reading newspapers. They get, get their news from social media. And most people only get news (laughs) in their little bubble. And so along comes now some people like Donald Trump who, who figured out ... among the first successful politicians to figure out how to really play this new game and become very, very influential. But I think that w- ... Donald Trump was a symp- ... Well, he, he, he took advantage of it. He didn't create ... The, the fundamental conditions were created by machine learning sort of taking over the news media. So this is what motivated my little COVID project here. So, you know, I said before, machine learning and tech in general is not evil, but it's also not good. It's just a tool-

Lex Fridman 1:00:30 Mm-hmm.

Max Tegmark 1:00:30 ... that you can use for good things or bad things. And as it happens, machine learning in news is mainly used by the big players, big tech, to manipulate people into watch as many ads as possible, which had this unintended consequence of really screwing up our democracy into ... and fragmenting it into filter bubbles.So, I thought, well, machine learning algorithm, they're basically free. They can run on your smartphone for free also if someone gives them away to you, right? There's no reason why they only have to help the big guy-

Lex Fridman 1:01:00 Mm-hmm.

Max Tegmark 1:01:01 ...to manipulate the little guy. They can just as well help the little guy to-

Lex Fridman 1:01:05 Mm-hmm.

Max Tegmark 1:01:05 ...see through all the manipulation attempts from the big guy. So, did this project, it's called im- you can go to improvethenews.org. It, the first thing we built is the li- just a little news aggregator. Looks a bit like Google News, except it has these sliders on it to help you break out of your filter bubble. So, if you're, you're reading, you can click, click and go to your favorite topic, and then, um, if you just slide the left/right slider away, all the way over to the left-

Lex Fridman 1:01:32 There's two sliders, right?

Max Tegmark 1:01:33 Yeah. Th- there's the one, the ob- most obvious one is the one that has left/right labeled on it. You go to the left, you get one set of articles, you go to the right, you see a very different truth-

Lex Fridman 1:01:43 Yeah.

Max Tegmark 1:01:43 ...appearing.

Lex Fridman 1:01:44 Oh, that's literally left and right on the-

Max Tegmark 1:01:46 Political spectrum.

Lex Fridman 1:01:47 ... to the po- on the political spectrum.

Max Tegmark 1:01:48 Yeah, so if you're reading about immigration, for example, it- it's very, very str- noticeable. And- and I think step one always if you wanna not get manipulated is just to be able to recognize the techniques people use, so it's very helpful to just see how they spin things on the two sides. Uh, I think, uh, many people are under the misconception that the main problem is fake news.

Lex Fridman 1:02:13 Mm-hmm.

Max Tegmark 1:02:14 It's not. Uh, we, I had an amazing team of MIT students where we did an academic project, used machine learning to detect the main kinds of bias over the summer, and yes, of course, sometimes there's fake news, where someone just claims something that's false-

Lex Fridman 1:02:29 Mm-hmm.

Max Tegmark 1:02:29 ...right? Like, "Oh, Hillary Clinton just got divorced," or something.

Lex Fridman 1:02:33 Yes.

Max Tegmark 1:02:34 But what we see much more of is actually just omissions. If you go to, uh, there's some stories which just won't be mentioned by the left or the right because it doesn't suit their agenda, and then they'll instead mention other ones very, very, very much.

Lex Fridman 1:02:50 Mm-hmm.

Max Tegmark 1:02:50 So, for example, we've, we've had a lo- a number of stories about the Trump family's financial dealings.

Lex Fridman 1:02:59 Mm-hmm.

Max Tegmark 1:03:00 And then there's been a some st- a bunch of stories about the Biden family's, Hunter Biden's financial dealings, right?

Lex Fridman 1:03:05 Mm-hmm.

Max Tegmark 1:03:06 Surprise, surprise, they don't get equal coverage on the left and the right.

Lex Fridman 1:03:09 That's right.

Max Tegmark 1:03:10 One side loves to cover the Biden, Hunter Biden stuff, and one side loves to cover the Trump-

Lex Fridman 1:03:16 You can't-

Max Tegmark 1:03:16 ... you can never guess which is which, right?

Lex Fridman 1:03:17 (laughs) Yeah.

Max Tegmark 1:03:18 These... But the great news is if you wanna, if you are a normal American citizen and you dislike corruption in all its forms, then slide, slide, you can just get, look at both of-

Lex Fridman 1:03:28 Yes.

Max Tegmark 1:03:29 ... the sides and you'll, you'll see all the corrupt- (laughs) those political corruption stories.

Lex Fridman 1:03:33 It's really liberating to just take in the both sides, the spin on both sides. It somehow unlocks your mind to like think on your own, to realize that, that... I don't know, it's- it's the same thing that was useful, right, in the Soviet Union times for when- when y- everybody was much more aware that they're surrounded by propaganda, right?

Max Tegmark 1:03:58 That is so interesting what you're saying, actually. Uh, so Noam Chomsky, you know, used to be our MIT colleague, once said that propaganda is to democracy what violence is to totalitarianism. And- and- and what he means by that is if- if you have a really totalitarian government, you don't need propaganda.

Lex Fridman 1:04:19 Right.

Max Tegmark 1:04:20 People will do what you tell (laughs) -

Lex Fridman 1:04:21 (laughs)

Max Tegmark 1:04:21 ...what you want them to do anyway but out of fear, right?

Lex Fridman 1:04:24 Yes.

Max Tegmark 1:04:25 But otherwise, you need propaganda. (laughs) So I- I would say, actually, that the propaganda is much higher quality in democracies-

Lex Fridman 1:04:33 (laughs)

Max Tegmark 1:04:33 ... much more believable, and it's really s-

Lex Fridman 1:04:35 That's brilliant. (laughs)

Max Tegmark 1:04:36 ... it- it's really striking. When I talk to colleagues-

Lex Fridman 1:04:39 Yeah.

Max Tegmark 1:04:39 ... science colleagues, like from Russia and China and so on, I notice they are actually much more aware of the propaganda in their own media than many of my American colleagues are about the propaganda in Western media.

Lex Fridman 1:04:51 That's brilliant. That means the propaganda in the Western media is just better.

Max Tegmark 1:04:55 Yes!

Lex Fridman 1:04:56 That's so brilliant. (laughs)

Max Tegmark 1:04:57 Everything's better in the West, even the propaganda. (laughs)

Lex Fridman 1:05:00 (laughs) Oh, man.

Max Tegmark 1:05:01 So- so- so- but the- the- (laughs)

Lex Fridman 1:05:04 (laughs) .

Max Tegmark 1:05:04 (laughs) But-

Lex Fridman 1:05:05 That's good.

Max Tegmark 1:05:05 ... but when you, when once you realize that, you realize there's also something very optimistic there that you can do about it, right? Because first of all, omissions, as long as there's no outright censorship, you can just look at both sides and pretty quickly s- piece together a much more accurate idea of what's actually going on, right?

Lex Fridman 1:05:26 And develop a natural skepticism, too.

Max Tegmark 1:05:28 Yeah.

Lex Fridman 1:05:28 Just a-

Max Tegmark 1:05:29 Yeah.

Lex Fridman 1:05:29 ... a- a- analytical, scientific mind-

Max Tegmark 1:05:31 Yeah.

Lex Fridman 1:05:31 ... about the way you take in information.

Max Tegmark 1:05:32 Yeah, and I- I think, I have to say, sometimes I feel that some of us in the academic bubble are- are too arrogant about this and somehow think, "Oh, it's just people who aren't as, 'oh, as educated as us,'" or, "Who-"

Lex Fridman 1:05:45 (laughs) Yeah.

Max Tegmark 1:05:45 ... when we are often just as gullible also, you know-

Lex Fridman 1:05:48 Yes.

Max Tegmark 1:05:48 ... 'cause we read only our media and- and don't see through things. Anyone who looks at both sides like this and compares a little will immediately start noticing the shenanigans being pulled, and- and, you know, I think wh- what I try to do with- with this app is that the big tech has to some extent, uh, try to blame the individual for being manipulated, much like big tobacco tried to blame the individuals entirely for smoking. And- and a little later on, you know, our government stepped up and said, "Actually, you know, you can't just blame little kids for starting to smoke. We have to have more responsible advertising," and this and that. I think it's a bit the same here. It's very convenient for big tech to blame, so, "It's just people who are so dumb and get fooled."

Lex Fridman 1:06:29 Yeah.

Max Tegmark 1:06:30 Uh, the blame usually comes in saying, "Oh, it's just human psychology. People just wanna hear what they already believe." But Professor David Rand at MIT actually partly debunked that with a really nice study showing that people are, tend to be interested in hearing things that go against what they believe if it's presented in a respectful way, like-

Lex Fridman 1:06:52 Yeah.

Max Tegmark 1:06:52 ... suppose, for example, that, um-You have a company, and you're just about to launch this project, and you're convinced it's gonna work, and someone says, "You know, Lex, I hate to tell you this, but this is gonna fail and here's why." Would you be like, "Shut up, I don't wanna hear it. La-la-la-la-la-la-la-la-la"?

Lex Fridman 1:07:10 (laughs) Yeah, yeah.

Max Tegmark 1:07:10 Would you? You would be interested, right?

Lex Fridman 1:07:13 Yes.

Max Tegmark 1:07:13 And also, if, if you are on an airplane, um, back in the CO- pre-COVID times, you know-

Lex Fridman 1:07:18 (laughs)

Max Tegmark 1:07:19 ... and the guy next to you is clearly from the opposite side of the political spectrum, but is very respectful and polite to you, wouldn't you be kind of interested to hear a bit about how h- h- he or she thinks about things?

Lex Fridman 1:07:32 Of course.

Max Tegmark 1:07:33 Right? But it's not so easy to find out respectful disagreement now, because, like, for example, if you are a Democrat, and you're like, "Oh, I wanna see something on the other side," so you just go to breitbart.com-

Lex Fridman 1:07:45 Mm-hmm.

Max Tegmark 1:07:46 ... and then after the first 10 seconds, you feel deeply insulted by something, and they ... It's- it's not gonna work. Or if you take someone who votes Republican, and they go to something on the left, and they just get very offended very quickly by them having put a deliberately ugly picture of Donald Trump on the front page or something, it doesn't really work. So this news aggregator also has a nuance slider-

Lex Fridman 1:08:10 Mm-hmm.

Max Tegmark 1:08:10 ... which you can pull to the right and then ... To make it easier to get exposed to actually more sort of academic-style or more respectful, um-

Lex Fridman 1:08:18 Mm-hmm.

Max Tegmark 1:08:19 ... port-

Lex Fridman 1:08:19 That's brilliant, yeah.

Max Tegmark 1:08:20 ... portrayals of, of different views. And, and finally, the one kind of bias I think people are mostly aware of is the left/right, right?

Lex Fridman 1:08:28 Mm-hmm.

Max Tegmark 1:08:28 Because it's so obvious. Because both left and right are very powerful here, right? Both of them have well-funded TV stations and newspapers, and it's kind of hard to miss. But there's another one, the establishment slider, which is, is also really fun. I, I, I, I love to play with it.

Lex Fridman 1:08:45 Yeah. I love that one.

Max Tegmark 1:08:45 And that's more about corruption.

Lex Fridman 1:08:47 Yes.

Max Tegmark 1:08:47 Because if you have a, a society that, where almost all the powerful entities want you to believe a certain thing, that's what you're gonna read in both the big media, mainstream media on the left and on the right, of course.

Lex Fridman 1:09:04 Mm-hmm.

Max Tegmark 1:09:04 And, uh, powerful companies can push back very hard, like tobacco companies pushed back very hard back in the day when peop- some newspapers started writing articles about tobacco being dangerous, so they wa- it was hard to get a lot of coverage about it initially. And also if you look geopolitically, right? Of course, in any country, when you read their media, you're mainly gonna be reading a lot about articles about how our country is the good guy-

Lex Fridman 1:09:26 Mm-hmm.

Max Tegmark 1:09:27 ... and the other countries are the bad guys.

Lex Fridman 1:09:29 Mm-hmm.

Max Tegmark 1:09:29 Right? So if you wanna have a really more nuanced understanding, you know, like, the Germans used to be told that the ... No, the f- the British used to be told that the French were the bad guys, and the French used to be told (laughs) that the British were the bad guys. Now they v- visit each other's countries a lot and have a much more nuanced understanding. I don't think there's gonna be any more wars between France and Germany, but on the geopolitical scale, it's just, just as much as ever, you know, big Cold War, now US, China, and so on. And if you wanna, if you wanna get a more nuanced understanding of what's happening geopolitically, then it's really fun to look at this establishment slider, because there are, it turns out there are tons of little newspapers, both on the left and on the right, who sometimes challenge, um, establishment and say, you know, "Maybe we shouldn't actually invade Iraq right now. Maybe this weapons of mass destruction thing is BS." If you look at the journalism research afterwards, you can actually see that quite clearly, that both CNN and Fox were very pro-"Let's in- get rid of Saddam. There are weapons of mass destruction." Then there were a lot of smaller newspapers, they were like, "Wait a minute. This evidence seems a bit sketchy, and maybe we ..." But of course, they were so hard to find. Most people didn't even know they existed, right? Yet it woulda been better for American national security if those voices had also come up.

Lex Fridman 1:10:49 Mm-hmm.

Max Tegmark 1:10:50 I think it harmed America's national security actually-

Lex Fridman 1:10:52 And-

Max Tegmark 1:10:52 ... that we invaded Iraq.

Lex Fridman 1:10:53 And arguably there's a-

Max Tegmark 1:10:54 ... that's not-

Lex Fridman 1:10:54 ... lot more interest in that kind of thinking too from those small sources. So, like, the, when you say big, it's more about kind of, uh, the reach of the broadcast, um-

Max Tegmark 1:11:08 Mm-hmm.

Lex Fridman 1:11:08 ... but it's not big in terms of the interest. I think-

Max Tegmark 1:11:12 Yeah.

Lex Fridman 1:11:12 I think there's a lot of interest in that kind of anti-establishment or, like, skepticism towards, you know, o- out of the box thinking.

Max Tegmark 1:11:20 Yeah. Yeah.

Lex Fridman 1:11:20 There's a lot of interest in that kind of thing. Do you see this, uh, news project or something like it being, um, basically taking over the world as, as the main way we consume information? Like, what's-

Max Tegmark 1:11:33 (laughs)

Lex Fridman 1:11:33 How do we get, how do we get there? Like, how, how do we ... You know. So, okay, the idea is brilliant. It's a, it's a bea- You cal- you're calling it your little project (laughs) -

Max Tegmark 1:11:42 (laughs)

Lex Fridman 1:11:43 ... in 2020, but how does that become the new way we consume information?

Max Tegmark 1:11:48 I hope, first of all, just to plant a little seed there, because normally the big barrier of doing anything in media is you, you need a ton of money, but-

Lex Fridman 1:11:57 Right.

Max Tegmark 1:11:57 ... this cost no money at all. I've just been paying (laughs) myself. You pay a tiny amount of money each month to Amazon to run the thing in their cloud.

Lex Fridman 1:12:04 Yeah.

Max Tegmark 1:12:04 We're not ... There never will never be any ads. The, the point is not to make any money off of it. And we've, we just train machine learning algorithms to classify the articles and stuff, so it just kinda runs by itself. So if it actually gets good enough at some point that it starts catching on, it could scale, and if other people carbon copy it and make other versions that are better, that's ... More, the more the merrier. I, I think there's a real opportunity for machine learning to empower the individual against, uh, this sty- the powerful f- players. Uh, it's b- as I said in the beginning here, it's been mostly the other way around so far, that the big-

Lex Fridman 1:12:43 Right.

Max Tegmark 1:12:43 ... players have the AI, and then they tell people, "This is the truth. This is how it is." But it can just as well go the other way around. And w- when the internet was born actually, a lot of people had this hope that maybe this will be a great thing for democracy, make it easier to find out about things.... and maybe machine learning and things like this can, uh, can actually help again. And I have to say, I think it's as imp- it's more important than ever now, right? Because this is very linked, also, to the whole future of life, as we discussed earlier, right? We're getting this ever more powerful tech, you know? It, it, frankly, it's pretty clear, if you look on the one or two generation, three generation times scale that there are only two ways this can end, geopolitically.

Lex Fridman 1:13:24 Yeah.

Max Tegmark 1:13:25 Either it ends great for all of humanity or (laughs) it ends terribly for all of us. There's, there's really no in between. It... And-

Lex Fridman 1:13:34 (laughs)

Max Tegmark 1:13:34 ... we're so stuck in, in that... Because, you know, technology knows no borders.

Lex Fridman 1:13:38 Mm-hmm.

Max Tegmark 1:13:39 And you can't have people fighting when the weapons just keep getting ever more powerful, uh, indefinitely. Eventually, uh, lux, the luck runs out. And, and, you know, the, the... Right, like right now we have... I love America, but the, the fact of the matter is, what's good for America is not opposite, in the long term, to what's good for other countries. It, it would be if this was some sort of zero sum game like it was thousands of years ago, when the only way one country could get more resources was to take land from other countries, 'cause that was basically the resource, right?

Lex Fridman 1:14:16 Mm-hmm.

Max Tegmark 1:14:17 Look at the map of Europe. Some countries kept getting bigger and smaller with endless wars. But then, since 1945, there hasn't been any war in Western Europe, and they all got way richer because of tech. So the, the optimistic outcome is that the big winner in this century is gonna be America and China and Russia and everybody else, because technology just makes us all healthier and wealthier, and we just find some way of keeping the peace on this planet. Uh, but I think, unfortunately, right, there are some pretty powerful forces right now that are pushing in exactly the opposite direction and trying to demonize other countries, uh, which is just... Makes it more likely that this ever more powerful tech we're building is gonna-

Lex Fridman 1:15:00 Will be used for-

Max Tegmark 1:15:01 ... be used in disastrous ways.

Lex Fridman 1:15:02 Yeah, for aggression versus cooperation, that kind of thing.

Max Tegmark 1:15:05 Yeah, even look at, look at just military AI now, right? Those 2020, um, it was so awesome to see these dancing robots. I loved it, right?

Lex Fridman 1:15:13 (laughs)

Max Tegmark 1:15:13 Uh, but one of the biggest growth areas in robotics now is of course autonomous weapons, right? And, and 2020 was like the best marketing, uh, year ever for autonomous weapons, because in both Libya Civil War and in Nagorno-Karabakh, they made the decisive difference, right? And everybody else is like watching this, "Oh yeah, we wanna build autonomous weapons too." In, in Libya, you had on one hand our ally, the United Arab Emirates, that were flying their autonomous weapons that they bought from China, bombing Libyans, and on the other side you had our other ally, Turkey, flying their drones. And they, they had no skin in the game, any of these other countries, and of course it was the Libyans who really got screwed. In Nagorno-Karabakh you had actually... Again, so now Turkey is sending drones built by this company that was actually founded by a guy who went to MIT AeroAstro. Do you know that?

Lex Fridman 1:16:17 No.

Max Tegmark 1:16:18 Bakroutiar? Yeah?

Lex Fridman 1:16:19 Mm-hmm.

Max Tegmark 1:16:19 So MIT has a direct responsibility for, ultimately, this, and a lot of civilians were killed there, you know? And s- so because it was militarily so effective, now, now suddenly there's like a huge push, "Oh yeah, yeah, let's go build ever more autonomy into these li- these weapons and it's gonna be great." A- a- and, uh... I think, actually, people who are obsessed about some sort of future Terminator scenario right now are... Should start focusing on the fact that we have two much more urgent threats happening from machine learning. One of them is the whole destruction of democracy that we've talked about now-

Lex Fridman 1:16:58 Yes.

Max Tegmark 1:16:59 ... where, where, where our flow of information is being manipulated by machine learning, and the other one is that right now, you know, this is the year when the big arms race in... Out of control arms race in at least autonomous weapons is gonna start or it's gonna stop.

Lex Fridman 1:17:14 So you have a sense that there is, uh... Like 2020 was a instrumental catalyst for the race of... For the autonomous weapons race?

Max Tegmark 1:17:24 Yeah, 'cause it was the first year when, when they proved decisive in the battlefield.

Lex Fridman 1:17:28 Oh, man.

Max Tegmark 1:17:28 And, and these ones are still not fully autonomous, mostly they're remote controlled, right? But, you know, we could very quickly make things about, you know, the size and cost of a smartphone, which you just put in the GPS coordinates or the face of the one you want to kill, a skin color or whatever, and it flies away and, you know, does it, and... The, the real good reason why the US and all the other superpowers should put the kibosh on this is the same reason we decided to put the kibosh on bioweapons. So, you know, we gave the Future of Life Award, that we can talk more about later-

Lex Fridman 1:18:06 Yes.

Max Tegmark 1:18:06 ... to Mathew Meselson from Harvard before for convincing Nixon to ban bioweapons, and I asked him, "How did you do it?" (laughs) And he was like, "Well, I just said, 'Look, we don't want there to be a $500 weapon of mass destruction that even, all our enemies can afford, even non-state actors.'" And Nixon was like, "Good point." (laughs) Y- you know, it's in America's interest that the powerful weapons are all really expensive, so only we can afford them, or maybe some more stable adversaries, right? Uh, nuclear weapons are like that. But bioweapons were not like that, that's why we banned them, and that's why you never hear about them now. That's why we love biology.

Lex Fridman 1:18:50 So y- you have a sense that it's possible-

Max Tegmark 1:18:52 Oh, yeah.

Lex Fridman 1:18:52 ... for, for the big powerhouses, in terms of the, the big nations in the world to agree that autonomous weapons is not a race we wanna be on, that it doesn't end well?

Max Tegmark 1:19:03 ... yeah, because we, we know it's just gonna end in mass proliferation, and every terrorist e- everywhere is gonna have these super cheap weapons that they will use against us. Um, it, and our pa- and our politicians have to constantly worry about being assassinated every time they go outdoors by some anonymous little mini drone, you know? We don't want that. And i- if, even if the US and China and everyone else could just agree that you can only build these weapons if they cost at least 10 million bucks-

Lex Fridman 1:19:30 Mm-hmm.

Max Tegmark 1:19:31 ... that would, that would be a huge win for the superpowers and, frankly, for everybody. Um, the, um, you don't ... And people often push back and say, "Well, it's so hard to prevent cheating."

Lex Fridman 1:19:42 Mm-hmm.

Max Tegmark 1:19:43 But hey, you could say the same about bioweapons, you know? Take any of your, our MIT colleagues in biology. Of course they could build some nasty bioweapon if they really wanted to. But first of all, they don't want to, 'cause they think it's disgusting, 'cause of the stigma. And second, even if they're some sort of nutcase and want to, it's very likely that some of their grad students or someone would rat them out because everyone else-

Lex Fridman 1:20:06 Yes.

Max Tegmark 1:20:06 ... thinks it's so disgusting, right?

Lex Fridman 1:20:07 Yes. Yeah.

Max Tegmark 1:20:08 And in fact, we now know there was even a fair bit of cheating on the bioweapons ban.

Lex Fridman 1:20:12 Mm-hmm.

Max Tegmark 1:20:13 But none, no countries used them because it was so stigmatized that it just wasn't worth revealing that they had cheated.

Lex Fridman 1:20:22 You, you talk about drones, but you kind of think that drones is like remote operation.

Max Tegmark 1:20:29 Which they are mostly still.

Lex Fridman 1:20:30 Yes. But you're not taking the next intellectual step of like, "Where does this go?" You're kind of saying the problem with drones is that you're removing yourself from di- direct violence, therefore you're not able to sort of maintain the common humanity required to make the proper decisions strategically. But that's the criticism, as opposed to like if this is automated and just exactly as you said, if you automate it and there's a race, then you just going to ... The technology's going to get better and better and better, which means getting cheaper and cheaper and cheaper.

Max Tegmark 1:21:03 Yeah.

Lex Fridman 1:21:03 And unlike perhaps nuclear weapons, which is connected to, uh, resources in a way.

Max Tegmark 1:21:10 Yeah.

Lex Fridman 1:21:10 Like, it's hard to get the, uh, it's hard to engineer.

Max Tegmark 1:21:12 Uranium or plutonium.

Lex Fridman 1:21:13 Yeah. It feels like it's, you know, um, there's too much overlap between the tech industry and autonomous weapons to where you could have smartphone type of, uh, cheapness.

Narrator 1:21:24 Mm-hmm.

Lex Fridman 1:21:24 If you look at drones, uh, you know, it's a, it's a ... You know, for $1,000 you can have an incredible system that's able to maintain flight autonomously for you and take pictures and stuff. You could see that going into the autonomous weapon space that, um ... But like why is that not thought about or discussed enough in the public do you think? You see those dancing Boston Dynamics robots, and everybody has this kind of, um, like as if this is like a far future.

Max Tegmark 1:21:55 Yeah.

Lex Fridman 1:21:55 They have this like fear like, "Oh, this'll be Terminator in like some, I don't know, unspecified 20, 30, 40 years." And they don't think about, well, this is like some much less dramatic version of that is actually happening now. It's not gonna have, it's not gonna be legged, it's not gonna be dancing, but it's, this already has the capability to use artificial intelligence to kill humans.

Max Tegmark 1:22:20 Yeah. The Boston Dynamics leg robots, I think the reason we imagine them holding guns is just 'cause you've all seen Arnold Schwarzenegger.

Lex Fridman 1:22:26 Right.

Max Tegmark 1:22:27 Right? That's our reference point.

Lex Fridman 1:22:30 (laughs)

Max Tegmark 1:22:30 That's pretty useless. Uh, that's not gonna be the main military v- use of them.

Lex Fridman 1:22:35 Yeah.

Max Tegmark 1:22:35 They might be useful in s- in law enforcement, uh, in the future, and then there's a whole debate about do