Michael Littman: Reinforcement Learning and the Future of AI | Lex Fridman Podcast #144

1. 0:00 – 2:30

   Introduction

   1. LFLex Fridman0:00

      The following is a conversation with Michael Littman, a computer science professor at Brown University, doing research on and teaching machine learning, reinforcement learning, and artificial intelligence. He enjoys being silly and light-hearted in conversation, so this was definitely a fun one. Quick mention of each sponsor, followed by some thoughts related to the episode. Thank you to SimpliSafe, a home security company I use to monitor and protect my apartment; ExpressVPN, the VPN I've used for many years to protect my privacy on the internet; Masterclass, online courses that I enjoy from some of the most amazing humans in history; and BetterHelp, online therapy with a licensed professional. Please check out these sponsors in the description to get a discount and to support this podcast. As a side note, let me say that I may experiment with doing some solo episodes in the coming month or two. The three ideas I have floating in my head currently is to use, one, a particular moment in history; two, a particular movie; or three, a book, to, uh, drive a conversation about a set of, uh, related concepts. For example, I could use 2001: A Space Odyssey or Ex Machina to talk about AGI for one, two, three hours. Or I could do an episode on the, uh, yes, rise and fall of Hitler and Stalin each in a separate episode, using relevant books and historical moments for reference. I find the format of a solo episode very uncomfortable and challenging, but that just tells me that it's something I definitely need to do and learn from the experience. Of course, I hope you come along for the ride. Also, since we have all this momentum built up on announcements, I'm giving a few lectures on machine learning at MIT this January. In general, if you have ideas for the episodes, for the lectures, or for just short videos on YouTube, let me know in the comments that I still definitely read despite my better judgment and the wise sage advice of the great Joe Rogan. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter @lexfridman. And now, here's my conversation with Michael Littman.

2. 2:30 – 4:50

   Robot and Frank

   1. LFLex Fridman2:31

      I saw a video of you talking to Charles Isbell about Westworld, the TV series. You guys were doing the kind of thing where you're watching new things together. But let's rewind back. Is there a sci-fi movie or book or shows that you. That was profound, that had an impact on you philosophically or just, like, specifically something you enjoyed nerding out about?

   2. MLMichael Littman2:55

      (laughs) Yeah, interesting. I think a lot of us have been inspired by robots in movies. The one that I really like is, uh, there's a movie called Robot and Frank, which I think is really interesting 'cause it's very near-term future, where, uh, robots are being deployed as, uh, helpers in people's homes.

   3. LFLex Fridman3:14

      Mm-hmm.

   4. MLMichael Littman3:14

      And it was- it was... And we don't know how to make robots like that at this point, but it seemed very plausible. It seemed very realistic or imaginable. And I thought that was really cool because they d- (laughs) they're awkward, they do funny things that raise some interesting issues, but it seemed like something that would ultimately be helpful and good if we could do it right.

   5. LFLex Fridman3:33

      Yeah, he was an older cranky gentleman, right?

   6. MLMichael Littman3:34

      (laughs) He was an older cranky, uh, jewel thief, yeah.

   7. LFLex Fridman3:37

      It's- it's kind of a funny little thing which is, you know, he's a jewel thief and so he pulls the robot into his life which is like- which is something you could imagine taking a home robotics thing and pulling into whatever quirky thing that's involved in your existence.

   8. MLMichael Littman3:55

      If it's meaningful to you.

   9. LFLex Fridman3:57

      Yeah.

   10. MLMichael Littman3:57

       Ex- exactly so. Yeah, and I think- I think from that perspective, I mean, not all of us are jewel thieves and so when we bring our robots into our lives (laughs) -

   11. LFLex Fridman4:02

       Speak for yourself, yeah.

   12. MLMichael Littman4:05

       Uh, it explains a lot about this apartment actually. Uh...

   13. LFLex Fridman4:07

       (laughs)

   14. MLMichael Littman4:07

       But, no, the idea that it w- that- that people should have the ability to, you know, make this technology their own, that- that- that it becomes part of their lives. And- and I think that's... It's hard for us as technologists to make that kind of technology. It's easier to mold people into what we need them to be. And, um, just th- that opposite vision, I think, is really inspiring.

   15. LFLex Fridman4:27

       And then there's a, uh, anthropomorphization where we project certain things on them, 'cause I think the robot was kinda dumb. But I have a bunch of Roombas I play with and they- you- you immediately project stuff onto them.

   16. MLMichael Littman4:38

       Mm-hmm.

   17. LFLex Fridman4:38

       Much greater level of intelligence. We probably do that with each other too.

   18. MLMichael Littman4:42

       (laughs)

   19. LFLex Fridman4:42

       Much- much great- (laughs) much greater degree of compassion.

   20. MLMichael Littman4:44

       That's right. One- one of the things we're learning from AI is where we are smart and where we are not smart.

   21. LFLex Fridman4:49

       Yeah.

3. 4:50 – 8:01

   Music

   1. LFLex Fridman4:51

      You also enjoy, as people can see, and I enjoy myself, uh, watching you sing and even dance a little bit? A little bit?

   2. MLMichael Littman5:00

      (laughs)

   3. LFLex Fridman5:00

      A little bit- a little bit of dancing?

   4. MLMichael Littman5:02

      A little bit of dancing.

   5. LFLex Fridman5:03

      Uh-

   6. MLMichael Littman5:03

      It's not quite my thing.

   7. LFLex Fridman5:04

      ... as a- as a method of education, uh, or just in life, c- uh, you know, in general.

   8. MLMichael Littman5:10

      Mm-hmm. Mm-hmm.

   9. LFLex Fridman5:10

      So easy question, what's the definitive, objectively speaking, top three songs of all time?

   10. MLMichael Littman5:17

       (laughs)

   11. LFLex Fridman5:17

       Maybe something that, you know, uh, to walk that back a little bit, maybe something that others might be surprised by. The three s- three songs that you kind of enjoy.

   12. MLMichael Littman5:28

       That is a great question that I cannot answer.

   13. LFLex Fridman5:31

       Yeah.

   14. MLMichael Littman5:31

       But instead, let me tell you a story.

   15. LFLex Fridman5:32

       (laughs)

   16. MLMichael Littman5:33

       So, um-

   17. LFLex Fridman5:33

       Pick a question you do want to answer. (laughs)

   18. MLMichael Littman5:35

       (laughs) That's right. I've been watching the presidential debates and vice presidential debates and turns out, yeah, it's really- you can just answer any question you want. So- so-

   19. LFLex Fridman5:42

       Let me interrupt you.

   20. MLMichael Littman5:42

       ... it's a related question. (laughs)

   21. LFLex Fridman5:43

       No, I'm just kidding. (laughs)

   22. MLMichael Littman5:45

       (laughs) Yeah, well said. I really like pop music. I've enjoyed pop music ever since I was- I was very young. So '60s music, '70s music, '80s music, this is all awesome. And then I had kids and I think I stopped listening to music. And I was starting to realize that the- like, my musical taste had sort of frozen out. And so I decided...... in 2011, I think, to start listening to the top 10 Billboard songs each week. So I'd be on the, on the treadmill and I would listen to that week's top 10 songs so I could find out what was popular now. And what I discovered is that I have no musical taste whatsoever.

   23. LFLex Fridman6:18

       (laughs)

   24. MLMichael Littman6:18

       I like what I'm familiar with. And so-

   25. LFLex Fridman6:21

       Yeah.

   26. MLMichael Littman6:21

       ... the first time I'd hear a song, it's the first week that it was on the charts, I'd be like, "Ugh." And then the second week, I was into it a little bit, and the third week, I was loving it, and by the fourth week, it was, like, just part of me. And so I'm afraid that I can't tell you the most... my favorite song of all time-

   27. LFLex Fridman6:36

       (laughs)

   28. MLMichael Littman6:36

       ... because it's whatever I heard most recently.

   29. LFLex Fridman6:38

       Yeah. It, that's interesting, uh. People have told me that, um, there, there's an art to listening to music as well, that you can start to, if you listen to a song just carefully, like, ex- explicitly just force yourself to really listen, you start to, uh... I did this when I was part of a jazz band, a fusion band in, in college, is there's, they... You, you tr- you start to hear the layers of the instruments.

   30. MLMichael Littman7:02

       Mm-hmm.
4. 8:01 – 18:14

   Starring in a TurboTax commercial

   1. LFLex Fridman8:01

      really interesting.

   2. MLMichael Littman8:01

      (laughs)

   3. LFLex Fridman8:02

      So maybe on that topic, I've seen your, um, your, you're a celebrity in multiple dimensions, but one of them is you've done cameos in different places. I've seen you in a TurboTax commercial (laughs) as like, I guess, the, the brilliant Einstein character, and the, like, the point is that TurboTax doesn't need somebody like you.

   4. MLMichael Littman8:22

      (laughs) That's right.

   5. LFLex Fridman8:24

      It doesn't (laughs) , it doesn't-

   6. MLMichael Littman8:24

      (laughs)

   7. LFLex Fridman8:24

      ... it doesn't need a brilliant, uh, person with a-

   8. MLMichael Littman8:25

      Very few things need someone like me. But yes-

   9. LFLex Fridman8:27

      (laughs)

   10. MLMichael Littman8:27

       ... they were specifically emphasizing the idea that you don't need to be a, like, a computer expert to be able to use their software.

   11. LFLex Fridman8:33

       How'd you end up in that world?

   12. MLMichael Littman8:35

       I think it's an interesting story. So I was teaching my class. Uh, it was an intro computer science class for non-concentrators, non-majors. And, uh, I... Sometimes when people would visit campus, they would check in to say, "Hey, we wanna see what a class is like. Can we sit in on your class?" So, uh, a person came to my class who was the daughter of the brother of the hu- hus- husband of the best friend of my wife (laughs) . Anyway-

   13. LFLex Fridman9:07

       Hmm.

   14. MLMichael Littman9:07

       ... uh, basically a family friend came to campus to, to check out Brown, and asked to come to my class, and, and came with her dad. Her dad is, uh, who I've known from various kinds of family events and so forth, but he also does advertising. And he said that he was recruiting scientists for this, this, this ad, this, this TurboTax, uh, set of ads. And he said, "We wrote the ad with the idea that we get, like, the most brilliant researchers, um, but they all said no. So can you help us"-

   15. LFLex Fridman9:39

       (laughs)

   16. MLMichael Littman9:39

       ..."find the, like, B-level scientists?"

   17. LFLex Fridman9:43

       (laughs)

   18. MLMichael Littman9:43

       And I'm like, "Sure, that's, that's who I hang out with-"

   19. LFLex Fridman9:46

       (laughs)

   20. MLMichael Littman9:46

       "... so that should be fine."

   21. LFLex Fridman9:48

       Yeah.

   22. MLMichael Littman9:48

       So I put together a list and I did what some people called a Dick Cheney. So I included myself on the list-

   23. LFLex Fridman9:53

       Mm-hmm.

   24. MLMichael Littman9:53

       ... of possible candidates, uh, you know, with a little blurb about each one and why I thought-

   25. LFLex Fridman9:57

       Yeah.

   26. MLMichael Littman9:57

       ... that it would make sense for them to, to do it. And they reached out to a handful of them, but then they, ultimately, they YouTube stalked me a little bit and they thought, "Oh, I think he could do this." And, um, they said, "Okay, we're, we're gonna offer you the commercial." I'm like, "What?" So, um, it was a, it was such an interesting experience 'cause it's, it's... They have another world. The people who do, like, nationwide kind of ad campaigns and, and television shows and movies and so forth, it's quite a, uh, a remarkable system that they have going because they-

   27. LFLex Fridman10:28

       Is it like a set? It's-

   28. MLMichael Littman10:29

       Yeah, so I went to, uh, it was just somebody's (laughs) house that they rented in New Jersey. Um, but it, in the, in the commercial, it's just me and this other woman. In reality, there were 50 people in that room-

   29. LFLex Fridman10:42

       Wow.

   30. MLMichael Littman10:42

       ... and another, I don't know, half a dozen kind of spread out around the house in various ways. There were peoples who, whose job it was to control the sun. They were in the backyard on ladders putting, uh, filters up to try to make sure that the sun didn't glare off the window in a way that would wreck the shot.
5. 18:14 – 36:36

   Existential risks of AI

   1. LFLex Fridman18:14

      So one of the other things Charles said is that, you know-... everyone knows you as, like, a super nice guy, super passionate about teaching and so on. Uh, what he said, don't know if it's true, that despite the fact that you're, you are s-

   2. MLMichael Littman18:30

      Killed a man in cold blood.

   3. LFLex Fridman18:31

      (laughs)

   4. MLMichael Littman18:32

      Like, okay, all right.

   5. LFLex Fridman18:33

      Just, it's-

   6. MLMichael Littman18:33

      I will admit this finally-

   7. LFLex Fridman18:34

      It's the Joh-

   8. MLMichael Littman18:34

      ... for the first time, that was, that was me.

   9. LFLex Fridman18:36

      It's the Johnny Cash song.

   10. MLMichael Littman18:37

       (laughs)

   11. LFLex Fridman18:38

       Went and killed a man in Reno just to watch him die. Uh, (laughs) that you actually do have, uh, some strong opinions on some topics. So if this in fact is true, what, uh, strong opinions would you say you have? Is there ideas you think maybe in artificial intelligence, machine learning, maybe in life that you believe is true that others might, you know, some number of people might disagree with you on?

   12. MLMichael Littman19:06

       So I try very hard to see things from multiple perspectives. There's a, there's this great Calvin and Harbs, uh, Calvin and Hobbes cartoon where Cal- Do you know the Cal-

   13. LFLex Fridman19:16

       Yeah.

   14. MLMichael Littman19:16

       Okay. So Calvin's dad is always kind of a bit of a foil, and he, he was, he talked Calvin into s- Calvin had done something wrong, the dad talks him into like seeing it from another perspective. And Calvin, like, this breaks Calvin because he's like, "Oh my gosh, now I can see the opposite sides of things." And so the, it's, it's, it becomes like a cubist cartoon-

   15. LFLex Fridman19:34

       Mm-hmm.

   16. MLMichael Littman19:34

       ... where there's no front and back, everything's just exposed, and it really freaks him out. And finally, he settles back down. He's like, "Oh, good. No, I can make that go away." But, like, I'm that, I'm that. I live in that world where I tr- I'm trying to see everything from every perspective all the time. So there are some things that I've formed opinions about that I, would be harder, I think, to disavow me of. One is, um, the super intelligence argument and the existential threat of AI is one where I feel pretty confident in my feeling about that one. Like, I'm willing to hear other arguments, but like, I am not particularly moved by the idea that if we're not careful, we will accidentally create a super intelligence that will destroy human life.

   17. LFLex Fridman20:15

       Let's talk about that one. Let's get you in trouble and record you on video.

   18. MLMichael Littman20:18

       (laughs)

   19. LFLex Fridman20:19

       S- saying, it's like Bill Gates, uh, I think he said, like, some quote about the internet, that that's just, uh, gonna be a small thing, it's not gonna really go anywhere. And then I think, uh, S-

   20. MLMichael Littman20:29

       Mm-hmm.

   21. LFLex Fridman20:29

       ... Steve Ballmer said, uh, I don't know why I'm sticking on Microsoft, uh, that something that, like, smartphones are useless, there's no reason why Microsoft should get into smartphones, that kind of... So let's get, let's talk about AGI. As, as AGI is destroying the world, we'll look back at this video and see. No, uh, I think it's really interesting to actually talk about because nobody really knows the future, so you have to use your best intuition. It's very difficult to predict it. But you have spoken about AGI and the, the existential risks around it and sort of basing your intuition that we're quite far away from that being a serious concern relative to the other concerns we have. Can you maybe, uh, unpack that a little bit?

   22. MLMichael Littman21:12

       Yeah, sure, sure, sure. So, so as, as I understand it, the, uh, for example, I read Bostrom's book (laughs) and a bunch of other reading material about this sort of general way of thinking about the world, and, and I think the story goes something like this, that we will at some point create computers that, uh, are smart enough that they can help design the next version of themselves, which itself will be smarter than the previous version of themselves, and eventually bootstrapped up to being smarter than us, at which point we are essentially at the mercy of this sort of more powerful intellect, which in principle, uh, we don't have any control over what its goals are. And so if its goals are at all out of sync with our goals, like, the exi- for example, the continued existence of humanity-

   23. LFLex Fridman22:05

       Mm-hmm.

   24. MLMichael Littman22:06

       ... we won't be able to stop it. It'll be way more powerful than us, and we will be toast. So there's some, I don't know, very smart people who have signed on to that story, and it's a, it's a compelling story. I wa- I once... (laughs) Now I can really get myself in trouble.

   25. LFLex Fridman22:23

       (laughs)

   26. MLMichael Littman22:23

       I once wrote an op-ed about this specifically responding to some quotes from Elon Musk, who has been, you know, on this very podcast, uh, more than once, and...

   27. LFLex Fridman22:33

       Uh, E- E- AI is summoning the demon. I forget. There's a-

   28. MLMichael Littman22:36

       That's a thing he said, but then he came to Providence, Rhode Island, which is where I live-

   29. LFLex Fridman22:39

       Mm-hmm.

   30. MLMichael Littman22:40

       ... and said, uh, to the governors of a- all the states, "You know, you're worried about entirely the wrong thing. You need to be worried about AI. You need to be very, very worried about AI." So, uh, and s- peop- uh, journalists kind of reacted to that and they wanted to get people's, people's take, and I was like, "Okay, my, my, (laughs) my belief is that one of the things that makes Elon Musk so successful and so remarkable as an individual is that he believes in the power of ideas." He believes that you can h- you can, if you- you know, if you have a really good idea for getting into space, you can get into space. If you have a really good idea for a company or for how to change the way that people drive, you just have to do it and it, and it can happen. It's really natural to apply that same idea to AI. You see these systems that are doing some pretty remarkable computational tricks, uh, demonstrations, and then to take that idea and just push it all the way to the limit and think, "Okay, where does this go? Where is this gonna take us next?" And if you're a deep believer in the power of ideas, then it's really natural to believe that those ideas could be taken to the extreme and, and kill us. So I think, you know, his strength is also his undoing because that doesn't mean it's true. (laughs) Like, it doesn't mean that that has to happen, but it's natural for him to think that.
6. 36:36 – 1:02:24

   Reinforcement learning

   1. LFLex Fridman36:37

      You mentioned reinforcement learning. So you've, have, uh, a couple of years in the field? No.

   2. MLMichael Littman36:42

      (laughs)

   3. LFLex Fridman36:42

      Uh, quite, you know-

   4. MLMichael Littman36:44

      A few.

   5. LFLex Fridman36:44

      (laughs) Quite a few. Qu- uh, quite a long career in artificial intelligence broadly but in reinforcement learning specifically. Uh, can you maybe give a hint about your sense of the history of the field? In, in some ways it's changed with the advent of deep learning, but has, uh, l- long roots. Like, how is it weaved in and out of your own life? How have you seen the community change or maybe the ideas that it's playing with change?

   6. MLMichael Littman37:11

      I've had the privilege, the pleasure of being, of, of having almost a front row seat to a lot of this stuff, and it's been really f- really fun and interesting. So, uh, when I was in college in the '80s, early '80s, uh, the neural net thing was starting to happen and, uh, I was taking a lot of psychology classes and a lot of computer science classes as a college student, and I thought, "You know, something that can play tic-tac-toe and just, like, learn to get better at it, that oughta be a really easy thing." So I spent almost, (laughs) almost all of my, what would've been vacations during college, like, hacking on my home computer, trying to teach it how to play tic-tac-toe and-

   7. LFLex Fridman37:49

      What programming language did you learn?

   8. MLMichael Littman37:50

      Basic. Oh, yeah.

   9. LFLex Fridman37:51

      Basic.

   10. MLMichael Littman37:51

       That's, that's, I was, I, that's my first language, that's my native language.

   11. LFLex Fridman37:55

       Is that when you first fell in love with computer science, just, like, programming in Basic on that, uh... What was, what was the computer? Do you remember?

   12. MLMichael Littman38:02

       I had, I had a TRS-80 Model I before they were called Model Is 'cause there was nothing else. Uh, I got my computer in 1979. Uh, instead, so I was, uh, it was, I would've been bar mitzvahed-

   13. LFLex Fridman38:19

       Mm-hmm.

   14. MLMichael Littman38:19

       ... but instead of having a big party that my parents threw on my behalf, they just got me a computer, 'cause that's what I really, really, really wanted. I saw 'em in the, in the, in the mall in Radio Shack and I thought, "What? How are they doing that?"

   15. LFLex Fridman38:31

       Yeah. (laughs)

   16. MLMichael Littman38:31

       I would try to stump them. I would give them math problems like one plus, and then in parentheses, two plus one.

   17. LFLex Fridman38:36

       Yeah.

   18. MLMichael Littman38:36

       And I would always get it right. I'm like, "How do you know so much?"

   19. LFLex Fridman38:39

       It's magic.

   20. MLMichael Littman38:40

       Like, "I've had to s- go to algebra class for the last few years to learn this stuff, and you just seem to know." So I was, I was, yeah, I was smitten and, uh, got a computer. And I think ages 13 to 15. I have no memory of those years. (laughs) I think I just was in my room with a computer-

   21. LFLex Fridman38:57

       Listening to Billy Joel.

   22. MLMichael Littman38:58

       ... communing. Possibly listening to the radio, listening to Billy Joel. That was the one album I had, uh, on vinyl at that time. And, um, and then I got it on cassette tape, and that was really helpful.

   23. LFLex Fridman39:08

       (laughs)

   24. MLMichael Littman39:09

       'Cause then I could play it. I didn't have to go down to my parents' wifi. Or hi-fi, sorry. Uh, and at age 15, I remember kind of walking out and like, "Okay. I'm ready to talk to people again. Like, I've learned what I need to learn here." (laughs)

   25. LFLex Fridman39:22

       Mm-hmm.

   26. MLMichael Littman39:22

       And, um, so yeah, so, so that was-

   27. LFLex Fridman39:24

       (laughs)

   28. MLMichael Littman39:24

       ... that was my home computer. And so I went to college and I was like, "Oh, I'm totally gonna study computer science." And I, I opted... The college I chose specifically had a computer science major. The one that w- I really wanted, the college I really wanted to go to didn't, so bye-bye to them.

   29. LFLex Fridman39:37

       Which, which college did you go to?

   30. MLMichael Littman39:38

       So I went to Yale. Uh, Princeton would've been way more convenient and it was just a beautiful campus and it was close enough to home and I was really excited about Princeton, and I visited and I said, "So, computer science major." They're like, "Well, we have computer engineering." I'm like, "Ooh, I don't like that word 'engineering'." (laughs)
7. 1:02:24 – 1:12:03

   AlphaGo and David Silver

   1. LFLex Fridman1:02:24

   2. MLMichael Littman1:02:24

      Mm-hmm.

   3. LFLex Fridman1:02:24

      So that is... I don't know what to make of it. I think it would be interesting to hear what your opinions are on just how exciting, surprising, profound, interesting-

   4. MLMichael Littman1:02:39

      Mm-hmm.

   5. LFLex Fridman1:02:39

      ... or boring the breakthrough, uh, performance of AlphaZero was.

   6. MLMichael Littman1:02:45

      Okay. So AlphaGo knocked my socks off. That was- that was so remarkable. The-

   7. LFLex Fridman1:02:50

      Which aspect of it?

   8. MLMichael Littman1:02:53

      That- that- that- that they- they got it to work.

   9. LFLex Fridman1:02:55

      Yeah (laughs) .

   10. MLMichael Littman1:02:55

       That they actually were able to leverage a whole bunch of different ideas, integrate them into one giant system. Just the software engineering aspect of it is mind-blowing. I don't- I- I've never been a part of a program as complicated as the program that they built for that. And, um, and just the, you know, like- like Jerry Tessaro is a neural net whisperer, like, you know, David Silver is a kind of (laughs) neural net whisperer too. He was able to coax these networks and these new way out there architectures to do these, you know, solve these problems that, um, as you say, w- you know, when we were learning from, uh, AI, no one had an idea how to make it work. It was- it was remarkable that, um, these, you know, these- these techniques that were so good at playing chess and they could beat the world champion in chess, couldn't beat, you know, your typical Go-playing teenager in Go.

   11. LFLex Fridman1:03:46

       Mm-hmm.

   12. MLMichael Littman1:03:46

       So the fact that- that, you know, in a very short number of years, we kind of ramped up to, uh, trouncing people in Go just blew me away.

   13. LFLex Fridman1:03:56

       So you- you're kind of focusing on the engineering aspect which is also very surprising. I mean, there's something different about large well-funded companies. I mean, there's a compute aspect to it too.

   14. MLMichael Littman1:04:07

       Sure.

   15. LFLex Fridman1:04:08

       Like, that, of course, I mean, that's similar to Deep Blue, right? With, uh, with IBM. Like there's something important to be learned and remembered about a large company taking the ideas that are already out there and investing a few million dollars into it, or- or more. And so you're kind of saying the engineering is kind of fascinating both on the- with AlphaGo is probably just gathering all the data, right? The- of the- of the expert games, like organizing everything, actually doing distributed, uh, supervised learning, and to me, see, the engineering I kind of took for granted.

   16. MLMichael Littman1:04:48

       (laughs) Mm-hmm.

   17. LFLex Fridman1:04:49

       To me, philosophically being able to persist in the- in the face of like long odds-

   18. MLMichael Littman1:04:57

       Mm-hmm.

   19. LFLex Fridman1:04:58

       ... because it feels like, for me, I'll be one of the skeptical people in the room thinking that you can learn your way to- to beat Go. Like, I- it sounded like, especially with David Silver, it sounded like David was not confident at all.

   20. MLMichael Littman1:05:11

       Hmm.

   21. LFLex Fridman1:05:11

       It's like it was, like not... It- it's funny how confidence works.

   22. MLMichael Littman1:05:18

       Yeah.

   23. LFLex Fridman1:05:18

       It's like you're not like cocky about it, like but-

   24. MLMichael Littman1:05:24

       Right. 'Cause if you're cocky about it, you s- you kind of stop and stall and don't get anywhere.

   25. LFLex Fridman1:05:28

       Yeah. But there's like a hope-

   26. MLMichael Littman1:05:30

       Yeah.

   27. LFLex Fridman1:05:30

       ... that's unbreakable. Maybe that's better than confidence.

   28. MLMichael Littman1:05:33

       Mm-hmm.

   29. LFLex Fridman1:05:33

       It's a kind of wishful hope and a little dream and you almost don't wanna do anything else, you kind of keep doing it. That's- that seems to be the story. And I-

   30. MLMichael Littman1:05:43

       But with enough skepticism that you're looking for where the problems are and fighting through them.
8. 1:12:03 – 1:24:30

   Will neural networks achieve AGI?

   1. MLMichael Littman1:12:04

   2. LFLex Fridman1:12:04

      How far do you think ... okay, this is where you kind of bring up the, the Elon Musks and the Sam Harris', right? How far is your intuition about these kinds of self-play mechanisms being able to take us? 'Cause it feels-

   3. MLMichael Littman1:12:16

      Mm.

   4. LFLex Fridman1:12:16

      ... one of the ominous s- but stated calmly things that when I talked to David Silver he said is that they have not yet discovered a ceiling for AlphaZero, for example, on the game of Go or chess.

   5. MLMichael Littman1:12:32

      Oh, okay.

   6. LFLex Fridman1:12:32

      Like, it's, it keeps, no matter how much they compute, they throw at it, it keeps improving.

   7. MLMichael Littman1:12:37

      Mm.

   8. LFLex Fridman1:12:37

      So it's possible, it's very possible that you th- i- if you throw, you know, the sum, like 10X compute, that it will improve by 5X or something like that and i- uh, when stated calmly, it's so like, "Oh, yeah, I guess so." But-

   9. MLMichael Littman1:12:54

      (laughs)

   10. LFLex Fridman1:12:54

       But like, and then you think, like, "Well, are, can we potentially have, like, uh, continuations of Moore's Law in totally different way?" Like, broadly defined Moore's Law.

   11. MLMichael Littman1:13:04

       Right.

   12. LFLex Fridman1:13:05

       Not the conceptu-

   13. MLMichael Littman1:13:05

       Exponential improvement too.

   14. LFLex Fridman1:13:06

       Yeah, exponential improvement, like are we going to have an AlphaZero that swallows the world? Uh, so that- that-

   15. MLMichael Littman1:13:13

       But notice, it's not getting better at other things. It's getting better at Go.

   16. LFLex Fridman1:13:16

       Yeah.

   17. MLMichael Littman1:13:16

       And I don't, I think it's a, that's a big leap to say, "Okay, well, therefore, it's..."... better at other things. (laughs)

   18. LFLex Fridman1:13:22

       Well, I mean, the, the question is, how much of the game of life can be turned into-

   19. MLMichael Littman1:13:27

       Right. So that's a, that, I think, is a really good question. And I think that we don't, I don't think we, as a, I don't know, community, really know the, the answer to this. But, um, g- so, okay, so, so I went, I went to a talk, uh, by some experts on computer chess. So, in particular, computer chess is really interesting because for, you know, for, of course, for 1,000 years, humans were the best chess-playing things on the planet. Um, and then computers, like, edged ahead of the best person, and they've been ahead ever since. It's not like people have, have overtaken computers. But, um, but computers and people together have overtaken computers.

   20. LFLex Fridman1:14:06

       Right.

   21. MLMichael Littman1:14:07

       So, at least last time I checked, I don't know what the very latest is, but last time I checked that there were teams of people who could work with computer programs to defeat the best computer programs.

   22. LFLex Fridman1:14:18

       In the game of Go?

   23. MLMichael Littman1:14:18

       In the game of chess.

   24. LFLex Fridman1:14:19

       In the game of chess.

   25. MLMichael Littman1:14:20

       Right. And so using the information about how these things called Elo scores, this sort of notion of how strong a player are you, there's a, there's kind of a range of possible scores. And the s- the, you, you increment in score basically if you can beat another player of that lower score 62% of the time or something like that. Like, there's some threshold of if you can somewhat consistently beat someone, then you are of a higher score than that person. And there's a question as to how many times can you do that in chess, right? And so we know that there's a range of human ability levels that cap out with the best-playing humans, and the computers went a step beyond that.

   26. LFLex Fridman1:14:59

       Mm-hmm.

   27. MLMichael Littman1:15:00

       And computers and people together have not gone, I think, a full step beyond that. It feels... The estimates that they have is that it's starting to asymptote, that we've reached kind of the maximum, the best possible chess playing. And so that means that there's kind of a finite strategic depth, right? At some point, you just can't get any better at this game.

Episode duration: 1:56:32